2024-04-24T15:18:29.585 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-24T15:18:29.585 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_yuriw@teuthology --name yuriw-2024-04-24_14:09:24-rbd-wip-yuriw4-testing-20240423.151325-quincy-distro-default-smithi --verbose --description rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zstd supported-random-distro$/{centos_8} thrashers/cache thrashosds-health workloads/rbd_fsx_rate_limit} --archive /home/teuthworker/archive/yuriw-2024-04-24_14:09:24-rbd-wip-yuriw4-testing-20240423.151325-quincy-distro-default-smithi/7671570 -- /home/teuthworker/archive/yuriw-2024-04-24_14:09:24-rbd-wip-yuriw4-testing-20240423.151325-quincy-distro-default-smithi/7671570/orig.config.yaml 2024-04-24T15:18:29.621 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-24T15:18:29.708 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2024-04-24_14:09:24-rbd-wip-yuriw4-testing-20240423.151325-quincy-distro-default-smithi/7671570 branch: wip-yuriw4-testing-20240423.151325-quincy description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zstd supported-random-distro$/{centos_8} thrashers/cache thrashosds-health workloads/rbd_fsx_rate_limit} email: yweinste@redhat.com first_in_suite: false job_id: '7671570' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: yuriw-2024-04-24_14:09:24-rbd-wip-yuriw4-testing-20240423.151325-quincy-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 - machine: cpus: 1 disk: 40 ram: 8000 volumes: count: 4 size: 30 os_type: centos os_version: 8.stream overrides: admin_socket: branch: wip-yuriw4-testing-20240423.151325-quincy ceph: conf: client: rbd qos iops burst: 100 rbd qos iops limit: 50 rbd qos schedule tick min: 100 global: mon client directed command retry: 5 mon warn on pool no app: false ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bluestore block size: 96636764160 bluestore compression algorithm: zstd bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(CACHE_POOL_NEAR_FULL\) - \(CACHE_POOL_NO_HIT_SET\) - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request sha1: 330067dbce47655d1f762ef14d546d2fde16a6a9 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: 330067dbce47655d1f762ef14d546d2fde16a6a9 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuriw4-testing-20240423.151325-quincy sha1: 330067dbce47655d1f762ef14d546d2fde16a6a9 owner: scheduled_yuriw@teuthology priority: 99 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b seed: 9954 sha1: 330067dbce47655d1f762ef14d546d2fde16a6a9 sleep_before_teardown: 0 subset: 111/120000 suite: rbd suite_branch: wip-yuriw4-testing-20240423.151325-quincy suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_330067dbce47655d1f762ef14d546d2fde16a6a9/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 330067dbce47655d1f762ef14d546d2fde16a6a9 targets: smithi096.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFy0JYDV0SvTWoBphPv+zUKpuktGpfetG3mJ5AU80JookSLWtK1UsnuAmBiY/OxjCLA1Xo55UDJufJWJ59xjtDI= smithi151.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL+p3hm6p4IoLnIjP8Kf65mobZFShOZg/nGai6no81y6bfOQ4/2atdoMISqXDp7VqOHXVLFJWSPm4fVtLzxIZSA= tasks: - install: null - ceph: null - exec: client.0: - sudo ceph osd pool create cache 4 - sudo ceph osd tier add rbd cache - sudo ceph osd tier cache-mode cache writeback - sudo ceph osd tier set-overlay rbd cache - sudo ceph osd pool set cache hit_set_type bloom - sudo ceph osd pool set cache hit_set_count 8 - sudo ceph osd pool set cache hit_set_period 60 - sudo ceph osd pool set cache target_max_objects 250 - thrashosds: timeout: 1200 - rbd_fsx: clients: - client.0 ops: 6000 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-24_14:09:24 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-24T15:18:29.709 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_330067dbce47655d1f762ef14d546d2fde16a6a9/qa; will attempt to use it 2024-04-24T15:18:29.709 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_330067dbce47655d1f762ef14d546d2fde16a6a9/qa/tasks 2024-04-24T15:18:29.709 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-24T15:18:29.711 INFO:teuthology.task.internal:Checking packages... 2024-04-24T15:18:29.736 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '330067dbce47655d1f762ef14d546d2fde16a6a9' 2024-04-24T15:18:29.736 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-24T15:18:29.736 INFO:teuthology.packaging:ref: None 2024-04-24T15:18:29.736 INFO:teuthology.packaging:tag: None 2024-04-24T15:18:29.737 INFO:teuthology.packaging:branch: wip-yuriw4-testing-20240423.151325-quincy 2024-04-24T15:18:29.737 INFO:teuthology.packaging:sha1: 330067dbce47655d1f762ef14d546d2fde16a6a9 2024-04-24T15:18:29.737 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-yuriw4-testing-20240423.151325-quincy 2024-04-24T15:18:29.966 INFO:teuthology.task.internal:Found packages for ceph version 17.2.7-915.g330067db 2024-04-24T15:18:29.967 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-24T15:18:29.972 INFO:teuthology.task.internal:no buildpackages task found 2024-04-24T15:18:29.973 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-24T15:18:29.992 INFO:teuthology.task.internal:Saving configuration 2024-04-24T15:18:30.007 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-24T15:18:30.012 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-24T15:18:30.040 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi096.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-04-24_14:09:24-rbd-wip-yuriw4-testing-20240423.151325-quincy-distro-default-smithi/7671570', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 15:13:11.632128', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFy0JYDV0SvTWoBphPv+zUKpuktGpfetG3mJ5AU80JookSLWtK1UsnuAmBiY/OxjCLA1Xo55UDJufJWJ59xjtDI='} 2024-04-24T15:18:30.061 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi151.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-04-24_14:09:24-rbd-wip-yuriw4-testing-20240423.151325-quincy-distro-default-smithi/7671570', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 15:13:11.633964', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL+p3hm6p4IoLnIjP8Kf65mobZFShOZg/nGai6no81y6bfOQ4/2atdoMISqXDp7VqOHXVLFJWSPm4fVtLzxIZSA='} 2024-04-24T15:18:30.061 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-24T15:18:30.067 INFO:teuthology.task.internal:roles: ubuntu@smithi096.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2024-04-24T15:18:30.067 INFO:teuthology.task.internal:roles: ubuntu@smithi151.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-04-24T15:18:30.067 INFO:teuthology.run_tasks:Running task console_log... 2024-04-24T15:18:30.140 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f709373df70>, signals=[15]) 2024-04-24T15:18:30.141 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-24T15:18:30.147 INFO:teuthology.task.internal:Opening connections... 2024-04-24T15:18:30.147 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi096.front.sepia.ceph.com 2024-04-24T15:18:30.149 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T15:18:30.224 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi151.front.sepia.ceph.com 2024-04-24T15:18:30.225 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T15:18:30.297 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-24T15:18:30.304 DEBUG:teuthology.orchestra.run.smithi096:> uname -m 2024-04-24T15:18:30.326 INFO:teuthology.orchestra.run.smithi096.stdout:x86_64 2024-04-24T15:18:30.326 DEBUG:teuthology.orchestra.run.smithi096:> cat /etc/os-release 2024-04-24T15:18:30.388 INFO:teuthology.orchestra.run.smithi096.stdout:NAME="CentOS Stream" 2024-04-24T15:18:30.388 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION="8" 2024-04-24T15:18:30.388 INFO:teuthology.orchestra.run.smithi096.stdout:ID="centos" 2024-04-24T15:18:30.388 INFO:teuthology.orchestra.run.smithi096.stdout:ID_LIKE="rhel fedora" 2024-04-24T15:18:30.388 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION_ID="8" 2024-04-24T15:18:30.388 INFO:teuthology.orchestra.run.smithi096.stdout:PLATFORM_ID="platform:el8" 2024-04-24T15:18:30.388 INFO:teuthology.orchestra.run.smithi096.stdout:PRETTY_NAME="CentOS Stream 8" 2024-04-24T15:18:30.388 INFO:teuthology.orchestra.run.smithi096.stdout:ANSI_COLOR="0;31" 2024-04-24T15:18:30.389 INFO:teuthology.orchestra.run.smithi096.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-04-24T15:18:30.389 INFO:teuthology.orchestra.run.smithi096.stdout:HOME_URL="https://centos.org/" 2024-04-24T15:18:30.389 INFO:teuthology.orchestra.run.smithi096.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-24T15:18:30.389 INFO:teuthology.orchestra.run.smithi096.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-04-24T15:18:30.389 INFO:teuthology.orchestra.run.smithi096.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-24T15:18:30.389 INFO:teuthology.lock.ops:Updating smithi096.front.sepia.ceph.com on lock server 2024-04-24T15:18:30.409 DEBUG:teuthology.orchestra.run.smithi151:> uname -m 2024-04-24T15:18:30.431 INFO:teuthology.orchestra.run.smithi151.stdout:x86_64 2024-04-24T15:18:30.431 DEBUG:teuthology.orchestra.run.smithi151:> cat /etc/os-release 2024-04-24T15:18:30.491 INFO:teuthology.orchestra.run.smithi151.stdout:NAME="CentOS Stream" 2024-04-24T15:18:30.492 INFO:teuthology.orchestra.run.smithi151.stdout:VERSION="8" 2024-04-24T15:18:30.492 INFO:teuthology.orchestra.run.smithi151.stdout:ID="centos" 2024-04-24T15:18:30.492 INFO:teuthology.orchestra.run.smithi151.stdout:ID_LIKE="rhel fedora" 2024-04-24T15:18:30.492 INFO:teuthology.orchestra.run.smithi151.stdout:VERSION_ID="8" 2024-04-24T15:18:30.492 INFO:teuthology.orchestra.run.smithi151.stdout:PLATFORM_ID="platform:el8" 2024-04-24T15:18:30.492 INFO:teuthology.orchestra.run.smithi151.stdout:PRETTY_NAME="CentOS Stream 8" 2024-04-24T15:18:30.492 INFO:teuthology.orchestra.run.smithi151.stdout:ANSI_COLOR="0;31" 2024-04-24T15:18:30.492 INFO:teuthology.orchestra.run.smithi151.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-04-24T15:18:30.492 INFO:teuthology.orchestra.run.smithi151.stdout:HOME_URL="https://centos.org/" 2024-04-24T15:18:30.492 INFO:teuthology.orchestra.run.smithi151.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-24T15:18:30.492 INFO:teuthology.orchestra.run.smithi151.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-04-24T15:18:30.492 INFO:teuthology.orchestra.run.smithi151.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-24T15:18:30.493 INFO:teuthology.lock.ops:Updating smithi151.front.sepia.ceph.com on lock server 2024-04-24T15:18:30.517 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-24T15:18:30.526 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-24T15:18:30.531 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-24T15:18:30.531 DEBUG:teuthology.orchestra.run.smithi096:> test '!' -e /home/ubuntu/cephtest 2024-04-24T15:18:30.533 DEBUG:teuthology.orchestra.run.smithi151:> test '!' -e /home/ubuntu/cephtest 2024-04-24T15:18:30.552 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-24T15:18:30.558 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-24T15:18:30.558 DEBUG:teuthology.orchestra.run.smithi096:> test -z $(ls -A /var/lib/ceph) 2024-04-24T15:18:30.595 DEBUG:teuthology.orchestra.run.smithi151:> test -z $(ls -A /var/lib/ceph) 2024-04-24T15:18:30.630 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-24T15:18:30.682 INFO:teuthology.run_tasks:Running task kernel... 2024-04-24T15:18:30.696 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-04-24T15:18:30.696 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-04-24T15:18:30.696 DEBUG:teuthology.orchestra.run.smithi096:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-24T15:18:30.697 DEBUG:teuthology.orchestra.run.smithi151:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-24T15:18:30.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T15:18:30.717 DEBUG:teuthology.orchestra.run.smithi151:> uname -r 2024-04-24T15:18:30.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T15:18:30.718 DEBUG:teuthology.orchestra.run.smithi096:> uname -r 2024-04-24T15:18:30.777 INFO:teuthology.orchestra.run.smithi151.stdout:4.18.0-489.el8.x86_64 2024-04-24T15:18:30.777 INFO:teuthology.task.kernel:Running kernel on smithi151: 4.18.0-489.el8.x86_64 2024-04-24T15:18:30.777 DEBUG:teuthology.orchestra.run.smithi151:> sudo yum install -y kernel 2024-04-24T15:18:30.778 INFO:teuthology.orchestra.run.smithi096.stdout:4.18.0-489.el8.x86_64 2024-04-24T15:18:30.778 INFO:teuthology.task.kernel:Running kernel on smithi096: 4.18.0-489.el8.x86_64 2024-04-24T15:18:30.778 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum install -y kernel 2024-04-24T15:18:35.948 INFO:teuthology.orchestra.run.smithi151.stdout:CentOS Stream 8 - AppStream 11 MB/s | 28 MB 00:02 2024-04-24T15:18:36.087 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 8 - AppStream 10 MB/s | 28 MB 00:02 2024-04-24T15:18:42.623 INFO:teuthology.orchestra.run.smithi151.stdout:CentOS Stream 8 - BaseOS 6.5 MB/s | 10 MB 00:01 2024-04-24T15:18:42.770 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 8 - BaseOS 6.2 MB/s | 10 MB 00:01 2024-04-24T15:18:44.523 INFO:teuthology.orchestra.run.smithi151.stdout:CentOS Stream 8 - Extras 73 kB/s | 18 kB 00:00 2024-04-24T15:18:44.637 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 8 - Extras 82 kB/s | 18 kB 00:00 2024-04-24T15:18:44.887 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 8 - Extras common packages 54 kB/s | 7.7 kB 00:00 2024-04-24T15:18:44.982 INFO:teuthology.orchestra.run.smithi151.stdout:CentOS Stream 8 - Extras common packages 22 kB/s | 7.7 kB 00:00 2024-04-24T15:18:45.155 INFO:teuthology.orchestra.run.smithi096.stdout:Copr repo for python3-asyncssh owned by ceph 19 kB/s | 3.5 kB 00:00 2024-04-24T15:18:45.209 INFO:teuthology.orchestra.run.smithi151.stdout:Copr repo for python3-asyncssh owned by ceph 26 kB/s | 3.5 kB 00:00 2024-04-24T15:18:47.409 INFO:teuthology.orchestra.run.smithi151.stdout:Extra Packages for Enterprise Linux 7.7 MB/s | 16 MB 00:02 2024-04-24T15:18:47.648 INFO:teuthology.orchestra.run.smithi096.stdout:Extra Packages for Enterprise Linux 6.8 MB/s | 16 MB 00:02 2024-04-24T15:18:51.218 INFO:teuthology.orchestra.run.smithi151.stdout:lab-extras 493 kB/s | 24 kB 00:00 2024-04-24T15:18:51.372 INFO:teuthology.orchestra.run.smithi096.stdout:lab-extras 488 kB/s | 24 kB 00:00 2024-04-24T15:18:54.452 INFO:teuthology.orchestra.run.smithi096.stdout:Package kernel-4.18.0-348.el8.x86_64 is already installed. 2024-04-24T15:18:54.452 INFO:teuthology.orchestra.run.smithi096.stdout:Package kernel-4.18.0-489.el8.x86_64 is already installed. 2024-04-24T15:18:54.456 INFO:teuthology.orchestra.run.smithi151.stdout:Package kernel-4.18.0-348.el8.x86_64 is already installed. 2024-04-24T15:18:54.456 INFO:teuthology.orchestra.run.smithi151.stdout:Package kernel-4.18.0-489.el8.x86_64 is already installed. 2024-04-24T15:18:54.579 INFO:teuthology.orchestra.run.smithi151.stdout:Dependencies resolved. 2024-04-24T15:18:54.580 INFO:teuthology.orchestra.run.smithi151.stdout:================================================================================ 2024-04-24T15:18:54.580 INFO:teuthology.orchestra.run.smithi151.stdout: Package Architecture Version Repository Size 2024-04-24T15:18:54.580 INFO:teuthology.orchestra.run.smithi151.stdout:================================================================================ 2024-04-24T15:18:54.580 INFO:teuthology.orchestra.run.smithi151.stdout:Installing: 2024-04-24T15:18:54.580 INFO:teuthology.orchestra.run.smithi151.stdout: kernel x86_64 4.18.0-552.el8 baseos 10 M 2024-04-24T15:18:54.580 INFO:teuthology.orchestra.run.smithi151.stdout:Installing dependencies: 2024-04-24T15:18:54.580 INFO:teuthology.orchestra.run.smithi151.stdout: kernel-core x86_64 4.18.0-552.el8 baseos 44 M 2024-04-24T15:18:54.580 INFO:teuthology.orchestra.run.smithi151.stdout: kernel-modules x86_64 4.18.0-552.el8 baseos 36 M 2024-04-24T15:18:54.580 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:18:54.580 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction Summary 2024-04-24T15:18:54.581 INFO:teuthology.orchestra.run.smithi151.stdout:================================================================================ 2024-04-24T15:18:54.581 INFO:teuthology.orchestra.run.smithi151.stdout:Install 3 Packages 2024-04-24T15:18:54.581 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:18:54.582 INFO:teuthology.orchestra.run.smithi151.stdout:Total download size: 90 M 2024-04-24T15:18:54.582 INFO:teuthology.orchestra.run.smithi151.stdout:Installed size: 96 M 2024-04-24T15:18:54.582 INFO:teuthology.orchestra.run.smithi151.stdout:Downloading Packages: 2024-04-24T15:18:54.582 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-04-24T15:18:54.582 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-04-24T15:18:54.582 INFO:teuthology.orchestra.run.smithi096.stdout: Package Architecture Version Repository Size 2024-04-24T15:18:54.583 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-04-24T15:18:54.583 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-04-24T15:18:54.583 INFO:teuthology.orchestra.run.smithi096.stdout: kernel x86_64 4.18.0-552.el8 baseos 10 M 2024-04-24T15:18:54.583 INFO:teuthology.orchestra.run.smithi096.stdout:Installing dependencies: 2024-04-24T15:18:54.583 INFO:teuthology.orchestra.run.smithi096.stdout: kernel-core x86_64 4.18.0-552.el8 baseos 44 M 2024-04-24T15:18:54.583 INFO:teuthology.orchestra.run.smithi096.stdout: kernel-modules x86_64 4.18.0-552.el8 baseos 36 M 2024-04-24T15:18:54.583 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:18:54.583 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-04-24T15:18:54.583 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-04-24T15:18:54.583 INFO:teuthology.orchestra.run.smithi096.stdout:Install 3 Packages 2024-04-24T15:18:54.583 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:18:54.583 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 90 M 2024-04-24T15:18:54.583 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 96 M 2024-04-24T15:18:54.584 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-04-24T15:18:56.073 INFO:teuthology.orchestra.run.smithi151.stdout:(1/3): kernel-4.18.0-552.el8.x86_64.rpm 7.2 MB/s | 10 MB 00:01 2024-04-24T15:18:56.115 INFO:teuthology.orchestra.run.smithi096.stdout:(1/3): kernel-4.18.0-552.el8.x86_64.rpm 7.0 MB/s | 10 MB 00:01 2024-04-24T15:18:57.432 INFO:teuthology.orchestra.run.smithi096.stdout:(2/3): kernel-modules-4.18.0-552.el8.x86_64.rpm 13 MB/s | 36 MB 00:02 2024-04-24T15:18:57.502 INFO:teuthology.orchestra.run.smithi151.stdout:(2/3): kernel-modules-4.18.0-552.el8.x86_64.rpm 13 MB/s | 36 MB 00:02 2024-04-24T15:18:58.069 INFO:teuthology.orchestra.run.smithi151.stdout:(3/3): kernel-core-4.18.0-552.el8.x86_64.rpm 13 MB/s | 44 MB 00:03 2024-04-24T15:18:58.070 INFO:teuthology.orchestra.run.smithi151.stdout:-------------------------------------------------------------------------------- 2024-04-24T15:18:58.070 INFO:teuthology.orchestra.run.smithi151.stdout:Total 26 MB/s | 90 MB 00:03 2024-04-24T15:18:58.199 INFO:teuthology.orchestra.run.smithi096.stdout:(3/3): kernel-core-4.18.0-552.el8.x86_64.rpm 12 MB/s | 44 MB 00:03 2024-04-24T15:18:58.199 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-04-24T15:18:58.199 INFO:teuthology.orchestra.run.smithi096.stdout:Total 25 MB/s | 90 MB 00:03 2024-04-24T15:18:58.752 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction check 2024-04-24T15:18:58.882 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-04-24T15:18:59.223 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction check succeeded. 2024-04-24T15:18:59.223 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction test 2024-04-24T15:18:59.302 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-04-24T15:18:59.302 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-04-24T15:19:00.479 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction test succeeded. 2024-04-24T15:19:00.500 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction 2024-04-24T15:19:00.530 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-04-24T15:19:00.555 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-04-24T15:19:03.315 INFO:teuthology.orchestra.run.smithi151.stdout: Preparing : 1/1 2024-04-24T15:19:03.437 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-04-24T15:19:03.874 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : kernel-core-4.18.0-552.el8.x86_64 1/3 2024-04-24T15:19:03.945 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : kernel-core-4.18.0-552.el8.x86_64 1/3 2024-04-24T15:19:07.445 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: kernel-core-4.18.0-552.el8.x86_64 1/3 2024-04-24T15:19:07.630 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: kernel-core-4.18.0-552.el8.x86_64 1/3 2024-04-24T15:19:07.815 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : kernel-modules-4.18.0-552.el8.x86_64 2/3 2024-04-24T15:19:08.010 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : kernel-modules-4.18.0-552.el8.x86_64 2/3 2024-04-24T15:19:11.257 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: kernel-modules-4.18.0-552.el8.x86_64 2/3 2024-04-24T15:19:11.444 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: kernel-modules-4.18.0-552.el8.x86_64 2/3 2024-04-24T15:19:11.530 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : kernel-4.18.0-552.el8.x86_64 3/3 2024-04-24T15:19:11.734 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : kernel-4.18.0-552.el8.x86_64 3/3 2024-04-24T15:19:33.589 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: kernel-core-4.18.0-552.el8.x86_64 3/3 2024-04-24T15:19:33.614 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: kernel-modules-4.18.0-552.el8.x86_64 3/3 2024-04-24T15:19:33.835 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: kernel-core-4.18.0-552.el8.x86_64 3/3 2024-04-24T15:19:33.861 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: kernel-modules-4.18.0-552.el8.x86_64 3/3 2024-04-24T15:19:33.962 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: kernel-4.18.0-552.el8.x86_64 3/3 2024-04-24T15:19:33.962 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : kernel-4.18.0-552.el8.x86_64 1/3 2024-04-24T15:19:33.962 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : kernel-core-4.18.0-552.el8.x86_64 2/3 2024-04-24T15:19:34.213 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: kernel-4.18.0-552.el8.x86_64 3/3 2024-04-24T15:19:34.213 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : kernel-4.18.0-552.el8.x86_64 1/3 2024-04-24T15:19:34.213 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : kernel-core-4.18.0-552.el8.x86_64 2/3 2024-04-24T15:19:34.490 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : kernel-modules-4.18.0-552.el8.x86_64 3/3 2024-04-24T15:19:34.490 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:19:34.491 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-04-24T15:19:34.491 INFO:teuthology.orchestra.run.smithi096.stdout: kernel-4.18.0-552.el8.x86_64 kernel-core-4.18.0-552.el8.x86_64 2024-04-24T15:19:34.491 INFO:teuthology.orchestra.run.smithi096.stdout: kernel-modules-4.18.0-552.el8.x86_64 2024-04-24T15:19:34.491 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:19:34.491 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-04-24T15:19:34.632 DEBUG:teuthology.orchestra.run.smithi096:> rpm -q kernel | sort -rV | head -n 1 2024-04-24T15:19:34.853 INFO:teuthology.orchestra.run.smithi096.stdout:kernel-4.18.0-552.el8.x86_64 2024-04-24T15:19:34.853 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-552.el8.x86_64 2024-04-24T15:19:34.853 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-489.el8.x86_64 Expected: 4.18.0-552.el8.x86_64 2024-04-24T15:19:34.853 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-24T15:19:34.853 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-24T15:19:34.854 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-04-24T15:19:34.854 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi096.front.sepia.ceph.com, path=None, version=distro) 2024-04-24T15:19:34.854 DEBUG:teuthology.orchestra.run.smithi096:> rpm -q kernel | sort -rV | head -n 1 2024-04-24T15:19:35.064 INFO:teuthology.orchestra.run.smithi096.stdout:kernel-4.18.0-552.el8.x86_64 2024-04-24T15:19:35.064 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-552.el8.x86_64 2024-04-24T15:19:35.065 DEBUG:teuthology.orchestra.run.smithi096:> sudo rpm -qi grub2-tools 2024-04-24T15:19:35.098 INFO:teuthology.orchestra.run.smithi096.stdout:Name : grub2-tools 2024-04-24T15:19:35.098 INFO:teuthology.orchestra.run.smithi096.stdout:Epoch : 1 2024-04-24T15:19:35.098 INFO:teuthology.orchestra.run.smithi096.stdout:Version : 2.02 2024-04-24T15:19:35.099 INFO:teuthology.orchestra.run.smithi096.stdout:Release : 129.el8 2024-04-24T15:19:35.099 INFO:teuthology.orchestra.run.smithi096.stdout:Architecture: x86_64 2024-04-24T15:19:35.099 INFO:teuthology.orchestra.run.smithi096.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2024-04-24T15:19:35.099 INFO:teuthology.orchestra.run.smithi096.stdout:Group : System Environment/Base 2024-04-24T15:19:35.099 INFO:teuthology.orchestra.run.smithi096.stdout:Size : 9265586 2024-04-24T15:19:35.099 INFO:teuthology.orchestra.run.smithi096.stdout:License : GPLv3+ 2024-04-24T15:19:35.099 INFO:teuthology.orchestra.run.smithi096.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2024-04-24T15:19:35.099 INFO:teuthology.orchestra.run.smithi096.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2024-04-24T15:19:35.099 INFO:teuthology.orchestra.run.smithi096.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2024-04-24T15:19:35.099 INFO:teuthology.orchestra.run.smithi096.stdout:Build Host : kbuilder.bsys.centos.org 2024-04-24T15:19:35.099 INFO:teuthology.orchestra.run.smithi096.stdout:Relocations : (not relocatable) 2024-04-24T15:19:35.099 INFO:teuthology.orchestra.run.smithi096.stdout:Packager : CentOS BuildSystem 2024-04-24T15:19:35.100 INFO:teuthology.orchestra.run.smithi096.stdout:Vendor : CentOS 2024-04-24T15:19:35.100 INFO:teuthology.orchestra.run.smithi096.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-24T15:19:35.100 INFO:teuthology.orchestra.run.smithi096.stdout:Summary : Support tools for GRUB. 2024-04-24T15:19:35.100 INFO:teuthology.orchestra.run.smithi096.stdout:Description : 2024-04-24T15:19:35.100 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:19:35.100 INFO:teuthology.orchestra.run.smithi096.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-24T15:19:35.100 INFO:teuthology.orchestra.run.smithi096.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-24T15:19:35.100 INFO:teuthology.orchestra.run.smithi096.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-24T15:19:35.100 INFO:teuthology.orchestra.run.smithi096.stdout:hardware devices. 2024-04-24T15:19:35.100 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:19:35.100 INFO:teuthology.orchestra.run.smithi096.stdout:This subpackage provides tools for support of all platforms. 2024-04-24T15:19:35.101 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-24T15:19:35.101 INFO:teuthology.task.kernel:Updating grub on smithi096 to boot 4.18.0-552.el8.x86_64 2024-04-24T15:19:35.101 DEBUG:teuthology.orchestra.run.smithi096:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-24T15:19:35.169 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : kernel-modules-4.18.0-552.el8.x86_64 3/3 2024-04-24T15:19:35.169 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:19:35.170 INFO:teuthology.orchestra.run.smithi151.stdout:Installed: 2024-04-24T15:19:35.170 INFO:teuthology.orchestra.run.smithi151.stdout: kernel-4.18.0-552.el8.x86_64 kernel-core-4.18.0-552.el8.x86_64 2024-04-24T15:19:35.170 INFO:teuthology.orchestra.run.smithi151.stdout: kernel-modules-4.18.0-552.el8.x86_64 2024-04-24T15:19:35.170 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:19:35.170 INFO:teuthology.orchestra.run.smithi151.stdout:Complete! 2024-04-24T15:19:35.328 DEBUG:teuthology.orchestra.run.smithi151:> rpm -q kernel | sort -rV | head -n 1 2024-04-24T15:19:35.546 INFO:teuthology.orchestra.run.smithi151.stdout:kernel-4.18.0-552.el8.x86_64 2024-04-24T15:19:35.546 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-552.el8.x86_64 2024-04-24T15:19:35.546 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-489.el8.x86_64 Expected: 4.18.0-552.el8.x86_64 2024-04-24T15:19:35.546 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-24T15:19:35.546 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-24T15:19:35.547 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-04-24T15:19:35.547 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi151.front.sepia.ceph.com, path=None, version=distro) 2024-04-24T15:19:35.547 DEBUG:teuthology.orchestra.run.smithi151:> rpm -q kernel | sort -rV | head -n 1 2024-04-24T15:19:35.758 INFO:teuthology.orchestra.run.smithi151.stdout:kernel-4.18.0-552.el8.x86_64 2024-04-24T15:19:35.758 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-552.el8.x86_64 2024-04-24T15:19:35.759 DEBUG:teuthology.orchestra.run.smithi151:> sudo rpm -qi grub2-tools 2024-04-24T15:19:35.792 INFO:teuthology.orchestra.run.smithi151.stdout:Name : grub2-tools 2024-04-24T15:19:35.792 INFO:teuthology.orchestra.run.smithi151.stdout:Epoch : 1 2024-04-24T15:19:35.792 INFO:teuthology.orchestra.run.smithi151.stdout:Version : 2.02 2024-04-24T15:19:35.793 INFO:teuthology.orchestra.run.smithi151.stdout:Release : 129.el8 2024-04-24T15:19:35.793 INFO:teuthology.orchestra.run.smithi151.stdout:Architecture: x86_64 2024-04-24T15:19:35.793 INFO:teuthology.orchestra.run.smithi151.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2024-04-24T15:19:35.793 INFO:teuthology.orchestra.run.smithi151.stdout:Group : System Environment/Base 2024-04-24T15:19:35.793 INFO:teuthology.orchestra.run.smithi151.stdout:Size : 9265586 2024-04-24T15:19:35.793 INFO:teuthology.orchestra.run.smithi151.stdout:License : GPLv3+ 2024-04-24T15:19:35.793 INFO:teuthology.orchestra.run.smithi151.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2024-04-24T15:19:35.793 INFO:teuthology.orchestra.run.smithi151.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2024-04-24T15:19:35.793 INFO:teuthology.orchestra.run.smithi151.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2024-04-24T15:19:35.793 INFO:teuthology.orchestra.run.smithi151.stdout:Build Host : kbuilder.bsys.centos.org 2024-04-24T15:19:35.793 INFO:teuthology.orchestra.run.smithi151.stdout:Relocations : (not relocatable) 2024-04-24T15:19:35.793 INFO:teuthology.orchestra.run.smithi151.stdout:Packager : CentOS BuildSystem 2024-04-24T15:19:35.793 INFO:teuthology.orchestra.run.smithi151.stdout:Vendor : CentOS 2024-04-24T15:19:35.794 INFO:teuthology.orchestra.run.smithi151.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-24T15:19:35.794 INFO:teuthology.orchestra.run.smithi151.stdout:Summary : Support tools for GRUB. 2024-04-24T15:19:35.794 INFO:teuthology.orchestra.run.smithi151.stdout:Description : 2024-04-24T15:19:35.794 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:19:35.794 INFO:teuthology.orchestra.run.smithi151.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-24T15:19:35.794 INFO:teuthology.orchestra.run.smithi151.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-24T15:19:35.794 INFO:teuthology.orchestra.run.smithi151.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-24T15:19:35.794 INFO:teuthology.orchestra.run.smithi151.stdout:hardware devices. 2024-04-24T15:19:35.794 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:19:35.794 INFO:teuthology.orchestra.run.smithi151.stdout:This subpackage provides tools for support of all platforms. 2024-04-24T15:19:35.795 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-24T15:19:35.795 INFO:teuthology.task.kernel:Updating grub on smithi151 to boot 4.18.0-552.el8.x86_64 2024-04-24T15:19:35.795 DEBUG:teuthology.orchestra.run.smithi151:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-24T15:19:36.951 INFO:teuthology.orchestra.run.smithi096.stderr:Generating grub configuration file ... 2024-04-24T15:19:37.210 INFO:teuthology.orchestra.run.smithi151.stderr:Generating grub configuration file ... 2024-04-24T15:19:39.390 INFO:teuthology.orchestra.run.smithi151.stderr:done 2024-04-24T15:19:39.392 DEBUG:teuthology.orchestra.run.smithi151:> mktemp 2024-04-24T15:19:39.432 INFO:teuthology.orchestra.run.smithi151.stdout:/tmp/tmp.OYjN1G2IA3 2024-04-24T15:19:39.432 DEBUG:teuthology.orchestra.run.smithi151:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.OYjN1G2IA3 2024-04-24T15:19:39.506 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod 0666 /tmp/tmp.OYjN1G2IA3 2024-04-24T15:19:39.564 INFO:teuthology.orchestra.run.smithi096.stderr:done 2024-04-24T15:19:39.565 DEBUG:teuthology.orchestra.run.smithi096:> mktemp 2024-04-24T15:19:39.607 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/tmp.byBvwLFoWG 2024-04-24T15:19:39.608 DEBUG:teuthology.orchestra.run.smithi096:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.byBvwLFoWG 2024-04-24T15:19:39.655 DEBUG:teuthology.orchestra.remote:smithi151:/tmp/tmp.OYjN1G2IA3 is 6KB 2024-04-24T15:19:39.685 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 0666 /tmp/tmp.byBvwLFoWG 2024-04-24T15:19:39.706 DEBUG:teuthology.orchestra.run.smithi151:> rm -fr /tmp/tmp.OYjN1G2IA3 2024-04-24T15:19:39.724 DEBUG:teuthology.orchestra.run.smithi151:> sudo /bin/ls /boot/loader/entries || true 2024-04-24T15:19:39.797 INFO:teuthology.orchestra.run.smithi151.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2024-04-24T15:19:39.797 INFO:teuthology.orchestra.run.smithi151.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2024-04-24T15:19:39.798 INFO:teuthology.orchestra.run.smithi151.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2024-04-24T15:19:39.798 INFO:teuthology.orchestra.run.smithi151.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-552.el8.x86_64.conf 2024-04-24T15:19:39.799 DEBUG:teuthology.orchestra.run.smithi151:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-4.18.0-552.el8.x86_64 2024-04-24T15:19:39.823 DEBUG:teuthology.orchestra.remote:smithi096:/tmp/tmp.byBvwLFoWG is 6KB 2024-04-24T15:19:39.871 DEBUG:teuthology.orchestra.run.smithi096:> rm -fr /tmp/tmp.byBvwLFoWG 2024-04-24T15:19:39.888 DEBUG:teuthology.orchestra.run.smithi096:> sudo /bin/ls /boot/loader/entries || true 2024-04-24T15:19:39.959 DEBUG:teuthology.orchestra.run.smithi151:> sudo shutdown -r now 2024-04-24T15:19:39.960 INFO:teuthology.orchestra.run.smithi096.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2024-04-24T15:19:39.960 INFO:teuthology.orchestra.run.smithi096.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2024-04-24T15:19:39.960 INFO:teuthology.orchestra.run.smithi096.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2024-04-24T15:19:39.960 INFO:teuthology.orchestra.run.smithi096.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-552.el8.x86_64.conf 2024-04-24T15:19:39.961 DEBUG:teuthology.orchestra.run.smithi096:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-4.18.0-552.el8.x86_64 2024-04-24T15:19:40.113 DEBUG:teuthology.orchestra.run.smithi096:> sudo shutdown -r now 2024-04-24T15:20:09.990 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-04-24T15:20:09.991 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi151.front.sepia.ceph.com' 2024-04-24T15:20:09.992 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T15:20:10.116 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-04-24T15:20:10.117 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-04-24T15:20:10.117 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T15:20:28.446 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.151 2024-04-24T15:20:28.447 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.96 2024-04-24T15:20:37.454 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi151.front.sepia.ceph.com' 2024-04-24T15:20:37.455 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T15:20:37.456 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-04-24T15:20:37.457 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T15:20:40.510 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.96 2024-04-24T15:20:40.511 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.151 2024-04-24T15:20:52.523 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-04-24T15:20:52.524 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T15:20:52.525 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi151.front.sepia.ceph.com' 2024-04-24T15:20:52.525 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T15:21:10.943 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.96 2024-04-24T15:21:10.943 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.151 2024-04-24T15:21:25.958 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-04-24T15:21:25.959 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T15:21:25.961 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi151.front.sepia.ceph.com' 2024-04-24T15:21:25.961 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T15:21:26.431 DEBUG:teuthology.orchestra.run.smithi151:> true 2024-04-24T15:21:26.747 DEBUG:teuthology.orchestra.run.smithi096:> true 2024-04-24T15:21:26.789 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi151.front.sepia.ceph.com' 2024-04-24T15:21:26.789 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "4.18.0-552.el8.x86_64"... 2024-04-24T15:21:26.790 DEBUG:teuthology.orchestra.run.smithi151:> uname -r 2024-04-24T15:21:26.806 INFO:teuthology.orchestra.run.smithi151.stdout:4.18.0-552.el8.x86_64 2024-04-24T15:21:26.807 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-552.el8.x86_64 vs 4.18.0-552.el8.x86_64 2024-04-24T15:21:26.807 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-24T15:21:26.807 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-24T15:21:27.808 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-04-24T15:21:27.808 DEBUG:teuthology.orchestra.run.smithi151:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-24T15:21:27.887 INFO:teuthology.orchestra.run.smithi151.stdout:ttyS1 2024-04-24T15:21:27.903 DEBUG:teuthology.parallel:result is None 2024-04-24T15:21:28.055 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-04-24T15:21:28.055 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "4.18.0-552.el8.x86_64"... 2024-04-24T15:21:28.055 DEBUG:teuthology.orchestra.run.smithi096:> uname -r 2024-04-24T15:21:28.216 INFO:teuthology.orchestra.run.smithi096.stdout:4.18.0-552.el8.x86_64 2024-04-24T15:21:28.216 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-552.el8.x86_64 vs 4.18.0-552.el8.x86_64 2024-04-24T15:21:28.216 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-24T15:21:28.216 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-24T15:21:29.217 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-24T15:21:29.217 DEBUG:teuthology.orchestra.run.smithi096:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-24T15:21:29.274 INFO:teuthology.orchestra.run.smithi096.stdout:ttyS1 2024-04-24T15:21:29.284 DEBUG:teuthology.parallel:result is None 2024-04-24T15:21:29.285 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-24T15:21:29.292 INFO:teuthology.task.internal:Creating test directory... 2024-04-24T15:21:29.292 DEBUG:teuthology.orchestra.run.smithi096:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-24T15:21:29.294 DEBUG:teuthology.orchestra.run.smithi151:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-24T15:21:29.312 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-24T15:21:29.319 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-24T15:21:29.328 INFO:teuthology.task.internal:Creating archive directory... 2024-04-24T15:21:29.329 DEBUG:teuthology.orchestra.run.smithi096:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-24T15:21:29.352 DEBUG:teuthology.orchestra.run.smithi151:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-24T15:21:29.398 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-24T15:21:29.486 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-24T15:21:29.486 DEBUG:teuthology.orchestra.run.smithi096:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-24T15:21:29.489 DEBUG:teuthology.orchestra.run.smithi151:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-24T15:21:29.520 INFO:teuthology.orchestra.run.smithi151.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T15:21:29.522 INFO:teuthology.orchestra.run.smithi096.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T15:21:29.530 INFO:teuthology.orchestra.run.smithi151.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T15:21:29.533 INFO:teuthology.orchestra.run.smithi096.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T15:21:29.534 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-24T15:21:29.573 INFO:teuthology.task.internal:Configuring sudo... 2024-04-24T15:21:29.574 DEBUG:teuthology.orchestra.run.smithi096:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-24T15:21:29.576 DEBUG:teuthology.orchestra.run.smithi151:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-24T15:21:29.602 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-24T15:21:29.612 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-24T15:21:29.612 DEBUG:teuthology.orchestra.run.smithi096:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-24T15:21:29.644 DEBUG:teuthology.orchestra.run.smithi151:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-24T15:21:29.660 DEBUG:teuthology.orchestra.run.smithi096:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T15:21:29.746 DEBUG:teuthology.orchestra.run.smithi096:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T15:21:29.800 DEBUG:teuthology.orchestra.run.smithi096:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T15:21:29.886 DEBUG:teuthology.orchestra.run.smithi096:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T15:21:29.926 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-24T15:21:29.927 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-24T15:21:29.994 DEBUG:teuthology.orchestra.run.smithi151:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T15:21:30.036 DEBUG:teuthology.orchestra.run.smithi151:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T15:21:30.104 DEBUG:teuthology.orchestra.run.smithi151:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T15:21:30.147 DEBUG:teuthology.orchestra.run.smithi151:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T15:21:30.189 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-24T15:21:30.190 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-24T15:21:30.256 DEBUG:teuthology.orchestra.run.smithi096:> sudo service rsyslog restart 2024-04-24T15:21:30.258 DEBUG:teuthology.orchestra.run.smithi151:> sudo service rsyslog restart 2024-04-24T15:21:30.312 INFO:teuthology.orchestra.run.smithi096.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T15:21:30.458 INFO:teuthology.orchestra.run.smithi151.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T15:21:30.789 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-24T15:21:30.797 INFO:teuthology.task.internal:Starting timer... 2024-04-24T15:21:30.797 INFO:teuthology.run_tasks:Running task pcp... 2024-04-24T15:21:30.808 INFO:teuthology.run_tasks:Running task selinux... 2024-04-24T15:21:30.817 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-04-24T15:21:30.818 DEBUG:teuthology.orchestra.run.smithi096:> sudo service auditd rotate 2024-04-24T15:21:30.920 INFO:teuthology.orchestra.run.smithi096.stdout:Rotating logs: 2024-04-24T15:21:30.921 DEBUG:teuthology.orchestra.run.smithi151:> sudo service auditd rotate 2024-04-24T15:21:31.232 INFO:teuthology.orchestra.run.smithi151.stdout:Rotating logs: 2024-04-24T15:21:31.246 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-24T15:21:31.246 DEBUG:teuthology.orchestra.run.smithi096:> /usr/sbin/getenforce 2024-04-24T15:21:31.271 INFO:teuthology.orchestra.run.smithi096.stdout:Permissive 2024-04-24T15:21:31.271 DEBUG:teuthology.orchestra.run.smithi151:> /usr/sbin/getenforce 2024-04-24T15:21:31.785 INFO:teuthology.orchestra.run.smithi151.stdout:Permissive 2024-04-24T15:21:31.785 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi096.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi151.front.sepia.ceph.com': 'permissive'} 2024-04-24T15:21:31.785 DEBUG:teuthology.orchestra.run.smithi096:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-24T15:21:31.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T15:21:31.813 DEBUG:teuthology.orchestra.run.smithi151:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-24T15:21:32.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T15:21:32.408 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-24T15:21:32.408 DEBUG:teuthology.orchestra.run.smithi096:> sudo /usr/sbin/setenforce permissive 2024-04-24T15:21:32.447 DEBUG:teuthology.orchestra.run.smithi151:> sudo /usr/sbin/setenforce permissive 2024-04-24T15:21:32.479 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-24T15:21:32.489 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-24T15:21:32.507 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-24T15:21:32.508 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi096.front.sepia.ceph.com,smithi151.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-24T15:27:25.826 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi096.front.sepia.ceph.com'), Remote(name='ubuntu@smithi151.front.sepia.ceph.com')] 2024-04-24T15:27:25.828 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-04-24T15:27:25.829 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T15:27:25.903 DEBUG:teuthology.orchestra.run.smithi096:> true 2024-04-24T15:27:25.969 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-04-24T15:27:25.969 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi151.front.sepia.ceph.com' 2024-04-24T15:27:25.969 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T15:27:26.040 DEBUG:teuthology.orchestra.run.smithi151:> true 2024-04-24T15:27:26.109 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi151.front.sepia.ceph.com' 2024-04-24T15:27:26.109 INFO:teuthology.run_tasks:Running task clock... 2024-04-24T15:27:26.120 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-24T15:27:26.120 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-24T15:27:26.121 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T15:27:26.123 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-24T15:27:26.123 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T15:27:26.151 INFO:teuthology.orchestra.run.smithi096.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-24T15:27:26.165 INFO:teuthology.orchestra.run.smithi096.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-24T15:27:26.180 INFO:teuthology.orchestra.run.smithi151.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-24T15:27:26.193 INFO:teuthology.orchestra.run.smithi151.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-24T15:27:26.228 INFO:teuthology.orchestra.run.smithi096.stderr:sudo: ntpd: command not found 2024-04-24T15:27:26.239 INFO:teuthology.orchestra.run.smithi096.stdout:506 Cannot talk to daemon 2024-04-24T15:27:26.252 INFO:teuthology.orchestra.run.smithi096.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-24T15:27:26.257 INFO:teuthology.orchestra.run.smithi151.stderr:sudo: ntpd: command not found 2024-04-24T15:27:26.265 INFO:teuthology.orchestra.run.smithi096.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-24T15:27:26.268 INFO:teuthology.orchestra.run.smithi151.stdout:506 Cannot talk to daemon 2024-04-24T15:27:26.282 INFO:teuthology.orchestra.run.smithi151.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-24T15:27:26.295 INFO:teuthology.orchestra.run.smithi151.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-24T15:27:26.332 INFO:teuthology.orchestra.run.smithi096.stderr:bash: ntpq: command not found 2024-04-24T15:27:26.335 INFO:teuthology.orchestra.run.smithi096.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T15:27:26.335 INFO:teuthology.orchestra.run.smithi096.stdout:=============================================================================== 2024-04-24T15:27:26.335 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T15:27:26.335 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T15:27:26.335 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T15:27:26.335 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T15:27:26.377 INFO:teuthology.orchestra.run.smithi151.stderr:bash: ntpq: command not found 2024-04-24T15:27:26.380 INFO:teuthology.orchestra.run.smithi151.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T15:27:26.380 INFO:teuthology.orchestra.run.smithi151.stdout:=============================================================================== 2024-04-24T15:27:26.380 INFO:teuthology.orchestra.run.smithi151.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T15:27:26.381 INFO:teuthology.orchestra.run.smithi151.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T15:27:26.381 INFO:teuthology.orchestra.run.smithi151.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T15:27:26.381 INFO:teuthology.orchestra.run.smithi151.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T15:27:26.381 INFO:teuthology.run_tasks:Running task install... 2024-04-24T15:27:26.393 DEBUG:teuthology.task.install:project ceph 2024-04-24T15:27:26.393 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '330067dbce47655d1f762ef14d546d2fde16a6a9'}} 2024-04-24T15:27:26.393 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '330067dbce47655d1f762ef14d546d2fde16a6a9'} 2024-04-24T15:27:26.394 INFO:teuthology.task.install:Using flavor: default 2024-04-24T15:27:26.400 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-24T15:27:26.400 INFO:teuthology.task.install:extra packages: [] 2024-04-24T15:27:26.400 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '330067dbce47655d1f762ef14d546d2fde16a6a9', 'tag': None, 'wait_for_package': False} 2024-04-24T15:27:26.401 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=330067dbce47655d1f762ef14d546d2fde16a6a9 2024-04-24T15:27:26.403 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '330067dbce47655d1f762ef14d546d2fde16a6a9', 'tag': None, 'wait_for_package': False} 2024-04-24T15:27:26.403 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=330067dbce47655d1f762ef14d546d2fde16a6a9 2024-04-24T15:27:26.546 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-yuriw4-testing-20240423.151325-quincy/330067dbce47655d1f762ef14d546d2fde16a6a9/centos/8/flavors/default/ 2024-04-24T15:27:26.546 INFO:teuthology.task.install.rpm:Package version is 17.2.7-915.g330067db 2024-04-24T15:27:26.670 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-yuriw4-testing-20240423.151325-quincy/330067dbce47655d1f762ef14d546d2fde16a6a9/centos/8/flavors/default/ 2024-04-24T15:27:26.670 INFO:teuthology.task.install.rpm:Package version is 17.2.7-915.g330067db 2024-04-24T15:27:26.747 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-yuriw4-testing-20240423.151325-quincy/330067dbce47655d1f762ef14d546d2fde16a6a9/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-yuriw4-testing-20240423.151325-quincy/330067dbce47655d1f762ef14d546d2fde16a6a9/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-yuriw4-testing-20240423.151325-quincy/330067dbce47655d1f762ef14d546d2fde16a6a9/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-24T15:27:26.747 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-24T15:27:26.748 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-24T15:27:26.775 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-04-24T15:27:26.776 DEBUG:teuthology.orchestra.run.smithi096:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/330067dbce47655d1f762ef14d546d2fde16a6a9/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-24T15:27:26.836 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-yuriw4-testing-20240423.151325-quincy/330067dbce47655d1f762ef14d546d2fde16a6a9/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-yuriw4-testing-20240423.151325-quincy/330067dbce47655d1f762ef14d546d2fde16a6a9/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-yuriw4-testing-20240423.151325-quincy/330067dbce47655d1f762ef14d546d2fde16a6a9/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-24T15:27:26.836 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-24T15:27:26.836 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-24T15:27:26.843 DEBUG:teuthology.orchestra.run.smithi096:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-24T15:27:26.864 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-04-24T15:27:26.865 DEBUG:teuthology.orchestra.run.smithi151:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/330067dbce47655d1f762ef14d546d2fde16a6a9/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-24T15:27:26.920 DEBUG:teuthology.orchestra.run.smithi096:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-24T15:27:26.933 DEBUG:teuthology.orchestra.run.smithi151:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-24T15:27:26.986 INFO:teuthology.orchestra.run.smithi096.stdout:check_obsoletes = 1 2024-04-24T15:27:26.987 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum clean all 2024-04-24T15:27:27.011 DEBUG:teuthology.orchestra.run.smithi151:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-24T15:27:27.078 INFO:teuthology.orchestra.run.smithi151.stdout:check_obsoletes = 1 2024-04-24T15:27:27.079 DEBUG:teuthology.orchestra.run.smithi151:> sudo yum clean all 2024-04-24T15:27:27.255 INFO:teuthology.orchestra.run.smithi096.stdout:78 files removed 2024-04-24T15:27:27.284 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install ceph-radosgw 2024-04-24T15:27:27.348 INFO:teuthology.orchestra.run.smithi151.stdout:78 files removed 2024-04-24T15:27:27.376 DEBUG:teuthology.orchestra.run.smithi151:> sudo yum -y install ceph-radosgw 2024-04-24T15:27:28.088 INFO:teuthology.orchestra.run.smithi151.stdout:ceph packages for x86_64 258 kB/s | 78 kB 00:00 2024-04-24T15:27:28.123 INFO:teuthology.orchestra.run.smithi096.stdout:ceph packages for x86_64 181 kB/s | 78 kB 00:00 2024-04-24T15:27:28.422 INFO:teuthology.orchestra.run.smithi151.stdout:ceph noarch packages 36 kB/s | 11 kB 00:00 2024-04-24T15:27:28.432 INFO:teuthology.orchestra.run.smithi096.stdout:ceph noarch packages 39 kB/s | 11 kB 00:00 2024-04-24T15:27:28.789 INFO:teuthology.orchestra.run.smithi151.stdout:ceph source packages 4.3 kB/s | 1.4 kB 00:00 2024-04-24T15:27:28.791 INFO:teuthology.orchestra.run.smithi096.stdout:ceph source packages 4.4 kB/s | 1.4 kB 00:00 2024-04-24T15:27:31.341 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 8 - AppStream 11 MB/s | 28 MB 00:02 2024-04-24T15:27:31.457 INFO:teuthology.orchestra.run.smithi151.stdout:CentOS Stream 8 - AppStream 11 MB/s | 28 MB 00:02 2024-04-24T15:27:37.825 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 8 - BaseOS 6.9 MB/s | 10 MB 00:01 2024-04-24T15:27:38.124 INFO:teuthology.orchestra.run.smithi151.stdout:CentOS Stream 8 - BaseOS 6.2 MB/s | 10 MB 00:01 2024-04-24T15:27:39.825 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 8 - Extras 45 kB/s | 18 kB 00:00 2024-04-24T15:27:40.083 INFO:teuthology.orchestra.run.smithi151.stdout:CentOS Stream 8 - Extras 48 kB/s | 18 kB 00:00 2024-04-24T15:27:40.234 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 8 - Extras common packages 26 kB/s | 7.7 kB 00:00 2024-04-24T15:27:40.367 INFO:teuthology.orchestra.run.smithi151.stdout:CentOS Stream 8 - Extras common packages 43 kB/s | 7.7 kB 00:00 2024-04-24T15:27:41.202 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 8 - PowerTools 5.6 MB/s | 5.0 MB 00:00 2024-04-24T15:27:41.369 INFO:teuthology.orchestra.run.smithi151.stdout:CentOS Stream 8 - PowerTools 5.4 MB/s | 5.0 MB 00:00 2024-04-24T15:27:42.271 INFO:teuthology.orchestra.run.smithi096.stdout:Copr repo for python3-asyncssh owned by ceph 25 kB/s | 3.5 kB 00:00 2024-04-24T15:27:42.429 INFO:teuthology.orchestra.run.smithi151.stdout:Copr repo for python3-asyncssh owned by ceph 24 kB/s | 3.5 kB 00:00 2024-04-24T15:27:44.015 INFO:teuthology.orchestra.run.smithi096.stdout:Extra Packages for Enterprise Linux 9.9 MB/s | 16 MB 00:01 2024-04-24T15:27:47.111 INFO:teuthology.orchestra.run.smithi151.stdout:Extra Packages for Enterprise Linux 3.6 MB/s | 16 MB 00:04 2024-04-24T15:27:47.797 INFO:teuthology.orchestra.run.smithi096.stdout:lab-extras 431 kB/s | 24 kB 00:00 2024-04-24T15:27:50.906 INFO:teuthology.orchestra.run.smithi151.stdout:lab-extras 379 kB/s | 24 kB 00:00 2024-04-24T15:27:51.473 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-04-24T15:27:51.475 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-04-24T15:27:51.476 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repository Size 2024-04-24T15:27:51.476 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-04-24T15:27:51.476 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-04-24T15:27:51.476 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-radosgw x86_64 2:17.2.7-915.g330067db.el8 ceph 17 M 2024-04-24T15:27:51.476 INFO:teuthology.orchestra.run.smithi096.stdout:Upgrading: 2024-04-24T15:27:51.476 INFO:teuthology.orchestra.run.smithi096.stdout: librados2 x86_64 2:17.2.7-915.g330067db.el8 ceph 3.5 M 2024-04-24T15:27:51.476 INFO:teuthology.orchestra.run.smithi096.stdout: librbd1 x86_64 2:17.2.7-915.g330067db.el8 ceph 3.6 M 2024-04-24T15:27:51.476 INFO:teuthology.orchestra.run.smithi096.stdout:Installing dependencies: 2024-04-24T15:27:51.476 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-base x86_64 2:17.2.7-915.g330067db.el8 ceph 5.3 M 2024-04-24T15:27:51.476 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-common x86_64 2:17.2.7-915.g330067db.el8 ceph 24 M 2024-04-24T15:27:51.477 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-selinux x86_64 2:17.2.7-915.g330067db.el8 ceph 24 k 2024-04-24T15:27:51.477 INFO:teuthology.orchestra.run.smithi096.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-04-24T15:27:51.477 INFO:teuthology.orchestra.run.smithi096.stdout: libcephfs2 x86_64 2:17.2.7-915.g330067db.el8 ceph 798 k 2024-04-24T15:27:51.477 INFO:teuthology.orchestra.run.smithi096.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-04-24T15:27:51.477 INFO:teuthology.orchestra.run.smithi096.stdout: librabbitmq x86_64 0.9.0-5.el8 baseos 47 k 2024-04-24T15:27:51.477 INFO:teuthology.orchestra.run.smithi096.stdout: libradosstriper1 x86_64 2:17.2.7-915.g330067db.el8 ceph 505 k 2024-04-24T15:27:51.478 INFO:teuthology.orchestra.run.smithi096.stdout: librdkafka x86_64 1.6.1-1.el8 appstream 677 k 2024-04-24T15:27:51.478 INFO:teuthology.orchestra.run.smithi096.stdout: librgw2 x86_64 2:17.2.7-915.g330067db.el8 ceph 5.6 M 2024-04-24T15:27:51.478 INFO:teuthology.orchestra.run.smithi096.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-04-24T15:27:51.478 INFO:teuthology.orchestra.run.smithi096.stdout: lttng-ust x86_64 2.8.1-11.el8 appstream 259 k 2024-04-24T15:27:51.478 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ceph-argparse x86_64 2:17.2.7-915.g330067db.el8 ceph 45 k 2024-04-24T15:27:51.478 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ceph-common x86_64 2:17.2.7-915.g330067db.el8 ceph 117 k 2024-04-24T15:27:51.478 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cephfs x86_64 2:17.2.7-915.g330067db.el8 ceph 190 k 2024-04-24T15:27:51.478 INFO:teuthology.orchestra.run.smithi096.stdout: python3-prettytable noarch 0.7.2-14.el8 appstream 44 k 2024-04-24T15:27:51.478 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rados x86_64 2:17.2.7-915.g330067db.el8 ceph 370 k 2024-04-24T15:27:51.478 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rbd x86_64 2:17.2.7-915.g330067db.el8 ceph 358 k 2024-04-24T15:27:51.479 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rgw x86_64 2:17.2.7-915.g330067db.el8 ceph 107 k 2024-04-24T15:27:51.479 INFO:teuthology.orchestra.run.smithi096.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-04-24T15:27:51.479 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:27:51.479 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-04-24T15:27:51.479 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-04-24T15:27:51.479 INFO:teuthology.orchestra.run.smithi096.stdout:Install 21 Packages 2024-04-24T15:27:51.479 INFO:teuthology.orchestra.run.smithi096.stdout:Upgrade 2 Packages 2024-04-24T15:27:51.479 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:27:51.480 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 64 M 2024-04-24T15:27:51.480 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-04-24T15:27:54.331 INFO:teuthology.orchestra.run.smithi096.stdout:(1/23): ceph-base-17.2.7-915.g330067db.el8.x86_ 2.7 MB/s | 5.3 MB 00:01 2024-04-24T15:27:54.414 INFO:teuthology.orchestra.run.smithi096.stdout:(2/23): ceph-selinux-17.2.7-915.g330067db.el8.x 292 kB/s | 24 kB 00:00 2024-04-24T15:27:54.597 INFO:teuthology.orchestra.run.smithi151.stdout:Dependencies resolved. 2024-04-24T15:27:54.600 INFO:teuthology.orchestra.run.smithi151.stdout:================================================================================ 2024-04-24T15:27:54.600 INFO:teuthology.orchestra.run.smithi151.stdout: Package Arch Version Repository Size 2024-04-24T15:27:54.600 INFO:teuthology.orchestra.run.smithi151.stdout:================================================================================ 2024-04-24T15:27:54.600 INFO:teuthology.orchestra.run.smithi151.stdout:Installing: 2024-04-24T15:27:54.600 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-radosgw x86_64 2:17.2.7-915.g330067db.el8 ceph 17 M 2024-04-24T15:27:54.600 INFO:teuthology.orchestra.run.smithi151.stdout:Upgrading: 2024-04-24T15:27:54.600 INFO:teuthology.orchestra.run.smithi151.stdout: librados2 x86_64 2:17.2.7-915.g330067db.el8 ceph 3.5 M 2024-04-24T15:27:54.600 INFO:teuthology.orchestra.run.smithi151.stdout: librbd1 x86_64 2:17.2.7-915.g330067db.el8 ceph 3.6 M 2024-04-24T15:27:54.600 INFO:teuthology.orchestra.run.smithi151.stdout:Installing dependencies: 2024-04-24T15:27:54.600 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-base x86_64 2:17.2.7-915.g330067db.el8 ceph 5.3 M 2024-04-24T15:27:54.600 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-common x86_64 2:17.2.7-915.g330067db.el8 ceph 24 M 2024-04-24T15:27:54.600 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-selinux x86_64 2:17.2.7-915.g330067db.el8 ceph 24 k 2024-04-24T15:27:54.601 INFO:teuthology.orchestra.run.smithi151.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-04-24T15:27:54.601 INFO:teuthology.orchestra.run.smithi151.stdout: libcephfs2 x86_64 2:17.2.7-915.g330067db.el8 ceph 798 k 2024-04-24T15:27:54.601 INFO:teuthology.orchestra.run.smithi151.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-04-24T15:27:54.601 INFO:teuthology.orchestra.run.smithi151.stdout: librabbitmq x86_64 0.9.0-5.el8 baseos 47 k 2024-04-24T15:27:54.601 INFO:teuthology.orchestra.run.smithi151.stdout: libradosstriper1 x86_64 2:17.2.7-915.g330067db.el8 ceph 505 k 2024-04-24T15:27:54.601 INFO:teuthology.orchestra.run.smithi151.stdout: librdkafka x86_64 1.6.1-1.el8 appstream 677 k 2024-04-24T15:27:54.601 INFO:teuthology.orchestra.run.smithi151.stdout: librgw2 x86_64 2:17.2.7-915.g330067db.el8 ceph 5.6 M 2024-04-24T15:27:54.601 INFO:teuthology.orchestra.run.smithi151.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-04-24T15:27:54.601 INFO:teuthology.orchestra.run.smithi151.stdout: lttng-ust x86_64 2.8.1-11.el8 appstream 259 k 2024-04-24T15:27:54.601 INFO:teuthology.orchestra.run.smithi151.stdout: python3-ceph-argparse x86_64 2:17.2.7-915.g330067db.el8 ceph 45 k 2024-04-24T15:27:54.601 INFO:teuthology.orchestra.run.smithi151.stdout: python3-ceph-common x86_64 2:17.2.7-915.g330067db.el8 ceph 117 k 2024-04-24T15:27:54.601 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cephfs x86_64 2:17.2.7-915.g330067db.el8 ceph 190 k 2024-04-24T15:27:54.601 INFO:teuthology.orchestra.run.smithi151.stdout: python3-prettytable noarch 0.7.2-14.el8 appstream 44 k 2024-04-24T15:27:54.601 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rados x86_64 2:17.2.7-915.g330067db.el8 ceph 370 k 2024-04-24T15:27:54.602 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rbd x86_64 2:17.2.7-915.g330067db.el8 ceph 358 k 2024-04-24T15:27:54.602 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rgw x86_64 2:17.2.7-915.g330067db.el8 ceph 107 k 2024-04-24T15:27:54.602 INFO:teuthology.orchestra.run.smithi151.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-04-24T15:27:54.602 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:27:54.602 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction Summary 2024-04-24T15:27:54.602 INFO:teuthology.orchestra.run.smithi151.stdout:================================================================================ 2024-04-24T15:27:54.602 INFO:teuthology.orchestra.run.smithi151.stdout:Install 21 Packages 2024-04-24T15:27:54.602 INFO:teuthology.orchestra.run.smithi151.stdout:Upgrade 2 Packages 2024-04-24T15:27:54.602 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:27:54.603 INFO:teuthology.orchestra.run.smithi151.stdout:Total download size: 64 M 2024-04-24T15:27:54.603 INFO:teuthology.orchestra.run.smithi151.stdout:Downloading Packages: 2024-04-24T15:27:54.640 INFO:teuthology.orchestra.run.smithi096.stdout:(3/23): libcephfs2-17.2.7-915.g330067db.el8.x86 3.5 MB/s | 798 kB 00:00 2024-04-24T15:27:54.807 INFO:teuthology.orchestra.run.smithi096.stdout:(4/23): libradosstriper1-17.2.7-915.g330067db.e 3.0 MB/s | 505 kB 00:00 2024-04-24T15:27:55.840 INFO:teuthology.orchestra.run.smithi096.stdout:(5/23): librgw2-17.2.7-915.g330067db.el8.x86_64 5.4 MB/s | 5.6 MB 00:01 2024-04-24T15:27:55.915 INFO:teuthology.orchestra.run.smithi096.stdout:(6/23): python3-ceph-argparse-17.2.7-915.g33006 599 kB/s | 45 kB 00:00 2024-04-24T15:27:56.024 INFO:teuthology.orchestra.run.smithi096.stdout:(7/23): python3-ceph-common-17.2.7-915.g330067d 1.1 MB/s | 117 kB 00:00 2024-04-24T15:27:56.116 INFO:teuthology.orchestra.run.smithi096.stdout:(8/23): python3-cephfs-17.2.7-915.g330067db.el8 2.0 MB/s | 190 kB 00:00 2024-04-24T15:27:56.225 INFO:teuthology.orchestra.run.smithi096.stdout:(9/23): python3-rados-17.2.7-915.g330067db.el8. 3.3 MB/s | 370 kB 00:00 2024-04-24T15:27:56.358 INFO:teuthology.orchestra.run.smithi096.stdout:(10/23): python3-rbd-17.2.7-915.g330067db.el8.x 2.6 MB/s | 358 kB 00:00 2024-04-24T15:27:56.450 INFO:teuthology.orchestra.run.smithi096.stdout:(11/23): python3-rgw-17.2.7-915.g330067db.el8.x 1.1 MB/s | 107 kB 00:00 2024-04-24T15:27:56.975 INFO:teuthology.orchestra.run.smithi096.stdout:(12/23): librdkafka-1.6.1-1.el8.x86_64.rpm 1.3 MB/s | 677 kB 00:00 2024-04-24T15:27:57.076 INFO:teuthology.orchestra.run.smithi096.stdout:(13/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 2.5 MB/s | 259 kB 00:00 2024-04-24T15:27:57.192 INFO:teuthology.orchestra.run.smithi096.stdout:(14/23): python3-prettytable-0.7.2-14.el8.noarc 376 kB/s | 44 kB 00:00 2024-04-24T15:27:57.568 INFO:teuthology.orchestra.run.smithi096.stdout:(15/23): ceph-radosgw-17.2.7-915.g330067db.el8. 3.2 MB/s | 17 MB 00:05 2024-04-24T15:27:57.593 INFO:teuthology.orchestra.run.smithi096.stdout:(16/23): librabbitmq-0.9.0-5.el8.x86_64.rpm 117 kB/s | 47 kB 00:00 2024-04-24T15:27:57.877 INFO:teuthology.orchestra.run.smithi096.stdout:(17/23): liboath-2.6.2-3.el8.x86_64.rpm 208 kB/s | 59 kB 00:00 2024-04-24T15:27:58.019 INFO:teuthology.orchestra.run.smithi096.stdout:(18/23): libunwind-1.3.1-3.el8.x86_64.rpm 530 kB/s | 75 kB 00:00 2024-04-24T15:27:58.044 INFO:teuthology.orchestra.run.smithi096.stdout:(19/23): gperftools-libs-2.7-9.el8.x86_64.rpm 644 kB/s | 306 kB 00:00 2024-04-24T15:27:58.241 INFO:teuthology.orchestra.run.smithi151.stdout:(1/23): ceph-base-17.2.7-915.g330067db.el8.x86_ 1.9 MB/s | 5.3 MB 00:02 2024-04-24T15:27:58.328 INFO:teuthology.orchestra.run.smithi096.stdout:(20/23): thrift-0.13.0-2.el8.x86_64.rpm 5.6 MB/s | 1.7 MB 00:00 2024-04-24T15:27:58.341 INFO:teuthology.orchestra.run.smithi151.stdout:(2/23): ceph-selinux-17.2.7-915.g330067db.el8.x 243 kB/s | 24 kB 00:00 2024-04-24T15:27:58.625 INFO:teuthology.orchestra.run.smithi151.stdout:(3/23): ceph-radosgw-17.2.7-915.g330067db.el8.x 5.2 MB/s | 17 MB 00:03 2024-04-24T15:27:58.776 INFO:teuthology.orchestra.run.smithi151.stdout:(4/23): libcephfs2-17.2.7-915.g330067db.el8.x86 1.8 MB/s | 798 kB 00:00 2024-04-24T15:27:58.895 INFO:teuthology.orchestra.run.smithi096.stdout:(21/23): librados2-17.2.7-915.g330067db.el8.x86 4.1 MB/s | 3.5 MB 00:00 2024-04-24T15:27:59.160 INFO:teuthology.orchestra.run.smithi151.stdout:(5/23): libradosstriper1-17.2.7-915.g330067db.e 947 kB/s | 505 kB 00:00 2024-04-24T15:27:59.276 INFO:teuthology.orchestra.run.smithi151.stdout:(6/23): python3-ceph-argparse-17.2.7-915.g33006 383 kB/s | 45 kB 00:00 2024-04-24T15:27:59.568 INFO:teuthology.orchestra.run.smithi151.stdout:(7/23): ceph-common-17.2.7-915.g330067db.el8.x8 5.7 MB/s | 24 MB 00:04 2024-04-24T15:27:59.594 INFO:teuthology.orchestra.run.smithi151.stdout:(8/23): python3-ceph-common-17.2.7-915.g330067d 369 kB/s | 117 kB 00:00 2024-04-24T15:27:59.595 INFO:teuthology.orchestra.run.smithi096.stdout:(22/23): librbd1-17.2.7-915.g330067db.el8.x86_6 2.8 MB/s | 3.6 MB 00:01 2024-04-24T15:27:59.652 INFO:teuthology.orchestra.run.smithi151.stdout:(9/23): python3-cephfs-17.2.7-915.g330067db.el8 2.2 MB/s | 190 kB 00:00 2024-04-24T15:27:59.702 INFO:teuthology.orchestra.run.smithi151.stdout:(10/23): python3-rbd-17.2.7-915.g330067db.el8.x 7.0 MB/s | 358 kB 00:00 2024-04-24T15:27:59.753 INFO:teuthology.orchestra.run.smithi151.stdout:(11/23): python3-rgw-17.2.7-915.g330067db.el8.x 2.1 MB/s | 107 kB 00:00 2024-04-24T15:27:59.845 INFO:teuthology.orchestra.run.smithi151.stdout:(12/23): python3-rados-17.2.7-915.g330067db.el8 1.4 MB/s | 370 kB 00:00 2024-04-24T15:28:00.279 INFO:teuthology.orchestra.run.smithi151.stdout:(13/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 599 kB/s | 259 kB 00:00 2024-04-24T15:28:00.304 INFO:teuthology.orchestra.run.smithi151.stdout:(14/23): librdkafka-1.6.1-1.el8.x86_64.rpm 1.2 MB/s | 677 kB 00:00 2024-04-24T15:28:00.371 INFO:teuthology.orchestra.run.smithi151.stdout:(15/23): python3-prettytable-0.7.2-14.el8.noarc 477 kB/s | 44 kB 00:00 2024-04-24T15:28:00.455 INFO:teuthology.orchestra.run.smithi151.stdout:(16/23): librgw2-17.2.7-915.g330067db.el8.x86_6 3.3 MB/s | 5.6 MB 00:01 2024-04-24T15:28:00.797 INFO:teuthology.orchestra.run.smithi151.stdout:(17/23): librabbitmq-0.9.0-5.el8.x86_64.rpm 95 kB/s | 47 kB 00:00 2024-04-24T15:28:00.823 INFO:teuthology.orchestra.run.smithi151.stdout:(18/23): liboath-2.6.2-3.el8.x86_64.rpm 160 kB/s | 59 kB 00:00 2024-04-24T15:28:00.965 INFO:teuthology.orchestra.run.smithi151.stdout:(19/23): gperftools-libs-2.7-9.el8.x86_64.rpm 516 kB/s | 306 kB 00:00 2024-04-24T15:28:01.148 INFO:teuthology.orchestra.run.smithi151.stdout:(20/23): libunwind-1.3.1-3.el8.x86_64.rpm 214 kB/s | 75 kB 00:00 2024-04-24T15:28:01.437 INFO:teuthology.orchestra.run.smithi096.stdout:(23/23): ceph-common-17.2.7-915.g330067db.el8.x 2.6 MB/s | 24 MB 00:09 2024-04-24T15:28:01.441 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-04-24T15:28:01.441 INFO:teuthology.orchestra.run.smithi096.stdout:Total 6.4 MB/s | 64 MB 00:09 2024-04-24T15:28:01.485 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-04-24T15:28:01.531 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-04-24T15:28:01.531 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-04-24T15:28:01.582 INFO:teuthology.orchestra.run.smithi151.stdout:(21/23): thrift-0.13.0-2.el8.x86_64.rpm 2.3 MB/s | 1.7 MB 00:00 2024-04-24T15:28:01.826 INFO:teuthology.orchestra.run.smithi151.stdout:(22/23): librbd1-17.2.7-915.g330067db.el8.x86_6 5.3 MB/s | 3.6 MB 00:00 2024-04-24T15:28:01.890 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-04-24T15:28:01.893 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-04-24T15:28:02.666 INFO:teuthology.orchestra.run.smithi151.stdout:(23/23): librados2-17.2.7-915.g330067db.el8.x86 2.0 MB/s | 3.5 MB 00:01 2024-04-24T15:28:02.668 INFO:teuthology.orchestra.run.smithi151.stdout:-------------------------------------------------------------------------------- 2024-04-24T15:28:02.669 INFO:teuthology.orchestra.run.smithi151.stdout:Total 7.9 MB/s | 64 MB 00:08 2024-04-24T15:28:02.711 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction check 2024-04-24T15:28:02.757 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction check succeeded. 2024-04-24T15:28:02.757 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction test 2024-04-24T15:28:03.001 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-04-24T15:28:03.119 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction test succeeded. 2024-04-24T15:28:03.122 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction 2024-04-24T15:28:03.193 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/25 2024-04-24T15:28:03.360 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 2/25 2024-04-24T15:28:03.752 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 2/25 2024-04-24T15:28:03.951 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : librados2-2:17.2.7-915.g330067db.el8.x86_64 3/25 2024-04-24T15:28:04.045 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librados2-2:17.2.7-915.g330067db.el8.x86_64 3/25 2024-04-24T15:28:04.202 INFO:teuthology.orchestra.run.smithi151.stdout: Preparing : 1/1 2024-04-24T15:28:04.318 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-rados-2:17.2.7-915.g330067db.el8.x86_64 4/25 2024-04-24T15:28:04.435 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/25 2024-04-24T15:28:04.501 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libcephfs2-2:17.2.7-915.g330067db.el8.x86_64 5/25 2024-04-24T15:28:04.613 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 2/25 2024-04-24T15:28:04.888 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: libcephfs2-2:17.2.7-915.g330067db.el8.x86_64 5/25 2024-04-24T15:28:05.021 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 2/25 2024-04-24T15:28:05.093 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : librbd1-2:17.2.7-915.g330067db.el8.x86_64 6/25 2024-04-24T15:28:05.196 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : librados2-2:17.2.7-915.g330067db.el8.x86_64 3/25 2024-04-24T15:28:05.297 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librbd1-2:17.2.7-915.g330067db.el8.x86_64 6/25 2024-04-24T15:28:05.315 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: librados2-2:17.2.7-915.g330067db.el8.x86_64 3/25 2024-04-24T15:28:05.548 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : thrift-0.13.0-2.el8.x86_64 7/25 2024-04-24T15:28:05.596 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-rados-2:17.2.7-915.g330067db.el8.x86_64 4/25 2024-04-24T15:28:05.754 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libcephfs2-2:17.2.7-915.g330067db.el8.x86_64 5/25 2024-04-24T15:28:05.777 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : librabbitmq-0.9.0-5.el8.x86_64 8/25 2024-04-24T15:28:06.151 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: libcephfs2-2:17.2.7-915.g330067db.el8.x86_64 5/25 2024-04-24T15:28:06.379 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : librbd1-2:17.2.7-915.g330067db.el8.x86_64 6/25 2024-04-24T15:28:06.462 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : librdkafka-1.6.1-1.el8.x86_64 9/25 2024-04-24T15:28:06.581 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: librbd1-2:17.2.7-915.g330067db.el8.x86_64 6/25 2024-04-24T15:28:06.626 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : librgw2-2:17.2.7-915.g330067db.el8.x86_64 10/25 2024-04-24T15:28:06.705 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librgw2-2:17.2.7-915.g330067db.el8.x86_64 10/25 2024-04-24T15:28:06.870 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-ceph-argparse-2:17.2.7-915.g330067db.el8.x 11/25 2024-04-24T15:28:06.918 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : thrift-0.13.0-2.el8.x86_64 7/25 2024-04-24T15:28:07.032 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cephfs-2:17.2.7-915.g330067db.el8.x86_64 12/25 2024-04-24T15:28:07.169 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : librabbitmq-0.9.0-5.el8.x86_64 8/25 2024-04-24T15:28:07.277 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-rgw-2:17.2.7-915.g330067db.el8.x86_64 13/25 2024-04-24T15:28:07.513 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-rbd-2:17.2.7-915.g330067db.el8.x86_64 14/25 2024-04-24T15:28:07.833 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : librdkafka-1.6.1-1.el8.x86_64 9/25 2024-04-24T15:28:07.988 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : librgw2-2:17.2.7-915.g330067db.el8.x86_64 10/25 2024-04-24T15:28:08.009 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libradosstriper1-2:17.2.7-915.g330067db.el8.x86_64 15/25 2024-04-24T15:28:08.054 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: librgw2-2:17.2.7-915.g330067db.el8.x86_64 10/25 2024-04-24T15:28:08.078 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: libradosstriper1-2:17.2.7-915.g330067db.el8.x86_64 15/25 2024-04-24T15:28:08.240 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-ceph-argparse-2:17.2.7-915.g330067db.el8.x 11/25 2024-04-24T15:28:08.302 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 16/25 2024-04-24T15:28:08.403 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cephfs-2:17.2.7-915.g330067db.el8.x86_64 12/25 2024-04-24T15:28:08.509 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 17/25 2024-04-24T15:28:08.639 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-rgw-2:17.2.7-915.g330067db.el8.x86_64 13/25 2024-04-24T15:28:08.696 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 18/25 2024-04-24T15:28:08.893 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-ceph-common-2:17.2.7-915.g330067db.el8.x86 19/25 2024-04-24T15:28:08.899 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-rbd-2:17.2.7-915.g330067db.el8.x86_64 14/25 2024-04-24T15:28:09.079 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libradosstriper1-2:17.2.7-915.g330067db.el8.x86_64 15/25 2024-04-24T15:28:09.152 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: libradosstriper1-2:17.2.7-915.g330067db.el8.x86_64 15/25 2024-04-24T15:28:09.380 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 16/25 2024-04-24T15:28:09.596 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 17/25 2024-04-24T15:28:09.821 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 18/25 2024-04-24T15:28:10.063 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-ceph-common-2:17.2.7-915.g330067db.el8.x86 19/25 2024-04-24T15:28:11.581 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-common-2:17.2.7-915.g330067db.el8.x86_64 20/25 2024-04-24T15:28:11.801 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-common-2:17.2.7-915.g330067db.el8.x86_64 20/25 2024-04-24T15:28:12.378 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-common-2:17.2.7-915.g330067db.el8.x86_64 20/25 2024-04-24T15:28:12.633 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-base-2:17.2.7-915.g330067db.el8.x86_64 21/25 2024-04-24T15:28:12.728 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-common-2:17.2.7-915.g330067db.el8.x86_64 20/25 2024-04-24T15:28:12.739 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-base-2:17.2.7-915.g330067db.el8.x86_64 21/25 2024-04-24T15:28:12.883 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-selinux-2:17.2.7-915.g330067db.el8.x86_64 22/25 2024-04-24T15:28:12.979 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-common-2:17.2.7-915.g330067db.el8.x86_64 20/25 2024-04-24T15:28:13.553 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-common-2:17.2.7-915.g330067db.el8.x86_64 20/25 2024-04-24T15:28:13.875 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-base-2:17.2.7-915.g330067db.el8.x86_64 21/25 2024-04-24T15:28:13.981 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-base-2:17.2.7-915.g330067db.el8.x86_64 21/25 2024-04-24T15:28:14.135 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-selinux-2:17.2.7-915.g330067db.el8.x86_64 22/25 2024-04-24T15:28:32.143 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-selinux-2:17.2.7-915.g330067db.el8.x86_64 22/25 2024-04-24T15:28:32.143 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /sys 2024-04-24T15:28:32.143 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /proc 2024-04-24T15:28:32.143 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /dev 2024-04-24T15:28:32.143 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /run 2024-04-24T15:28:32.143 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /mnt 2024-04-24T15:28:32.144 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /var/tmp 2024-04-24T15:28:32.144 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /home 2024-04-24T15:28:32.144 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /tmp 2024-04-24T15:28:32.144 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /dev 2024-04-24T15:28:32.144 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:28:33.525 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-selinux-2:17.2.7-915.g330067db.el8.x86_64 22/25 2024-04-24T15:28:33.525 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /sys 2024-04-24T15:28:33.526 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /proc 2024-04-24T15:28:33.526 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /dev 2024-04-24T15:28:33.526 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /run 2024-04-24T15:28:33.526 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /mnt 2024-04-24T15:28:33.526 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /var/tmp 2024-04-24T15:28:33.526 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /home 2024-04-24T15:28:33.526 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /tmp 2024-04-24T15:28:33.526 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /dev 2024-04-24T15:28:33.526 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:28:33.845 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-radosgw-2:17.2.7-915.g330067db.el8.x86_64 23/25 2024-04-24T15:28:33.925 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-915.g330067db.el8.x86_64 23/25 2024-04-24T15:28:33.925 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2024-04-24T15:28:34.130 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2024-04-24T15:28:34.131 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2024-04-24T15:28:34.793 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2024-04-24T15:28:34.793 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-base-2:17.2.7-915.g330067db.el8.x86_64 1/25 2024-04-24T15:28:34.793 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-common-2:17.2.7-915.g330067db.el8.x86_64 2/25 2024-04-24T15:28:34.793 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-radosgw-2:17.2.7-915.g330067db.el8.x86_64 3/25 2024-04-24T15:28:34.793 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-selinux-2:17.2.7-915.g330067db.el8.x86_64 4/25 2024-04-24T15:28:34.793 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libcephfs2-2:17.2.7-915.g330067db.el8.x86_64 5/25 2024-04-24T15:28:34.793 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libradosstriper1-2:17.2.7-915.g330067db.el8.x86_64 6/25 2024-04-24T15:28:34.794 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librgw2-2:17.2.7-915.g330067db.el8.x86_64 7/25 2024-04-24T15:28:34.794 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-ceph-argparse-2:17.2.7-915.g330067db.el8.x 8/25 2024-04-24T15:28:34.794 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-ceph-common-2:17.2.7-915.g330067db.el8.x86 9/25 2024-04-24T15:28:34.794 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cephfs-2:17.2.7-915.g330067db.el8.x86_64 10/25 2024-04-24T15:28:34.794 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-rados-2:17.2.7-915.g330067db.el8.x86_64 11/25 2024-04-24T15:28:34.794 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-rbd-2:17.2.7-915.g330067db.el8.x86_64 12/25 2024-04-24T15:28:34.794 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-rgw-2:17.2.7-915.g330067db.el8.x86_64 13/25 2024-04-24T15:28:34.794 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librdkafka-1.6.1-1.el8.x86_64 14/25 2024-04-24T15:28:34.794 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 15/25 2024-04-24T15:28:34.794 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 16/25 2024-04-24T15:28:34.794 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librabbitmq-0.9.0-5.el8.x86_64 17/25 2024-04-24T15:28:34.794 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 18/25 2024-04-24T15:28:34.794 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 19/25 2024-04-24T15:28:34.795 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 20/25 2024-04-24T15:28:34.795 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 21/25 2024-04-24T15:28:34.795 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librados2-2:17.2.7-915.g330067db.el8.x86_64 22/25 2024-04-24T15:28:34.795 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2024-04-24T15:28:34.795 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librbd1-2:17.2.7-915.g330067db.el8.x86_64 24/25 2024-04-24T15:28:35.239 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-radosgw-2:17.2.7-915.g330067db.el8.x86_64 23/25 2024-04-24T15:28:35.303 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-915.g330067db.el8.x86_64 23/25 2024-04-24T15:28:35.304 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2024-04-24T15:28:35.535 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2024-04-24T15:28:35.535 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2024-04-24T15:28:35.650 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2024-04-24T15:28:35.650 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:28:35.650 INFO:teuthology.orchestra.run.smithi096.stdout:Upgraded: 2024-04-24T15:28:35.650 INFO:teuthology.orchestra.run.smithi096.stdout: librados2-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T15:28:35.650 INFO:teuthology.orchestra.run.smithi096.stdout: librbd1-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T15:28:35.650 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-04-24T15:28:35.650 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-base-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T15:28:35.650 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-common-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T15:28:35.650 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-radosgw-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T15:28:35.650 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-selinux-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T15:28:35.651 INFO:teuthology.orchestra.run.smithi096.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-04-24T15:28:35.651 INFO:teuthology.orchestra.run.smithi096.stdout: libcephfs2-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T15:28:35.651 INFO:teuthology.orchestra.run.smithi096.stdout: liboath-2.6.2-3.el8.x86_64 2024-04-24T15:28:35.651 INFO:teuthology.orchestra.run.smithi096.stdout: librabbitmq-0.9.0-5.el8.x86_64 2024-04-24T15:28:35.651 INFO:teuthology.orchestra.run.smithi096.stdout: libradosstriper1-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T15:28:35.651 INFO:teuthology.orchestra.run.smithi096.stdout: librdkafka-1.6.1-1.el8.x86_64 2024-04-24T15:28:35.651 INFO:teuthology.orchestra.run.smithi096.stdout: librgw2-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T15:28:35.651 INFO:teuthology.orchestra.run.smithi096.stdout: libunwind-1.3.1-3.el8.x86_64 2024-04-24T15:28:35.651 INFO:teuthology.orchestra.run.smithi096.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-04-24T15:28:35.651 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ceph-argparse-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T15:28:35.651 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ceph-common-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T15:28:35.651 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cephfs-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T15:28:35.651 INFO:teuthology.orchestra.run.smithi096.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-04-24T15:28:35.651 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rados-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T15:28:35.651 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rbd-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T15:28:35.652 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rgw-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T15:28:35.652 INFO:teuthology.orchestra.run.smithi096.stdout: thrift-0.13.0-2.el8.x86_64 2024-04-24T15:28:35.652 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:28:35.652 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-04-24T15:28:35.977 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install ceph-test 2024-04-24T15:28:36.212 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2024-04-24T15:28:36.212 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-base-2:17.2.7-915.g330067db.el8.x86_64 1/25 2024-04-24T15:28:36.212 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-common-2:17.2.7-915.g330067db.el8.x86_64 2/25 2024-04-24T15:28:36.212 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-radosgw-2:17.2.7-915.g330067db.el8.x86_64 3/25 2024-04-24T15:28:36.212 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-selinux-2:17.2.7-915.g330067db.el8.x86_64 4/25 2024-04-24T15:28:36.213 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libcephfs2-2:17.2.7-915.g330067db.el8.x86_64 5/25 2024-04-24T15:28:36.213 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libradosstriper1-2:17.2.7-915.g330067db.el8.x86_64 6/25 2024-04-24T15:28:36.213 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librgw2-2:17.2.7-915.g330067db.el8.x86_64 7/25 2024-04-24T15:28:36.213 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-ceph-argparse-2:17.2.7-915.g330067db.el8.x 8/25 2024-04-24T15:28:36.213 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-ceph-common-2:17.2.7-915.g330067db.el8.x86 9/25 2024-04-24T15:28:36.213 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cephfs-2:17.2.7-915.g330067db.el8.x86_64 10/25 2024-04-24T15:28:36.213 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-rados-2:17.2.7-915.g330067db.el8.x86_64 11/25 2024-04-24T15:28:36.213 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-rbd-2:17.2.7-915.g330067db.el8.x86_64 12/25 2024-04-24T15:28:36.213 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-rgw-2:17.2.7-915.g330067db.el8.x86_64 13/25 2024-04-24T15:28:36.213 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librdkafka-1.6.1-1.el8.x86_64 14/25 2024-04-24T15:28:36.213 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 15/25 2024-04-24T15:28:36.213 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 16/25 2024-04-24T15:28:36.213 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librabbitmq-0.9.0-5.el8.x86_64 17/25 2024-04-24T15:28:36.213 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 18/25 2024-04-24T15:28:36.213 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 19/25 2024-04-24T15:28:36.214 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 20/25 2024-04-24T15:28:36.214 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 21/25 2024-04-24T15:28:36.214 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librados2-2:17.2.7-915.g330067db.el8.x86_64 22/25 2024-04-24T15:28:36.214 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2024-04-24T15:28:36.214 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librbd1-2:17.2.7-915.g330067db.el8.x86_64 24/25 2024-04-24T15:28:36.708 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:00:49 ago on Wed 24 Apr 2024 03:27:47 PM UTC. 2024-04-24T15:28:37.088 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2024-04-24T15:28:37.088 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:28:37.088 INFO:teuthology.orchestra.run.smithi151.stdout:Upgraded: 2024-04-24T15:28:37.088 INFO:teuthology.orchestra.run.smithi151.stdout: librados2-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T15:28:37.088 INFO:teuthology.orchestra.run.smithi151.stdout: librbd1-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T15:28:37.089 INFO:teuthology.orchestra.run.smithi151.stdout:Installed: 2024-04-24T15:28:37.089 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-base-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T15:28:37.089 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-common-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T15:28:37.089 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-radosgw-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T15:28:37.089 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-selinux-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T15:28:37.089 INFO:teuthology.orchestra.run.smithi151.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-04-24T15:28:37.089 INFO:teuthology.orchestra.run.smithi151.stdout: libcephfs2-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T15:28:37.089 INFO:teuthology.orchestra.run.smithi151.stdout: liboath-2.6.2-3.el8.x86_64 2024-04-24T15:28:37.089 INFO:teuthology.orchestra.run.smithi151.stdout: librabbitmq-0.9.0-5.el8.x86_64 2024-04-24T15:28:37.089 INFO:teuthology.orchestra.run.smithi151.stdout: libradosstriper1-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T15:28:37.089 INFO:teuthology.orchestra.run.smithi151.stdout: librdkafka-1.6.1-1.el8.x86_64 2024-04-24T15:28:37.089 INFO:teuthology.orchestra.run.smithi151.stdout: librgw2-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T15:28:37.090 INFO:teuthology.orchestra.run.smithi151.stdout: libunwind-1.3.1-3.el8.x86_64 2024-04-24T15:28:37.090 INFO:teuthology.orchestra.run.smithi151.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-04-24T15:28:37.090 INFO:teuthology.orchestra.run.smithi151.stdout: python3-ceph-argparse-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T15:28:37.090 INFO:teuthology.orchestra.run.smithi151.stdout: python3-ceph-common-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T15:28:37.090 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cephfs-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T15:28:37.090 INFO:teuthology.orchestra.run.smithi151.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-04-24T15:28:37.090 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rados-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T15:28:37.090 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rbd-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T15:28:37.090 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rgw-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T15:28:37.090 INFO:teuthology.orchestra.run.smithi151.stdout: thrift-0.13.0-2.el8.x86_64 2024-04-24T15:28:37.090 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:28:37.090 INFO:teuthology.orchestra.run.smithi151.stdout:Complete! 2024-04-24T15:28:37.450 DEBUG:teuthology.orchestra.run.smithi151:> sudo yum -y install ceph-test 2024-04-24T15:28:38.185 INFO:teuthology.orchestra.run.smithi151.stdout:Last metadata expiration check: 0:00:48 ago on Wed 24 Apr 2024 03:27:50 PM UTC. 2024-04-24T15:28:38.352 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-04-24T15:28:38.353 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-04-24T15:28:38.354 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repository Size 2024-04-24T15:28:38.354 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-04-24T15:28:38.354 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-04-24T15:28:38.354 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-test x86_64 2:17.2.7-915.g330067db.el8 ceph 53 M 2024-04-24T15:28:38.354 INFO:teuthology.orchestra.run.smithi096.stdout:Installing dependencies: 2024-04-24T15:28:38.354 INFO:teuthology.orchestra.run.smithi096.stdout: jq x86_64 1.6-8.el8 appstream 203 k 2024-04-24T15:28:38.354 INFO:teuthology.orchestra.run.smithi096.stdout: libcephsqlite x86_64 2:17.2.7-915.g330067db.el8 ceph 179 k 2024-04-24T15:28:38.354 INFO:teuthology.orchestra.run.smithi096.stdout: oniguruma x86_64 6.8.2-3.el8 appstream 188 k 2024-04-24T15:28:38.354 INFO:teuthology.orchestra.run.smithi096.stdout: socat x86_64 1.7.4.1-1.el8 appstream 323 k 2024-04-24T15:28:38.354 INFO:teuthology.orchestra.run.smithi096.stdout: xmlstarlet x86_64 1.6.1-20.el8 appstream 71 k 2024-04-24T15:28:38.355 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:28:38.355 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-04-24T15:28:38.355 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-04-24T15:28:38.355 INFO:teuthology.orchestra.run.smithi096.stdout:Install 6 Packages 2024-04-24T15:28:38.355 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:28:38.355 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 54 M 2024-04-24T15:28:38.356 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 236 M 2024-04-24T15:28:38.356 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-04-24T15:28:38.925 INFO:teuthology.orchestra.run.smithi096.stdout:(1/6): jq-1.6-8.el8.x86_64.rpm 535 kB/s | 203 kB 00:00 2024-04-24T15:28:39.025 INFO:teuthology.orchestra.run.smithi096.stdout:(2/6): oniguruma-6.8.2-3.el8.x86_64.rpm 1.8 MB/s | 188 kB 00:00 2024-04-24T15:28:39.201 INFO:teuthology.orchestra.run.smithi096.stdout:(3/6): socat-1.7.4.1-1.el8.x86_64.rpm 1.8 MB/s | 323 kB 00:00 2024-04-24T15:28:39.276 INFO:teuthology.orchestra.run.smithi096.stdout:(4/6): libcephsqlite-17.2.7-915.g330067db.el8.x 246 kB/s | 179 kB 00:00 2024-04-24T15:28:39.301 INFO:teuthology.orchestra.run.smithi096.stdout:(5/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 706 kB/s | 71 kB 00:00 2024-04-24T15:28:39.837 INFO:teuthology.orchestra.run.smithi151.stdout:Dependencies resolved. 2024-04-24T15:28:39.838 INFO:teuthology.orchestra.run.smithi151.stdout:================================================================================ 2024-04-24T15:28:39.838 INFO:teuthology.orchestra.run.smithi151.stdout: Package Arch Version Repository Size 2024-04-24T15:28:39.838 INFO:teuthology.orchestra.run.smithi151.stdout:================================================================================ 2024-04-24T15:28:39.838 INFO:teuthology.orchestra.run.smithi151.stdout:Installing: 2024-04-24T15:28:39.838 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-test x86_64 2:17.2.7-915.g330067db.el8 ceph 53 M 2024-04-24T15:28:39.838 INFO:teuthology.orchestra.run.smithi151.stdout:Installing dependencies: 2024-04-24T15:28:39.838 INFO:teuthology.orchestra.run.smithi151.stdout: jq x86_64 1.6-8.el8 appstream 203 k 2024-04-24T15:28:39.838 INFO:teuthology.orchestra.run.smithi151.stdout: libcephsqlite x86_64 2:17.2.7-915.g330067db.el8 ceph 179 k 2024-04-24T15:28:39.838 INFO:teuthology.orchestra.run.smithi151.stdout: oniguruma x86_64 6.8.2-3.el8 appstream 188 k 2024-04-24T15:28:39.838 INFO:teuthology.orchestra.run.smithi151.stdout: socat x86_64 1.7.4.1-1.el8 appstream 323 k 2024-04-24T15:28:39.839 INFO:teuthology.orchestra.run.smithi151.stdout: xmlstarlet x86_64 1.6.1-20.el8 appstream 71 k 2024-04-24T15:28:39.839 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:28:39.839 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction Summary 2024-04-24T15:28:39.839 INFO:teuthology.orchestra.run.smithi151.stdout:================================================================================ 2024-04-24T15:28:39.839 INFO:teuthology.orchestra.run.smithi151.stdout:Install 6 Packages 2024-04-24T15:28:39.839 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:28:39.839 INFO:teuthology.orchestra.run.smithi151.stdout:Total download size: 54 M 2024-04-24T15:28:39.840 INFO:teuthology.orchestra.run.smithi151.stdout:Installed size: 236 M 2024-04-24T15:28:39.840 INFO:teuthology.orchestra.run.smithi151.stdout:Downloading Packages: 2024-04-24T15:28:40.257 INFO:teuthology.orchestra.run.smithi151.stdout:(1/6): jq-1.6-8.el8.x86_64.rpm 557 kB/s | 203 kB 00:00 2024-04-24T15:28:40.349 INFO:teuthology.orchestra.run.smithi151.stdout:(2/6): oniguruma-6.8.2-3.el8.x86_64.rpm 2.0 MB/s | 188 kB 00:00 2024-04-24T15:28:40.533 INFO:teuthology.orchestra.run.smithi151.stdout:(3/6): socat-1.7.4.1-1.el8.x86_64.rpm 1.7 MB/s | 323 kB 00:00 2024-04-24T15:28:40.566 INFO:teuthology.orchestra.run.smithi151.stdout:(4/6): libcephsqlite-17.2.7-915.g330067db.el8.x 266 kB/s | 179 kB 00:00 2024-04-24T15:28:40.633 INFO:teuthology.orchestra.run.smithi151.stdout:(5/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 702 kB/s | 71 kB 00:00 2024-04-24T15:28:46.091 INFO:teuthology.orchestra.run.smithi151.stdout:(6/6): ceph-test-17.2.7-915.g330067db.el8.x86_6 8.5 MB/s | 53 MB 00:06 2024-04-24T15:28:46.093 INFO:teuthology.orchestra.run.smithi151.stdout:-------------------------------------------------------------------------------- 2024-04-24T15:28:46.094 INFO:teuthology.orchestra.run.smithi151.stdout:Total 8.6 MB/s | 54 MB 00:06 2024-04-24T15:28:46.134 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction check 2024-04-24T15:28:46.156 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction check succeeded. 2024-04-24T15:28:46.156 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction test 2024-04-24T15:28:46.476 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction test succeeded. 2024-04-24T15:28:46.479 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction 2024-04-24T15:28:46.970 INFO:teuthology.orchestra.run.smithi151.stdout: Preparing : 1/1 2024-04-24T15:28:47.217 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 1/6 2024-04-24T15:28:47.406 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 2/6 2024-04-24T15:28:47.556 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : oniguruma-6.8.2-3.el8.x86_64 3/6 2024-04-24T15:28:47.630 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: oniguruma-6.8.2-3.el8.x86_64 3/6 2024-04-24T15:28:47.839 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : jq-1.6-8.el8.x86_64 4/6 2024-04-24T15:28:47.973 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libcephsqlite-2:17.2.7-915.g330067db.el8.x86_64 5/6 2024-04-24T15:28:53.014 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: libcephsqlite-2:17.2.7-915.g330067db.el8.x86_64 5/6 2024-04-24T15:28:53.279 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-test-2:17.2.7-915.g330067db.el8.x86_64 6/6 2024-04-24T15:28:53.571 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-test-2:17.2.7-915.g330067db.el8.x86_64 6/6 2024-04-24T15:28:53.571 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-test-2:17.2.7-915.g330067db.el8.x86_64 1/6 2024-04-24T15:28:53.571 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libcephsqlite-2:17.2.7-915.g330067db.el8.x86_64 2/6 2024-04-24T15:28:53.571 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : jq-1.6-8.el8.x86_64 3/6 2024-04-24T15:28:53.572 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : oniguruma-6.8.2-3.el8.x86_64 4/6 2024-04-24T15:28:53.572 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 5/6 2024-04-24T15:28:53.965 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 6/6 2024-04-24T15:28:53.965 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:28:53.965 INFO:teuthology.orchestra.run.smithi151.stdout:Installed: 2024-04-24T15:28:53.965 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-test-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T15:28:53.965 INFO:teuthology.orchestra.run.smithi151.stdout: jq-1.6-8.el8.x86_64 2024-04-24T15:28:53.965 INFO:teuthology.orchestra.run.smithi151.stdout: libcephsqlite-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T15:28:53.965 INFO:teuthology.orchestra.run.smithi151.stdout: oniguruma-6.8.2-3.el8.x86_64 2024-04-24T15:28:53.965 INFO:teuthology.orchestra.run.smithi151.stdout: socat-1.7.4.1-1.el8.x86_64 2024-04-24T15:28:53.966 INFO:teuthology.orchestra.run.smithi151.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-04-24T15:28:53.966 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:28:53.966 INFO:teuthology.orchestra.run.smithi151.stdout:Complete! 2024-04-24T15:28:54.217 DEBUG:teuthology.orchestra.run.smithi151:> sudo yum -y install ceph 2024-04-24T15:28:54.963 INFO:teuthology.orchestra.run.smithi151.stdout:Last metadata expiration check: 0:01:04 ago on Wed 24 Apr 2024 03:27:50 PM UTC. 2024-04-24T15:28:55.615 INFO:teuthology.orchestra.run.smithi096.stdout:(6/6): ceph-test-17.2.7-915.g330067db.el8.x86_6 3.1 MB/s | 53 MB 00:17 2024-04-24T15:28:55.617 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-04-24T15:28:55.618 INFO:teuthology.orchestra.run.smithi096.stdout:Total 3.1 MB/s | 54 MB 00:17 2024-04-24T15:28:55.659 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-04-24T15:28:55.679 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-04-24T15:28:55.679 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-04-24T15:28:55.998 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-04-24T15:28:56.002 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-04-24T15:28:56.469 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-04-24T15:28:56.613 INFO:teuthology.orchestra.run.smithi151.stdout:Dependencies resolved. 2024-04-24T15:28:56.616 INFO:teuthology.orchestra.run.smithi151.stdout:================================================================================ 2024-04-24T15:28:56.616 INFO:teuthology.orchestra.run.smithi151.stdout: Package Arch Version Repository Size 2024-04-24T15:28:56.616 INFO:teuthology.orchestra.run.smithi151.stdout:================================================================================ 2024-04-24T15:28:56.617 INFO:teuthology.orchestra.run.smithi151.stdout:Installing: 2024-04-24T15:28:56.617 INFO:teuthology.orchestra.run.smithi151.stdout: ceph x86_64 2:17.2.7-915.g330067db.el8 ceph 6.4 k 2024-04-24T15:28:56.617 INFO:teuthology.orchestra.run.smithi151.stdout:Installing dependencies: 2024-04-24T15:28:56.617 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mds x86_64 2:17.2.7-915.g330067db.el8 ceph 2.3 M 2024-04-24T15:28:56.617 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr x86_64 2:17.2.7-915.g330067db.el8 ceph 1.6 M 2024-04-24T15:28:56.617 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-modules-core noarch 2:17.2.7-915.g330067db.el8 ceph-noarch 240 k 2024-04-24T15:28:56.617 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mon x86_64 2:17.2.7-915.g330067db.el8 ceph 4.0 M 2024-04-24T15:28:56.617 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-osd x86_64 2:17.2.7-915.g330067db.el8 ceph 17 M 2024-04-24T15:28:56.617 INFO:teuthology.orchestra.run.smithi151.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-04-24T15:28:56.617 INFO:teuthology.orchestra.run.smithi151.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-04-24T15:28:56.617 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cffi x86_64 1.11.5-6.el8 baseos 249 k 2024-04-24T15:28:56.617 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-04-24T15:28:56.617 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-04-24T15:28:56.618 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cryptography x86_64 3.2.1-7.el8 baseos 641 k 2024-04-24T15:28:56.618 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-04-24T15:28:56.618 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-04-24T15:28:56.618 INFO:teuthology.orchestra.run.smithi151.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-04-24T15:28:56.618 INFO:teuthology.orchestra.run.smithi151.stdout: python3-mako noarch 1.0.6-14.el8 appstream 157 k 2024-04-24T15:28:56.618 INFO:teuthology.orchestra.run.smithi151.stdout: python3-markupsafe x86_64 0.23-19.el8 appstream 39 k 2024-04-24T15:28:56.618 INFO:teuthology.orchestra.run.smithi151.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-04-24T15:28:56.618 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-04-24T15:28:56.618 INFO:teuthology.orchestra.run.smithi151.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-04-24T15:28:56.618 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 appstream 103 k 2024-04-24T15:28:56.618 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pycparser noarch 2.14-14.el8 baseos 109 k 2024-04-24T15:28:56.618 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pytz noarch 2017.2-11.el8 appstream 57 k 2024-04-24T15:28:56.618 INFO:teuthology.orchestra.run.smithi151.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-04-24T15:28:56.619 INFO:teuthology.orchestra.run.smithi151.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-04-24T15:28:56.619 INFO:teuthology.orchestra.run.smithi151.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-04-24T15:28:56.619 INFO:teuthology.orchestra.run.smithi151.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-04-24T15:28:56.619 INFO:teuthology.orchestra.run.smithi151.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-04-24T15:28:56.619 INFO:teuthology.orchestra.run.smithi151.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-04-24T15:28:56.619 INFO:teuthology.orchestra.run.smithi151.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-04-24T15:28:56.619 INFO:teuthology.orchestra.run.smithi151.stdout: python3-werkzeug noarch 0.12.2-4.el8 appstream 457 k 2024-04-24T15:28:56.619 INFO:teuthology.orchestra.run.smithi151.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-04-24T15:28:56.619 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:28:56.619 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction Summary 2024-04-24T15:28:56.619 INFO:teuthology.orchestra.run.smithi151.stdout:================================================================================ 2024-04-24T15:28:56.619 INFO:teuthology.orchestra.run.smithi151.stdout:Install 32 Packages 2024-04-24T15:28:56.619 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:28:56.620 INFO:teuthology.orchestra.run.smithi151.stdout:Total download size: 29 M 2024-04-24T15:28:56.620 INFO:teuthology.orchestra.run.smithi151.stdout:Installed size: 113 M 2024-04-24T15:28:56.620 INFO:teuthology.orchestra.run.smithi151.stdout:Downloading Packages: 2024-04-24T15:28:56.667 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 1/6 2024-04-24T15:28:56.883 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 2/6 2024-04-24T15:28:56.998 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : oniguruma-6.8.2-3.el8.x86_64 3/6 2024-04-24T15:28:57.077 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: oniguruma-6.8.2-3.el8.x86_64 3/6 2024-04-24T15:28:57.255 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : jq-1.6-8.el8.x86_64 4/6 2024-04-24T15:28:57.298 INFO:teuthology.orchestra.run.smithi151.stdout:(1/32): ceph-17.2.7-915.g330067db.el8.x86_64.rp 42 kB/s | 6.4 kB 00:00 2024-04-24T15:28:57.389 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libcephsqlite-2:17.2.7-915.g330067db.el8.x86_64 5/6 2024-04-24T15:28:57.682 INFO:teuthology.orchestra.run.smithi151.stdout:(2/32): ceph-mgr-17.2.7-915.g330067db.el8.x86_6 2.9 MB/s | 1.6 MB 00:00 2024-04-24T15:28:57.832 INFO:teuthology.orchestra.run.smithi151.stdout:(3/32): ceph-mds-17.2.7-915.g330067db.el8.x86_6 3.4 MB/s | 2.3 MB 00:00 2024-04-24T15:28:57.932 INFO:teuthology.orchestra.run.smithi151.stdout:(4/32): ceph-mgr-modules-core-17.2.7-915.g33006 2.3 MB/s | 240 kB 00:00 2024-04-24T15:28:58.299 INFO:teuthology.orchestra.run.smithi151.stdout:(5/32): python3-mako-1.0.6-14.el8.noarch.rpm 428 kB/s | 157 kB 00:00 2024-04-24T15:28:58.408 INFO:teuthology.orchestra.run.smithi151.stdout:(6/32): python3-markupsafe-0.23-19.el8.x86_64.r 358 kB/s | 39 kB 00:00 2024-04-24T15:28:58.575 INFO:teuthology.orchestra.run.smithi151.stdout:(7/32): ceph-mon-17.2.7-915.g330067db.el8.x86_6 3.2 MB/s | 4.0 MB 00:01 2024-04-24T15:28:58.601 INFO:teuthology.orchestra.run.smithi151.stdout:(8/32): python3-pyOpenSSL-19.0.0-1.el8.noarch.r 538 kB/s | 103 kB 00:00 2024-04-24T15:28:58.951 INFO:teuthology.orchestra.run.smithi151.stdout:(9/32): ceph-osd-17.2.7-915.g330067db.el8.x86_6 14 MB/s | 17 MB 00:01 2024-04-24T15:28:58.985 INFO:teuthology.orchestra.run.smithi151.stdout:(10/32): python3-pytz-2017.2-11.el8.noarch.rpm 139 kB/s | 57 kB 00:00 2024-04-24T15:28:59.010 INFO:teuthology.orchestra.run.smithi151.stdout:(11/32): python3-werkzeug-0.12.2-4.el8.noarch.r 1.1 MB/s | 457 kB 00:00 2024-04-24T15:28:59.436 INFO:teuthology.orchestra.run.smithi151.stdout:(12/32): python3-pycparser-2.14-14.el8.noarch.r 256 kB/s | 109 kB 00:00 2024-04-24T15:28:59.547 INFO:teuthology.orchestra.run.smithi151.stdout:(13/32): python3-cffi-1.11.5-6.el8.x86_64.rpm 418 kB/s | 249 kB 00:00 2024-04-24T15:28:59.611 INFO:teuthology.orchestra.run.smithi151.stdout:(14/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rp 248 kB/s | 44 kB 00:00 2024-04-24T15:28:59.670 INFO:teuthology.orchestra.run.smithi151.stdout:(15/32): python3-cryptography-3.2.1-7.el8.x86_6 936 kB/s | 641 kB 00:00 2024-04-24T15:28:59.746 INFO:teuthology.orchestra.run.smithi151.stdout:(16/32): python3-beautifulsoup4-4.6.3-2.el8.1.n 934 kB/s | 185 kB 00:00 2024-04-24T15:28:59.813 INFO:teuthology.orchestra.run.smithi151.stdout:(17/32): python3-jaraco-6.2-6.el8.noarch.rpm 167 kB/s | 11 kB 00:00 2024-04-24T15:28:59.880 INFO:teuthology.orchestra.run.smithi151.stdout:(18/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 644 kB/s | 173 kB 00:00 2024-04-24T15:28:59.963 INFO:teuthology.orchestra.run.smithi151.stdout:(19/32): python3-cherrypy-18.4.0-1.el8.noarch.r 1.3 MB/s | 384 kB 00:00 2024-04-24T15:28:59.980 INFO:teuthology.orchestra.run.smithi151.stdout:(20/32): python3-jaraco-functools-2.0-4.el8.noa 105 kB/s | 18 kB 00:00 2024-04-24T15:29:00.056 INFO:teuthology.orchestra.run.smithi151.stdout:(21/32): python3-logutils-0.3.5-11.el8.noarch.r 277 kB/s | 49 kB 00:00 2024-04-24T15:29:00.106 INFO:teuthology.orchestra.run.smithi151.stdout:(22/32): python3-more-itertools-7.2.0-3.el8.noa 412 kB/s | 59 kB 00:00 2024-04-24T15:29:00.180 INFO:teuthology.orchestra.run.smithi151.stdout:(23/32): python3-portend-2.6-1.el8.noarch.rpm 126 kB/s | 16 kB 00:00 2024-04-24T15:29:00.282 INFO:teuthology.orchestra.run.smithi151.stdout:(24/32): python3-simplegeneric-0.8.1-17.el8.noa 106 kB/s | 19 kB 00:00 2024-04-24T15:29:00.316 INFO:teuthology.orchestra.run.smithi151.stdout:(25/32): python3-singledispatch-3.4.0.3-18.el8. 177 kB/s | 24 kB 00:00 2024-04-24T15:29:00.366 INFO:teuthology.orchestra.run.smithi151.stdout:(26/32): python3-pecan-1.3.2-9.el8.noarch.rpm 734 kB/s | 283 kB 00:00 2024-04-24T15:29:00.392 INFO:teuthology.orchestra.run.smithi151.stdout:(27/32): python3-tempora-1.14.1-5.el8.noarch.rp 268 kB/s | 29 kB 00:00 2024-04-24T15:29:00.442 INFO:teuthology.orchestra.run.smithi151.stdout:(28/32): python3-trustme-0.6.0-4.el8.noarch.rpm 219 kB/s | 27 kB 00:00 2024-04-24T15:29:00.667 INFO:teuthology.orchestra.run.smithi151.stdout:(29/32): python3-webtest-2.0.33-1.el8.noarch.rp 379 kB/s | 85 kB 00:00 2024-04-24T15:29:00.734 INFO:teuthology.orchestra.run.smithi151.stdout:(30/32): python3-waitress-1.4.3-1.el8.noarch.rp 694 kB/s | 255 kB 00:00 2024-04-24T15:29:00.768 INFO:teuthology.orchestra.run.smithi151.stdout:(31/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 668 kB/s | 251 kB 00:00 2024-04-24T15:29:00.827 INFO:teuthology.orchestra.run.smithi151.stdout:(32/32): python3-zc-lockfile-2.0-2.el8.noarch.r 144 kB/s | 23 kB 00:00 2024-04-24T15:29:00.830 INFO:teuthology.orchestra.run.smithi151.stdout:-------------------------------------------------------------------------------- 2024-04-24T15:29:00.830 INFO:teuthology.orchestra.run.smithi151.stdout:Total 6.9 MB/s | 29 MB 00:04 2024-04-24T15:29:00.911 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction check 2024-04-24T15:29:00.947 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction check succeeded. 2024-04-24T15:29:00.947 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction test 2024-04-24T15:29:01.254 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction test succeeded. 2024-04-24T15:29:01.257 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction 2024-04-24T15:29:02.411 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: libcephsqlite-2:17.2.7-915.g330067db.el8.x86_64 5/6 2024-04-24T15:29:02.412 INFO:teuthology.orchestra.run.smithi151.stdout: Preparing : 1/1 2024-04-24T15:29:02.627 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2024-04-24T15:29:02.645 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-test-2:17.2.7-915.g330067db.el8.x86_64 6/6 2024-04-24T15:29:02.862 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2024-04-24T15:29:02.926 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-test-2:17.2.7-915.g330067db.el8.x86_64 6/6 2024-04-24T15:29:02.926 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-test-2:17.2.7-915.g330067db.el8.x86_64 1/6 2024-04-24T15:29:02.926 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libcephsqlite-2:17.2.7-915.g330067db.el8.x86_64 2/6 2024-04-24T15:29:02.926 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : jq-1.6-8.el8.x86_64 3/6 2024-04-24T15:29:02.926 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : oniguruma-6.8.2-3.el8.x86_64 4/6 2024-04-24T15:29:02.926 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 5/6 2024-04-24T15:29:03.112 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 3/32 2024-04-24T15:29:03.335 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 4/32 2024-04-24T15:29:03.373 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 6/6 2024-04-24T15:29:03.373 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:29:03.373 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-04-24T15:29:03.373 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-test-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T15:29:03.373 INFO:teuthology.orchestra.run.smithi096.stdout: jq-1.6-8.el8.x86_64 2024-04-24T15:29:03.373 INFO:teuthology.orchestra.run.smithi096.stdout: libcephsqlite-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T15:29:03.373 INFO:teuthology.orchestra.run.smithi096.stdout: oniguruma-6.8.2-3.el8.x86_64 2024-04-24T15:29:03.373 INFO:teuthology.orchestra.run.smithi096.stdout: socat-1.7.4.1-1.el8.x86_64 2024-04-24T15:29:03.374 INFO:teuthology.orchestra.run.smithi096.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-04-24T15:29:03.374 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:29:03.374 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-04-24T15:29:03.566 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 5/32 2024-04-24T15:29:03.620 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install ceph 2024-04-24T15:29:03.789 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 6/32 2024-04-24T15:29:04.015 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 7/32 2024-04-24T15:29:04.193 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 8/32 2024-04-24T15:29:04.362 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:01:17 ago on Wed 24 Apr 2024 03:27:47 PM UTC. 2024-04-24T15:29:04.461 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 9/32 2024-04-24T15:29:04.748 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 10/32 2024-04-24T15:29:05.003 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 11/32 2024-04-24T15:29:05.242 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 12/32 2024-04-24T15:29:05.634 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 13/32 2024-04-24T15:29:05.931 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cryptography-3.2.1-7.el8.x86_64 14/32 2024-04-24T15:29:06.022 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-04-24T15:29:06.026 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-04-24T15:29:06.026 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repository Size 2024-04-24T15:29:06.026 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-04-24T15:29:06.026 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-04-24T15:29:06.026 INFO:teuthology.orchestra.run.smithi096.stdout: ceph x86_64 2:17.2.7-915.g330067db.el8 ceph 6.4 k 2024-04-24T15:29:06.026 INFO:teuthology.orchestra.run.smithi096.stdout:Installing dependencies: 2024-04-24T15:29:06.026 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mds x86_64 2:17.2.7-915.g330067db.el8 ceph 2.3 M 2024-04-24T15:29:06.026 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr x86_64 2:17.2.7-915.g330067db.el8 ceph 1.6 M 2024-04-24T15:29:06.026 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-modules-core noarch 2:17.2.7-915.g330067db.el8 ceph-noarch 240 k 2024-04-24T15:29:06.027 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mon x86_64 2:17.2.7-915.g330067db.el8 ceph 4.0 M 2024-04-24T15:29:06.027 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-osd x86_64 2:17.2.7-915.g330067db.el8 ceph 17 M 2024-04-24T15:29:06.027 INFO:teuthology.orchestra.run.smithi096.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-04-24T15:29:06.027 INFO:teuthology.orchestra.run.smithi096.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-04-24T15:29:06.027 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cffi x86_64 1.11.5-6.el8 baseos 249 k 2024-04-24T15:29:06.027 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-04-24T15:29:06.027 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-04-24T15:29:06.027 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cryptography x86_64 3.2.1-7.el8 baseos 641 k 2024-04-24T15:29:06.027 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-04-24T15:29:06.027 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-04-24T15:29:06.027 INFO:teuthology.orchestra.run.smithi096.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-04-24T15:29:06.027 INFO:teuthology.orchestra.run.smithi096.stdout: python3-mako noarch 1.0.6-14.el8 appstream 157 k 2024-04-24T15:29:06.027 INFO:teuthology.orchestra.run.smithi096.stdout: python3-markupsafe x86_64 0.23-19.el8 appstream 39 k 2024-04-24T15:29:06.027 INFO:teuthology.orchestra.run.smithi096.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-04-24T15:29:06.028 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-04-24T15:29:06.028 INFO:teuthology.orchestra.run.smithi096.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-04-24T15:29:06.028 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 appstream 103 k 2024-04-24T15:29:06.028 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pycparser noarch 2.14-14.el8 baseos 109 k 2024-04-24T15:29:06.028 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pytz noarch 2017.2-11.el8 appstream 57 k 2024-04-24T15:29:06.028 INFO:teuthology.orchestra.run.smithi096.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-04-24T15:29:06.028 INFO:teuthology.orchestra.run.smithi096.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-04-24T15:29:06.028 INFO:teuthology.orchestra.run.smithi096.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-04-24T15:29:06.028 INFO:teuthology.orchestra.run.smithi096.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-04-24T15:29:06.028 INFO:teuthology.orchestra.run.smithi096.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-04-24T15:29:06.028 INFO:teuthology.orchestra.run.smithi096.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-04-24T15:29:06.028 INFO:teuthology.orchestra.run.smithi096.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-04-24T15:29:06.028 INFO:teuthology.orchestra.run.smithi096.stdout: python3-werkzeug noarch 0.12.2-4.el8 appstream 457 k 2024-04-24T15:29:06.029 INFO:teuthology.orchestra.run.smithi096.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-04-24T15:29:06.029 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:29:06.029 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-04-24T15:29:06.029 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-04-24T15:29:06.029 INFO:teuthology.orchestra.run.smithi096.stdout:Install 32 Packages 2024-04-24T15:29:06.029 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:29:06.030 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 29 M 2024-04-24T15:29:06.030 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 113 M 2024-04-24T15:29:06.030 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-04-24T15:29:06.111 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 15/32 2024-04-24T15:29:06.378 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 16/32 2024-04-24T15:29:06.630 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 17/32 2024-04-24T15:29:06.650 INFO:teuthology.orchestra.run.smithi096.stdout:(1/32): ceph-17.2.7-915.g330067db.el8.x86_64.rp 36 kB/s | 6.4 kB 00:00 2024-04-24T15:29:06.997 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 18/32 2024-04-24T15:29:07.209 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/32 2024-04-24T15:29:07.351 INFO:teuthology.orchestra.run.smithi096.stdout:(2/32): ceph-mds-17.2.7-915.g330067db.el8.x86_6 2.6 MB/s | 2.3 MB 00:00 2024-04-24T15:29:07.405 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 20/32 2024-04-24T15:29:07.601 INFO:teuthology.orchestra.run.smithi096.stdout:(3/32): ceph-mon-17.2.7-915.g330067db.el8.x86_6 4.2 MB/s | 4.0 MB 00:00 2024-04-24T15:29:07.641 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2024-04-24T15:29:07.668 INFO:teuthology.orchestra.run.smithi096.stdout:(4/32): ceph-mgr-modules-core-17.2.7-915.g33006 3.5 MB/s | 240 kB 00:00 2024-04-24T15:29:07.884 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2024-04-24T15:29:08.035 INFO:teuthology.orchestra.run.smithi096.stdout:(5/32): python3-mako-1.0.6-14.el8.noarch.rpm 428 kB/s | 157 kB 00:00 2024-04-24T15:29:08.127 INFO:teuthology.orchestra.run.smithi096.stdout:(6/32): python3-markupsafe-0.23-19.el8.x86_64.r 424 kB/s | 39 kB 00:00 2024-04-24T15:29:08.219 INFO:teuthology.orchestra.run.smithi096.stdout:(7/32): python3-pyOpenSSL-19.0.0-1.el8.noarch.r 1.1 MB/s | 103 kB 00:00 2024-04-24T15:29:08.303 INFO:teuthology.orchestra.run.smithi096.stdout:(8/32): python3-pytz-2017.2-11.el8.noarch.rpm 684 kB/s | 57 kB 00:00 2024-04-24T15:29:08.478 INFO:teuthology.orchestra.run.smithi096.stdout:(9/32): python3-werkzeug-0.12.2-4.el8.noarch.rp 2.6 MB/s | 457 kB 00:00 2024-04-24T15:29:08.822 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2024-04-24T15:29:09.078 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/32 2024-04-24T15:29:09.278 INFO:teuthology.orchestra.run.smithi096.stdout:(10/32): python3-cffi-1.11.5-6.el8.x86_64.rpm 311 kB/s | 249 kB 00:00 2024-04-24T15:29:09.440 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 25/32 2024-04-24T15:29:09.637 INFO:teuthology.orchestra.run.smithi096.stdout:(11/32): python3-cryptography-3.2.1-7.el8.x86_6 1.7 MB/s | 641 kB 00:00 2024-04-24T15:29:09.754 INFO:teuthology.orchestra.run.smithi096.stdout:(12/32): python3-pycparser-2.14-14.el8.noarch.r 975 kB/s | 109 kB 00:00 2024-04-24T15:29:09.793 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2024-04-24T15:29:09.962 INFO:teuthology.orchestra.run.smithi096.stdout:(13/32): ceph-mgr-17.2.7-915.g330067db.el8.x86_ 461 kB/s | 1.6 MB 00:03 2024-04-24T15:29:10.130 INFO:teuthology.orchestra.run.smithi096.stdout:(14/32): ceph-osd-17.2.7-915.g330067db.el8.x86_ 6.2 MB/s | 17 MB 00:02 2024-04-24T15:29:10.199 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-915.g330067db.el8.n 27/32 2024-04-24T15:29:10.205 INFO:teuthology.orchestra.run.smithi096.stdout:(15/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rp 97 kB/s | 44 kB 00:00 2024-04-24T15:29:10.442 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mgr-2:17.2.7-915.g330067db.el8.x86_64 28/32 2024-04-24T15:29:10.588 INFO:teuthology.orchestra.run.smithi096.stdout:(16/32): python3-cherrypy-18.4.0-1.el8.noarch.r 1.0 MB/s | 384 kB 00:00 2024-04-24T15:29:10.630 INFO:teuthology.orchestra.run.smithi096.stdout:(17/32): python3-beautifulsoup4-4.6.3-2.el8.1.n 278 kB/s | 185 kB 00:00 2024-04-24T15:29:10.697 INFO:teuthology.orchestra.run.smithi096.stdout:(18/32): python3-jaraco-6.2-6.el8.noarch.rpm 102 kB/s | 11 kB 00:00 2024-04-24T15:29:10.722 INFO:teuthology.orchestra.run.smithi096.stdout:(19/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 294 kB/s | 173 kB 00:00 2024-04-24T15:29:10.748 INFO:teuthology.orchestra.run.smithi096.stdout:(20/32): python3-jaraco-functools-2.0-4.el8.noa 150 kB/s | 18 kB 00:00 2024-04-24T15:29:10.807 INFO:teuthology.orchestra.run.smithi096.stdout:(21/32): python3-logutils-0.3.5-11.el8.noarch.r 447 kB/s | 49 kB 00:00 2024-04-24T15:29:10.849 INFO:teuthology.orchestra.run.smithi096.stdout:(22/32): python3-more-itertools-7.2.0-3.el8.noa 467 kB/s | 59 kB 00:00 2024-04-24T15:29:10.915 INFO:teuthology.orchestra.run.smithi096.stdout:(23/32): python3-portend-2.6-1.el8.noarch.rpm 144 kB/s | 16 kB 00:00 2024-04-24T15:29:10.949 INFO:teuthology.orchestra.run.smithi096.stdout:(24/32): python3-pecan-1.3.2-9.el8.noarch.rpm 1.4 MB/s | 283 kB 00:00 2024-04-24T15:29:10.974 INFO:teuthology.orchestra.run.smithi096.stdout:(25/32): python3-simplegeneric-0.8.1-17.el8.noa 149 kB/s | 19 kB 00:00 2024-04-24T15:29:11.025 INFO:teuthology.orchestra.run.smithi096.stdout:(26/32): python3-singledispatch-3.4.0.3-18.el8. 219 kB/s | 24 kB 00:00 2024-04-24T15:29:11.058 INFO:teuthology.orchestra.run.smithi096.stdout:(27/32): python3-tempora-1.14.1-5.el8.noarch.rp 268 kB/s | 29 kB 00:00 2024-04-24T15:29:11.092 INFO:teuthology.orchestra.run.smithi096.stdout:(28/32): python3-trustme-0.6.0-4.el8.noarch.rpm 235 kB/s | 27 kB 00:00 2024-04-24T15:29:11.167 INFO:teuthology.orchestra.run.smithi096.stdout:(29/32): python3-waitress-1.4.3-1.el8.noarch.rp 1.8 MB/s | 255 kB 00:00 2024-04-24T15:29:11.193 INFO:teuthology.orchestra.run.smithi096.stdout:(30/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 1.8 MB/s | 251 kB 00:00 2024-04-24T15:29:11.226 INFO:teuthology.orchestra.run.smithi096.stdout:(31/32): python3-webtest-2.0.33-1.el8.noarch.rp 638 kB/s | 85 kB 00:00 2024-04-24T15:29:11.301 INFO:teuthology.orchestra.run.smithi096.stdout:(32/32): python3-zc-lockfile-2.0-2.el8.noarch.r 171 kB/s | 23 kB 00:00 2024-04-24T15:29:11.306 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-04-24T15:29:11.306 INFO:teuthology.orchestra.run.smithi096.stdout:Total 5.5 MB/s | 29 MB 00:05 2024-04-24T15:29:11.388 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-04-24T15:29:11.424 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-04-24T15:29:11.424 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-04-24T15:29:11.743 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-04-24T15:29:11.747 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-04-24T15:29:11.986 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mgr-2:17.2.7-915.g330067db.el8.x86_64 28/32 2024-04-24T15:29:12.166 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-osd-2:17.2.7-915.g330067db.el8.x86_64 29/32 2024-04-24T15:29:12.570 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-osd-2:17.2.7-915.g330067db.el8.x86_64 29/32 2024-04-24T15:29:12.769 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mon-2:17.2.7-915.g330067db.el8.x86_64 30/32 2024-04-24T15:29:12.894 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-04-24T15:29:12.988 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mon-2:17.2.7-915.g330067db.el8.x86_64 30/32 2024-04-24T15:29:13.150 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2024-04-24T15:29:13.177 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mds-2:17.2.7-915.g330067db.el8.x86_64 31/32 2024-04-24T15:29:13.195 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mds-2:17.2.7-915.g330067db.el8.x86_64 31/32 2024-04-24T15:29:13.310 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-2:17.2.7-915.g330067db.el8.x86_64 32/32 2024-04-24T15:29:13.360 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2024-04-24T15:29:13.568 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 3/32 2024-04-24T15:29:13.750 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-2:17.2.7-915.g330067db.el8.x86_64 32/32 2024-04-24T15:29:13.750 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-2:17.2.7-915.g330067db.el8.x86_64 1/32 2024-04-24T15:29:13.750 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mds-2:17.2.7-915.g330067db.el8.x86_64 2/32 2024-04-24T15:29:13.750 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mgr-2:17.2.7-915.g330067db.el8.x86_64 3/32 2024-04-24T15:29:13.750 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mon-2:17.2.7-915.g330067db.el8.x86_64 4/32 2024-04-24T15:29:13.750 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-osd-2:17.2.7-915.g330067db.el8.x86_64 5/32 2024-04-24T15:29:13.750 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-915.g330067db.el8.n 6/32 2024-04-24T15:29:13.750 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 7/32 2024-04-24T15:29:13.750 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 8/32 2024-04-24T15:29:13.751 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 9/32 2024-04-24T15:29:13.751 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 10/32 2024-04-24T15:29:13.751 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 11/32 2024-04-24T15:29:13.751 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 12/32 2024-04-24T15:29:13.751 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cryptography-3.2.1-7.el8.x86_64 13/32 2024-04-24T15:29:13.751 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 14/32 2024-04-24T15:29:13.751 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 15/32 2024-04-24T15:29:13.751 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 16/32 2024-04-24T15:29:13.751 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 17/32 2024-04-24T15:29:13.751 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 18/32 2024-04-24T15:29:13.751 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 19/32 2024-04-24T15:29:13.751 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 20/32 2024-04-24T15:29:13.751 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 21/32 2024-04-24T15:29:13.752 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 22/32 2024-04-24T15:29:13.752 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 23/32 2024-04-24T15:29:13.752 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-portend-2.6-1.el8.noarch 24/32 2024-04-24T15:29:13.752 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 25/32 2024-04-24T15:29:13.753 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 26/32 2024-04-24T15:29:13.753 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 27/32 2024-04-24T15:29:13.753 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 28/32 2024-04-24T15:29:13.753 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 29/32 2024-04-24T15:29:13.753 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 30/32 2024-04-24T15:29:13.753 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 31/32 2024-04-24T15:29:13.809 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 4/32 2024-04-24T15:29:14.023 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 5/32 2024-04-24T15:29:14.204 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 6/32 2024-04-24T15:29:14.372 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 7/32 2024-04-24T15:29:14.533 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 8/32 2024-04-24T15:29:14.623 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 32/32 2024-04-24T15:29:14.623 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:29:14.624 INFO:teuthology.orchestra.run.smithi151.stdout:Installed: 2024-04-24T15:29:14.624 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T15:29:14.624 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mds-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T15:29:14.624 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T15:29:14.624 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-modules-core-2:17.2.7-915.g330067db.el8.noarch 2024-04-24T15:29:14.624 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mon-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T15:29:14.624 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-osd-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T15:29:14.624 INFO:teuthology.orchestra.run.smithi151.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-04-24T15:29:14.624 INFO:teuthology.orchestra.run.smithi151.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-04-24T15:29:14.624 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-04-24T15:29:14.624 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-04-24T15:29:14.624 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-04-24T15:29:14.624 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cryptography-3.2.1-7.el8.x86_64 2024-04-24T15:29:14.625 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-6.2-6.el8.noarch 2024-04-24T15:29:14.625 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-04-24T15:29:14.625 INFO:teuthology.orchestra.run.smithi151.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-04-24T15:29:14.625 INFO:teuthology.orchestra.run.smithi151.stdout: python3-mako-1.0.6-14.el8.noarch 2024-04-24T15:29:14.625 INFO:teuthology.orchestra.run.smithi151.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-04-24T15:29:14.625 INFO:teuthology.orchestra.run.smithi151.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-04-24T15:29:14.625 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-04-24T15:29:14.625 INFO:teuthology.orchestra.run.smithi151.stdout: python3-portend-2.6-1.el8.noarch 2024-04-24T15:29:14.625 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-04-24T15:29:14.625 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pycparser-2.14-14.el8.noarch 2024-04-24T15:29:14.625 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pytz-2017.2-11.el8.noarch 2024-04-24T15:29:14.625 INFO:teuthology.orchestra.run.smithi151.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-04-24T15:29:14.626 INFO:teuthology.orchestra.run.smithi151.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-04-24T15:29:14.626 INFO:teuthology.orchestra.run.smithi151.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-04-24T15:29:14.626 INFO:teuthology.orchestra.run.smithi151.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-04-24T15:29:14.626 INFO:teuthology.orchestra.run.smithi151.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-04-24T15:29:14.626 INFO:teuthology.orchestra.run.smithi151.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-04-24T15:29:14.626 INFO:teuthology.orchestra.run.smithi151.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-04-24T15:29:14.626 INFO:teuthology.orchestra.run.smithi151.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-04-24T15:29:14.626 INFO:teuthology.orchestra.run.smithi151.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-04-24T15:29:14.626 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:29:14.626 INFO:teuthology.orchestra.run.smithi151.stdout:Complete! 2024-04-24T15:29:14.792 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 9/32 2024-04-24T15:29:14.953 DEBUG:teuthology.orchestra.run.smithi151:> sudo yum -y install ceph-base 2024-04-24T15:29:15.038 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 10/32 2024-04-24T15:29:15.284 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 11/32 2024-04-24T15:29:15.550 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 12/32 2024-04-24T15:29:15.715 INFO:teuthology.orchestra.run.smithi151.stdout:Last metadata expiration check: 0:01:25 ago on Wed 24 Apr 2024 03:27:50 PM UTC. 2024-04-24T15:29:15.917 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 13/32 2024-04-24T15:29:16.196 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cryptography-3.2.1-7.el8.x86_64 14/32 2024-04-24T15:29:16.402 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 15/32 2024-04-24T15:29:16.627 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 16/32 2024-04-24T15:29:16.862 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 17/32 2024-04-24T15:29:17.219 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 18/32 2024-04-24T15:29:17.303 INFO:teuthology.orchestra.run.smithi151.stdout:Package ceph-base-2:17.2.7-915.g330067db.el8.x86_64 is already installed. 2024-04-24T15:29:17.352 INFO:teuthology.orchestra.run.smithi151.stdout:Dependencies resolved. 2024-04-24T15:29:17.353 INFO:teuthology.orchestra.run.smithi151.stdout:Nothing to do. 2024-04-24T15:29:17.353 INFO:teuthology.orchestra.run.smithi151.stdout:Complete! 2024-04-24T15:29:17.432 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/32 2024-04-24T15:29:17.469 DEBUG:teuthology.orchestra.run.smithi151:> sudo yum -y install cephadm 2024-04-24T15:29:17.612 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 20/32 2024-04-24T15:29:17.815 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2024-04-24T15:29:18.074 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2024-04-24T15:29:18.224 INFO:teuthology.orchestra.run.smithi151.stdout:Last metadata expiration check: 0:01:28 ago on Wed 24 Apr 2024 03:27:50 PM UTC. 2024-04-24T15:29:18.313 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2024-04-24T15:29:18.520 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/32 2024-04-24T15:29:18.850 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 25/32 2024-04-24T15:29:19.183 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2024-04-24T15:29:19.549 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-915.g330067db.el8.n 27/32 2024-04-24T15:29:19.793 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-2:17.2.7-915.g330067db.el8.x86_64 28/32 2024-04-24T15:29:19.870 INFO:teuthology.orchestra.run.smithi151.stdout:Dependencies resolved. 2024-04-24T15:29:19.871 INFO:teuthology.orchestra.run.smithi151.stdout:================================================================================ 2024-04-24T15:29:19.871 INFO:teuthology.orchestra.run.smithi151.stdout: Package Arch Version Repository Size 2024-04-24T15:29:19.871 INFO:teuthology.orchestra.run.smithi151.stdout:================================================================================ 2024-04-24T15:29:19.871 INFO:teuthology.orchestra.run.smithi151.stdout:Installing: 2024-04-24T15:29:19.871 INFO:teuthology.orchestra.run.smithi151.stdout: cephadm noarch 2:17.2.7-915.g330067db.el8 ceph-noarch 82 k 2024-04-24T15:29:19.871 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:29:19.871 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction Summary 2024-04-24T15:29:19.871 INFO:teuthology.orchestra.run.smithi151.stdout:================================================================================ 2024-04-24T15:29:19.871 INFO:teuthology.orchestra.run.smithi151.stdout:Install 1 Package 2024-04-24T15:29:19.871 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:29:19.871 INFO:teuthology.orchestra.run.smithi151.stdout:Total download size: 82 k 2024-04-24T15:29:19.871 INFO:teuthology.orchestra.run.smithi151.stdout:Installed size: 372 k 2024-04-24T15:29:19.871 INFO:teuthology.orchestra.run.smithi151.stdout:Downloading Packages: 2024-04-24T15:29:20.194 INFO:teuthology.orchestra.run.smithi151.stdout:cephadm-17.2.7-915.g330067db.el8.noarch.rpm 254 kB/s | 82 kB 00:00 2024-04-24T15:29:20.195 INFO:teuthology.orchestra.run.smithi151.stdout:-------------------------------------------------------------------------------- 2024-04-24T15:29:20.195 INFO:teuthology.orchestra.run.smithi151.stdout:Total 253 kB/s | 82 kB 00:00 2024-04-24T15:29:20.195 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction check 2024-04-24T15:29:20.203 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction check succeeded. 2024-04-24T15:29:20.203 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction test 2024-04-24T15:29:20.205 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction test succeeded. 2024-04-24T15:29:20.208 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction 2024-04-24T15:29:20.253 INFO:teuthology.orchestra.run.smithi151.stdout: Preparing : 1/1 2024-04-24T15:29:20.872 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: cephadm-2:17.2.7-915.g330067db.el8.noarch 1/1 2024-04-24T15:29:21.111 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : cephadm-2:17.2.7-915.g330067db.el8.noarch 1/1 2024-04-24T15:29:21.324 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-2:17.2.7-915.g330067db.el8.x86_64 28/32 2024-04-24T15:29:21.344 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: cephadm-2:17.2.7-915.g330067db.el8.noarch 1/1 2024-04-24T15:29:21.526 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-osd-2:17.2.7-915.g330067db.el8.x86_64 29/32 2024-04-24T15:29:21.789 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : cephadm-2:17.2.7-915.g330067db.el8.noarch 1/1 2024-04-24T15:29:21.789 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:29:21.790 INFO:teuthology.orchestra.run.smithi151.stdout:Installed: 2024-04-24T15:29:21.790 INFO:teuthology.orchestra.run.smithi151.stdout: cephadm-2:17.2.7-915.g330067db.el8.noarch 2024-04-24T15:29:21.790 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:29:21.790 INFO:teuthology.orchestra.run.smithi151.stdout:Complete! 2024-04-24T15:29:21.927 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-osd-2:17.2.7-915.g330067db.el8.x86_64 29/32 2024-04-24T15:29:22.043 DEBUG:teuthology.orchestra.run.smithi151:> sudo yum -y install ceph-immutable-object-cache 2024-04-24T15:29:22.143 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mon-2:17.2.7-915.g330067db.el8.x86_64 30/32 2024-04-24T15:29:22.367 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mon-2:17.2.7-915.g330067db.el8.x86_64 30/32 2024-04-24T15:29:22.518 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mds-2:17.2.7-915.g330067db.el8.x86_64 31/32 2024-04-24T15:29:22.535 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mds-2:17.2.7-915.g330067db.el8.x86_64 31/32 2024-04-24T15:29:22.642 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-2:17.2.7-915.g330067db.el8.x86_64 32/32 2024-04-24T15:29:22.810 INFO:teuthology.orchestra.run.smithi151.stdout:Last metadata expiration check: 0:01:32 ago on Wed 24 Apr 2024 03:27:50 PM UTC. 2024-04-24T15:29:23.062 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-2:17.2.7-915.g330067db.el8.x86_64 32/32 2024-04-24T15:29:23.062 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-2:17.2.7-915.g330067db.el8.x86_64 1/32 2024-04-24T15:29:23.062 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mds-2:17.2.7-915.g330067db.el8.x86_64 2/32 2024-04-24T15:29:23.062 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-2:17.2.7-915.g330067db.el8.x86_64 3/32 2024-04-24T15:29:23.062 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mon-2:17.2.7-915.g330067db.el8.x86_64 4/32 2024-04-24T15:29:23.062 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-osd-2:17.2.7-915.g330067db.el8.x86_64 5/32 2024-04-24T15:29:23.063 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-915.g330067db.el8.n 6/32 2024-04-24T15:29:23.063 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 7/32 2024-04-24T15:29:23.063 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 8/32 2024-04-24T15:29:23.063 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 9/32 2024-04-24T15:29:23.063 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 10/32 2024-04-24T15:29:23.063 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 11/32 2024-04-24T15:29:23.063 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 12/32 2024-04-24T15:29:23.063 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cryptography-3.2.1-7.el8.x86_64 13/32 2024-04-24T15:29:23.063 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 14/32 2024-04-24T15:29:23.063 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 15/32 2024-04-24T15:29:23.063 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 16/32 2024-04-24T15:29:23.063 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 17/32 2024-04-24T15:29:23.063 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 18/32 2024-04-24T15:29:23.064 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 19/32 2024-04-24T15:29:23.064 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 20/32 2024-04-24T15:29:23.064 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 21/32 2024-04-24T15:29:23.064 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 22/32 2024-04-24T15:29:23.064 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 23/32 2024-04-24T15:29:23.064 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-portend-2.6-1.el8.noarch 24/32 2024-04-24T15:29:23.064 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 25/32 2024-04-24T15:29:23.064 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 26/32 2024-04-24T15:29:23.064 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 27/32 2024-04-24T15:29:23.065 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 28/32 2024-04-24T15:29:23.065 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 29/32 2024-04-24T15:29:23.065 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 30/32 2024-04-24T15:29:23.065 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 31/32 2024-04-24T15:29:23.898 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 32/32 2024-04-24T15:29:23.898 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:29:23.898 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-04-24T15:29:23.898 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T15:29:23.898 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mds-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T15:29:23.898 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T15:29:23.898 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-modules-core-2:17.2.7-915.g330067db.el8.noarch 2024-04-24T15:29:23.898 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mon-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T15:29:23.899 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-osd-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T15:29:23.899 INFO:teuthology.orchestra.run.smithi096.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-04-24T15:29:23.899 INFO:teuthology.orchestra.run.smithi096.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-04-24T15:29:23.899 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-04-24T15:29:23.899 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-04-24T15:29:23.899 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-04-24T15:29:23.899 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cryptography-3.2.1-7.el8.x86_64 2024-04-24T15:29:23.899 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-6.2-6.el8.noarch 2024-04-24T15:29:23.899 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-04-24T15:29:23.899 INFO:teuthology.orchestra.run.smithi096.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-04-24T15:29:23.899 INFO:teuthology.orchestra.run.smithi096.stdout: python3-mako-1.0.6-14.el8.noarch 2024-04-24T15:29:23.899 INFO:teuthology.orchestra.run.smithi096.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-04-24T15:29:23.899 INFO:teuthology.orchestra.run.smithi096.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-04-24T15:29:23.900 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-04-24T15:29:23.900 INFO:teuthology.orchestra.run.smithi096.stdout: python3-portend-2.6-1.el8.noarch 2024-04-24T15:29:23.900 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-04-24T15:29:23.900 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pycparser-2.14-14.el8.noarch 2024-04-24T15:29:23.900 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pytz-2017.2-11.el8.noarch 2024-04-24T15:29:23.900 INFO:teuthology.orchestra.run.smithi096.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-04-24T15:29:23.900 INFO:teuthology.orchestra.run.smithi096.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-04-24T15:29:23.900 INFO:teuthology.orchestra.run.smithi096.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-04-24T15:29:23.900 INFO:teuthology.orchestra.run.smithi096.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-04-24T15:29:23.900 INFO:teuthology.orchestra.run.smithi096.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-04-24T15:29:23.900 INFO:teuthology.orchestra.run.smithi096.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-04-24T15:29:23.900 INFO:teuthology.orchestra.run.smithi096.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-04-24T15:29:23.900 INFO:teuthology.orchestra.run.smithi096.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-04-24T15:29:23.900 INFO:teuthology.orchestra.run.smithi096.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-04-24T15:29:23.900 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:29:23.901 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-04-24T15:29:24.160 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install ceph-base 2024-04-24T15:29:24.457 INFO:teuthology.orchestra.run.smithi151.stdout:Dependencies resolved. 2024-04-24T15:29:24.457 INFO:teuthology.orchestra.run.smithi151.stdout:================================================================================ 2024-04-24T15:29:24.457 INFO:teuthology.orchestra.run.smithi151.stdout: Package Arch Version Repo Size 2024-04-24T15:29:24.457 INFO:teuthology.orchestra.run.smithi151.stdout:================================================================================ 2024-04-24T15:29:24.457 INFO:teuthology.orchestra.run.smithi151.stdout:Installing: 2024-04-24T15:29:24.458 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-915.g330067db.el8 ceph 161 k 2024-04-24T15:29:24.458 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:29:24.458 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction Summary 2024-04-24T15:29:24.458 INFO:teuthology.orchestra.run.smithi151.stdout:================================================================================ 2024-04-24T15:29:24.458 INFO:teuthology.orchestra.run.smithi151.stdout:Install 1 Package 2024-04-24T15:29:24.458 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:29:24.459 INFO:teuthology.orchestra.run.smithi151.stdout:Total download size: 161 k 2024-04-24T15:29:24.459 INFO:teuthology.orchestra.run.smithi151.stdout:Installed size: 494 k 2024-04-24T15:29:24.459 INFO:teuthology.orchestra.run.smithi151.stdout:Downloading Packages: 2024-04-24T15:29:24.644 INFO:teuthology.orchestra.run.smithi151.stdout:ceph-immutable-object-cache-17.2.7-915.g330067d 870 kB/s | 161 kB 00:00 2024-04-24T15:29:24.645 INFO:teuthology.orchestra.run.smithi151.stdout:-------------------------------------------------------------------------------- 2024-04-24T15:29:24.645 INFO:teuthology.orchestra.run.smithi151.stdout:Total 864 kB/s | 161 kB 00:00 2024-04-24T15:29:24.645 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction check 2024-04-24T15:29:24.656 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction check succeeded. 2024-04-24T15:29:24.656 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction test 2024-04-24T15:29:24.702 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction test succeeded. 2024-04-24T15:29:24.705 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction 2024-04-24T15:29:24.815 INFO:teuthology.orchestra.run.smithi151.stdout: Preparing : 1/1 2024-04-24T15:29:24.919 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:01:37 ago on Wed 24 Apr 2024 03:27:47 PM UTC. 2024-04-24T15:29:26.234 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-915.g330067db.e 1/1 2024-04-24T15:29:26.524 INFO:teuthology.orchestra.run.smithi096.stdout:Package ceph-base-2:17.2.7-915.g330067db.el8.x86_64 is already installed. 2024-04-24T15:29:26.572 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-04-24T15:29:26.573 INFO:teuthology.orchestra.run.smithi096.stdout:Nothing to do. 2024-04-24T15:29:26.573 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-04-24T15:29:26.648 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-915.g330067db.e 1/1 2024-04-24T15:29:26.690 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install cephadm 2024-04-24T15:29:27.097 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-915.g330067db.e 1/1 2024-04-24T15:29:27.097 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:29:27.097 INFO:teuthology.orchestra.run.smithi151.stdout:Installed: 2024-04-24T15:29:27.097 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-immutable-object-cache-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T15:29:27.097 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:29:27.097 INFO:teuthology.orchestra.run.smithi151.stdout:Complete! 2024-04-24T15:29:27.324 DEBUG:teuthology.orchestra.run.smithi151:> sudo yum -y install ceph-mgr 2024-04-24T15:29:27.448 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:01:40 ago on Wed 24 Apr 2024 03:27:47 PM UTC. 2024-04-24T15:29:28.101 INFO:teuthology.orchestra.run.smithi151.stdout:Last metadata expiration check: 0:01:38 ago on Wed 24 Apr 2024 03:27:50 PM UTC. 2024-04-24T15:29:29.086 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-04-24T15:29:29.087 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-04-24T15:29:29.087 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repository Size 2024-04-24T15:29:29.087 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-04-24T15:29:29.087 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-04-24T15:29:29.087 INFO:teuthology.orchestra.run.smithi096.stdout: cephadm noarch 2:17.2.7-915.g330067db.el8 ceph-noarch 82 k 2024-04-24T15:29:29.087 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:29:29.087 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-04-24T15:29:29.087 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-04-24T15:29:29.087 INFO:teuthology.orchestra.run.smithi096.stdout:Install 1 Package 2024-04-24T15:29:29.087 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:29:29.088 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 82 k 2024-04-24T15:29:29.088 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 372 k 2024-04-24T15:29:29.088 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-04-24T15:29:29.333 INFO:teuthology.orchestra.run.smithi096.stdout:cephadm-17.2.7-915.g330067db.el8.noarch.rpm 334 kB/s | 82 kB 00:00 2024-04-24T15:29:29.335 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-04-24T15:29:29.335 INFO:teuthology.orchestra.run.smithi096.stdout:Total 332 kB/s | 82 kB 00:00 2024-04-24T15:29:29.335 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-04-24T15:29:29.342 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-04-24T15:29:29.342 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-04-24T15:29:29.344 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-04-24T15:29:29.347 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-04-24T15:29:29.391 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-04-24T15:29:29.704 INFO:teuthology.orchestra.run.smithi151.stdout:Package ceph-mgr-2:17.2.7-915.g330067db.el8.x86_64 is already installed. 2024-04-24T15:29:29.752 INFO:teuthology.orchestra.run.smithi151.stdout:Dependencies resolved. 2024-04-24T15:29:29.753 INFO:teuthology.orchestra.run.smithi151.stdout:Nothing to do. 2024-04-24T15:29:29.753 INFO:teuthology.orchestra.run.smithi151.stdout:Complete! 2024-04-24T15:29:29.868 DEBUG:teuthology.orchestra.run.smithi151:> sudo yum -y install ceph-mgr-dashboard 2024-04-24T15:29:29.892 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: cephadm-2:17.2.7-915.g330067db.el8.noarch 1/1 2024-04-24T15:29:30.037 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : cephadm-2:17.2.7-915.g330067db.el8.noarch 1/1 2024-04-24T15:29:30.256 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: cephadm-2:17.2.7-915.g330067db.el8.noarch 1/1 2024-04-24T15:29:30.618 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : cephadm-2:17.2.7-915.g330067db.el8.noarch 1/1 2024-04-24T15:29:30.619 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:29:30.619 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-04-24T15:29:30.619 INFO:teuthology.orchestra.run.smithi096.stdout: cephadm-2:17.2.7-915.g330067db.el8.noarch 2024-04-24T15:29:30.619 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:29:30.619 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-04-24T15:29:30.622 INFO:teuthology.orchestra.run.smithi151.stdout:Last metadata expiration check: 0:01:40 ago on Wed 24 Apr 2024 03:27:50 PM UTC. 2024-04-24T15:29:30.831 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install ceph-immutable-object-cache 2024-04-24T15:29:31.596 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:01:44 ago on Wed 24 Apr 2024 03:27:47 PM UTC. 2024-04-24T15:29:32.290 INFO:teuthology.orchestra.run.smithi151.stdout:Dependencies resolved. 2024-04-24T15:29:32.291 INFO:teuthology.orchestra.run.smithi151.stdout:================================================================================ 2024-04-24T15:29:32.291 INFO:teuthology.orchestra.run.smithi151.stdout: Package Arch Version Repository Size 2024-04-24T15:29:32.291 INFO:teuthology.orchestra.run.smithi151.stdout:================================================================================ 2024-04-24T15:29:32.291 INFO:teuthology.orchestra.run.smithi151.stdout:Installing: 2024-04-24T15:29:32.291 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-dashboard noarch 2:17.2.7-915.g330067db.el8 ceph-noarch 3.4 M 2024-04-24T15:29:32.291 INFO:teuthology.orchestra.run.smithi151.stdout:Installing dependencies: 2024-04-24T15:29:32.291 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-grafana-dashboards noarch 2:17.2.7-915.g330067db.el8 ceph-noarch 24 k 2024-04-24T15:29:32.291 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-prometheus-alerts noarch 2:17.2.7-915.g330067db.el8 ceph-noarch 14 k 2024-04-24T15:29:32.292 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jwt noarch 1.6.1-2.el8 baseos 43 k 2024-04-24T15:29:32.292 INFO:teuthology.orchestra.run.smithi151.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-04-24T15:29:32.292 INFO:teuthology.orchestra.run.smithi151.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-04-24T15:29:32.292 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:29:32.292 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction Summary 2024-04-24T15:29:32.292 INFO:teuthology.orchestra.run.smithi151.stdout:================================================================================ 2024-04-24T15:29:32.292 INFO:teuthology.orchestra.run.smithi151.stdout:Install 6 Packages 2024-04-24T15:29:32.292 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:29:32.292 INFO:teuthology.orchestra.run.smithi151.stdout:Total download size: 3.7 M 2024-04-24T15:29:32.293 INFO:teuthology.orchestra.run.smithi151.stdout:Installed size: 80 M 2024-04-24T15:29:32.293 INFO:teuthology.orchestra.run.smithi151.stdout:Downloading Packages: 2024-04-24T15:29:32.810 INFO:teuthology.orchestra.run.smithi151.stdout:(1/6): ceph-prometheus-alerts-17.2.7-915.g33006 95 kB/s | 14 kB 00:00 2024-04-24T15:29:32.835 INFO:teuthology.orchestra.run.smithi151.stdout:(2/6): ceph-grafana-dashboards-17.2.7-915.g3300 137 kB/s | 24 kB 00:00 2024-04-24T15:29:33.161 INFO:teuthology.orchestra.run.smithi151.stdout:(3/6): python3-jwt-1.6.1-2.el8.noarch.rpm 122 kB/s | 43 kB 00:00 2024-04-24T15:29:33.177 INFO:teuthology.orchestra.run.smithi151.stdout:(4/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 98 kB/s | 34 kB 00:00 2024-04-24T15:29:33.220 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-04-24T15:29:33.221 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-04-24T15:29:33.221 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repo Size 2024-04-24T15:29:33.221 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-04-24T15:29:33.221 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-04-24T15:29:33.221 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-915.g330067db.el8 ceph 161 k 2024-04-24T15:29:33.221 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:29:33.221 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-04-24T15:29:33.222 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-04-24T15:29:33.222 INFO:teuthology.orchestra.run.smithi096.stdout:Install 1 Package 2024-04-24T15:29:33.222 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:29:33.222 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 161 k 2024-04-24T15:29:33.222 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 494 k 2024-04-24T15:29:33.222 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-04-24T15:29:33.288 INFO:teuthology.orchestra.run.smithi151.stdout:(5/6): ceph-mgr-dashboard-17.2.7-915.g330067db. 5.3 MB/s | 3.4 MB 00:00 2024-04-24T15:29:33.663 INFO:teuthology.orchestra.run.smithi096.stdout:ceph-immutable-object-cache-17.2.7-915.g330067d 366 kB/s | 161 kB 00:00 2024-04-24T15:29:33.664 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-04-24T15:29:33.664 INFO:teuthology.orchestra.run.smithi096.stdout:Total 365 kB/s | 161 kB 00:00 2024-04-24T15:29:33.664 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-04-24T15:29:33.674 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-04-24T15:29:33.675 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-04-24T15:29:33.694 INFO:teuthology.orchestra.run.smithi151.stdout:(6/6): python3-routes-2.4.1-12.el8.noarch.rpm 368 kB/s | 196 kB 00:00 2024-04-24T15:29:33.698 INFO:teuthology.orchestra.run.smithi151.stdout:-------------------------------------------------------------------------------- 2024-04-24T15:29:33.698 INFO:teuthology.orchestra.run.smithi151.stdout:Total 2.6 MB/s | 3.7 MB 00:01 2024-04-24T15:29:33.709 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction check 2024-04-24T15:29:33.719 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction check succeeded. 2024-04-24T15:29:33.719 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction test 2024-04-24T15:29:33.720 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-04-24T15:29:33.724 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-04-24T15:29:33.810 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction test succeeded. 2024-04-24T15:29:33.814 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction 2024-04-24T15:29:34.106 INFO:teuthology.orchestra.run.smithi151.stdout: Preparing : 1/1 2024-04-24T15:29:34.322 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 1/6 2024-04-24T15:29:34.391 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-04-24T15:29:34.525 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 2/6 2024-04-24T15:29:34.727 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 3/6 2024-04-24T15:29:34.872 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-915.g330067db.el8.no 4/6 2024-04-24T15:29:35.507 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-915.g330067db.e 1/1 2024-04-24T15:29:35.947 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-915.g330067db.el8.n 5/6 2024-04-24T15:29:35.957 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-915.g330067db.e 1/1 2024-04-24T15:29:36.212 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-915.g330067db.el8.noarch 6/6 2024-04-24T15:29:36.301 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-915.g330067db.e 1/1 2024-04-24T15:29:36.301 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:29:36.302 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-04-24T15:29:36.302 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-immutable-object-cache-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T15:29:36.302 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:29:36.302 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-04-24T15:29:36.498 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-915.g330067db.el8.noarch 6/6 2024-04-24T15:29:36.498 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-915.g330067db.el8.n 1/6 2024-04-24T15:29:36.498 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-915.g330067db.el8.noarch 2/6 2024-04-24T15:29:36.498 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-915.g330067db.el8.no 3/6 2024-04-24T15:29:36.499 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 4/6 2024-04-24T15:29:36.499 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 5/6 2024-04-24T15:29:36.539 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install ceph-mgr 2024-04-24T15:29:37.185 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 6/6 2024-04-24T15:29:37.185 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:29:37.186 INFO:teuthology.orchestra.run.smithi151.stdout:Installed: 2024-04-24T15:29:37.186 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-grafana-dashboards-2:17.2.7-915.g330067db.el8.noarch 2024-04-24T15:29:37.186 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-dashboard-2:17.2.7-915.g330067db.el8.noarch 2024-04-24T15:29:37.186 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-prometheus-alerts-2:17.2.7-915.g330067db.el8.noarch 2024-04-24T15:29:37.186 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-04-24T15:29:37.186 INFO:teuthology.orchestra.run.smithi151.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-04-24T15:29:37.186 INFO:teuthology.orchestra.run.smithi151.stdout: python3-routes-2.4.1-12.el8.noarch 2024-04-24T15:29:37.186 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:29:37.186 INFO:teuthology.orchestra.run.smithi151.stdout:Complete! 2024-04-24T15:29:37.302 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:01:50 ago on Wed 24 Apr 2024 03:27:47 PM UTC. 2024-04-24T15:29:37.316 DEBUG:teuthology.orchestra.run.smithi151:> sudo yum -y install ceph-mgr-diskprediction-local 2024-04-24T15:29:38.071 INFO:teuthology.orchestra.run.smithi151.stdout:Last metadata expiration check: 0:01:48 ago on Wed 24 Apr 2024 03:27:50 PM UTC. 2024-04-24T15:29:38.912 INFO:teuthology.orchestra.run.smithi096.stdout:Package ceph-mgr-2:17.2.7-915.g330067db.el8.x86_64 is already installed. 2024-04-24T15:29:38.960 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-04-24T15:29:38.961 INFO:teuthology.orchestra.run.smithi096.stdout:Nothing to do. 2024-04-24T15:29:38.962 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-04-24T15:29:39.084 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install ceph-mgr-dashboard 2024-04-24T15:29:39.698 INFO:teuthology.orchestra.run.smithi151.stdout:Dependencies resolved. 2024-04-24T15:29:39.699 INFO:teuthology.orchestra.run.smithi151.stdout:============================================================================================ 2024-04-24T15:29:39.699 INFO:teuthology.orchestra.run.smithi151.stdout: Package Arch Version Repository Size 2024-04-24T15:29:39.699 INFO:teuthology.orchestra.run.smithi151.stdout:============================================================================================ 2024-04-24T15:29:39.699 INFO:teuthology.orchestra.run.smithi151.stdout:Installing: 2024-04-24T15:29:39.699 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-915.g330067db.el8 ceph-noarch 7.4 M 2024-04-24T15:29:39.699 INFO:teuthology.orchestra.run.smithi151.stdout:Installing dependencies: 2024-04-24T15:29:39.700 INFO:teuthology.orchestra.run.smithi151.stdout: libgfortran x86_64 8.5.0-21.el8 baseos 645 k 2024-04-24T15:29:39.700 INFO:teuthology.orchestra.run.smithi151.stdout: libquadmath x86_64 8.5.0-21.el8 baseos 172 k 2024-04-24T15:29:39.700 INFO:teuthology.orchestra.run.smithi151.stdout: openblas x86_64 0.3.15-4.el8 appstream 4.7 M 2024-04-24T15:29:39.700 INFO:teuthology.orchestra.run.smithi151.stdout: openblas-threads x86_64 0.3.15-4.el8 appstream 4.9 M 2024-04-24T15:29:39.700 INFO:teuthology.orchestra.run.smithi151.stdout: python3-numpy x86_64 1:1.14.3-10.el8 appstream 3.7 M 2024-04-24T15:29:39.700 INFO:teuthology.orchestra.run.smithi151.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 appstream 225 k 2024-04-24T15:29:39.700 INFO:teuthology.orchestra.run.smithi151.stdout: python3-scipy x86_64 1.0.0-21.module_el8+762+77bd8591 appstream 14 M 2024-04-24T15:29:39.700 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:29:39.700 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction Summary 2024-04-24T15:29:39.700 INFO:teuthology.orchestra.run.smithi151.stdout:============================================================================================ 2024-04-24T15:29:39.701 INFO:teuthology.orchestra.run.smithi151.stdout:Install 8 Packages 2024-04-24T15:29:39.701 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:29:39.701 INFO:teuthology.orchestra.run.smithi151.stdout:Total download size: 36 M 2024-04-24T15:29:39.701 INFO:teuthology.orchestra.run.smithi151.stdout:Installed size: 210 M 2024-04-24T15:29:39.701 INFO:teuthology.orchestra.run.smithi151.stdout:Downloading Packages: 2024-04-24T15:29:39.829 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:01:52 ago on Wed 24 Apr 2024 03:27:47 PM UTC. 2024-04-24T15:29:40.861 INFO:teuthology.orchestra.run.smithi151.stdout:(1/8): openblas-0.3.15-4.el8.x86_64.rpm 5.1 MB/s | 4.7 MB 00:00 2024-04-24T15:29:40.938 INFO:teuthology.orchestra.run.smithi151.stdout:(2/8): openblas-threads-0.3.15-4.el8.x86_64.rpm 4.9 MB/s | 4.9 MB 00:01 2024-04-24T15:29:41.030 INFO:teuthology.orchestra.run.smithi151.stdout:(3/8): python3-numpy-f2py-1.14.3-10.el8.x86_64. 2.4 MB/s | 225 kB 00:00 2024-04-24T15:29:41.231 INFO:teuthology.orchestra.run.smithi151.stdout:(4/8): python3-numpy-1.14.3-10.el8.x86_64.rpm 10 MB/s | 3.7 MB 00:00 2024-04-24T15:29:41.486 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-04-24T15:29:41.487 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-04-24T15:29:41.487 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repository Size 2024-04-24T15:29:41.487 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-04-24T15:29:41.487 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-04-24T15:29:41.488 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-dashboard noarch 2:17.2.7-915.g330067db.el8 ceph-noarch 3.4 M 2024-04-24T15:29:41.488 INFO:teuthology.orchestra.run.smithi096.stdout:Installing dependencies: 2024-04-24T15:29:41.488 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-grafana-dashboards noarch 2:17.2.7-915.g330067db.el8 ceph-noarch 24 k 2024-04-24T15:29:41.488 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-prometheus-alerts noarch 2:17.2.7-915.g330067db.el8 ceph-noarch 14 k 2024-04-24T15:29:41.488 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jwt noarch 1.6.1-2.el8 baseos 43 k 2024-04-24T15:29:41.488 INFO:teuthology.orchestra.run.smithi096.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-04-24T15:29:41.488 INFO:teuthology.orchestra.run.smithi096.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-04-24T15:29:41.488 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:29:41.488 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-04-24T15:29:41.488 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-04-24T15:29:41.488 INFO:teuthology.orchestra.run.smithi096.stdout:Install 6 Packages 2024-04-24T15:29:41.488 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:29:41.489 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 3.7 M 2024-04-24T15:29:41.489 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 80 M 2024-04-24T15:29:41.489 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-04-24T15:29:41.622 INFO:teuthology.orchestra.run.smithi151.stdout:(5/8): ceph-mgr-diskprediction-local-17.2.7-915 4.4 MB/s | 7.4 MB 00:01 2024-04-24T15:29:41.898 INFO:teuthology.orchestra.run.smithi151.stdout:(6/8): libgfortran-8.5.0-21.el8.x86_64.rpm 967 kB/s | 645 kB 00:00 2024-04-24T15:29:42.017 INFO:teuthology.orchestra.run.smithi096.stdout:(1/6): ceph-prometheus-alerts-17.2.7-915.g33006 95 kB/s | 14 kB 00:00 2024-04-24T15:29:42.125 INFO:teuthology.orchestra.run.smithi096.stdout:(2/6): ceph-grafana-dashboards-17.2.7-915.g3300 93 kB/s | 24 kB 00:00 2024-04-24T15:29:42.243 INFO:teuthology.orchestra.run.smithi151.stdout:(7/8): python3-scipy-1.0.0-21.module_el8+762+77 12 MB/s | 14 MB 00:01 2024-04-24T15:29:42.351 INFO:teuthology.orchestra.run.smithi096.stdout:(3/6): python3-jwt-1.6.1-2.el8.noarch.rpm 128 kB/s | 43 kB 00:00 2024-04-24T15:29:42.376 INFO:teuthology.orchestra.run.smithi151.stdout:(8/8): libquadmath-8.5.0-21.el8.x86_64.rpm 229 kB/s | 172 kB 00:00 2024-04-24T15:29:42.376 INFO:teuthology.orchestra.run.smithi151.stdout:-------------------------------------------------------------------------------- 2024-04-24T15:29:42.376 INFO:teuthology.orchestra.run.smithi151.stdout:Total 13 MB/s | 36 MB 00:02 2024-04-24T15:29:42.610 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction check 2024-04-24T15:29:42.628 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction check succeeded. 2024-04-24T15:29:42.628 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction test 2024-04-24T15:29:42.659 INFO:teuthology.orchestra.run.smithi096.stdout:(4/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 63 kB/s | 34 kB 00:00 2024-04-24T15:29:43.084 INFO:teuthology.orchestra.run.smithi096.stdout:(5/6): python3-routes-2.4.1-12.el8.noarch.rpm 268 kB/s | 196 kB 00:00 2024-04-24T15:29:43.233 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction test succeeded. 2024-04-24T15:29:43.236 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction 2024-04-24T15:29:43.969 INFO:teuthology.orchestra.run.smithi151.stdout: Preparing : 1/1 2024-04-24T15:29:44.173 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libquadmath-8.5.0-21.el8.x86_64 1/8 2024-04-24T15:29:44.282 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: libquadmath-8.5.0-21.el8.x86_64 1/8 2024-04-24T15:29:44.318 INFO:teuthology.orchestra.run.smithi096.stdout:(6/6): ceph-mgr-dashboard-17.2.7-915.g330067db. 1.4 MB/s | 3.4 MB 00:02 2024-04-24T15:29:44.321 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-04-24T15:29:44.321 INFO:teuthology.orchestra.run.smithi096.stdout:Total 1.3 MB/s | 3.7 MB 00:02 2024-04-24T15:29:44.333 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-04-24T15:29:44.343 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-04-24T15:29:44.343 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-04-24T15:29:44.429 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-04-24T15:29:44.432 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-04-24T15:29:44.464 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libgfortran-8.5.0-21.el8.x86_64 2/8 2024-04-24T15:29:44.693 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-04-24T15:29:44.884 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 1/6 2024-04-24T15:29:45.067 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: libgfortran-8.5.0-21.el8.x86_64 2/8 2024-04-24T15:29:45.096 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 2/6 2024-04-24T15:29:45.256 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : openblas-0.3.15-4.el8.x86_64 3/8 2024-04-24T15:29:45.273 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 3/6 2024-04-24T15:29:45.410 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-915.g330067db.el8.no 4/6 2024-04-24T15:29:45.876 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 3/8 2024-04-24T15:29:46.056 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 4/8 2024-04-24T15:29:46.537 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-915.g330067db.el8.n 5/6 2024-04-24T15:29:46.753 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-915.g330067db.el8.noarch 6/6 2024-04-24T15:29:46.754 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 4/8 2024-04-24T15:29:47.035 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-915.g330067db.el8.noarch 6/6 2024-04-24T15:29:47.035 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-915.g330067db.el8.n 1/6 2024-04-24T15:29:47.035 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-915.g330067db.el8.noarch 2/6 2024-04-24T15:29:47.035 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-915.g330067db.el8.no 3/6 2024-04-24T15:29:47.035 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 4/6 2024-04-24T15:29:47.035 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 5/6 2024-04-24T15:29:47.134 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2024-04-24T15:29:47.397 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2024-04-24T15:29:47.413 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 6/6 2024-04-24T15:29:47.413 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:29:47.414 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-04-24T15:29:47.414 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-grafana-dashboards-2:17.2.7-915.g330067db.el8.noarch 2024-04-24T15:29:47.414 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-dashboard-2:17.2.7-915.g330067db.el8.noarch 2024-04-24T15:29:47.414 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-prometheus-alerts-2:17.2.7-915.g330067db.el8.noarch 2024-04-24T15:29:47.414 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-04-24T15:29:47.414 INFO:teuthology.orchestra.run.smithi096.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-04-24T15:29:47.414 INFO:teuthology.orchestra.run.smithi096.stdout: python3-routes-2.4.1-12.el8.noarch 2024-04-24T15:29:47.414 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:29:47.414 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-04-24T15:29:47.547 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install ceph-mgr-diskprediction-local 2024-04-24T15:29:48.312 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:02:01 ago on Wed 24 Apr 2024 03:27:47 PM UTC. 2024-04-24T15:29:49.177 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2024-04-24T15:29:49.951 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-04-24T15:29:49.952 INFO:teuthology.orchestra.run.smithi096.stdout:============================================================================================ 2024-04-24T15:29:49.953 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repository Size 2024-04-24T15:29:49.953 INFO:teuthology.orchestra.run.smithi096.stdout:============================================================================================ 2024-04-24T15:29:49.953 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-04-24T15:29:49.953 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-915.g330067db.el8 ceph-noarch 7.4 M 2024-04-24T15:29:49.953 INFO:teuthology.orchestra.run.smithi096.stdout:Installing dependencies: 2024-04-24T15:29:49.953 INFO:teuthology.orchestra.run.smithi096.stdout: libgfortran x86_64 8.5.0-21.el8 baseos 645 k 2024-04-24T15:29:49.953 INFO:teuthology.orchestra.run.smithi096.stdout: libquadmath x86_64 8.5.0-21.el8 baseos 172 k 2024-04-24T15:29:49.953 INFO:teuthology.orchestra.run.smithi096.stdout: openblas x86_64 0.3.15-4.el8 appstream 4.7 M 2024-04-24T15:29:49.953 INFO:teuthology.orchestra.run.smithi096.stdout: openblas-threads x86_64 0.3.15-4.el8 appstream 4.9 M 2024-04-24T15:29:49.953 INFO:teuthology.orchestra.run.smithi096.stdout: python3-numpy x86_64 1:1.14.3-10.el8 appstream 3.7 M 2024-04-24T15:29:49.953 INFO:teuthology.orchestra.run.smithi096.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 appstream 225 k 2024-04-24T15:29:49.953 INFO:teuthology.orchestra.run.smithi096.stdout: python3-scipy x86_64 1.0.0-21.module_el8+762+77bd8591 appstream 14 M 2024-04-24T15:29:49.954 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:29:49.954 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-04-24T15:29:49.954 INFO:teuthology.orchestra.run.smithi096.stdout:============================================================================================ 2024-04-24T15:29:49.954 INFO:teuthology.orchestra.run.smithi096.stdout:Install 8 Packages 2024-04-24T15:29:49.954 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:29:49.954 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 36 M 2024-04-24T15:29:49.954 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 210 M 2024-04-24T15:29:49.955 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-04-24T15:29:50.423 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_6 7/8 2024-04-24T15:29:50.605 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-915.g330067db 8/8 2024-04-24T15:29:50.954 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-915.g330067db 8/8 2024-04-24T15:29:50.954 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-915.g330067db 1/8 2024-04-24T15:29:50.954 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 2/8 2024-04-24T15:29:50.954 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 3/8 2024-04-24T15:29:50.955 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 4/8 2024-04-24T15:29:50.955 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 5/8 2024-04-24T15:29:50.955 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_6 6/8 2024-04-24T15:29:50.955 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libgfortran-8.5.0-21.el8.x86_64 7/8 2024-04-24T15:29:51.084 INFO:teuthology.orchestra.run.smithi096.stdout:(1/8): openblas-0.3.15-4.el8.x86_64.rpm 5.3 MB/s | 4.7 MB 00:00 2024-04-24T15:29:51.151 INFO:teuthology.orchestra.run.smithi096.stdout:(2/8): openblas-threads-0.3.15-4.el8.x86_64.rpm 5.1 MB/s | 4.9 MB 00:00 2024-04-24T15:29:51.251 INFO:teuthology.orchestra.run.smithi096.stdout:(3/8): python3-numpy-f2py-1.14.3-10.el8.x86_64. 2.2 MB/s | 225 kB 00:00 2024-04-24T15:29:51.394 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libquadmath-8.5.0-21.el8.x86_64 8/8 2024-04-24T15:29:51.394 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:29:51.394 INFO:teuthology.orchestra.run.smithi151.stdout:Installed: 2024-04-24T15:29:51.394 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-diskprediction-local-2:17.2.7-915.g330067db.el8.noarch 2024-04-24T15:29:51.394 INFO:teuthology.orchestra.run.smithi151.stdout: libgfortran-8.5.0-21.el8.x86_64 2024-04-24T15:29:51.395 INFO:teuthology.orchestra.run.smithi151.stdout: libquadmath-8.5.0-21.el8.x86_64 2024-04-24T15:29:51.395 INFO:teuthology.orchestra.run.smithi151.stdout: openblas-0.3.15-4.el8.x86_64 2024-04-24T15:29:51.395 INFO:teuthology.orchestra.run.smithi151.stdout: openblas-threads-0.3.15-4.el8.x86_64 2024-04-24T15:29:51.395 INFO:teuthology.orchestra.run.smithi151.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-04-24T15:29:51.395 INFO:teuthology.orchestra.run.smithi151.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-04-24T15:29:51.395 INFO:teuthology.orchestra.run.smithi151.stdout: python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_64 2024-04-24T15:29:51.395 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:29:51.395 INFO:teuthology.orchestra.run.smithi151.stdout:Complete! 2024-04-24T15:29:51.452 INFO:teuthology.orchestra.run.smithi096.stdout:(4/8): python3-numpy-1.14.3-10.el8.x86_64.rpm 10 MB/s | 3.7 MB 00:00 2024-04-24T15:29:51.532 DEBUG:teuthology.orchestra.run.smithi151:> sudo yum -y install ceph-mgr-rook 2024-04-24T15:29:52.284 INFO:teuthology.orchestra.run.smithi151.stdout:Last metadata expiration check: 0:02:02 ago on Wed 24 Apr 2024 03:27:50 PM UTC. 2024-04-24T15:29:52.318 INFO:teuthology.orchestra.run.smithi096.stdout:(5/8): libgfortran-8.5.0-21.el8.x86_64.rpm 745 kB/s | 645 kB 00:00 2024-04-24T15:29:52.479 INFO:teuthology.orchestra.run.smithi096.stdout:(6/8): python3-scipy-1.0.0-21.module_el8+762+77 12 MB/s | 14 MB 00:01 2024-04-24T15:29:52.504 INFO:teuthology.orchestra.run.smithi096.stdout:(7/8): libquadmath-8.5.0-21.el8.x86_64.rpm 928 kB/s | 172 kB 00:00 2024-04-24T15:29:53.163 INFO:teuthology.orchestra.run.smithi096.stdout:(8/8): ceph-mgr-diskprediction-local-17.2.7-915 2.5 MB/s | 7.4 MB 00:02 2024-04-24T15:29:53.164 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-04-24T15:29:53.164 INFO:teuthology.orchestra.run.smithi096.stdout:Total 11 MB/s | 36 MB 00:03 2024-04-24T15:29:53.399 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-04-24T15:29:53.417 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-04-24T15:29:53.417 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-04-24T15:29:53.815 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-04-24T15:29:53.818 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-04-24T15:29:53.930 INFO:teuthology.orchestra.run.smithi151.stdout:Dependencies resolved. 2024-04-24T15:29:53.932 INFO:teuthology.orchestra.run.smithi151.stdout:================================================================================ 2024-04-24T15:29:53.932 INFO:teuthology.orchestra.run.smithi151.stdout: Package Arch Version Repository Size 2024-04-24T15:29:53.932 INFO:teuthology.orchestra.run.smithi151.stdout:================================================================================ 2024-04-24T15:29:53.932 INFO:teuthology.orchestra.run.smithi151.stdout:Installing: 2024-04-24T15:29:53.932 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-rook noarch 2:17.2.7-915.g330067db.el8 ceph-noarch 47 k 2024-04-24T15:29:53.933 INFO:teuthology.orchestra.run.smithi151.stdout:Installing dependencies: 2024-04-24T15:29:53.933 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-04-24T15:29:53.933 INFO:teuthology.orchestra.run.smithi151.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-04-24T15:29:53.933 INFO:teuthology.orchestra.run.smithi151.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-04-24T15:29:53.933 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jsonpatch noarch 1.21-2.el8 appstream 27 k 2024-04-24T15:29:53.933 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jsonpointer noarch 1.10-11.el8 appstream 20 k 2024-04-24T15:29:53.933 INFO:teuthology.orchestra.run.smithi151.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-04-24T15:29:53.933 INFO:teuthology.orchestra.run.smithi151.stdout: python3-oauthlib noarch 2.1.0-1.el8 baseos 155 k 2024-04-24T15:29:53.933 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyasn1 noarch 0.3.7-6.el8 appstream 126 k 2024-04-24T15:29:53.933 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 appstream 110 k 2024-04-24T15:29:53.933 INFO:teuthology.orchestra.run.smithi151.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 baseos 43 k 2024-04-24T15:29:53.933 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-04-24T15:29:53.933 INFO:teuthology.orchestra.run.smithi151.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-04-24T15:29:53.934 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:29:53.934 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction Summary 2024-04-24T15:29:53.934 INFO:teuthology.orchestra.run.smithi151.stdout:================================================================================ 2024-04-24T15:29:53.934 INFO:teuthology.orchestra.run.smithi151.stdout:Install 13 Packages 2024-04-24T15:29:53.934 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:29:53.934 INFO:teuthology.orchestra.run.smithi151.stdout:Total download size: 1.9 M 2024-04-24T15:29:53.935 INFO:teuthology.orchestra.run.smithi151.stdout:Installed size: 25 M 2024-04-24T15:29:53.935 INFO:teuthology.orchestra.run.smithi151.stdout:Downloading Packages: 2024-04-24T15:29:54.389 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-04-24T15:29:54.519 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libquadmath-8.5.0-21.el8.x86_64 1/8 2024-04-24T15:29:54.649 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: libquadmath-8.5.0-21.el8.x86_64 1/8 2024-04-24T15:29:54.679 INFO:teuthology.orchestra.run.smithi151.stdout:(1/13): python3-jsonpointer-1.10-11.el8.noarch. 85 kB/s | 20 kB 00:00 2024-04-24T15:29:54.697 INFO:teuthology.orchestra.run.smithi151.stdout:(2/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 105 kB/s | 27 kB 00:00 2024-04-24T15:29:54.818 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libgfortran-8.5.0-21.el8.x86_64 2/8 2024-04-24T15:29:54.921 INFO:teuthology.orchestra.run.smithi151.stdout:(3/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 522 kB/s | 126 kB 00:00 2024-04-24T15:29:54.938 INFO:teuthology.orchestra.run.smithi151.stdout:(4/13): ceph-mgr-rook-17.2.7-915.g330067db.el8. 96 kB/s | 47 kB 00:00 2024-04-24T15:29:54.972 INFO:teuthology.orchestra.run.smithi151.stdout:(5/13): python3-pyasn1-modules-0.3.7-6.el8.noar 398 kB/s | 110 kB 00:00 2024-04-24T15:29:55.315 INFO:teuthology.orchestra.run.smithi151.stdout:(6/13): python3-requests-oauthlib-1.0.0-1.el8.n 125 kB/s | 43 kB 00:00 2024-04-24T15:29:55.340 INFO:teuthology.orchestra.run.smithi151.stdout:(7/13): python3-cachetools-3.1.1-4.el8.noarch.r 90 kB/s | 33 kB 00:00 2024-04-24T15:29:55.410 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: libgfortran-8.5.0-21.el8.x86_64 2/8 2024-04-24T15:29:55.457 INFO:teuthology.orchestra.run.smithi151.stdout:(8/13): python3-oauthlib-2.1.0-1.el8.noarch.rpm 289 kB/s | 155 kB 00:00 2024-04-24T15:29:55.549 INFO:teuthology.orchestra.run.smithi151.stdout:(9/13): python3-google-auth-1.1.1-10.el8.noarch 389 kB/s | 81 kB 00:00 2024-04-24T15:29:55.585 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : openblas-0.3.15-4.el8.x86_64 3/8 2024-04-24T15:29:55.633 INFO:teuthology.orchestra.run.smithi151.stdout:(10/13): python3-certifi-2018.10.15-7.el8.noarc 51 kB/s | 16 kB 00:00 2024-04-24T15:29:55.683 INFO:teuthology.orchestra.run.smithi151.stdout:(11/13): python3-rsa-4.9-2.el8.noarch.rpm 497 kB/s | 64 kB 00:00 2024-04-24T15:29:55.875 INFO:teuthology.orchestra.run.smithi151.stdout:(12/13): python3-websocket-client-0.56.0-5.el8. 253 kB/s | 61 kB 00:00 2024-04-24T15:29:56.200 INFO:teuthology.orchestra.run.smithi151.stdout:(13/13): python3-kubernetes-11.0.0-6.el8.noarch 1.6 MB/s | 1.2 MB 00:00 2024-04-24T15:29:56.203 INFO:teuthology.orchestra.run.smithi151.stdout:-------------------------------------------------------------------------------- 2024-04-24T15:29:56.203 INFO:teuthology.orchestra.run.smithi151.stdout:Total 871 kB/s | 1.9 MB 00:02 2024-04-24T15:29:56.218 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 3/8 2024-04-24T15:29:56.259 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction check 2024-04-24T15:29:56.275 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction check succeeded. 2024-04-24T15:29:56.275 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction test 2024-04-24T15:29:56.362 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction test succeeded. 2024-04-24T15:29:56.365 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction 2024-04-24T15:29:56.375 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 4/8 2024-04-24T15:29:56.946 INFO:teuthology.orchestra.run.smithi151.stdout: Preparing : 1/1 2024-04-24T15:29:57.046 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 4/8 2024-04-24T15:29:57.205 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2024-04-24T15:29:57.412 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 2/13 2024-04-24T15:29:57.480 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2024-04-24T15:29:57.631 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-rsa-4.9-2.el8.noarch 3/13 2024-04-24T15:29:57.701 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2024-04-24T15:29:57.860 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 4/13 2024-04-24T15:29:58.092 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 5/13 2024-04-24T15:29:58.349 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 6/13 2024-04-24T15:29:58.615 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 7/13 2024-04-24T15:29:58.857 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 8/13 2024-04-24T15:29:59.448 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2024-04-24T15:29:59.915 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 9/13 2024-04-24T15:30:00.185 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 10/13 2024-04-24T15:30:00.393 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 11/13 2024-04-24T15:30:00.637 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 12/13 2024-04-24T15:30:00.681 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_6 7/8 2024-04-24T15:30:00.845 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mgr-rook-2:17.2.7-915.g330067db.el8.noarch 13/13 2024-04-24T15:30:00.850 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-915.g330067db 8/8 2024-04-24T15:30:01.139 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-915.g330067db.el8.noarch 13/13 2024-04-24T15:30:01.139 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mgr-rook-2:17.2.7-915.g330067db.el8.noarch 1/13 2024-04-24T15:30:01.140 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 2/13 2024-04-24T15:30:01.140 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 3/13 2024-04-24T15:30:01.140 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 4/13 2024-04-24T15:30:01.140 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 5/13 2024-04-24T15:30:01.140 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 6/13 2024-04-24T15:30:01.140 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 7/13 2024-04-24T15:30:01.140 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 8/13 2024-04-24T15:30:01.140 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 9/13 2024-04-24T15:30:01.140 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 10/13 2024-04-24T15:30:01.140 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 11/13 2024-04-24T15:30:01.140 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 12/13 2024-04-24T15:30:01.163 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-915.g330067db 8/8 2024-04-24T15:30:01.164 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-915.g330067db 1/8 2024-04-24T15:30:01.164 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 2/8 2024-04-24T15:30:01.164 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 3/8 2024-04-24T15:30:01.164 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 4/8 2024-04-24T15:30:01.164 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 5/8 2024-04-24T15:30:01.164 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_6 6/8 2024-04-24T15:30:01.164 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libgfortran-8.5.0-21.el8.x86_64 7/8 2024-04-24T15:30:01.579 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libquadmath-8.5.0-21.el8.x86_64 8/8 2024-04-24T15:30:01.580 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:30:01.580 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-04-24T15:30:01.580 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-diskprediction-local-2:17.2.7-915.g330067db.el8.noarch 2024-04-24T15:30:01.580 INFO:teuthology.orchestra.run.smithi096.stdout: libgfortran-8.5.0-21.el8.x86_64 2024-04-24T15:30:01.580 INFO:teuthology.orchestra.run.smithi096.stdout: libquadmath-8.5.0-21.el8.x86_64 2024-04-24T15:30:01.580 INFO:teuthology.orchestra.run.smithi096.stdout: openblas-0.3.15-4.el8.x86_64 2024-04-24T15:30:01.580 INFO:teuthology.orchestra.run.smithi096.stdout: openblas-threads-0.3.15-4.el8.x86_64 2024-04-24T15:30:01.580 INFO:teuthology.orchestra.run.smithi096.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-04-24T15:30:01.580 INFO:teuthology.orchestra.run.smithi096.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-04-24T15:30:01.580 INFO:teuthology.orchestra.run.smithi096.stdout: python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_64 2024-04-24T15:30:01.580 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:30:01.580 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-04-24T15:30:01.595 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 13/13 2024-04-24T15:30:01.596 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:30:01.596 INFO:teuthology.orchestra.run.smithi151.stdout:Installed: 2024-04-24T15:30:01.596 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-rook-2:17.2.7-915.g330067db.el8.noarch 2024-04-24T15:30:01.596 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-04-24T15:30:01.596 INFO:teuthology.orchestra.run.smithi151.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-04-24T15:30:01.596 INFO:teuthology.orchestra.run.smithi151.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-04-24T15:30:01.596 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-04-24T15:30:01.596 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-04-24T15:30:01.596 INFO:teuthology.orchestra.run.smithi151.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-04-24T15:30:01.596 INFO:teuthology.orchestra.run.smithi151.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-04-24T15:30:01.596 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-04-24T15:30:01.596 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-04-24T15:30:01.596 INFO:teuthology.orchestra.run.smithi151.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-04-24T15:30:01.596 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rsa-4.9-2.el8.noarch 2024-04-24T15:30:01.597 INFO:teuthology.orchestra.run.smithi151.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-04-24T15:30:01.597 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:30:01.597 INFO:teuthology.orchestra.run.smithi151.stdout:Complete! 2024-04-24T15:30:01.701 DEBUG:teuthology.orchestra.run.smithi151:> sudo yum -y install ceph-mgr-cephadm 2024-04-24T15:30:01.712 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install ceph-mgr-rook 2024-04-24T15:30:02.462 INFO:teuthology.orchestra.run.smithi151.stdout:Last metadata expiration check: 0:02:12 ago on Wed 24 Apr 2024 03:27:50 PM UTC. 2024-04-24T15:30:02.470 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:02:15 ago on Wed 24 Apr 2024 03:27:47 PM UTC. 2024-04-24T15:30:04.095 INFO:teuthology.orchestra.run.smithi151.stdout:Dependencies resolved. 2024-04-24T15:30:04.095 INFO:teuthology.orchestra.run.smithi151.stdout:================================================================================ 2024-04-24T15:30:04.095 INFO:teuthology.orchestra.run.smithi151.stdout: Package Arch Version Repository Size 2024-04-24T15:30:04.095 INFO:teuthology.orchestra.run.smithi151.stdout:================================================================================ 2024-04-24T15:30:04.095 INFO:teuthology.orchestra.run.smithi151.stdout:Installing: 2024-04-24T15:30:04.095 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-cephadm noarch 2:17.2.7-915.g330067db.el8 ceph-noarch 123 k 2024-04-24T15:30:04.095 INFO:teuthology.orchestra.run.smithi151.stdout:Installing dependencies: 2024-04-24T15:30:04.095 INFO:teuthology.orchestra.run.smithi151.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-04-24T15:30:04.096 INFO:teuthology.orchestra.run.smithi151.stdout: python3-babel noarch 2.5.1-7.el8 appstream 4.8 M 2024-04-24T15:30:04.096 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jinja2 noarch 2.10.1-4.el8 appstream 578 k 2024-04-24T15:30:04.096 INFO:teuthology.orchestra.run.smithi151.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-04-24T15:30:04.096 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:30:04.096 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction Summary 2024-04-24T15:30:04.096 INFO:teuthology.orchestra.run.smithi151.stdout:================================================================================ 2024-04-24T15:30:04.096 INFO:teuthology.orchestra.run.smithi151.stdout:Install 5 Packages 2024-04-24T15:30:04.096 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:30:04.096 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-04-24T15:30:04.096 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-04-24T15:30:04.097 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repository Size 2024-04-24T15:30:04.097 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-04-24T15:30:04.097 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-04-24T15:30:04.097 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-rook noarch 2:17.2.7-915.g330067db.el8 ceph-noarch 47 k 2024-04-24T15:30:04.097 INFO:teuthology.orchestra.run.smithi096.stdout:Installing dependencies: 2024-04-24T15:30:04.097 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-04-24T15:30:04.097 INFO:teuthology.orchestra.run.smithi096.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-04-24T15:30:04.097 INFO:teuthology.orchestra.run.smithi096.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-04-24T15:30:04.097 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jsonpatch noarch 1.21-2.el8 appstream 27 k 2024-04-24T15:30:04.097 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jsonpointer noarch 1.10-11.el8 appstream 20 k 2024-04-24T15:30:04.097 INFO:teuthology.orchestra.run.smithi096.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-04-24T15:30:04.097 INFO:teuthology.orchestra.run.smithi096.stdout: python3-oauthlib noarch 2.1.0-1.el8 baseos 155 k 2024-04-24T15:30:04.098 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyasn1 noarch 0.3.7-6.el8 appstream 126 k 2024-04-24T15:30:04.098 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 appstream 110 k 2024-04-24T15:30:04.098 INFO:teuthology.orchestra.run.smithi096.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 baseos 43 k 2024-04-24T15:30:04.098 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-04-24T15:30:04.098 INFO:teuthology.orchestra.run.smithi096.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-04-24T15:30:04.098 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:30:04.098 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-04-24T15:30:04.098 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-04-24T15:30:04.098 INFO:teuthology.orchestra.run.smithi096.stdout:Install 13 Packages 2024-04-24T15:30:04.098 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:30:04.098 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 1.9 M 2024-04-24T15:30:04.098 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 25 M 2024-04-24T15:30:04.099 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-04-24T15:30:04.099 INFO:teuthology.orchestra.run.smithi151.stdout:Total download size: 6.0 M 2024-04-24T15:30:04.099 INFO:teuthology.orchestra.run.smithi151.stdout:Installed size: 26 M 2024-04-24T15:30:04.099 INFO:teuthology.orchestra.run.smithi151.stdout:Downloading Packages: 2024-04-24T15:30:04.921 INFO:teuthology.orchestra.run.smithi096.stdout:(1/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 115 kB/s | 27 kB 00:00 2024-04-24T15:30:04.947 INFO:teuthology.orchestra.run.smithi096.stdout:(2/13): python3-jsonpointer-1.10-11.el8.noarch. 78 kB/s | 20 kB 00:00 2024-04-24T15:30:05.147 INFO:teuthology.orchestra.run.smithi096.stdout:(3/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 561 kB/s | 126 kB 00:00 2024-04-24T15:30:05.172 INFO:teuthology.orchestra.run.smithi096.stdout:(4/13): python3-pyasn1-modules-0.3.7-6.el8.noar 488 kB/s | 110 kB 00:00 2024-04-24T15:30:05.322 INFO:teuthology.orchestra.run.smithi096.stdout:(5/13): ceph-mgr-rook-17.2.7-915.g330067db.el8. 75 kB/s | 47 kB 00:00 2024-04-24T15:30:05.414 INFO:teuthology.orchestra.run.smithi096.stdout:(6/13): python3-cachetools-3.1.1-4.el8.noarch.r 359 kB/s | 33 kB 00:00 2024-04-24T15:30:05.465 INFO:teuthology.orchestra.run.smithi096.stdout:(7/13): python3-certifi-2018.10.15-7.el8.noarch 326 kB/s | 16 kB 00:00 2024-04-24T15:30:05.540 INFO:teuthology.orchestra.run.smithi096.stdout:(8/13): python3-google-auth-1.1.1-10.el8.noarch 1.1 MB/s | 81 kB 00:00 2024-04-24T15:30:05.566 INFO:teuthology.orchestra.run.smithi096.stdout:(9/13): python3-requests-oauthlib-1.0.0-1.el8.n 108 kB/s | 43 kB 00:00 2024-04-24T15:30:05.658 INFO:teuthology.orchestra.run.smithi096.stdout:(10/13): python3-rsa-4.9-2.el8.noarch.rpm 699 kB/s | 64 kB 00:00 2024-04-24T15:30:05.699 INFO:teuthology.orchestra.run.smithi096.stdout:(11/13): python3-oauthlib-2.1.0-1.el8.noarch.rp 280 kB/s | 155 kB 00:00 2024-04-24T15:30:05.725 INFO:teuthology.orchestra.run.smithi096.stdout:(12/13): python3-websocket-client-0.56.0-5.el8. 919 kB/s | 61 kB 00:00 2024-04-24T15:30:05.917 INFO:teuthology.orchestra.run.smithi096.stdout:(13/13): python3-kubernetes-11.0.0-6.el8.noarch 3.1 MB/s | 1.2 MB 00:00 2024-04-24T15:30:05.917 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-04-24T15:30:05.917 INFO:teuthology.orchestra.run.smithi096.stdout:Total 1.1 MB/s | 1.9 MB 00:01 2024-04-24T15:30:05.975 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-04-24T15:30:05.991 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-04-24T15:30:05.991 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-04-24T15:30:06.078 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-04-24T15:30:06.081 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-04-24T15:30:06.673 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-04-24T15:30:06.912 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2024-04-24T15:30:07.090 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 2/13 2024-04-24T15:30:07.334 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-rsa-4.9-2.el8.noarch 3/13 2024-04-24T15:30:07.589 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 4/13 2024-04-24T15:30:07.796 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 5/13 2024-04-24T15:30:08.028 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 6/13 2024-04-24T15:30:08.345 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 7/13 2024-04-24T15:30:08.595 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 8/13 2024-04-24T15:30:08.646 INFO:teuthology.orchestra.run.smithi151.stdout:(1/5): ceph-mgr-cephadm-17.2.7-915.g330067db.el 416 kB/s | 123 kB 00:00 2024-04-24T15:30:08.906 INFO:teuthology.orchestra.run.smithi151.stdout:(2/5): python3-jinja2-2.10.1-4.el8.noarch.rpm 1.0 MB/s | 578 kB 00:00 2024-04-24T15:30:09.615 INFO:teuthology.orchestra.run.smithi151.stdout:(3/5): python3-babel-2.5.1-7.el8.noarch.rpm 3.8 MB/s | 4.8 MB 00:01 2024-04-24T15:30:09.628 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 9/13 2024-04-24T15:30:09.865 INFO:teuthology.orchestra.run.smithi151.stdout:(4/5): python3-natsort-7.1.1-2.el8.noarch.rpm 63 kB/s | 60 kB 00:00 2024-04-24T15:30:09.913 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 10/13 2024-04-24T15:30:10.113 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 11/13 2024-04-24T15:30:10.199 INFO:teuthology.orchestra.run.smithi151.stdout:(5/5): python3-asyncssh-2.7.0-2.el8.noarch.rpm 307 kB/s | 476 kB 00:01 2024-04-24T15:30:10.200 INFO:teuthology.orchestra.run.smithi151.stdout:-------------------------------------------------------------------------------- 2024-04-24T15:30:10.200 INFO:teuthology.orchestra.run.smithi151.stdout:Total 1.0 MB/s | 6.0 MB 00:06 2024-04-24T15:30:10.255 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction check 2024-04-24T15:30:10.267 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction check succeeded. 2024-04-24T15:30:10.267 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction test 2024-04-24T15:30:10.315 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 12/13 2024-04-24T15:30:10.371 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction test succeeded. 2024-04-24T15:30:10.374 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction 2024-04-24T15:30:10.465 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-rook-2:17.2.7-915.g330067db.el8.noarch 13/13 2024-04-24T15:30:10.647 INFO:teuthology.orchestra.run.smithi151.stdout: Preparing : 1/1 2024-04-24T15:30:10.760 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-915.g330067db.el8.noarch 13/13 2024-04-24T15:30:10.760 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-rook-2:17.2.7-915.g330067db.el8.noarch 1/13 2024-04-24T15:30:10.761 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 2/13 2024-04-24T15:30:10.761 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 3/13 2024-04-24T15:30:10.761 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 4/13 2024-04-24T15:30:10.761 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 5/13 2024-04-24T15:30:10.761 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 6/13 2024-04-24T15:30:10.761 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 7/13 2024-04-24T15:30:10.761 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 8/13 2024-04-24T15:30:10.761 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 9/13 2024-04-24T15:30:10.761 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 10/13 2024-04-24T15:30:10.761 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 11/13 2024-04-24T15:30:10.761 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 12/13 2024-04-24T15:30:11.024 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 1/5 2024-04-24T15:30:11.210 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 13/13 2024-04-24T15:30:11.210 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:30:11.210 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-04-24T15:30:11.210 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-rook-2:17.2.7-915.g330067db.el8.noarch 2024-04-24T15:30:11.210 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-04-24T15:30:11.210 INFO:teuthology.orchestra.run.smithi096.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-04-24T15:30:11.210 INFO:teuthology.orchestra.run.smithi096.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-04-24T15:30:11.210 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-04-24T15:30:11.210 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-04-24T15:30:11.210 INFO:teuthology.orchestra.run.smithi096.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-04-24T15:30:11.211 INFO:teuthology.orchestra.run.smithi096.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-04-24T15:30:11.211 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-04-24T15:30:11.211 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-04-24T15:30:11.211 INFO:teuthology.orchestra.run.smithi096.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-04-24T15:30:11.211 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rsa-4.9-2.el8.noarch 2024-04-24T15:30:11.211 INFO:teuthology.orchestra.run.smithi096.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-04-24T15:30:11.211 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:30:11.211 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-04-24T15:30:11.330 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install ceph-mgr-cephadm 2024-04-24T15:30:12.039 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 2/5 2024-04-24T15:30:12.083 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:02:25 ago on Wed 24 Apr 2024 03:27:47 PM UTC. 2024-04-24T15:30:12.414 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 3/5 2024-04-24T15:30:12.720 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jinja2-2.10.1-4.el8.noarch 4/5 2024-04-24T15:30:12.959 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-915.g330067db.el8.noarch 5/5 2024-04-24T15:30:13.242 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-915.g330067db.el8.noarch 5/5 2024-04-24T15:30:13.242 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-915.g330067db.el8.noarch 1/5 2024-04-24T15:30:13.242 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 2/5 2024-04-24T15:30:13.242 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jinja2-2.10.1-4.el8.noarch 3/5 2024-04-24T15:30:13.242 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 4/5 2024-04-24T15:30:13.625 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 5/5 2024-04-24T15:30:13.625 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:30:13.625 INFO:teuthology.orchestra.run.smithi151.stdout:Installed: 2024-04-24T15:30:13.625 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-cephadm-2:17.2.7-915.g330067db.el8.noarch 2024-04-24T15:30:13.625 INFO:teuthology.orchestra.run.smithi151.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-04-24T15:30:13.625 INFO:teuthology.orchestra.run.smithi151.stdout: python3-babel-2.5.1-7.el8.noarch 2024-04-24T15:30:13.625 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jinja2-2.10.1-4.el8.noarch 2024-04-24T15:30:13.625 INFO:teuthology.orchestra.run.smithi151.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-04-24T15:30:13.626 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:30:13.626 INFO:teuthology.orchestra.run.smithi151.stdout:Complete! 2024-04-24T15:30:13.732 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-04-24T15:30:13.733 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-04-24T15:30:13.733 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repository Size 2024-04-24T15:30:13.733 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-04-24T15:30:13.733 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-04-24T15:30:13.733 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-cephadm noarch 2:17.2.7-915.g330067db.el8 ceph-noarch 123 k 2024-04-24T15:30:13.733 INFO:teuthology.orchestra.run.smithi096.stdout:Installing dependencies: 2024-04-24T15:30:13.733 INFO:teuthology.orchestra.run.smithi096.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-04-24T15:30:13.733 INFO:teuthology.orchestra.run.smithi096.stdout: python3-babel noarch 2.5.1-7.el8 appstream 4.8 M 2024-04-24T15:30:13.733 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jinja2 noarch 2.10.1-4.el8 appstream 578 k 2024-04-24T15:30:13.733 INFO:teuthology.orchestra.run.smithi096.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-04-24T15:30:13.734 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:30:13.734 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-04-24T15:30:13.734 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-04-24T15:30:13.734 INFO:teuthology.orchestra.run.smithi096.stdout:Install 5 Packages 2024-04-24T15:30:13.734 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:30:13.734 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 6.0 M 2024-04-24T15:30:13.735 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 26 M 2024-04-24T15:30:13.735 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-04-24T15:30:13.747 DEBUG:teuthology.orchestra.run.smithi151:> sudo yum -y install ceph-fuse 2024-04-24T15:30:14.316 INFO:teuthology.orchestra.run.smithi096.stdout:(1/5): ceph-mgr-cephadm-17.2.7-915.g330067db.el 669 kB/s | 123 kB 00:00 2024-04-24T15:30:14.505 INFO:teuthology.orchestra.run.smithi151.stdout:Last metadata expiration check: 0:02:24 ago on Wed 24 Apr 2024 03:27:50 PM UTC. 2024-04-24T15:30:14.625 INFO:teuthology.orchestra.run.smithi096.stdout:(2/5): python3-jinja2-2.10.1-4.el8.noarch.rpm 1.1 MB/s | 578 kB 00:00 2024-04-24T15:30:14.708 INFO:teuthology.orchestra.run.smithi096.stdout:(3/5): python3-asyncssh-2.7.0-2.el8.noarch.rpm 1.2 MB/s | 476 kB 00:00 2024-04-24T15:30:14.900 INFO:teuthology.orchestra.run.smithi096.stdout:(4/5): python3-natsort-7.1.1-2.el8.noarch.rpm 219 kB/s | 60 kB 00:00 2024-04-24T15:30:15.001 INFO:teuthology.orchestra.run.smithi096.stdout:(5/5): python3-babel-2.5.1-7.el8.noarch.rpm 5.5 MB/s | 4.8 MB 00:00 2024-04-24T15:30:15.003 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-04-24T15:30:15.004 INFO:teuthology.orchestra.run.smithi096.stdout:Total 4.7 MB/s | 6.0 MB 00:01 2024-04-24T15:30:15.057 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-04-24T15:30:15.069 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-04-24T15:30:15.069 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-04-24T15:30:15.173 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-04-24T15:30:15.176 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-04-24T15:30:15.445 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-04-24T15:30:15.795 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 1/5 2024-04-24T15:30:16.143 INFO:teuthology.orchestra.run.smithi151.stdout:Dependencies resolved. 2024-04-24T15:30:16.143 INFO:teuthology.orchestra.run.smithi151.stdout:================================================================================ 2024-04-24T15:30:16.143 INFO:teuthology.orchestra.run.smithi151.stdout: Package Arch Version Repository Size 2024-04-24T15:30:16.144 INFO:teuthology.orchestra.run.smithi151.stdout:================================================================================ 2024-04-24T15:30:16.144 INFO:teuthology.orchestra.run.smithi151.stdout:Installing: 2024-04-24T15:30:16.144 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-fuse x86_64 2:17.2.7-915.g330067db.el8 ceph 876 k 2024-04-24T15:30:16.144 INFO:teuthology.orchestra.run.smithi151.stdout:Installing dependencies: 2024-04-24T15:30:16.144 INFO:teuthology.orchestra.run.smithi151.stdout: fuse x86_64 2.9.7-19.el8 baseos 83 k 2024-04-24T15:30:16.144 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:30:16.144 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction Summary 2024-04-24T15:30:16.144 INFO:teuthology.orchestra.run.smithi151.stdout:================================================================================ 2024-04-24T15:30:16.144 INFO:teuthology.orchestra.run.smithi151.stdout:Install 2 Packages 2024-04-24T15:30:16.144 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:30:16.145 INFO:teuthology.orchestra.run.smithi151.stdout:Total download size: 959 k 2024-04-24T15:30:16.145 INFO:teuthology.orchestra.run.smithi151.stdout:Installed size: 3.1 M 2024-04-24T15:30:16.145 INFO:teuthology.orchestra.run.smithi151.stdout:Downloading Packages: 2024-04-24T15:30:16.743 INFO:teuthology.orchestra.run.smithi151.stdout:(1/2): fuse-2.9.7-19.el8.x86_64.rpm 202 kB/s | 83 kB 00:00 2024-04-24T15:30:16.754 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 2/5 2024-04-24T15:30:16.776 INFO:teuthology.orchestra.run.smithi151.stdout:(2/2): ceph-fuse-17.2.7-915.g330067db.el8.x86_6 1.9 MB/s | 876 kB 00:00 2024-04-24T15:30:16.776 INFO:teuthology.orchestra.run.smithi151.stdout:-------------------------------------------------------------------------------- 2024-04-24T15:30:16.777 INFO:teuthology.orchestra.run.smithi151.stdout:Total 1.5 MB/s | 959 kB 00:00 2024-04-24T15:30:16.788 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction check 2024-04-24T15:30:16.809 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction check succeeded. 2024-04-24T15:30:16.809 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction test 2024-04-24T15:30:16.898 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction test succeeded. 2024-04-24T15:30:16.902 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction 2024-04-24T15:30:17.066 INFO:teuthology.orchestra.run.smithi151.stdout: Preparing : 1/1 2024-04-24T15:30:17.102 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 3/5 2024-04-24T15:30:17.335 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : fuse-2.9.7-19.el8.x86_64 1/2 2024-04-24T15:30:17.407 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jinja2-2.10.1-4.el8.noarch 4/5 2024-04-24T15:30:17.517 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-fuse-2:17.2.7-915.g330067db.el8.x86_64 2/2 2024-04-24T15:30:17.797 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-915.g330067db.el8.noarch 5/5 2024-04-24T15:30:17.913 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-fuse-2:17.2.7-915.g330067db.el8.x86_64 2/2 2024-04-24T15:30:17.913 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-fuse-2:17.2.7-915.g330067db.el8.x86_64 1/2 2024-04-24T15:30:18.366 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : fuse-2.9.7-19.el8.x86_64 2/2 2024-04-24T15:30:18.366 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:30:18.366 INFO:teuthology.orchestra.run.smithi151.stdout:Installed: 2024-04-24T15:30:18.366 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-fuse-2:17.2.7-915.g330067db.el8.x86_64 fuse-2.9.7-19.el8.x86_64 2024-04-24T15:30:18.367 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:30:18.367 INFO:teuthology.orchestra.run.smithi151.stdout:Complete! 2024-04-24T15:30:18.447 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-915.g330067db.el8.noarch 5/5 2024-04-24T15:30:18.447 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-915.g330067db.el8.noarch 1/5 2024-04-24T15:30:18.447 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 2/5 2024-04-24T15:30:18.448 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jinja2-2.10.1-4.el8.noarch 3/5 2024-04-24T15:30:18.448 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 4/5 2024-04-24T15:30:18.591 DEBUG:teuthology.orchestra.run.smithi151:> sudo yum -y install ceph-volume 2024-04-24T15:30:19.121 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 5/5 2024-04-24T15:30:19.121 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:30:19.121 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-04-24T15:30:19.121 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-cephadm-2:17.2.7-915.g330067db.el8.noarch 2024-04-24T15:30:19.121 INFO:teuthology.orchestra.run.smithi096.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-04-24T15:30:19.121 INFO:teuthology.orchestra.run.smithi096.stdout: python3-babel-2.5.1-7.el8.noarch 2024-04-24T15:30:19.121 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jinja2-2.10.1-4.el8.noarch 2024-04-24T15:30:19.121 INFO:teuthology.orchestra.run.smithi096.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-04-24T15:30:19.121 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:30:19.122 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-04-24T15:30:19.242 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install ceph-fuse 2024-04-24T15:30:19.360 INFO:teuthology.orchestra.run.smithi151.stdout:Last metadata expiration check: 0:02:29 ago on Wed 24 Apr 2024 03:27:50 PM UTC. 2024-04-24T15:30:20.003 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:02:33 ago on Wed 24 Apr 2024 03:27:47 PM UTC. 2024-04-24T15:30:21.155 INFO:teuthology.orchestra.run.smithi151.stdout:Dependencies resolved. 2024-04-24T15:30:21.156 INFO:teuthology.orchestra.run.smithi151.stdout:================================================================================ 2024-04-24T15:30:21.156 INFO:teuthology.orchestra.run.smithi151.stdout: Package Arch Version Repository Size 2024-04-24T15:30:21.156 INFO:teuthology.orchestra.run.smithi151.stdout:================================================================================ 2024-04-24T15:30:21.156 INFO:teuthology.orchestra.run.smithi151.stdout:Installing: 2024-04-24T15:30:21.156 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-volume noarch 2:17.2.7-915.g330067db.el8 ceph-noarch 267 k 2024-04-24T15:30:21.156 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:30:21.156 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction Summary 2024-04-24T15:30:21.156 INFO:teuthology.orchestra.run.smithi151.stdout:================================================================================ 2024-04-24T15:30:21.156 INFO:teuthology.orchestra.run.smithi151.stdout:Install 1 Package 2024-04-24T15:30:21.156 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:30:21.156 INFO:teuthology.orchestra.run.smithi151.stdout:Total download size: 267 k 2024-04-24T15:30:21.156 INFO:teuthology.orchestra.run.smithi151.stdout:Installed size: 1.3 M 2024-04-24T15:30:21.156 INFO:teuthology.orchestra.run.smithi151.stdout:Downloading Packages: 2024-04-24T15:30:21.660 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-04-24T15:30:21.661 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-04-24T15:30:21.661 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repository Size 2024-04-24T15:30:21.661 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-04-24T15:30:21.661 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-04-24T15:30:21.661 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-fuse x86_64 2:17.2.7-915.g330067db.el8 ceph 876 k 2024-04-24T15:30:21.661 INFO:teuthology.orchestra.run.smithi096.stdout:Installing dependencies: 2024-04-24T15:30:21.661 INFO:teuthology.orchestra.run.smithi096.stdout: fuse x86_64 2.9.7-19.el8 baseos 83 k 2024-04-24T15:30:21.661 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:30:21.661 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-04-24T15:30:21.662 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-04-24T15:30:21.662 INFO:teuthology.orchestra.run.smithi096.stdout:Install 2 Packages 2024-04-24T15:30:21.662 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:30:21.662 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 959 k 2024-04-24T15:30:21.662 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 3.1 M 2024-04-24T15:30:21.662 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-04-24T15:30:22.201 INFO:teuthology.orchestra.run.smithi096.stdout:(1/2): fuse-2.9.7-19.el8.x86_64.rpm 205 kB/s | 83 kB 00:00 2024-04-24T15:30:22.256 INFO:teuthology.orchestra.run.smithi151.stdout:ceph-volume-17.2.7-915.g330067db.el8.noarch.rpm 243 kB/s | 267 kB 00:01 2024-04-24T15:30:22.257 INFO:teuthology.orchestra.run.smithi151.stdout:-------------------------------------------------------------------------------- 2024-04-24T15:30:22.258 INFO:teuthology.orchestra.run.smithi151.stdout:Total 243 kB/s | 267 kB 00:01 2024-04-24T15:30:22.258 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction check 2024-04-24T15:30:22.270 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction check succeeded. 2024-04-24T15:30:22.270 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction test 2024-04-24T15:30:22.310 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction test succeeded. 2024-04-24T15:30:22.314 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction 2024-04-24T15:30:22.541 INFO:teuthology.orchestra.run.smithi151.stdout: Preparing : 1/1 2024-04-24T15:30:22.585 INFO:teuthology.orchestra.run.smithi096.stdout:(2/2): ceph-fuse-17.2.7-915.g330067db.el8.x86_6 1.1 MB/s | 876 kB 00:00 2024-04-24T15:30:22.585 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-04-24T15:30:22.586 INFO:teuthology.orchestra.run.smithi096.stdout:Total 1.0 MB/s | 959 kB 00:00 2024-04-24T15:30:22.597 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-04-24T15:30:22.618 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-04-24T15:30:22.618 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-04-24T15:30:22.705 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-04-24T15:30:22.709 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-04-24T15:30:22.833 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-volume-2:17.2.7-915.g330067db.el8.noarch 1/1 2024-04-24T15:30:22.879 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-04-24T15:30:23.180 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : fuse-2.9.7-19.el8.x86_64 1/2 2024-04-24T15:30:23.290 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-volume-2:17.2.7-915.g330067db.el8.noarch 1/1 2024-04-24T15:30:23.395 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-fuse-2:17.2.7-915.g330067db.el8.x86_64 2/2 2024-04-24T15:30:23.697 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-volume-2:17.2.7-915.g330067db.el8.noarch 1/1 2024-04-24T15:30:23.697 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:30:23.697 INFO:teuthology.orchestra.run.smithi151.stdout:Installed: 2024-04-24T15:30:23.697 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-volume-2:17.2.7-915.g330067db.el8.noarch 2024-04-24T15:30:23.697 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:30:23.697 INFO:teuthology.orchestra.run.smithi151.stdout:Complete! 2024-04-24T15:30:23.782 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-fuse-2:17.2.7-915.g330067db.el8.x86_64 2/2 2024-04-24T15:30:23.782 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-fuse-2:17.2.7-915.g330067db.el8.x86_64 1/2 2024-04-24T15:30:23.961 DEBUG:teuthology.orchestra.run.smithi151:> sudo yum -y install librados-devel 2024-04-24T15:30:24.157 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : fuse-2.9.7-19.el8.x86_64 2/2 2024-04-24T15:30:24.157 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:30:24.157 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-04-24T15:30:24.157 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-fuse-2:17.2.7-915.g330067db.el8.x86_64 fuse-2.9.7-19.el8.x86_64 2024-04-24T15:30:24.157 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:30:24.157 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-04-24T15:30:24.387 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install ceph-volume 2024-04-24T15:30:24.734 INFO:teuthology.orchestra.run.smithi151.stdout:Last metadata expiration check: 0:02:34 ago on Wed 24 Apr 2024 03:27:50 PM UTC. 2024-04-24T15:30:25.156 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:02:38 ago on Wed 24 Apr 2024 03:27:47 PM UTC. 2024-04-24T15:30:26.385 INFO:teuthology.orchestra.run.smithi151.stdout:Dependencies resolved. 2024-04-24T15:30:26.385 INFO:teuthology.orchestra.run.smithi151.stdout:================================================================================ 2024-04-24T15:30:26.385 INFO:teuthology.orchestra.run.smithi151.stdout: Package Arch Version Repo Size 2024-04-24T15:30:26.385 INFO:teuthology.orchestra.run.smithi151.stdout:================================================================================ 2024-04-24T15:30:26.385 INFO:teuthology.orchestra.run.smithi151.stdout:Installing: 2024-04-24T15:30:26.385 INFO:teuthology.orchestra.run.smithi151.stdout: librados-devel x86_64 2:17.2.7-915.g330067db.el8 ceph 130 k 2024-04-24T15:30:26.386 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:30:26.386 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction Summary 2024-04-24T15:30:26.386 INFO:teuthology.orchestra.run.smithi151.stdout:================================================================================ 2024-04-24T15:30:26.386 INFO:teuthology.orchestra.run.smithi151.stdout:Install 1 Package 2024-04-24T15:30:26.386 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:30:26.386 INFO:teuthology.orchestra.run.smithi151.stdout:Total download size: 130 k 2024-04-24T15:30:26.386 INFO:teuthology.orchestra.run.smithi151.stdout:Installed size: 464 k 2024-04-24T15:30:26.387 INFO:teuthology.orchestra.run.smithi151.stdout:Downloading Packages: 2024-04-24T15:30:26.810 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-04-24T15:30:26.811 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-04-24T15:30:26.811 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repository Size 2024-04-24T15:30:26.811 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-04-24T15:30:26.811 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-04-24T15:30:26.811 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-volume noarch 2:17.2.7-915.g330067db.el8 ceph-noarch 267 k 2024-04-24T15:30:26.811 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:30:26.811 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-04-24T15:30:26.811 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-04-24T15:30:26.811 INFO:teuthology.orchestra.run.smithi096.stdout:Install 1 Package 2024-04-24T15:30:26.811 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:30:26.812 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 267 k 2024-04-24T15:30:26.812 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 1.3 M 2024-04-24T15:30:26.812 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-04-24T15:30:26.959 INFO:teuthology.orchestra.run.smithi151.stdout:librados-devel-17.2.7-915.g330067db.el8.x86_64. 227 kB/s | 130 kB 00:00 2024-04-24T15:30:26.960 INFO:teuthology.orchestra.run.smithi151.stdout:-------------------------------------------------------------------------------- 2024-04-24T15:30:26.960 INFO:teuthology.orchestra.run.smithi151.stdout:Total 226 kB/s | 130 kB 00:00 2024-04-24T15:30:26.960 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction check 2024-04-24T15:30:26.965 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction check succeeded. 2024-04-24T15:30:26.966 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction test 2024-04-24T15:30:27.015 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction test succeeded. 2024-04-24T15:30:27.018 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction 2024-04-24T15:30:27.141 INFO:teuthology.orchestra.run.smithi151.stdout: Preparing : 1/1 2024-04-24T15:30:27.306 INFO:teuthology.orchestra.run.smithi096.stdout:ceph-volume-17.2.7-915.g330067db.el8.noarch.rpm 541 kB/s | 267 kB 00:00 2024-04-24T15:30:27.307 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-04-24T15:30:27.307 INFO:teuthology.orchestra.run.smithi096.stdout:Total 540 kB/s | 267 kB 00:00 2024-04-24T15:30:27.307 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-04-24T15:30:27.318 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-04-24T15:30:27.318 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-04-24T15:30:27.350 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-04-24T15:30:27.353 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-04-24T15:30:27.398 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : librados-devel-2:17.2.7-915.g330067db.el8.x86_64 1/1 2024-04-24T15:30:27.535 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-04-24T15:30:27.690 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: librados-devel-2:17.2.7-915.g330067db.el8.x86_64 1/1 2024-04-24T15:30:27.795 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-volume-2:17.2.7-915.g330067db.el8.noarch 1/1 2024-04-24T15:30:28.103 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librados-devel-2:17.2.7-915.g330067db.el8.x86_64 1/1 2024-04-24T15:30:28.103 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:30:28.103 INFO:teuthology.orchestra.run.smithi151.stdout:Installed: 2024-04-24T15:30:28.103 INFO:teuthology.orchestra.run.smithi151.stdout: librados-devel-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T15:30:28.103 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:30:28.103 INFO:teuthology.orchestra.run.smithi151.stdout:Complete! 2024-04-24T15:30:28.183 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-volume-2:17.2.7-915.g330067db.el8.noarch 1/1 2024-04-24T15:30:28.352 DEBUG:teuthology.orchestra.run.smithi151:> sudo yum -y install libcephfs2 2024-04-24T15:30:28.544 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-volume-2:17.2.7-915.g330067db.el8.noarch 1/1 2024-04-24T15:30:28.544 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:30:28.544 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-04-24T15:30:28.544 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-volume-2:17.2.7-915.g330067db.el8.noarch 2024-04-24T15:30:28.544 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:30:28.544 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-04-24T15:30:28.739 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install librados-devel 2024-04-24T15:30:29.114 INFO:teuthology.orchestra.run.smithi151.stdout:Last metadata expiration check: 0:02:39 ago on Wed 24 Apr 2024 03:27:50 PM UTC. 2024-04-24T15:30:29.511 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:02:42 ago on Wed 24 Apr 2024 03:27:47 PM UTC. 2024-04-24T15:30:30.723 INFO:teuthology.orchestra.run.smithi151.stdout:Package libcephfs2-2:17.2.7-915.g330067db.el8.x86_64 is already installed. 2024-04-24T15:30:30.770 INFO:teuthology.orchestra.run.smithi151.stdout:Dependencies resolved. 2024-04-24T15:30:30.771 INFO:teuthology.orchestra.run.smithi151.stdout:Nothing to do. 2024-04-24T15:30:30.771 INFO:teuthology.orchestra.run.smithi151.stdout:Complete! 2024-04-24T15:30:30.885 DEBUG:teuthology.orchestra.run.smithi151:> sudo yum -y install libcephfs-devel 2024-04-24T15:30:31.179 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-04-24T15:30:31.180 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-04-24T15:30:31.180 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repo Size 2024-04-24T15:30:31.180 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-04-24T15:30:31.180 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-04-24T15:30:31.180 INFO:teuthology.orchestra.run.smithi096.stdout: librados-devel x86_64 2:17.2.7-915.g330067db.el8 ceph 130 k 2024-04-24T15:30:31.180 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:30:31.181 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-04-24T15:30:31.181 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-04-24T15:30:31.181 INFO:teuthology.orchestra.run.smithi096.stdout:Install 1 Package 2024-04-24T15:30:31.181 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:30:31.181 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 130 k 2024-04-24T15:30:31.181 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 464 k 2024-04-24T15:30:31.181 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-04-24T15:30:31.556 INFO:teuthology.orchestra.run.smithi096.stdout:librados-devel-17.2.7-915.g330067db.el8.x86_64. 346 kB/s | 130 kB 00:00 2024-04-24T15:30:31.557 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-04-24T15:30:31.557 INFO:teuthology.orchestra.run.smithi096.stdout:Total 345 kB/s | 130 kB 00:00 2024-04-24T15:30:31.557 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-04-24T15:30:31.562 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-04-24T15:30:31.563 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-04-24T15:30:31.613 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-04-24T15:30:31.615 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-04-24T15:30:31.637 INFO:teuthology.orchestra.run.smithi151.stdout:Last metadata expiration check: 0:02:41 ago on Wed 24 Apr 2024 03:27:50 PM UTC. 2024-04-24T15:30:31.733 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-04-24T15:30:31.943 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : librados-devel-2:17.2.7-915.g330067db.el8.x86_64 1/1 2024-04-24T15:30:32.220 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librados-devel-2:17.2.7-915.g330067db.el8.x86_64 1/1 2024-04-24T15:30:32.578 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librados-devel-2:17.2.7-915.g330067db.el8.x86_64 1/1 2024-04-24T15:30:32.578 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:30:32.578 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-04-24T15:30:32.578 INFO:teuthology.orchestra.run.smithi096.stdout: librados-devel-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T15:30:32.578 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:30:32.578 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-04-24T15:30:32.792 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install libcephfs2 2024-04-24T15:30:33.249 INFO:teuthology.orchestra.run.smithi151.stdout:Dependencies resolved. 2024-04-24T15:30:33.250 INFO:teuthology.orchestra.run.smithi151.stdout:================================================================================ 2024-04-24T15:30:33.250 INFO:teuthology.orchestra.run.smithi151.stdout: Package Arch Version Repo Size 2024-04-24T15:30:33.250 INFO:teuthology.orchestra.run.smithi151.stdout:================================================================================ 2024-04-24T15:30:33.250 INFO:teuthology.orchestra.run.smithi151.stdout:Installing: 2024-04-24T15:30:33.250 INFO:teuthology.orchestra.run.smithi151.stdout: libcephfs-devel x86_64 2:17.2.7-915.g330067db.el8 ceph 25 k 2024-04-24T15:30:33.250 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:30:33.250 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction Summary 2024-04-24T15:30:33.250 INFO:teuthology.orchestra.run.smithi151.stdout:================================================================================ 2024-04-24T15:30:33.250 INFO:teuthology.orchestra.run.smithi151.stdout:Install 1 Package 2024-04-24T15:30:33.251 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:30:33.251 INFO:teuthology.orchestra.run.smithi151.stdout:Total download size: 25 k 2024-04-24T15:30:33.251 INFO:teuthology.orchestra.run.smithi151.stdout:Installed size: 109 k 2024-04-24T15:30:33.251 INFO:teuthology.orchestra.run.smithi151.stdout:Downloading Packages: 2024-04-24T15:30:33.550 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:02:46 ago on Wed 24 Apr 2024 03:27:47 PM UTC. 2024-04-24T15:30:33.706 INFO:teuthology.orchestra.run.smithi151.stdout:libcephfs-devel-17.2.7-915.g330067db.el8.x86_64 55 kB/s | 25 kB 00:00 2024-04-24T15:30:33.706 INFO:teuthology.orchestra.run.smithi151.stdout:-------------------------------------------------------------------------------- 2024-04-24T15:30:33.707 INFO:teuthology.orchestra.run.smithi151.stdout:Total 54 kB/s | 25 kB 00:00 2024-04-24T15:30:33.707 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction check 2024-04-24T15:30:33.711 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction check succeeded. 2024-04-24T15:30:33.711 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction test 2024-04-24T15:30:33.713 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction test succeeded. 2024-04-24T15:30:33.716 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction 2024-04-24T15:30:33.766 INFO:teuthology.orchestra.run.smithi151.stdout: Preparing : 1/1 2024-04-24T15:30:33.997 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libcephfs-devel-2:17.2.7-915.g330067db.el8.x86_64 1/1 2024-04-24T15:30:34.254 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-915.g330067db.el8.x86_64 1/1 2024-04-24T15:30:34.623 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libcephfs-devel-2:17.2.7-915.g330067db.el8.x86_64 1/1 2024-04-24T15:30:34.623 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:30:34.623 INFO:teuthology.orchestra.run.smithi151.stdout:Installed: 2024-04-24T15:30:34.623 INFO:teuthology.orchestra.run.smithi151.stdout: libcephfs-devel-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T15:30:34.623 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:30:34.623 INFO:teuthology.orchestra.run.smithi151.stdout:Complete! 2024-04-24T15:30:34.754 DEBUG:teuthology.orchestra.run.smithi151:> sudo yum -y install librados2 2024-04-24T15:30:35.141 INFO:teuthology.orchestra.run.smithi096.stdout:Package libcephfs2-2:17.2.7-915.g330067db.el8.x86_64 is already installed. 2024-04-24T15:30:35.188 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-04-24T15:30:35.190 INFO:teuthology.orchestra.run.smithi096.stdout:Nothing to do. 2024-04-24T15:30:35.190 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-04-24T15:30:35.301 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install libcephfs-devel 2024-04-24T15:30:35.502 INFO:teuthology.orchestra.run.smithi151.stdout:Last metadata expiration check: 0:02:45 ago on Wed 24 Apr 2024 03:27:50 PM UTC. 2024-04-24T15:30:36.063 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:02:49 ago on Wed 24 Apr 2024 03:27:47 PM UTC. 2024-04-24T15:30:37.094 INFO:teuthology.orchestra.run.smithi151.stdout:Package librados2-2:17.2.7-915.g330067db.el8.x86_64 is already installed. 2024-04-24T15:30:37.145 INFO:teuthology.orchestra.run.smithi151.stdout:Dependencies resolved. 2024-04-24T15:30:37.146 INFO:teuthology.orchestra.run.smithi151.stdout:Nothing to do. 2024-04-24T15:30:37.146 INFO:teuthology.orchestra.run.smithi151.stdout:Complete! 2024-04-24T15:30:37.265 DEBUG:teuthology.orchestra.run.smithi151:> sudo yum -y install librbd1 2024-04-24T15:30:37.722 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-04-24T15:30:37.722 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-04-24T15:30:37.722 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repo Size 2024-04-24T15:30:37.722 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-04-24T15:30:37.723 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-04-24T15:30:37.723 INFO:teuthology.orchestra.run.smithi096.stdout: libcephfs-devel x86_64 2:17.2.7-915.g330067db.el8 ceph 25 k 2024-04-24T15:30:37.723 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:30:37.723 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-04-24T15:30:37.723 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-04-24T15:30:37.723 INFO:teuthology.orchestra.run.smithi096.stdout:Install 1 Package 2024-04-24T15:30:37.723 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:30:37.724 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 25 k 2024-04-24T15:30:37.724 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 109 k 2024-04-24T15:30:37.724 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-04-24T15:30:38.015 INFO:teuthology.orchestra.run.smithi151.stdout:Last metadata expiration check: 0:02:48 ago on Wed 24 Apr 2024 03:27:50 PM UTC. 2024-04-24T15:30:38.034 INFO:teuthology.orchestra.run.smithi096.stdout:libcephfs-devel-17.2.7-915.g330067db.el8.x86_64 80 kB/s | 25 kB 00:00 2024-04-24T15:30:38.035 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-04-24T15:30:38.035 INFO:teuthology.orchestra.run.smithi096.stdout:Total 79 kB/s | 25 kB 00:00 2024-04-24T15:30:38.036 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-04-24T15:30:38.039 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-04-24T15:30:38.039 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-04-24T15:30:38.041 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-04-24T15:30:38.044 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-04-24T15:30:38.094 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-04-24T15:30:38.307 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libcephfs-devel-2:17.2.7-915.g330067db.el8.x86_64 1/1 2024-04-24T15:30:38.582 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-915.g330067db.el8.x86_64 1/1 2024-04-24T15:30:38.918 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libcephfs-devel-2:17.2.7-915.g330067db.el8.x86_64 1/1 2024-04-24T15:30:38.918 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:30:38.918 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-04-24T15:30:38.918 INFO:teuthology.orchestra.run.smithi096.stdout: libcephfs-devel-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T15:30:38.918 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:30:38.918 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-04-24T15:30:39.057 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install librados2 2024-04-24T15:30:39.635 INFO:teuthology.orchestra.run.smithi151.stdout:Package librbd1-2:17.2.7-915.g330067db.el8.x86_64 is already installed. 2024-04-24T15:30:39.684 INFO:teuthology.orchestra.run.smithi151.stdout:Dependencies resolved. 2024-04-24T15:30:39.686 INFO:teuthology.orchestra.run.smithi151.stdout:Nothing to do. 2024-04-24T15:30:39.686 INFO:teuthology.orchestra.run.smithi151.stdout:Complete! 2024-04-24T15:30:39.804 DEBUG:teuthology.orchestra.run.smithi151:> sudo yum -y install python3-rados 2024-04-24T15:30:39.807 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:02:52 ago on Wed 24 Apr 2024 03:27:47 PM UTC. 2024-04-24T15:30:40.555 INFO:teuthology.orchestra.run.smithi151.stdout:Last metadata expiration check: 0:02:50 ago on Wed 24 Apr 2024 03:27:50 PM UTC. 2024-04-24T15:30:41.375 INFO:teuthology.orchestra.run.smithi096.stdout:Package librados2-2:17.2.7-915.g330067db.el8.x86_64 is already installed. 2024-04-24T15:30:41.424 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-04-24T15:30:41.425 INFO:teuthology.orchestra.run.smithi096.stdout:Nothing to do. 2024-04-24T15:30:41.425 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-04-24T15:30:41.529 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install librbd1 2024-04-24T15:30:42.131 INFO:teuthology.orchestra.run.smithi151.stdout:Package python3-rados-2:17.2.7-915.g330067db.el8.x86_64 is already installed. 2024-04-24T15:30:42.179 INFO:teuthology.orchestra.run.smithi151.stdout:Dependencies resolved. 2024-04-24T15:30:42.180 INFO:teuthology.orchestra.run.smithi151.stdout:Nothing to do. 2024-04-24T15:30:42.181 INFO:teuthology.orchestra.run.smithi151.stdout:Complete! 2024-04-24T15:30:42.277 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:02:55 ago on Wed 24 Apr 2024 03:27:47 PM UTC. 2024-04-24T15:30:42.297 DEBUG:teuthology.orchestra.run.smithi151:> sudo yum -y install python3-rgw 2024-04-24T15:30:43.063 INFO:teuthology.orchestra.run.smithi151.stdout:Last metadata expiration check: 0:02:53 ago on Wed 24 Apr 2024 03:27:50 PM UTC. 2024-04-24T15:30:43.899 INFO:teuthology.orchestra.run.smithi096.stdout:Package librbd1-2:17.2.7-915.g330067db.el8.x86_64 is already installed. 2024-04-24T15:30:43.948 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-04-24T15:30:43.950 INFO:teuthology.orchestra.run.smithi096.stdout:Nothing to do. 2024-04-24T15:30:43.950 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-04-24T15:30:44.065 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install python3-rados 2024-04-24T15:30:44.676 INFO:teuthology.orchestra.run.smithi151.stdout:Package python3-rgw-2:17.2.7-915.g330067db.el8.x86_64 is already installed. 2024-04-24T15:30:44.724 INFO:teuthology.orchestra.run.smithi151.stdout:Dependencies resolved. 2024-04-24T15:30:44.725 INFO:teuthology.orchestra.run.smithi151.stdout:Nothing to do. 2024-04-24T15:30:44.725 INFO:teuthology.orchestra.run.smithi151.stdout:Complete! 2024-04-24T15:30:44.812 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:02:57 ago on Wed 24 Apr 2024 03:27:47 PM UTC. 2024-04-24T15:30:44.838 DEBUG:teuthology.orchestra.run.smithi151:> sudo yum -y install python3-cephfs 2024-04-24T15:30:45.590 INFO:teuthology.orchestra.run.smithi151.stdout:Last metadata expiration check: 0:02:55 ago on Wed 24 Apr 2024 03:27:50 PM UTC. 2024-04-24T15:30:46.390 INFO:teuthology.orchestra.run.smithi096.stdout:Package python3-rados-2:17.2.7-915.g330067db.el8.x86_64 is already installed. 2024-04-24T15:30:46.438 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-04-24T15:30:46.439 INFO:teuthology.orchestra.run.smithi096.stdout:Nothing to do. 2024-04-24T15:30:46.439 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-04-24T15:30:46.552 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install python3-rgw 2024-04-24T15:30:47.202 INFO:teuthology.orchestra.run.smithi151.stdout:Package python3-cephfs-2:17.2.7-915.g330067db.el8.x86_64 is already installed. 2024-04-24T15:30:47.250 INFO:teuthology.orchestra.run.smithi151.stdout:Dependencies resolved. 2024-04-24T15:30:47.251 INFO:teuthology.orchestra.run.smithi151.stdout:Nothing to do. 2024-04-24T15:30:47.251 INFO:teuthology.orchestra.run.smithi151.stdout:Complete! 2024-04-24T15:30:47.299 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:03:00 ago on Wed 24 Apr 2024 03:27:47 PM UTC. 2024-04-24T15:30:47.363 DEBUG:teuthology.orchestra.run.smithi151:> sudo yum -y install python3-rbd 2024-04-24T15:30:48.115 INFO:teuthology.orchestra.run.smithi151.stdout:Last metadata expiration check: 0:02:58 ago on Wed 24 Apr 2024 03:27:50 PM UTC. 2024-04-24T15:30:48.899 INFO:teuthology.orchestra.run.smithi096.stdout:Package python3-rgw-2:17.2.7-915.g330067db.el8.x86_64 is already installed. 2024-04-24T15:30:48.947 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-04-24T15:30:48.948 INFO:teuthology.orchestra.run.smithi096.stdout:Nothing to do. 2024-04-24T15:30:48.948 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-04-24T15:30:49.060 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install python3-cephfs 2024-04-24T15:30:49.710 INFO:teuthology.orchestra.run.smithi151.stdout:Package python3-rbd-2:17.2.7-915.g330067db.el8.x86_64 is already installed. 2024-04-24T15:30:49.757 INFO:teuthology.orchestra.run.smithi151.stdout:Dependencies resolved. 2024-04-24T15:30:49.758 INFO:teuthology.orchestra.run.smithi151.stdout:Nothing to do. 2024-04-24T15:30:49.758 INFO:teuthology.orchestra.run.smithi151.stdout:Complete! 2024-04-24T15:30:49.807 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:03:02 ago on Wed 24 Apr 2024 03:27:47 PM UTC. 2024-04-24T15:30:49.880 DEBUG:teuthology.orchestra.run.smithi151:> sudo yum -y install rbd-fuse 2024-04-24T15:30:50.633 INFO:teuthology.orchestra.run.smithi151.stdout:Last metadata expiration check: 0:03:00 ago on Wed 24 Apr 2024 03:27:50 PM UTC. 2024-04-24T15:30:51.384 INFO:teuthology.orchestra.run.smithi096.stdout:Package python3-cephfs-2:17.2.7-915.g330067db.el8.x86_64 is already installed. 2024-04-24T15:30:51.432 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-04-24T15:30:51.433 INFO:teuthology.orchestra.run.smithi096.stdout:Nothing to do. 2024-04-24T15:30:51.433 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-04-24T15:30:51.547 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install python3-rbd 2024-04-24T15:30:52.280 INFO:teuthology.orchestra.run.smithi151.stdout:Dependencies resolved. 2024-04-24T15:30:52.281 INFO:teuthology.orchestra.run.smithi151.stdout:================================================================================ 2024-04-24T15:30:52.281 INFO:teuthology.orchestra.run.smithi151.stdout: Package Architecture Version Repository Size 2024-04-24T15:30:52.282 INFO:teuthology.orchestra.run.smithi151.stdout:================================================================================ 2024-04-24T15:30:52.282 INFO:teuthology.orchestra.run.smithi151.stdout:Installing: 2024-04-24T15:30:52.282 INFO:teuthology.orchestra.run.smithi151.stdout: rbd-fuse x86_64 2:17.2.7-915.g330067db.el8 ceph 91 k 2024-04-24T15:30:52.282 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:30:52.282 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction Summary 2024-04-24T15:30:52.282 INFO:teuthology.orchestra.run.smithi151.stdout:================================================================================ 2024-04-24T15:30:52.282 INFO:teuthology.orchestra.run.smithi151.stdout:Install 1 Package 2024-04-24T15:30:52.282 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:30:52.282 INFO:teuthology.orchestra.run.smithi151.stdout:Total download size: 91 k 2024-04-24T15:30:52.283 INFO:teuthology.orchestra.run.smithi151.stdout:Installed size: 268 k 2024-04-24T15:30:52.283 INFO:teuthology.orchestra.run.smithi151.stdout:Downloading Packages: 2024-04-24T15:30:52.301 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:03:05 ago on Wed 24 Apr 2024 03:27:47 PM UTC. 2024-04-24T15:30:52.542 INFO:teuthology.orchestra.run.smithi151.stdout:rbd-fuse-17.2.7-915.g330067db.el8.x86_64.rpm 348 kB/s | 91 kB 00:00 2024-04-24T15:30:52.543 INFO:teuthology.orchestra.run.smithi151.stdout:-------------------------------------------------------------------------------- 2024-04-24T15:30:52.543 INFO:teuthology.orchestra.run.smithi151.stdout:Total 346 kB/s | 91 kB 00:00 2024-04-24T15:30:52.543 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction check 2024-04-24T15:30:52.553 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction check succeeded. 2024-04-24T15:30:52.553 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction test 2024-04-24T15:30:52.602 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction test succeeded. 2024-04-24T15:30:52.605 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction 2024-04-24T15:30:52.707 INFO:teuthology.orchestra.run.smithi151.stdout: Preparing : 1/1 2024-04-24T15:30:52.886 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : rbd-fuse-2:17.2.7-915.g330067db.el8.x86_64 1/1 2024-04-24T15:30:53.158 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: rbd-fuse-2:17.2.7-915.g330067db.el8.x86_64 1/1 2024-04-24T15:30:53.542 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : rbd-fuse-2:17.2.7-915.g330067db.el8.x86_64 1/1 2024-04-24T15:30:53.542 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:30:53.542 INFO:teuthology.orchestra.run.smithi151.stdout:Installed: 2024-04-24T15:30:53.543 INFO:teuthology.orchestra.run.smithi151.stdout: rbd-fuse-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T15:30:53.543 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:30:53.543 INFO:teuthology.orchestra.run.smithi151.stdout:Complete! 2024-04-24T15:30:53.809 DEBUG:teuthology.orchestra.run.smithi151:> sudo yum -y install rbd-mirror 2024-04-24T15:30:53.893 INFO:teuthology.orchestra.run.smithi096.stdout:Package python3-rbd-2:17.2.7-915.g330067db.el8.x86_64 is already installed. 2024-04-24T15:30:53.940 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-04-24T15:30:53.941 INFO:teuthology.orchestra.run.smithi096.stdout:Nothing to do. 2024-04-24T15:30:53.941 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-04-24T15:30:54.053 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install rbd-fuse 2024-04-24T15:30:54.568 INFO:teuthology.orchestra.run.smithi151.stdout:Last metadata expiration check: 0:03:04 ago on Wed 24 Apr 2024 03:27:50 PM UTC. 2024-04-24T15:30:54.799 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:03:07 ago on Wed 24 Apr 2024 03:27:47 PM UTC. 2024-04-24T15:30:56.212 INFO:teuthology.orchestra.run.smithi151.stdout:Dependencies resolved. 2024-04-24T15:30:56.212 INFO:teuthology.orchestra.run.smithi151.stdout:================================================================================ 2024-04-24T15:30:56.212 INFO:teuthology.orchestra.run.smithi151.stdout: Package Arch Version Repo Size 2024-04-24T15:30:56.213 INFO:teuthology.orchestra.run.smithi151.stdout:================================================================================ 2024-04-24T15:30:56.213 INFO:teuthology.orchestra.run.smithi151.stdout:Installing: 2024-04-24T15:30:56.213 INFO:teuthology.orchestra.run.smithi151.stdout: rbd-mirror x86_64 2:17.2.7-915.g330067db.el8 ceph 3.5 M 2024-04-24T15:30:56.213 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:30:56.213 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction Summary 2024-04-24T15:30:56.213 INFO:teuthology.orchestra.run.smithi151.stdout:================================================================================ 2024-04-24T15:30:56.213 INFO:teuthology.orchestra.run.smithi151.stdout:Install 1 Package 2024-04-24T15:30:56.213 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:30:56.214 INFO:teuthology.orchestra.run.smithi151.stdout:Total download size: 3.5 M 2024-04-24T15:30:56.214 INFO:teuthology.orchestra.run.smithi151.stdout:Installed size: 16 M 2024-04-24T15:30:56.214 INFO:teuthology.orchestra.run.smithi151.stdout:Downloading Packages: 2024-04-24T15:30:56.454 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-04-24T15:30:56.455 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-04-24T15:30:56.455 INFO:teuthology.orchestra.run.smithi096.stdout: Package Architecture Version Repository Size 2024-04-24T15:30:56.455 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-04-24T15:30:56.455 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-04-24T15:30:56.455 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-fuse x86_64 2:17.2.7-915.g330067db.el8 ceph 91 k 2024-04-24T15:30:56.455 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:30:56.456 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-04-24T15:30:56.456 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-04-24T15:30:56.456 INFO:teuthology.orchestra.run.smithi096.stdout:Install 1 Package 2024-04-24T15:30:56.456 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:30:56.456 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 91 k 2024-04-24T15:30:56.456 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 268 k 2024-04-24T15:30:56.457 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-04-24T15:30:56.620 INFO:teuthology.orchestra.run.smithi151.stdout:rbd-mirror-17.2.7-915.g330067db.el8.x86_64.rpm 8.5 MB/s | 3.5 MB 00:00 2024-04-24T15:30:56.620 INFO:teuthology.orchestra.run.smithi151.stdout:-------------------------------------------------------------------------------- 2024-04-24T15:30:56.621 INFO:teuthology.orchestra.run.smithi151.stdout:Total 8.5 MB/s | 3.5 MB 00:00 2024-04-24T15:30:56.621 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction check 2024-04-24T15:30:56.632 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction check succeeded. 2024-04-24T15:30:56.632 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction test 2024-04-24T15:30:56.694 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction test succeeded. 2024-04-24T15:30:56.697 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction 2024-04-24T15:30:56.741 INFO:teuthology.orchestra.run.smithi096.stdout:rbd-fuse-17.2.7-915.g330067db.el8.x86_64.rpm 318 kB/s | 91 kB 00:00 2024-04-24T15:30:56.742 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-04-24T15:30:56.742 INFO:teuthology.orchestra.run.smithi096.stdout:Total 317 kB/s | 91 kB 00:00 2024-04-24T15:30:56.742 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-04-24T15:30:56.752 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-04-24T15:30:56.752 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-04-24T15:30:56.801 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-04-24T15:30:56.804 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-04-24T15:30:56.909 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-04-24T15:30:57.120 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : rbd-fuse-2:17.2.7-915.g330067db.el8.x86_64 1/1 2024-04-24T15:30:57.128 INFO:teuthology.orchestra.run.smithi151.stdout: Preparing : 1/1 2024-04-24T15:30:57.311 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : rbd-mirror-2:17.2.7-915.g330067db.el8.x86_64 1/1 2024-04-24T15:30:57.407 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: rbd-fuse-2:17.2.7-915.g330067db.el8.x86_64 1/1 2024-04-24T15:30:57.728 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: rbd-mirror-2:17.2.7-915.g330067db.el8.x86_64 1/1 2024-04-24T15:30:57.781 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : rbd-fuse-2:17.2.7-915.g330067db.el8.x86_64 1/1 2024-04-24T15:30:57.782 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:30:57.782 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-04-24T15:30:57.782 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-fuse-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T15:30:57.782 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:30:57.782 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-04-24T15:30:58.003 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install rbd-mirror 2024-04-24T15:30:58.177 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : rbd-mirror-2:17.2.7-915.g330067db.el8.x86_64 1/1 2024-04-24T15:30:58.177 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:30:58.177 INFO:teuthology.orchestra.run.smithi151.stdout:Installed: 2024-04-24T15:30:58.178 INFO:teuthology.orchestra.run.smithi151.stdout: rbd-mirror-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T15:30:58.178 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:30:58.178 INFO:teuthology.orchestra.run.smithi151.stdout:Complete! 2024-04-24T15:30:58.452 DEBUG:teuthology.orchestra.run.smithi151:> sudo yum -y install rbd-nbd 2024-04-24T15:30:58.767 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:03:11 ago on Wed 24 Apr 2024 03:27:47 PM UTC. 2024-04-24T15:30:59.228 INFO:teuthology.orchestra.run.smithi151.stdout:Last metadata expiration check: 0:03:09 ago on Wed 24 Apr 2024 03:27:50 PM UTC. 2024-04-24T15:31:00.412 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-04-24T15:31:00.412 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-04-24T15:31:00.413 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repo Size 2024-04-24T15:31:00.413 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-04-24T15:31:00.413 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-04-24T15:31:00.413 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-mirror x86_64 2:17.2.7-915.g330067db.el8 ceph 3.5 M 2024-04-24T15:31:00.413 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:31:00.413 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-04-24T15:31:00.413 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-04-24T15:31:00.413 INFO:teuthology.orchestra.run.smithi096.stdout:Install 1 Package 2024-04-24T15:31:00.413 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:31:00.413 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 3.5 M 2024-04-24T15:31:00.413 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 16 M 2024-04-24T15:31:00.413 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-04-24T15:31:00.882 INFO:teuthology.orchestra.run.smithi151.stdout:Dependencies resolved. 2024-04-24T15:31:00.883 INFO:teuthology.orchestra.run.smithi151.stdout:================================================================================ 2024-04-24T15:31:00.883 INFO:teuthology.orchestra.run.smithi151.stdout: Package Architecture Version Repository Size 2024-04-24T15:31:00.883 INFO:teuthology.orchestra.run.smithi151.stdout:================================================================================ 2024-04-24T15:31:00.883 INFO:teuthology.orchestra.run.smithi151.stdout:Installing: 2024-04-24T15:31:00.883 INFO:teuthology.orchestra.run.smithi151.stdout: rbd-nbd x86_64 2:17.2.7-915.g330067db.el8 ceph 175 k 2024-04-24T15:31:00.883 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:31:00.883 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction Summary 2024-04-24T15:31:00.883 INFO:teuthology.orchestra.run.smithi151.stdout:================================================================================ 2024-04-24T15:31:00.883 INFO:teuthology.orchestra.run.smithi151.stdout:Install 1 Package 2024-04-24T15:31:00.883 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:31:00.884 INFO:teuthology.orchestra.run.smithi151.stdout:Total download size: 175 k 2024-04-24T15:31:00.884 INFO:teuthology.orchestra.run.smithi151.stdout:Installed size: 556 k 2024-04-24T15:31:00.884 INFO:teuthology.orchestra.run.smithi151.stdout:Downloading Packages: 2024-04-24T15:31:01.044 INFO:teuthology.orchestra.run.smithi096.stdout:rbd-mirror-17.2.7-915.g330067db.el8.x86_64.rpm 5.5 MB/s | 3.5 MB 00:00 2024-04-24T15:31:01.045 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-04-24T15:31:01.045 INFO:teuthology.orchestra.run.smithi096.stdout:Total 5.5 MB/s | 3.5 MB 00:00 2024-04-24T15:31:01.046 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-04-24T15:31:01.057 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-04-24T15:31:01.057 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-04-24T15:31:01.119 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-04-24T15:31:01.122 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-04-24T15:31:01.196 INFO:teuthology.orchestra.run.smithi151.stdout:rbd-nbd-17.2.7-915.g330067db.el8.x86_64.rpm 560 kB/s | 175 kB 00:00 2024-04-24T15:31:01.197 INFO:teuthology.orchestra.run.smithi151.stdout:-------------------------------------------------------------------------------- 2024-04-24T15:31:01.197 INFO:teuthology.orchestra.run.smithi151.stdout:Total 557 kB/s | 175 kB 00:00 2024-04-24T15:31:01.197 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction check 2024-04-24T15:31:01.209 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction check succeeded. 2024-04-24T15:31:01.209 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction test 2024-04-24T15:31:01.258 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction test succeeded. 2024-04-24T15:31:01.261 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction 2024-04-24T15:31:01.395 INFO:teuthology.orchestra.run.smithi151.stdout: Preparing : 1/1 2024-04-24T15:31:01.555 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-04-24T15:31:01.609 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : rbd-nbd-2:17.2.7-915.g330067db.el8.x86_64 1/1 2024-04-24T15:31:01.745 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : rbd-mirror-2:17.2.7-915.g330067db.el8.x86_64 1/1 2024-04-24T15:31:01.887 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: rbd-nbd-2:17.2.7-915.g330067db.el8.x86_64 1/1 2024-04-24T15:31:02.146 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: rbd-mirror-2:17.2.7-915.g330067db.el8.x86_64 1/1 2024-04-24T15:31:02.332 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : rbd-nbd-2:17.2.7-915.g330067db.el8.x86_64 1/1 2024-04-24T15:31:02.332 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:31:02.332 INFO:teuthology.orchestra.run.smithi151.stdout:Installed: 2024-04-24T15:31:02.333 INFO:teuthology.orchestra.run.smithi151.stdout: rbd-nbd-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T15:31:02.333 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:31:02.333 INFO:teuthology.orchestra.run.smithi151.stdout:Complete! 2024-04-24T15:31:02.521 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : rbd-mirror-2:17.2.7-915.g330067db.el8.x86_64 1/1 2024-04-24T15:31:02.521 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:31:02.521 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-04-24T15:31:02.521 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-mirror-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T15:31:02.521 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:31:02.521 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-04-24T15:31:02.612 DEBUG:teuthology.parallel:result is None 2024-04-24T15:31:02.755 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install rbd-nbd 2024-04-24T15:31:03.516 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:03:16 ago on Wed 24 Apr 2024 03:27:47 PM UTC. 2024-04-24T15:31:05.142 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-04-24T15:31:05.143 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-04-24T15:31:05.143 INFO:teuthology.orchestra.run.smithi096.stdout: Package Architecture Version Repository Size 2024-04-24T15:31:05.143 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-04-24T15:31:05.144 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-04-24T15:31:05.144 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-nbd x86_64 2:17.2.7-915.g330067db.el8 ceph 175 k 2024-04-24T15:31:05.144 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:31:05.144 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-04-24T15:31:05.144 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-04-24T15:31:05.144 INFO:teuthology.orchestra.run.smithi096.stdout:Install 1 Package 2024-04-24T15:31:05.144 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:31:05.144 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 175 k 2024-04-24T15:31:05.144 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 556 k 2024-04-24T15:31:05.144 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-04-24T15:31:05.473 INFO:teuthology.orchestra.run.smithi096.stdout:rbd-nbd-17.2.7-915.g330067db.el8.x86_64.rpm 533 kB/s | 175 kB 00:00 2024-04-24T15:31:05.473 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-04-24T15:31:05.473 INFO:teuthology.orchestra.run.smithi096.stdout:Total 530 kB/s | 175 kB 00:00 2024-04-24T15:31:05.473 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-04-24T15:31:05.484 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-04-24T15:31:05.484 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-04-24T15:31:05.532 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-04-24T15:31:05.535 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-04-24T15:31:05.671 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-04-24T15:31:05.910 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : rbd-nbd-2:17.2.7-915.g330067db.el8.x86_64 1/1 2024-04-24T15:31:06.182 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: rbd-nbd-2:17.2.7-915.g330067db.el8.x86_64 1/1 2024-04-24T15:31:06.537 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : rbd-nbd-2:17.2.7-915.g330067db.el8.x86_64 1/1 2024-04-24T15:31:06.537 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:31:06.538 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-04-24T15:31:06.538 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-nbd-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T15:31:06.538 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:31:06.538 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-04-24T15:31:06.752 DEBUG:teuthology.parallel:result is None 2024-04-24T15:31:06.753 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=330067dbce47655d1f762ef14d546d2fde16a6a9 2024-04-24T15:31:06.975 DEBUG:teuthology.orchestra.run.smithi096:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-24T15:31:06.998 INFO:teuthology.orchestra.run.smithi096.stdout:17.2.7-915.g330067db.el8 2024-04-24T15:31:06.999 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-915.g330067db.el8 2024-04-24T15:31:06.999 INFO:teuthology.task.install:The correct ceph version 17.2.7-915.g330067db is installed. 2024-04-24T15:31:07.000 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=330067dbce47655d1f762ef14d546d2fde16a6a9 2024-04-24T15:31:07.145 DEBUG:teuthology.orchestra.run.smithi151:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-24T15:31:07.173 INFO:teuthology.orchestra.run.smithi151.stdout:17.2.7-915.g330067db.el8 2024-04-24T15:31:07.173 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-915.g330067db.el8 2024-04-24T15:31:07.173 INFO:teuthology.task.install:The correct ceph version 17.2.7-915.g330067db is installed. 2024-04-24T15:31:07.175 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-24T15:31:07.175 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-24T15:31:07.175 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-24T15:31:07.203 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-24T15:31:07.203 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-24T15:31:07.247 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-24T15:31:07.247 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-24T15:31:07.247 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/usr/bin/daemon-helper 2024-04-24T15:31:07.273 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-24T15:31:07.339 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-24T15:31:07.339 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/usr/bin/daemon-helper 2024-04-24T15:31:07.369 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-24T15:31:07.439 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-24T15:31:07.440 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-24T15:31:07.440 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-24T15:31:07.466 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-24T15:31:07.532 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-24T15:31:07.532 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-24T15:31:07.561 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-24T15:31:07.630 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-24T15:31:07.630 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-24T15:31:07.630 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/usr/bin/stdin-killer 2024-04-24T15:31:07.656 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-24T15:31:07.722 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-24T15:31:07.722 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/usr/bin/stdin-killer 2024-04-24T15:31:07.750 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-24T15:31:07.820 INFO:teuthology.run_tasks:Running task ceph... 2024-04-24T15:31:07.900 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-24T15:31:07.900 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 777 /var/log/ceph 2024-04-24T15:31:07.902 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod 777 /var/log/ceph 2024-04-24T15:31:07.931 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-24T15:31:07.931 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-24T15:31:07.969 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-24T15:31:07.998 INFO:tasks.ceph:Creating extra log directories... 2024-04-24T15:31:07.999 DEBUG:teuthology.orchestra.run.smithi096:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-24T15:31:08.035 DEBUG:teuthology.orchestra.run.smithi151:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-24T15:31:08.075 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-24T15:31:08.075 INFO:tasks.ceph:config {'conf': {'client': {'rbd qos iops burst': 100, 'rbd qos iops limit': 50, 'rbd qos schedule tick min': 100}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(CACHE_POOL_NO_HIT_SET\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-24T15:31:08.075 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2024-04-24_14:09:24-rbd-wip-yuriw4-testing-20240423.151325-quincy-distro-default-smithi/7671570', 'branch': 'wip-yuriw4-testing-20240423.151325-quincy', 'description': 'rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zstd supported-random-distro$/{centos_8} thrashers/cache thrashosds-health workloads/rbd_fsx_rate_limit}', 'email': 'yweinste@redhat.com', 'first_in_suite': False, 'job_id': '7671570', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'yuriw-2024-04-24_14:09:24-rbd-wip-yuriw4-testing-20240423.151325-quincy-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'machine': {'cpus': 1, 'disk': 40, 'ram': 8000}, 'volumes': {'count': 4, 'size': 30}}], 'os_type': 'centos', 'os_version': '8.stream', 'overrides': {'admin_socket': {'branch': 'wip-yuriw4-testing-20240423.151325-quincy'}, 'ceph': {'conf': {'client': {'rbd qos iops burst': 100, 'rbd qos iops limit': 50, 'rbd qos schedule tick min': 100}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(CACHE_POOL_NO_HIT_SET\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request'], 'sha1': '330067dbce47655d1f762ef14d546d2fde16a6a9'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '330067dbce47655d1f762ef14d546d2fde16a6a9'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuriw4-testing-20240423.151325-quincy', 'sha1': '330067dbce47655d1f762ef14d546d2fde16a6a9'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 99, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 9954, 'sha1': '330067dbce47655d1f762ef14d546d2fde16a6a9', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'rbd', 'suite_branch': 'wip-yuriw4-testing-20240423.151325-quincy', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_330067dbce47655d1f762ef14d546d2fde16a6a9/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '330067dbce47655d1f762ef14d546d2fde16a6a9', 'targets': {'smithi096.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFy0JYDV0SvTWoBphPv+zUKpuktGpfetG3mJ5AU80JookSLWtK1UsnuAmBiY/OxjCLA1Xo55UDJufJWJ59xjtDI=', 'smithi151.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL+p3hm6p4IoLnIjP8Kf65mobZFShOZg/nGai6no81y6bfOQ4/2atdoMISqXDp7VqOHXVLFJWSPm4fVtLzxIZSA='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'exec': {'client.0': ['sudo ceph osd pool create cache 4', 'sudo ceph osd tier add rbd cache', 'sudo ceph osd tier cache-mode cache writeback', 'sudo ceph osd tier set-overlay rbd cache', 'sudo ceph osd pool set cache hit_set_type bloom', 'sudo ceph osd pool set cache hit_set_count 8', 'sudo ceph osd pool set cache hit_set_period 60', 'sudo ceph osd pool set cache target_max_objects 250']}}, {'thrashosds': {'timeout': 1200}}, {'rbd_fsx': {'clients': ['client.0'], 'ops': 6000}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-24_14:09:24', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-24T15:31:08.076 DEBUG:teuthology.orchestra.run.smithi096:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-24T15:31:08.102 DEBUG:teuthology.orchestra.run.smithi151:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-24T15:31:08.123 DEBUG:teuthology.orchestra.run.smithi096:> sudo install -d -m0777 -- /var/run/ceph 2024-04-24T15:31:08.161 DEBUG:teuthology.orchestra.run.smithi151:> sudo install -d -m0777 -- /var/run/ceph 2024-04-24T15:31:08.191 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-24T15:31:08.191 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/scratch_devs of=/dev/stdout 2024-04-24T15:31:08.243 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-24T15:31:08.243 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_1 2024-04-24T15:31:08.301 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-24T15:31:08.301 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T15:31:08.301 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 6h/6d Inode: 165484 Links: 1 2024-04-24T15:31:08.301 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T15:31:08.301 INFO:teuthology.orchestra.run.smithi096.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T15:31:08.301 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-04-24 15:31:01.906200254 +0000 2024-04-24T15:31:08.301 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-04-24 15:26:50.579223278 +0000 2024-04-24T15:31:08.301 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-04-24 15:26:50.579223278 +0000 2024-04-24T15:31:08.301 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: - 2024-04-24T15:31:08.301 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-24T15:31:08.366 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-04-24T15:31:08.366 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-04-24T15:31:08.367 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000593262 s, 863 kB/s 2024-04-24T15:31:08.367 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-24T15:31:08.427 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_2 2024-04-24T15:31:08.484 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-24T15:31:08.484 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T15:31:08.484 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 6h/6d Inode: 165550 Links: 1 2024-04-24T15:31:08.484 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T15:31:08.484 INFO:teuthology.orchestra.run.smithi096.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T15:31:08.484 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-04-24 15:31:01.906200254 +0000 2024-04-24T15:31:08.484 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-04-24 15:26:50.917215784 +0000 2024-04-24T15:31:08.484 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-04-24 15:26:50.917215784 +0000 2024-04-24T15:31:08.485 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: - 2024-04-24T15:31:08.485 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-24T15:31:08.549 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-04-24T15:31:08.549 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-04-24T15:31:08.549 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000215466 s, 2.4 MB/s 2024-04-24T15:31:08.550 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-24T15:31:08.608 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_3 2024-04-24T15:31:08.666 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-24T15:31:08.667 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T15:31:08.667 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 6h/6d Inode: 165644 Links: 1 2024-04-24T15:31:08.667 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T15:31:08.667 INFO:teuthology.orchestra.run.smithi096.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T15:31:08.667 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-04-24 15:31:01.906200254 +0000 2024-04-24T15:31:08.667 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-04-24 15:26:51.266208046 +0000 2024-04-24T15:31:08.667 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-04-24 15:26:51.266208046 +0000 2024-04-24T15:31:08.667 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: - 2024-04-24T15:31:08.667 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-24T15:31:08.732 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-04-24T15:31:08.732 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-04-24T15:31:08.732 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000165895 s, 3.1 MB/s 2024-04-24T15:31:08.733 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-24T15:31:08.791 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_4 2024-04-24T15:31:08.848 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-24T15:31:08.848 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T15:31:08.848 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 6h/6d Inode: 169287 Links: 1 2024-04-24T15:31:08.849 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T15:31:08.849 INFO:teuthology.orchestra.run.smithi096.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T15:31:08.849 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-04-24 15:31:01.906200254 +0000 2024-04-24T15:31:08.849 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-04-24 15:26:51.596200729 +0000 2024-04-24T15:31:08.849 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-04-24 15:26:51.596200729 +0000 2024-04-24T15:31:08.849 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: - 2024-04-24T15:31:08.849 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-24T15:31:08.914 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-04-24T15:31:08.914 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-04-24T15:31:08.914 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000234556 s, 2.2 MB/s 2024-04-24T15:31:08.915 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-24T15:31:08.973 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-24T15:31:08.974 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-24T15:31:08.974 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/scratch_devs of=/dev/stdout 2024-04-24T15:31:08.995 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-24T15:31:08.996 DEBUG:teuthology.orchestra.run.smithi151:> stat /dev/vg_nvme/lv_1 2024-04-24T15:31:09.057 INFO:teuthology.orchestra.run.smithi151.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-24T15:31:09.057 INFO:teuthology.orchestra.run.smithi151.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T15:31:09.057 INFO:teuthology.orchestra.run.smithi151.stdout:Device: 6h/6d Inode: 169209 Links: 1 2024-04-24T15:31:09.057 INFO:teuthology.orchestra.run.smithi151.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T15:31:09.057 INFO:teuthology.orchestra.run.smithi151.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T15:31:09.057 INFO:teuthology.orchestra.run.smithi151.stdout:Access: 2024-04-24 15:30:57.479258891 +0000 2024-04-24T15:31:09.057 INFO:teuthology.orchestra.run.smithi151.stdout:Modify: 2024-04-24 15:26:54.586004634 +0000 2024-04-24T15:31:09.057 INFO:teuthology.orchestra.run.smithi151.stdout:Change: 2024-04-24 15:26:54.586004634 +0000 2024-04-24T15:31:09.058 INFO:teuthology.orchestra.run.smithi151.stdout: Birth: - 2024-04-24T15:31:09.058 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-24T15:31:09.127 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records in 2024-04-24T15:31:09.127 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records out 2024-04-24T15:31:09.127 INFO:teuthology.orchestra.run.smithi151.stderr:512 bytes copied, 0.000217764 s, 2.4 MB/s 2024-04-24T15:31:09.128 DEBUG:teuthology.orchestra.run.smithi151:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-24T15:31:09.190 DEBUG:teuthology.orchestra.run.smithi151:> stat /dev/vg_nvme/lv_2 2024-04-24T15:31:09.249 INFO:teuthology.orchestra.run.smithi151.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-24T15:31:09.249 INFO:teuthology.orchestra.run.smithi151.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T15:31:09.250 INFO:teuthology.orchestra.run.smithi151.stdout:Device: 6h/6d Inode: 167525 Links: 1 2024-04-24T15:31:09.250 INFO:teuthology.orchestra.run.smithi151.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T15:31:09.250 INFO:teuthology.orchestra.run.smithi151.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T15:31:09.250 INFO:teuthology.orchestra.run.smithi151.stdout:Access: 2024-04-24 15:30:57.479258891 +0000 2024-04-24T15:31:09.250 INFO:teuthology.orchestra.run.smithi151.stdout:Modify: 2024-04-24 15:26:54.947000385 +0000 2024-04-24T15:31:09.250 INFO:teuthology.orchestra.run.smithi151.stdout:Change: 2024-04-24 15:26:54.947000385 +0000 2024-04-24T15:31:09.250 INFO:teuthology.orchestra.run.smithi151.stdout: Birth: - 2024-04-24T15:31:09.250 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-24T15:31:09.317 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records in 2024-04-24T15:31:09.317 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records out 2024-04-24T15:31:09.317 INFO:teuthology.orchestra.run.smithi151.stderr:512 bytes copied, 0.000222367 s, 2.3 MB/s 2024-04-24T15:31:09.318 DEBUG:teuthology.orchestra.run.smithi151:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-24T15:31:09.378 DEBUG:teuthology.orchestra.run.smithi151:> stat /dev/vg_nvme/lv_3 2024-04-24T15:31:09.437 INFO:teuthology.orchestra.run.smithi151.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-24T15:31:09.437 INFO:teuthology.orchestra.run.smithi151.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T15:31:09.437 INFO:teuthology.orchestra.run.smithi151.stdout:Device: 6h/6d Inode: 160750 Links: 1 2024-04-24T15:31:09.437 INFO:teuthology.orchestra.run.smithi151.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T15:31:09.437 INFO:teuthology.orchestra.run.smithi151.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T15:31:09.437 INFO:teuthology.orchestra.run.smithi151.stdout:Access: 2024-04-24 15:30:57.479258891 +0000 2024-04-24T15:31:09.437 INFO:teuthology.orchestra.run.smithi151.stdout:Modify: 2024-04-24 15:26:55.280996453 +0000 2024-04-24T15:31:09.438 INFO:teuthology.orchestra.run.smithi151.stdout:Change: 2024-04-24 15:26:55.280996453 +0000 2024-04-24T15:31:09.438 INFO:teuthology.orchestra.run.smithi151.stdout: Birth: - 2024-04-24T15:31:09.438 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-24T15:31:09.503 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records in 2024-04-24T15:31:09.503 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records out 2024-04-24T15:31:09.503 INFO:teuthology.orchestra.run.smithi151.stderr:512 bytes copied, 0.000260397 s, 2.0 MB/s 2024-04-24T15:31:09.505 DEBUG:teuthology.orchestra.run.smithi151:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-24T15:31:09.563 DEBUG:teuthology.orchestra.run.smithi151:> stat /dev/vg_nvme/lv_4 2024-04-24T15:31:09.621 INFO:teuthology.orchestra.run.smithi151.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-24T15:31:09.622 INFO:teuthology.orchestra.run.smithi151.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T15:31:09.622 INFO:teuthology.orchestra.run.smithi151.stdout:Device: 6h/6d Inode: 169379 Links: 1 2024-04-24T15:31:09.622 INFO:teuthology.orchestra.run.smithi151.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T15:31:09.622 INFO:teuthology.orchestra.run.smithi151.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T15:31:09.622 INFO:teuthology.orchestra.run.smithi151.stdout:Access: 2024-04-24 15:30:57.479258891 +0000 2024-04-24T15:31:09.622 INFO:teuthology.orchestra.run.smithi151.stdout:Modify: 2024-04-24 15:26:55.613992533 +0000 2024-04-24T15:31:09.622 INFO:teuthology.orchestra.run.smithi151.stdout:Change: 2024-04-24 15:26:55.613992533 +0000 2024-04-24T15:31:09.622 INFO:teuthology.orchestra.run.smithi151.stdout: Birth: - 2024-04-24T15:31:09.622 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-24T15:31:09.687 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records in 2024-04-24T15:31:09.687 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records out 2024-04-24T15:31:09.687 INFO:teuthology.orchestra.run.smithi151.stderr:512 bytes copied, 0.000232436 s, 2.2 MB/s 2024-04-24T15:31:09.688 DEBUG:teuthology.orchestra.run.smithi151:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-24T15:31:09.748 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-24T15:31:09.748 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi096.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi151.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-24T15:31:09.748 INFO:tasks.ceph:Generating config... 2024-04-24T15:31:09.749 INFO:tasks.ceph:[client] rbd qos iops burst = 100 2024-04-24T15:31:09.749 INFO:tasks.ceph:[client] rbd qos iops limit = 50 2024-04-24T15:31:09.749 INFO:tasks.ceph:[client] rbd qos schedule tick min = 100 2024-04-24T15:31:09.749 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-04-24T15:31:09.749 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-04-24T15:31:09.749 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-04-24T15:31:09.749 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-24T15:31:09.750 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-24T15:31:09.750 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-24T15:31:09.750 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-24T15:31:09.750 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-24T15:31:09.750 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-24T15:31:09.750 INFO:tasks.ceph:[osd] bluestore compression algorithm = zstd 2024-04-24T15:31:09.750 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2024-04-24T15:31:09.750 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-24T15:31:09.750 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-04-24T15:31:09.750 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-04-24T15:31:09.750 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-24T15:31:09.750 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-24T15:31:09.750 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-04-24T15:31:09.750 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-24T15:31:09.751 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-24T15:31:09.751 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-24T15:31:09.751 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-24T15:31:09.751 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-04-24T15:31:09.751 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-24T15:31:09.751 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-04-24T15:31:09.751 INFO:tasks.ceph:Setting up mon.a... 2024-04-24T15:31:09.751 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-24T15:31:09.804 INFO:teuthology.orchestra.run.smithi096.stdout:creating /etc/ceph/ceph.keyring 2024-04-24T15:31:09.806 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-24T15:31:09.850 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-24T15:31:09.917 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.96'), ('mon.c', '[v2:172.21.15.96:3301,v1:172.21.15.96:6790]'), ('mon.b', '172.21.15.151')] 2024-04-24T15:31:09.917 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.96,[v2:172.21.15.96:3301,v1:172.21.15.96:6790],172.21.15.151', 'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'rbd qos iops burst': 100, 'rbd qos iops limit': 50, 'rbd qos schedule tick min': 100}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-04-24T15:31:09.918 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-24T15:31:09.918 DEBUG:teuthology.orchestra.run.smithi096:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-24T15:31:09.975 DEBUG:teuthology.orchestra.run.smithi096:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.96 --addv c '[v2:172.21.15.96:3301,v1:172.21.15.96:6790]' --add b 172.21.15.151 --print /home/ubuntu/cephtest/ceph.monmap 2024-04-24T15:31:10.062 INFO:teuthology.orchestra.run.smithi096.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-24T15:31:10.062 INFO:teuthology.orchestra.run.smithi096.stdout:monmaptool: generated fsid 253d426c-62cb-4194-8514-6cd970fd85c7 2024-04-24T15:31:10.062 INFO:teuthology.orchestra.run.smithi096.stdout:setting min_mon_release = octopus 2024-04-24T15:31:10.063 INFO:teuthology.orchestra.run.smithi096.stdout:epoch 0 2024-04-24T15:31:10.063 INFO:teuthology.orchestra.run.smithi096.stdout:fsid 253d426c-62cb-4194-8514-6cd970fd85c7 2024-04-24T15:31:10.063 INFO:teuthology.orchestra.run.smithi096.stdout:last_changed 2024-04-24T15:31:10.063756+0000 2024-04-24T15:31:10.063 INFO:teuthology.orchestra.run.smithi096.stdout:created 2024-04-24T15:31:10.063756+0000 2024-04-24T15:31:10.063 INFO:teuthology.orchestra.run.smithi096.stdout:min_mon_release 15 (octopus) 2024-04-24T15:31:10.063 INFO:teuthology.orchestra.run.smithi096.stdout:election_strategy: 1 2024-04-24T15:31:10.063 INFO:teuthology.orchestra.run.smithi096.stdout:0: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.a 2024-04-24T15:31:10.063 INFO:teuthology.orchestra.run.smithi096.stdout:1: [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon.b 2024-04-24T15:31:10.063 INFO:teuthology.orchestra.run.smithi096.stdout:2: [v2:172.21.15.96:3301/0,v1:172.21.15.96:6790/0] mon.c 2024-04-24T15:31:10.063 INFO:teuthology.orchestra.run.smithi096.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-24T15:31:10.064 DEBUG:teuthology.orchestra.run.smithi096:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-24T15:31:10.081 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 253d426c-62cb-4194-8514-6cd970fd85c7... 2024-04-24T15:31:10.082 DEBUG:teuthology.orchestra.run.smithi096:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-24T15:31:10.124 DEBUG:teuthology.orchestra.run.smithi151:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-24T15:31:10.165 INFO:teuthology.orchestra.run.smithi096.stdout:[global] 2024-04-24T15:31:10.166 INFO:teuthology.orchestra.run.smithi096.stdout: chdir = "" 2024-04-24T15:31:10.166 INFO:teuthology.orchestra.run.smithi096.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-24T15:31:10.166 INFO:teuthology.orchestra.run.smithi096.stdout: auth supported = cephx 2024-04-24T15:31:10.166 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:31:10.166 INFO:teuthology.orchestra.run.smithi096.stdout: filestore xattr use omap = true 2024-04-24T15:31:10.166 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:31:10.166 INFO:teuthology.orchestra.run.smithi096.stdout: mon clock drift allowed = 1.000 2024-04-24T15:31:10.166 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:31:10.166 INFO:teuthology.orchestra.run.smithi096.stdout: osd crush chooseleaf type = 0 2024-04-24T15:31:10.166 INFO:teuthology.orchestra.run.smithi096.stdout: auth debug = true 2024-04-24T15:31:10.166 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:31:10.166 INFO:teuthology.orchestra.run.smithi096.stdout: ms die on old message = true 2024-04-24T15:31:10.166 INFO:teuthology.orchestra.run.smithi096.stdout: ms die on bug = true 2024-04-24T15:31:10.166 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:31:10.167 INFO:teuthology.orchestra.run.smithi096.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-24T15:31:10.167 INFO:teuthology.orchestra.run.smithi096.stdout: mon pg warn max object skew = 0 2024-04-24T15:31:10.167 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:31:10.167 INFO:teuthology.orchestra.run.smithi096.stdout: # disable pg_autoscaler by default for new pools 2024-04-24T15:31:10.167 INFO:teuthology.orchestra.run.smithi096.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-24T15:31:10.167 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:31:10.167 INFO:teuthology.orchestra.run.smithi096.stdout: osd pool default size = 2 2024-04-24T15:31:10.167 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:31:10.167 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd allow primary affinity = true 2024-04-24T15:31:10.167 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd allow pg remap = true 2024-04-24T15:31:10.167 INFO:teuthology.orchestra.run.smithi096.stdout: mon warn on legacy crush tunables = false 2024-04-24T15:31:10.167 INFO:teuthology.orchestra.run.smithi096.stdout: mon warn on crush straw calc version zero = false 2024-04-24T15:31:10.167 INFO:teuthology.orchestra.run.smithi096.stdout: mon warn on no sortbitwise = false 2024-04-24T15:31:10.167 INFO:teuthology.orchestra.run.smithi096.stdout: mon warn on osd down out interval zero = false 2024-04-24T15:31:10.167 INFO:teuthology.orchestra.run.smithi096.stdout: mon warn on too few osds = false 2024-04-24T15:31:10.167 INFO:teuthology.orchestra.run.smithi096.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-24T15:31:10.168 INFO:teuthology.orchestra.run.smithi096.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-24T15:31:10.168 INFO:teuthology.orchestra.run.smithi096.stdout: mon_allow_pool_size_one = true 2024-04-24T15:31:10.168 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:31:10.168 INFO:teuthology.orchestra.run.smithi096.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-24T15:31:10.168 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:31:10.168 INFO:teuthology.orchestra.run.smithi096.stdout: osd default data pool replay window = 5 2024-04-24T15:31:10.168 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:31:10.168 INFO:teuthology.orchestra.run.smithi096.stdout: mon allow pool delete = true 2024-04-24T15:31:10.168 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:31:10.168 INFO:teuthology.orchestra.run.smithi096.stdout: mon cluster log file level = debug 2024-04-24T15:31:10.168 INFO:teuthology.orchestra.run.smithi096.stdout: debug asserts on shutdown = true 2024-04-24T15:31:10.168 INFO:teuthology.orchestra.run.smithi096.stdout: mon health detail to clog = false 2024-04-24T15:31:10.168 INFO:teuthology.orchestra.run.smithi096.stdout: mon host = "172.21.15.96,[v2:172.21.15.96:3301,v1:172.21.15.96:6790],172.21.15.151" 2024-04-24T15:31:10.168 INFO:teuthology.orchestra.run.smithi096.stdout: mon client directed command retry = 5 2024-04-24T15:31:10.168 INFO:teuthology.orchestra.run.smithi096.stdout: mon warn on pool no app = False 2024-04-24T15:31:10.168 INFO:teuthology.orchestra.run.smithi096.stdout: ms inject socket failures = 5000 2024-04-24T15:31:10.169 INFO:teuthology.orchestra.run.smithi096.stdout: fsid = 253d426c-62cb-4194-8514-6cd970fd85c7 2024-04-24T15:31:10.169 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:31:10.169 INFO:teuthology.orchestra.run.smithi096.stdout:[osd] 2024-04-24T15:31:10.169 INFO:teuthology.orchestra.run.smithi096.stdout: osd journal size = 100 2024-04-24T15:31:10.169 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:31:10.169 INFO:teuthology.orchestra.run.smithi096.stdout: osd scrub load threshold = 5.0 2024-04-24T15:31:10.169 INFO:teuthology.orchestra.run.smithi096.stdout: osd scrub max interval = 600 2024-04-24T15:31:10.169 INFO:teuthology.orchestra.run.smithi096.stdout: osd mclock profile = high_recovery_ops 2024-04-24T15:31:10.169 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:31:10.169 INFO:teuthology.orchestra.run.smithi096.stdout: osd recover clone overlap = true 2024-04-24T15:31:10.169 INFO:teuthology.orchestra.run.smithi096.stdout: osd recovery max chunk = 1048576 2024-04-24T15:31:10.169 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:31:10.169 INFO:teuthology.orchestra.run.smithi096.stdout: osd debug shutdown = true 2024-04-24T15:31:10.169 INFO:teuthology.orchestra.run.smithi096.stdout: osd debug op order = true 2024-04-24T15:31:10.169 INFO:teuthology.orchestra.run.smithi096.stdout: osd debug verify stray on activate = true 2024-04-24T15:31:10.169 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:31:10.170 INFO:teuthology.orchestra.run.smithi096.stdout: osd open classes on start = true 2024-04-24T15:31:10.170 INFO:teuthology.orchestra.run.smithi096.stdout: osd debug pg log writeout = true 2024-04-24T15:31:10.170 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:31:10.170 INFO:teuthology.orchestra.run.smithi096.stdout: osd deep scrub update digest min age = 30 2024-04-24T15:31:10.170 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:31:10.170 INFO:teuthology.orchestra.run.smithi096.stdout: osd map max advance = 10 2024-04-24T15:31:10.170 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:31:10.170 INFO:teuthology.orchestra.run.smithi096.stdout: journal zero on create = true 2024-04-24T15:31:10.170 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:31:10.170 INFO:teuthology.orchestra.run.smithi096.stdout: filestore ondisk finisher threads = 3 2024-04-24T15:31:10.170 INFO:teuthology.orchestra.run.smithi096.stdout: filestore apply finisher threads = 3 2024-04-24T15:31:10.170 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:31:10.170 INFO:teuthology.orchestra.run.smithi096.stdout: bdev debug aio = true 2024-04-24T15:31:10.170 INFO:teuthology.orchestra.run.smithi096.stdout: osd debug misdirected ops = true 2024-04-24T15:31:10.170 INFO:teuthology.orchestra.run.smithi096.stdout: bluestore block size = 96636764160 2024-04-24T15:31:10.170 INFO:teuthology.orchestra.run.smithi096.stdout: bluestore compression algorithm = zstd 2024-04-24T15:31:10.171 INFO:teuthology.orchestra.run.smithi096.stdout: bluestore compression mode = aggressive 2024-04-24T15:31:10.171 INFO:teuthology.orchestra.run.smithi096.stdout: bluestore fsck on mount = True 2024-04-24T15:31:10.171 INFO:teuthology.orchestra.run.smithi096.stdout: debug bluefs = 1/20 2024-04-24T15:31:10.171 INFO:teuthology.orchestra.run.smithi096.stdout: debug bluestore = 1/20 2024-04-24T15:31:10.171 INFO:teuthology.orchestra.run.smithi096.stdout: debug ms = 1 2024-04-24T15:31:10.171 INFO:teuthology.orchestra.run.smithi096.stdout: debug osd = 20 2024-04-24T15:31:10.171 INFO:teuthology.orchestra.run.smithi096.stdout: debug rocksdb = 4/10 2024-04-24T15:31:10.171 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd backfillfull_ratio = 0.85 2024-04-24T15:31:10.171 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd full ratio = 0.9 2024-04-24T15:31:10.171 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd nearfull ratio = 0.8 2024-04-24T15:31:10.171 INFO:teuthology.orchestra.run.smithi096.stdout: osd failsafe full ratio = 0.95 2024-04-24T15:31:10.171 INFO:teuthology.orchestra.run.smithi096.stdout: osd max markdown count = 1000 2024-04-24T15:31:10.171 INFO:teuthology.orchestra.run.smithi096.stdout: osd objectstore = bluestore 2024-04-24T15:31:10.171 INFO:teuthology.orchestra.run.smithi096.stdout: osd shutdown pgref assert = True 2024-04-24T15:31:10.171 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:31:10.172 INFO:teuthology.orchestra.run.smithi096.stdout:[mgr] 2024-04-24T15:31:10.172 INFO:teuthology.orchestra.run.smithi096.stdout: debug ms = 1 2024-04-24T15:31:10.172 INFO:teuthology.orchestra.run.smithi096.stdout: debug mgr = 20 2024-04-24T15:31:10.172 INFO:teuthology.orchestra.run.smithi096.stdout: debug mon = 20 2024-04-24T15:31:10.172 INFO:teuthology.orchestra.run.smithi096.stdout: debug auth = 20 2024-04-24T15:31:10.172 INFO:teuthology.orchestra.run.smithi096.stdout: mon reweight min pgs per osd = 4 2024-04-24T15:31:10.172 INFO:teuthology.orchestra.run.smithi096.stdout: mon reweight min bytes per osd = 10 2024-04-24T15:31:10.172 INFO:teuthology.orchestra.run.smithi096.stdout: mgr/telemetry/nag = false 2024-04-24T15:31:10.172 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:31:10.172 INFO:teuthology.orchestra.run.smithi096.stdout:[mon] 2024-04-24T15:31:10.172 INFO:teuthology.orchestra.run.smithi096.stdout: debug ms = 1 2024-04-24T15:31:10.172 INFO:teuthology.orchestra.run.smithi096.stdout: debug mon = 20 2024-04-24T15:31:10.172 INFO:teuthology.orchestra.run.smithi096.stdout: debug paxos = 20 2024-04-24T15:31:10.172 INFO:teuthology.orchestra.run.smithi096.stdout: debug auth = 20 2024-04-24T15:31:10.172 INFO:teuthology.orchestra.run.smithi096.stdout: mon data avail warn = 5 2024-04-24T15:31:10.172 INFO:teuthology.orchestra.run.smithi096.stdout: mon mgr mkfs grace = 240 2024-04-24T15:31:10.172 INFO:teuthology.orchestra.run.smithi096.stdout: mon reweight min pgs per osd = 4 2024-04-24T15:31:10.173 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd reporter subtree level = osd 2024-04-24T15:31:10.173 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd prime pg temp = true 2024-04-24T15:31:10.173 INFO:teuthology.orchestra.run.smithi096.stdout: mon reweight min bytes per osd = 10 2024-04-24T15:31:10.173 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:31:10.173 INFO:teuthology.orchestra.run.smithi096.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-24T15:31:10.173 INFO:teuthology.orchestra.run.smithi096.stdout: auth mon ticket ttl = 660 # 11m 2024-04-24T15:31:10.173 INFO:teuthology.orchestra.run.smithi096.stdout: auth service ticket ttl = 240 # 4m 2024-04-24T15:31:10.173 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:31:10.173 INFO:teuthology.orchestra.run.smithi096.stdout: # don't complain about insecure global_id in the test suite 2024-04-24T15:31:10.173 INFO:teuthology.orchestra.run.smithi096.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-24T15:31:10.173 INFO:teuthology.orchestra.run.smithi096.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-24T15:31:10.173 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:31:10.173 INFO:teuthology.orchestra.run.smithi096.stdout: # 1m isn't quite enough 2024-04-24T15:31:10.173 INFO:teuthology.orchestra.run.smithi096.stdout: mon_down_mkfs_grace = 2m 2024-04-24T15:31:10.173 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:31:10.173 INFO:teuthology.orchestra.run.smithi096.stdout: mon_warn_on_filestore_osds = false 2024-04-24T15:31:10.174 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:31:10.174 INFO:teuthology.orchestra.run.smithi096.stdout:[client] 2024-04-24T15:31:10.174 INFO:teuthology.orchestra.run.smithi096.stdout: rgw cache enabled = true 2024-04-24T15:31:10.174 INFO:teuthology.orchestra.run.smithi096.stdout: rgw enable ops log = true 2024-04-24T15:31:10.174 INFO:teuthology.orchestra.run.smithi096.stdout: rgw enable usage log = true 2024-04-24T15:31:10.174 INFO:teuthology.orchestra.run.smithi096.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-24T15:31:10.174 INFO:teuthology.orchestra.run.smithi096.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-24T15:31:10.174 INFO:teuthology.orchestra.run.smithi096.stdout: rbd qos iops burst = 100 2024-04-24T15:31:10.174 INFO:teuthology.orchestra.run.smithi096.stdout: rbd qos iops limit = 50 2024-04-24T15:31:10.174 INFO:teuthology.orchestra.run.smithi096.stdout: rbd qos schedule tick min = 100 2024-04-24T15:31:10.174 INFO:teuthology.orchestra.run.smithi096.stdout:[mon.a] 2024-04-24T15:31:10.174 INFO:teuthology.orchestra.run.smithi096.stdout:[mon.c] 2024-04-24T15:31:10.174 INFO:teuthology.orchestra.run.smithi096.stdout:[mon.b] 2024-04-24T15:31:10.175 INFO:teuthology.orchestra.run.smithi151.stdout:[global] 2024-04-24T15:31:10.175 INFO:teuthology.orchestra.run.smithi151.stdout: chdir = "" 2024-04-24T15:31:10.175 INFO:teuthology.orchestra.run.smithi151.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-24T15:31:10.175 INFO:teuthology.orchestra.run.smithi151.stdout: auth supported = cephx 2024-04-24T15:31:10.175 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:31:10.175 INFO:teuthology.orchestra.run.smithi151.stdout: filestore xattr use omap = true 2024-04-24T15:31:10.175 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:31:10.175 INFO:teuthology.orchestra.run.smithi151.stdout: mon clock drift allowed = 1.000 2024-04-24T15:31:10.175 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:31:10.175 INFO:teuthology.orchestra.run.smithi151.stdout: osd crush chooseleaf type = 0 2024-04-24T15:31:10.175 INFO:teuthology.orchestra.run.smithi151.stdout: auth debug = true 2024-04-24T15:31:10.175 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:31:10.176 INFO:teuthology.orchestra.run.smithi151.stdout: ms die on old message = true 2024-04-24T15:31:10.176 INFO:teuthology.orchestra.run.smithi151.stdout: ms die on bug = true 2024-04-24T15:31:10.176 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:31:10.176 INFO:teuthology.orchestra.run.smithi151.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-24T15:31:10.176 INFO:teuthology.orchestra.run.smithi151.stdout: mon pg warn max object skew = 0 2024-04-24T15:31:10.176 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:31:10.176 INFO:teuthology.orchestra.run.smithi151.stdout: # disable pg_autoscaler by default for new pools 2024-04-24T15:31:10.176 INFO:teuthology.orchestra.run.smithi151.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-24T15:31:10.176 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:31:10.176 INFO:teuthology.orchestra.run.smithi151.stdout: osd pool default size = 2 2024-04-24T15:31:10.176 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:31:10.176 INFO:teuthology.orchestra.run.smithi151.stdout: mon osd allow primary affinity = true 2024-04-24T15:31:10.176 INFO:teuthology.orchestra.run.smithi151.stdout: mon osd allow pg remap = true 2024-04-24T15:31:10.176 INFO:teuthology.orchestra.run.smithi151.stdout: mon warn on legacy crush tunables = false 2024-04-24T15:31:10.176 INFO:teuthology.orchestra.run.smithi151.stdout: mon warn on crush straw calc version zero = false 2024-04-24T15:31:10.176 INFO:teuthology.orchestra.run.smithi151.stdout: mon warn on no sortbitwise = false 2024-04-24T15:31:10.177 INFO:teuthology.orchestra.run.smithi151.stdout: mon warn on osd down out interval zero = false 2024-04-24T15:31:10.177 INFO:teuthology.orchestra.run.smithi151.stdout: mon warn on too few osds = false 2024-04-24T15:31:10.177 INFO:teuthology.orchestra.run.smithi151.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-24T15:31:10.177 INFO:teuthology.orchestra.run.smithi151.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-24T15:31:10.177 INFO:teuthology.orchestra.run.smithi151.stdout: mon_allow_pool_size_one = true 2024-04-24T15:31:10.177 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:31:10.177 INFO:teuthology.orchestra.run.smithi151.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-24T15:31:10.177 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:31:10.177 INFO:teuthology.orchestra.run.smithi151.stdout: osd default data pool replay window = 5 2024-04-24T15:31:10.177 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:31:10.177 INFO:teuthology.orchestra.run.smithi151.stdout: mon allow pool delete = true 2024-04-24T15:31:10.177 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:31:10.177 INFO:teuthology.orchestra.run.smithi151.stdout: mon cluster log file level = debug 2024-04-24T15:31:10.177 INFO:teuthology.orchestra.run.smithi151.stdout: debug asserts on shutdown = true 2024-04-24T15:31:10.177 INFO:teuthology.orchestra.run.smithi151.stdout: mon health detail to clog = false 2024-04-24T15:31:10.178 INFO:teuthology.orchestra.run.smithi151.stdout: mon host = "172.21.15.96,[v2:172.21.15.96:3301,v1:172.21.15.96:6790],172.21.15.151" 2024-04-24T15:31:10.178 INFO:teuthology.orchestra.run.smithi151.stdout: mon client directed command retry = 5 2024-04-24T15:31:10.178 INFO:teuthology.orchestra.run.smithi151.stdout: mon warn on pool no app = False 2024-04-24T15:31:10.178 INFO:teuthology.orchestra.run.smithi151.stdout: ms inject socket failures = 5000 2024-04-24T15:31:10.178 INFO:teuthology.orchestra.run.smithi151.stdout: fsid = 253d426c-62cb-4194-8514-6cd970fd85c7 2024-04-24T15:31:10.178 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:31:10.178 INFO:teuthology.orchestra.run.smithi151.stdout:[osd] 2024-04-24T15:31:10.178 INFO:teuthology.orchestra.run.smithi151.stdout: osd journal size = 100 2024-04-24T15:31:10.178 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:31:10.178 INFO:teuthology.orchestra.run.smithi151.stdout: osd scrub load threshold = 5.0 2024-04-24T15:31:10.178 INFO:teuthology.orchestra.run.smithi151.stdout: osd scrub max interval = 600 2024-04-24T15:31:10.178 INFO:teuthology.orchestra.run.smithi151.stdout: osd mclock profile = high_recovery_ops 2024-04-24T15:31:10.178 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:31:10.178 INFO:teuthology.orchestra.run.smithi151.stdout: osd recover clone overlap = true 2024-04-24T15:31:10.178 INFO:teuthology.orchestra.run.smithi151.stdout: osd recovery max chunk = 1048576 2024-04-24T15:31:10.178 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:31:10.178 INFO:teuthology.orchestra.run.smithi151.stdout: osd debug shutdown = true 2024-04-24T15:31:10.179 INFO:teuthology.orchestra.run.smithi151.stdout: osd debug op order = true 2024-04-24T15:31:10.179 INFO:teuthology.orchestra.run.smithi151.stdout: osd debug verify stray on activate = true 2024-04-24T15:31:10.179 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:31:10.179 INFO:teuthology.orchestra.run.smithi151.stdout: osd open classes on start = true 2024-04-24T15:31:10.179 INFO:teuthology.orchestra.run.smithi151.stdout: osd debug pg log writeout = true 2024-04-24T15:31:10.179 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:31:10.179 INFO:teuthology.orchestra.run.smithi151.stdout: osd deep scrub update digest min age = 30 2024-04-24T15:31:10.179 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:31:10.179 INFO:teuthology.orchestra.run.smithi151.stdout: osd map max advance = 10 2024-04-24T15:31:10.179 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:31:10.179 INFO:teuthology.orchestra.run.smithi151.stdout: journal zero on create = true 2024-04-24T15:31:10.179 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:31:10.179 INFO:teuthology.orchestra.run.smithi151.stdout: filestore ondisk finisher threads = 3 2024-04-24T15:31:10.179 INFO:teuthology.orchestra.run.smithi151.stdout: filestore apply finisher threads = 3 2024-04-24T15:31:10.179 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:31:10.179 INFO:teuthology.orchestra.run.smithi151.stdout: bdev debug aio = true 2024-04-24T15:31:10.180 INFO:teuthology.orchestra.run.smithi151.stdout: osd debug misdirected ops = true 2024-04-24T15:31:10.180 INFO:teuthology.orchestra.run.smithi151.stdout: bluestore block size = 96636764160 2024-04-24T15:31:10.180 INFO:teuthology.orchestra.run.smithi151.stdout: bluestore compression algorithm = zstd 2024-04-24T15:31:10.180 INFO:teuthology.orchestra.run.smithi151.stdout: bluestore compression mode = aggressive 2024-04-24T15:31:10.180 INFO:teuthology.orchestra.run.smithi151.stdout: bluestore fsck on mount = True 2024-04-24T15:31:10.180 INFO:teuthology.orchestra.run.smithi151.stdout: debug bluefs = 1/20 2024-04-24T15:31:10.180 INFO:teuthology.orchestra.run.smithi151.stdout: debug bluestore = 1/20 2024-04-24T15:31:10.180 INFO:teuthology.orchestra.run.smithi151.stdout: debug ms = 1 2024-04-24T15:31:10.180 INFO:teuthology.orchestra.run.smithi151.stdout: debug osd = 20 2024-04-24T15:31:10.180 INFO:teuthology.orchestra.run.smithi151.stdout: debug rocksdb = 4/10 2024-04-24T15:31:10.180 INFO:teuthology.orchestra.run.smithi151.stdout: mon osd backfillfull_ratio = 0.85 2024-04-24T15:31:10.180 INFO:teuthology.orchestra.run.smithi151.stdout: mon osd full ratio = 0.9 2024-04-24T15:31:10.180 INFO:teuthology.orchestra.run.smithi151.stdout: mon osd nearfull ratio = 0.8 2024-04-24T15:31:10.180 INFO:teuthology.orchestra.run.smithi151.stdout: osd failsafe full ratio = 0.95 2024-04-24T15:31:10.180 INFO:teuthology.orchestra.run.smithi151.stdout: osd max markdown count = 1000 2024-04-24T15:31:10.181 INFO:teuthology.orchestra.run.smithi151.stdout: osd objectstore = bluestore 2024-04-24T15:31:10.181 INFO:teuthology.orchestra.run.smithi151.stdout: osd shutdown pgref assert = True 2024-04-24T15:31:10.181 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:31:10.181 INFO:teuthology.orchestra.run.smithi151.stdout:[mgr] 2024-04-24T15:31:10.181 INFO:teuthology.orchestra.run.smithi151.stdout: debug ms = 1 2024-04-24T15:31:10.181 INFO:teuthology.orchestra.run.smithi151.stdout: debug mgr = 20 2024-04-24T15:31:10.181 INFO:teuthology.orchestra.run.smithi151.stdout: debug mon = 20 2024-04-24T15:31:10.181 INFO:teuthology.orchestra.run.smithi151.stdout: debug auth = 20 2024-04-24T15:31:10.181 INFO:teuthology.orchestra.run.smithi151.stdout: mon reweight min pgs per osd = 4 2024-04-24T15:31:10.181 INFO:teuthology.orchestra.run.smithi151.stdout: mon reweight min bytes per osd = 10 2024-04-24T15:31:10.181 INFO:teuthology.orchestra.run.smithi151.stdout: mgr/telemetry/nag = false 2024-04-24T15:31:10.181 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:31:10.181 INFO:teuthology.orchestra.run.smithi151.stdout:[mon] 2024-04-24T15:31:10.181 INFO:teuthology.orchestra.run.smithi151.stdout: debug ms = 1 2024-04-24T15:31:10.181 INFO:teuthology.orchestra.run.smithi151.stdout: debug mon = 20 2024-04-24T15:31:10.181 INFO:teuthology.orchestra.run.smithi151.stdout: debug paxos = 20 2024-04-24T15:31:10.181 INFO:teuthology.orchestra.run.smithi151.stdout: debug auth = 20 2024-04-24T15:31:10.182 INFO:teuthology.orchestra.run.smithi151.stdout: mon data avail warn = 5 2024-04-24T15:31:10.182 INFO:teuthology.orchestra.run.smithi151.stdout: mon mgr mkfs grace = 240 2024-04-24T15:31:10.182 INFO:teuthology.orchestra.run.smithi151.stdout: mon reweight min pgs per osd = 4 2024-04-24T15:31:10.182 INFO:teuthology.orchestra.run.smithi151.stdout: mon osd reporter subtree level = osd 2024-04-24T15:31:10.182 INFO:teuthology.orchestra.run.smithi151.stdout: mon osd prime pg temp = true 2024-04-24T15:31:10.182 INFO:teuthology.orchestra.run.smithi151.stdout: mon reweight min bytes per osd = 10 2024-04-24T15:31:10.182 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:31:10.182 INFO:teuthology.orchestra.run.smithi151.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-24T15:31:10.182 INFO:teuthology.orchestra.run.smithi151.stdout: auth mon ticket ttl = 660 # 11m 2024-04-24T15:31:10.182 INFO:teuthology.orchestra.run.smithi151.stdout: auth service ticket ttl = 240 # 4m 2024-04-24T15:31:10.182 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:31:10.182 INFO:teuthology.orchestra.run.smithi151.stdout: # don't complain about insecure global_id in the test suite 2024-04-24T15:31:10.182 INFO:teuthology.orchestra.run.smithi151.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-24T15:31:10.182 INFO:teuthology.orchestra.run.smithi151.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-24T15:31:10.182 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:31:10.182 INFO:teuthology.orchestra.run.smithi151.stdout: # 1m isn't quite enough 2024-04-24T15:31:10.183 INFO:teuthology.orchestra.run.smithi151.stdout: mon_down_mkfs_grace = 2m 2024-04-24T15:31:10.183 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:31:10.183 INFO:teuthology.orchestra.run.smithi151.stdout: mon_warn_on_filestore_osds = false 2024-04-24T15:31:10.183 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-24T15:31:10.183 INFO:teuthology.orchestra.run.smithi151.stdout:[client] 2024-04-24T15:31:10.183 INFO:teuthology.orchestra.run.smithi151.stdout: rgw cache enabled = true 2024-04-24T15:31:10.183 INFO:teuthology.orchestra.run.smithi151.stdout: rgw enable ops log = true 2024-04-24T15:31:10.183 INFO:teuthology.orchestra.run.smithi151.stdout: rgw enable usage log = true 2024-04-24T15:31:10.183 INFO:teuthology.orchestra.run.smithi151.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-24T15:31:10.183 INFO:teuthology.orchestra.run.smithi151.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-24T15:31:10.183 INFO:teuthology.orchestra.run.smithi151.stdout: rbd qos iops burst = 100 2024-04-24T15:31:10.183 INFO:teuthology.orchestra.run.smithi151.stdout: rbd qos iops limit = 50 2024-04-24T15:31:10.183 INFO:teuthology.orchestra.run.smithi151.stdout: rbd qos schedule tick min = 100 2024-04-24T15:31:10.183 INFO:teuthology.orchestra.run.smithi151.stdout:[mon.a] 2024-04-24T15:31:10.183 INFO:teuthology.orchestra.run.smithi151.stdout:[mon.c] 2024-04-24T15:31:10.183 INFO:teuthology.orchestra.run.smithi151.stdout:[mon.b] 2024-04-24T15:31:10.185 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-24T15:31:10.185 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-24T15:31:10.277 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-24T15:31:10.278 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-24T15:31:10.278 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-24T15:31:10.295 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-24T15:31:10.295 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-24T15:31:10.354 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi096.front.sepia.ceph.com 2024-04-24T15:31:10.355 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-24T15:31:10.355 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-24T15:31:10.355 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-24T15:31:10.429 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-24T15:31:10.429 DEBUG:teuthology.orchestra.run.smithi096:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-24T15:31:10.486 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi151.front.sepia.ceph.com 2024-04-24T15:31:10.486 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-24T15:31:10.486 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-24T15:31:10.486 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-24T15:31:10.523 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-24T15:31:10.523 DEBUG:teuthology.orchestra.run.smithi151:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-24T15:31:10.581 INFO:tasks.ceph:Setting up mon nodes... 2024-04-24T15:31:10.582 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-24T15:31:10.582 DEBUG:teuthology.orchestra.run.smithi096:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-24T15:31:10.630 INFO:teuthology.orchestra.run.smithi096.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-24T15:31:10.633 DEBUG:teuthology.orchestra.run.smithi151:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-24T15:31:10.716 INFO:teuthology.orchestra.run.smithi151.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-24T15:31:10.718 INFO:tasks.ceph:Setting up mds nodes... 2024-04-24T15:31:10.718 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-24T15:31:10.719 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-24T15:31:10.758 INFO:teuthology.orchestra.run.smithi096.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-24T15:31:10.769 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-24T15:31:10.810 INFO:teuthology.orchestra.run.smithi151.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-24T15:31:10.821 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-24T15:31:10.821 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi096.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi151.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-24T15:31:10.822 DEBUG:teuthology.orchestra.run.smithi096:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-24T15:31:10.848 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-24T15:31:10.848 INFO:tasks.ceph:role: osd.0 2024-04-24T15:31:10.848 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi096.front.sepia.ceph.com 2024-04-24T15:31:10.848 DEBUG:teuthology.orchestra.run.smithi096:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-24T15:31:10.922 INFO:teuthology.orchestra.run.smithi096.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-24T15:31:10.922 INFO:teuthology.orchestra.run.smithi096.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-24T15:31:10.922 INFO:teuthology.orchestra.run.smithi096.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-24T15:31:10.922 INFO:teuthology.orchestra.run.smithi096.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-24T15:31:10.922 INFO:teuthology.orchestra.run.smithi096.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-24T15:31:10.922 INFO:teuthology.orchestra.run.smithi096.stdout: = sunit=0 swidth=0 blks 2024-04-24T15:31:10.922 INFO:teuthology.orchestra.run.smithi096.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-24T15:31:10.922 INFO:teuthology.orchestra.run.smithi096.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-24T15:31:10.923 INFO:teuthology.orchestra.run.smithi096.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-24T15:31:10.923 INFO:teuthology.orchestra.run.smithi096.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-24T15:31:11.352 INFO:teuthology.orchestra.run.smithi096.stdout:Discarding blocks...Done. 2024-04-24T15:31:11.353 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi096.front.sepia.ceph.com -o noatime 2024-04-24T15:31:11.354 DEBUG:teuthology.orchestra.run.smithi096:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-24T15:31:11.519 DEBUG:teuthology.orchestra.run.smithi096:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-24T15:31:11.552 DEBUG:teuthology.orchestra.run.smithi096:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-24T15:31:11.619 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-24T15:31:11.619 INFO:tasks.ceph:role: osd.1 2024-04-24T15:31:11.619 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi096.front.sepia.ceph.com 2024-04-24T15:31:11.619 DEBUG:teuthology.orchestra.run.smithi096:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-24T15:31:11.693 INFO:teuthology.orchestra.run.smithi096.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-24T15:31:11.694 INFO:teuthology.orchestra.run.smithi096.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-24T15:31:11.694 INFO:teuthology.orchestra.run.smithi096.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-24T15:31:11.694 INFO:teuthology.orchestra.run.smithi096.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-24T15:31:11.694 INFO:teuthology.orchestra.run.smithi096.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-24T15:31:11.694 INFO:teuthology.orchestra.run.smithi096.stdout: = sunit=0 swidth=0 blks 2024-04-24T15:31:11.694 INFO:teuthology.orchestra.run.smithi096.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-24T15:31:11.694 INFO:teuthology.orchestra.run.smithi096.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-24T15:31:11.694 INFO:teuthology.orchestra.run.smithi096.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-24T15:31:11.694 INFO:teuthology.orchestra.run.smithi096.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-24T15:31:12.117 INFO:teuthology.orchestra.run.smithi096.stdout:Discarding blocks...Done. 2024-04-24T15:31:12.118 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi096.front.sepia.ceph.com -o noatime 2024-04-24T15:31:12.119 DEBUG:teuthology.orchestra.run.smithi096:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-24T15:31:12.223 DEBUG:teuthology.orchestra.run.smithi096:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-24T15:31:12.255 DEBUG:teuthology.orchestra.run.smithi096:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-24T15:31:12.322 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-24T15:31:12.322 INFO:tasks.ceph:role: osd.2 2024-04-24T15:31:12.322 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi096.front.sepia.ceph.com 2024-04-24T15:31:12.322 DEBUG:teuthology.orchestra.run.smithi096:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-24T15:31:12.396 INFO:teuthology.orchestra.run.smithi096.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-24T15:31:12.396 INFO:teuthology.orchestra.run.smithi096.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-24T15:31:12.396 INFO:teuthology.orchestra.run.smithi096.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-24T15:31:12.396 INFO:teuthology.orchestra.run.smithi096.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-24T15:31:12.396 INFO:teuthology.orchestra.run.smithi096.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-24T15:31:12.396 INFO:teuthology.orchestra.run.smithi096.stdout: = sunit=0 swidth=0 blks 2024-04-24T15:31:12.396 INFO:teuthology.orchestra.run.smithi096.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-24T15:31:12.396 INFO:teuthology.orchestra.run.smithi096.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-24T15:31:12.396 INFO:teuthology.orchestra.run.smithi096.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-24T15:31:12.396 INFO:teuthology.orchestra.run.smithi096.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-24T15:31:12.859 INFO:teuthology.orchestra.run.smithi096.stdout:Discarding blocks...Done. 2024-04-24T15:31:12.860 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi096.front.sepia.ceph.com -o noatime 2024-04-24T15:31:12.860 DEBUG:teuthology.orchestra.run.smithi096:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-24T15:31:12.998 DEBUG:teuthology.orchestra.run.smithi096:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-24T15:31:13.029 DEBUG:teuthology.orchestra.run.smithi096:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-24T15:31:13.095 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-24T15:31:13.096 INFO:tasks.ceph:role: osd.3 2024-04-24T15:31:13.096 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi096.front.sepia.ceph.com 2024-04-24T15:31:13.096 DEBUG:teuthology.orchestra.run.smithi096:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-24T15:31:13.170 INFO:teuthology.orchestra.run.smithi096.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-24T15:31:13.170 INFO:teuthology.orchestra.run.smithi096.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-24T15:31:13.170 INFO:teuthology.orchestra.run.smithi096.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-24T15:31:13.170 INFO:teuthology.orchestra.run.smithi096.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-24T15:31:13.170 INFO:teuthology.orchestra.run.smithi096.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-24T15:31:13.170 INFO:teuthology.orchestra.run.smithi096.stdout: = sunit=0 swidth=0 blks 2024-04-24T15:31:13.170 INFO:teuthology.orchestra.run.smithi096.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-24T15:31:13.170 INFO:teuthology.orchestra.run.smithi096.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-24T15:31:13.170 INFO:teuthology.orchestra.run.smithi096.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-24T15:31:13.170 INFO:teuthology.orchestra.run.smithi096.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-24T15:31:13.611 INFO:teuthology.orchestra.run.smithi096.stdout:Discarding blocks...Done. 2024-04-24T15:31:13.613 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi096.front.sepia.ceph.com -o noatime 2024-04-24T15:31:13.613 DEBUG:teuthology.orchestra.run.smithi096:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-24T15:31:13.797 DEBUG:teuthology.orchestra.run.smithi096:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-24T15:31:13.830 DEBUG:teuthology.orchestra.run.smithi096:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-24T15:31:13.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-24T15:31:13.921+0000 7ff792176540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-24T15:31:13.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-24T15:31:13.921+0000 7ff792176540 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-24T15:31:13.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-24T15:31:13.921+0000 7ff792176540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-24T15:31:13.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-24T15:31:13.921+0000 7ff792176540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-24T15:31:13.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-24T15:31:13.921+0000 7ff792176540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-24T15:31:13.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-24T15:31:13.922+0000 7ff792176540 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-24T15:31:14.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-24T15:31:14.445+0000 7ff792176540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-04-24T15:31:15.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-24T15:31:15.210+0000 7ff792176540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-04-24T15:31:15.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-24T15:31:15.976+0000 7ff792176540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-04-24T15:31:16.768 DEBUG:teuthology.orchestra.run.smithi096:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-24T15:31:16.800 DEBUG:teuthology.orchestra.run.smithi096:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-24T15:31:16.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-24T15:31:16.887+0000 7fd134ec3540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-24T15:31:16.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-24T15:31:16.887+0000 7fd134ec3540 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-24T15:31:16.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-24T15:31:16.887+0000 7fd134ec3540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-24T15:31:16.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-24T15:31:16.887+0000 7fd134ec3540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-24T15:31:16.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-24T15:31:16.887+0000 7fd134ec3540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-24T15:31:16.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-24T15:31:16.888+0000 7fd134ec3540 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-24T15:31:17.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-24T15:31:17.415+0000 7fd134ec3540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-04-24T15:31:18.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-24T15:31:18.177+0000 7fd134ec3540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-04-24T15:31:18.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-24T15:31:18.940+0000 7fd134ec3540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-04-24T15:31:19.739 DEBUG:teuthology.orchestra.run.smithi096:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-24T15:31:19.773 DEBUG:teuthology.orchestra.run.smithi096:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-24T15:31:19.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-24T15:31:19.861+0000 7fc2a4722540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-24T15:31:19.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-24T15:31:19.861+0000 7fc2a4722540 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-24T15:31:19.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-24T15:31:19.861+0000 7fc2a4722540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-24T15:31:19.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-24T15:31:19.861+0000 7fc2a4722540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-24T15:31:19.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-24T15:31:19.861+0000 7fc2a4722540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-24T15:31:19.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-24T15:31:19.861+0000 7fc2a4722540 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-24T15:31:20.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-24T15:31:20.391+0000 7fc2a4722540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-04-24T15:31:21.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-24T15:31:21.155+0000 7fc2a4722540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-04-24T15:31:21.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-24T15:31:21.917+0000 7fc2a4722540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-04-24T15:31:22.711 DEBUG:teuthology.orchestra.run.smithi096:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-24T15:31:22.743 DEBUG:teuthology.orchestra.run.smithi096:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-24T15:31:22.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-24T15:31:22.831+0000 7f124af06540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-24T15:31:22.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-24T15:31:22.831+0000 7f124af06540 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-24T15:31:22.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-24T15:31:22.831+0000 7f124af06540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-24T15:31:22.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-24T15:31:22.831+0000 7f124af06540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-24T15:31:22.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-24T15:31:22.831+0000 7f124af06540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-24T15:31:22.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-24T15:31:22.832+0000 7f124af06540 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-24T15:31:23.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-24T15:31:23.357+0000 7f124af06540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-04-24T15:31:24.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-24T15:31:24.124+0000 7f124af06540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-04-24T15:31:24.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-24T15:31:24.887+0000 7f124af06540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-04-24T15:31:25.680 DEBUG:teuthology.orchestra.run.smithi096:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-24T15:31:25.712 DEBUG:teuthology.orchestra.run.smithi151:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-24T15:31:25.746 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-24T15:31:25.747 INFO:tasks.ceph:role: osd.4 2024-04-24T15:31:25.747 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi151.front.sepia.ceph.com 2024-04-24T15:31:25.747 DEBUG:teuthology.orchestra.run.smithi151:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-24T15:31:25.824 INFO:teuthology.orchestra.run.smithi151.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-24T15:31:25.825 INFO:teuthology.orchestra.run.smithi151.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-24T15:31:25.825 INFO:teuthology.orchestra.run.smithi151.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-24T15:31:25.825 INFO:teuthology.orchestra.run.smithi151.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-24T15:31:25.825 INFO:teuthology.orchestra.run.smithi151.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-24T15:31:25.825 INFO:teuthology.orchestra.run.smithi151.stdout: = sunit=0 swidth=0 blks 2024-04-24T15:31:25.825 INFO:teuthology.orchestra.run.smithi151.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-24T15:31:25.825 INFO:teuthology.orchestra.run.smithi151.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-24T15:31:25.825 INFO:teuthology.orchestra.run.smithi151.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-24T15:31:25.825 INFO:teuthology.orchestra.run.smithi151.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-24T15:31:26.262 INFO:teuthology.orchestra.run.smithi151.stdout:Discarding blocks...Done. 2024-04-24T15:31:26.264 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi151.front.sepia.ceph.com -o noatime 2024-04-24T15:31:26.264 DEBUG:teuthology.orchestra.run.smithi151:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-24T15:31:26.405 DEBUG:teuthology.orchestra.run.smithi151:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-24T15:31:26.443 DEBUG:teuthology.orchestra.run.smithi151:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-24T15:31:26.513 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-24T15:31:26.513 INFO:tasks.ceph:role: osd.5 2024-04-24T15:31:26.513 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi151.front.sepia.ceph.com 2024-04-24T15:31:26.513 DEBUG:teuthology.orchestra.run.smithi151:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-24T15:31:26.590 INFO:teuthology.orchestra.run.smithi151.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-24T15:31:26.590 INFO:teuthology.orchestra.run.smithi151.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-24T15:31:26.590 INFO:teuthology.orchestra.run.smithi151.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-24T15:31:26.590 INFO:teuthology.orchestra.run.smithi151.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-24T15:31:26.591 INFO:teuthology.orchestra.run.smithi151.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-24T15:31:26.591 INFO:teuthology.orchestra.run.smithi151.stdout: = sunit=0 swidth=0 blks 2024-04-24T15:31:26.591 INFO:teuthology.orchestra.run.smithi151.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-24T15:31:26.591 INFO:teuthology.orchestra.run.smithi151.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-24T15:31:26.591 INFO:teuthology.orchestra.run.smithi151.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-24T15:31:26.591 INFO:teuthology.orchestra.run.smithi151.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-24T15:31:27.007 INFO:teuthology.orchestra.run.smithi151.stdout:Discarding blocks...Done. 2024-04-24T15:31:27.009 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi151.front.sepia.ceph.com -o noatime 2024-04-24T15:31:27.009 DEBUG:teuthology.orchestra.run.smithi151:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-24T15:31:27.100 DEBUG:teuthology.orchestra.run.smithi151:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-24T15:31:27.133 DEBUG:teuthology.orchestra.run.smithi151:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-24T15:31:27.202 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-24T15:31:27.203 INFO:tasks.ceph:role: osd.6 2024-04-24T15:31:27.203 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi151.front.sepia.ceph.com 2024-04-24T15:31:27.203 DEBUG:teuthology.orchestra.run.smithi151:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-24T15:31:27.278 INFO:teuthology.orchestra.run.smithi151.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-24T15:31:27.278 INFO:teuthology.orchestra.run.smithi151.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-24T15:31:27.278 INFO:teuthology.orchestra.run.smithi151.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-24T15:31:27.278 INFO:teuthology.orchestra.run.smithi151.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-24T15:31:27.278 INFO:teuthology.orchestra.run.smithi151.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-24T15:31:27.278 INFO:teuthology.orchestra.run.smithi151.stdout: = sunit=0 swidth=0 blks 2024-04-24T15:31:27.279 INFO:teuthology.orchestra.run.smithi151.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-24T15:31:27.279 INFO:teuthology.orchestra.run.smithi151.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-24T15:31:27.279 INFO:teuthology.orchestra.run.smithi151.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-24T15:31:27.279 INFO:teuthology.orchestra.run.smithi151.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-24T15:31:27.720 INFO:teuthology.orchestra.run.smithi151.stdout:Discarding blocks...Done. 2024-04-24T15:31:27.722 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi151.front.sepia.ceph.com -o noatime 2024-04-24T15:31:27.722 DEBUG:teuthology.orchestra.run.smithi151:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-24T15:31:27.853 DEBUG:teuthology.orchestra.run.smithi151:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-24T15:31:27.885 DEBUG:teuthology.orchestra.run.smithi151:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-24T15:31:27.952 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-24T15:31:27.952 INFO:tasks.ceph:role: osd.7 2024-04-24T15:31:27.952 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi151.front.sepia.ceph.com 2024-04-24T15:31:27.952 DEBUG:teuthology.orchestra.run.smithi151:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-24T15:31:28.028 INFO:teuthology.orchestra.run.smithi151.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-24T15:31:28.028 INFO:teuthology.orchestra.run.smithi151.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-24T15:31:28.028 INFO:teuthology.orchestra.run.smithi151.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-24T15:31:28.028 INFO:teuthology.orchestra.run.smithi151.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-24T15:31:28.028 INFO:teuthology.orchestra.run.smithi151.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-24T15:31:28.029 INFO:teuthology.orchestra.run.smithi151.stdout: = sunit=0 swidth=0 blks 2024-04-24T15:31:28.029 INFO:teuthology.orchestra.run.smithi151.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-24T15:31:28.029 INFO:teuthology.orchestra.run.smithi151.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-24T15:31:28.029 INFO:teuthology.orchestra.run.smithi151.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-24T15:31:28.029 INFO:teuthology.orchestra.run.smithi151.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-24T15:31:28.465 INFO:teuthology.orchestra.run.smithi151.stdout:Discarding blocks...Done. 2024-04-24T15:31:28.466 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi151.front.sepia.ceph.com -o noatime 2024-04-24T15:31:28.467 DEBUG:teuthology.orchestra.run.smithi151:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-24T15:31:28.632 DEBUG:teuthology.orchestra.run.smithi151:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-24T15:31:28.666 DEBUG:teuthology.orchestra.run.smithi151:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-24T15:31:28.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-24T15:31:28.780+0000 7f3b13e56540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-24T15:31:28.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-24T15:31:28.781+0000 7f3b13e56540 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-24T15:31:28.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-24T15:31:28.781+0000 7f3b13e56540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-24T15:31:28.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-24T15:31:28.781+0000 7f3b13e56540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-24T15:31:28.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-24T15:31:28.781+0000 7f3b13e56540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-24T15:31:28.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-24T15:31:28.786+0000 7f3b13e56540 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-24T15:31:29.311 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-24T15:31:29.311+0000 7f3b13e56540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-04-24T15:31:30.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-24T15:31:30.080+0000 7f3b13e56540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-04-24T15:31:30.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-24T15:31:30.842+0000 7f3b13e56540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-04-24T15:31:31.638 DEBUG:teuthology.orchestra.run.smithi151:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-24T15:31:31.671 DEBUG:teuthology.orchestra.run.smithi151:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-24T15:31:31.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-24T15:31:31.758+0000 7ff25d2a8540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-24T15:31:31.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-24T15:31:31.758+0000 7ff25d2a8540 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-24T15:31:31.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-24T15:31:31.758+0000 7ff25d2a8540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-24T15:31:31.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-24T15:31:31.758+0000 7ff25d2a8540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-24T15:31:31.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-24T15:31:31.758+0000 7ff25d2a8540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-24T15:31:31.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-24T15:31:31.759+0000 7ff25d2a8540 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-24T15:31:32.283 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-24T15:31:32.283+0000 7ff25d2a8540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-04-24T15:31:33.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-24T15:31:33.050+0000 7ff25d2a8540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-04-24T15:31:33.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-24T15:31:33.818+0000 7ff25d2a8540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-04-24T15:31:34.611 DEBUG:teuthology.orchestra.run.smithi151:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-24T15:31:34.643 DEBUG:teuthology.orchestra.run.smithi151:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-24T15:31:34.731 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-24T15:31:34.731+0000 7f5540ce3540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-24T15:31:34.731 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-24T15:31:34.731+0000 7f5540ce3540 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-24T15:31:34.731 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-24T15:31:34.731+0000 7f5540ce3540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-24T15:31:34.731 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-24T15:31:34.731+0000 7f5540ce3540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-24T15:31:34.732 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-24T15:31:34.731+0000 7f5540ce3540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-24T15:31:34.732 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-24T15:31:34.731+0000 7f5540ce3540 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-24T15:31:35.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-24T15:31:35.258+0000 7f5540ce3540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-04-24T15:31:36.024 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-24T15:31:36.024+0000 7f5540ce3540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-04-24T15:31:36.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-24T15:31:36.787+0000 7f5540ce3540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-04-24T15:31:37.581 DEBUG:teuthology.orchestra.run.smithi151:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-24T15:31:37.611 DEBUG:teuthology.orchestra.run.smithi151:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-24T15:31:37.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-24T15:31:37.698+0000 7f9f158da540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-24T15:31:37.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-24T15:31:37.698+0000 7f9f158da540 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-24T15:31:37.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-24T15:31:37.699+0000 7f9f158da540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-24T15:31:37.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-24T15:31:37.699+0000 7f9f158da540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-24T15:31:37.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-24T15:31:37.699+0000 7f9f158da540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-24T15:31:37.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-24T15:31:37.699+0000 7f9f158da540 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-24T15:31:38.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-24T15:31:38.225+0000 7f9f158da540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-04-24T15:31:38.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-24T15:31:38.990+0000 7f9f158da540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-04-24T15:31:39.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-24T15:31:39.751+0000 7f9f158da540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-04-24T15:31:40.547 DEBUG:teuthology.orchestra.run.smithi151:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-24T15:31:40.578 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-24T15:31:40.579 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-24T15:31:40.579 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-24T15:31:40.612 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-24T15:31:40.612 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-24T15:31:40.682 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-24T15:31:40.682 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-24T15:31:40.751 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-24T15:31:40.751 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-24T15:31:40.819 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-24T15:31:40.819 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-24T15:31:40.887 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-24T15:31:40.887 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-24T15:31:40.919 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-24T15:31:40.919 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-24T15:31:40.989 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-24T15:31:40.989 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-24T15:31:41.058 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-24T15:31:41.058 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-24T15:31:41.127 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-24T15:31:41.127 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-24T15:31:41.196 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-24T15:31:41.196 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-24T15:31:41.216 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-24T15:31:41.216 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-24T15:31:41.254 INFO:tasks.ceph:Adding keys to all mons... 2024-04-24T15:31:41.254 DEBUG:teuthology.orchestra.run.smithi096:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-24T15:31:41.258 DEBUG:teuthology.orchestra.run.smithi151:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-24T15:31:41.298 INFO:teuthology.orchestra.run.smithi096.stdout:[mgr.y] 2024-04-24T15:31:41.298 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQC+JSlmM+2wJRAA5lAUJs+sIbWx9bICRvh4Yg== 2024-04-24T15:31:41.298 INFO:teuthology.orchestra.run.smithi096.stdout:[osd.0] 2024-04-24T15:31:41.298 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQDBJSlmFKEFNxAANDfyexoX4aAgvuS9DWovTA== 2024-04-24T15:31:41.298 INFO:teuthology.orchestra.run.smithi096.stdout:[osd.1] 2024-04-24T15:31:41.298 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQDEJSlm6Av+NBAA1jLoJ920bMYDcYTdVwRRzg== 2024-04-24T15:31:41.298 INFO:teuthology.orchestra.run.smithi096.stdout:[osd.2] 2024-04-24T15:31:41.298 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQDHJSlmt0lqMxAASzbO6YbuaHzIGDOFBXoFlA== 2024-04-24T15:31:41.299 INFO:teuthology.orchestra.run.smithi096.stdout:[osd.3] 2024-04-24T15:31:41.299 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQDKJSlmrXWjMRAAxPR4FDyqgtJG1iB+w3PjNg== 2024-04-24T15:31:41.299 INFO:teuthology.orchestra.run.smithi096.stdout:[mgr.x] 2024-04-24T15:31:41.299 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQC+JSlmETjEKhAA9wh+5cmx1SihtNg6fOhaKQ== 2024-04-24T15:31:41.299 INFO:teuthology.orchestra.run.smithi096.stdout:[osd.4] 2024-04-24T15:31:41.299 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQDQJSlm+KmeLhAA0/gnYzjUM6Zib8nJh8TgyA== 2024-04-24T15:31:41.299 INFO:teuthology.orchestra.run.smithi096.stdout:[osd.5] 2024-04-24T15:31:41.299 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQDTJSlmkoxILRAA1uFjsxz9+Q1NHiNabtSB+Q== 2024-04-24T15:31:41.299 INFO:teuthology.orchestra.run.smithi096.stdout:[osd.6] 2024-04-24T15:31:41.299 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQDWJSlmC4WnKxAA/hOFdcUdVWsLuUj8FYBxiQ== 2024-04-24T15:31:41.299 INFO:teuthology.orchestra.run.smithi096.stdout:[osd.7] 2024-04-24T15:31:41.299 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQDZJSlm6Wa4KRAADju1ApyiijXgFAOOIW4qLg== 2024-04-24T15:31:41.300 INFO:teuthology.orchestra.run.smithi096.stdout:[client.0] 2024-04-24T15:31:41.300 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQC+JSlmdmBILRAAo6q6vJhbd+CfoVFSgb18dw== 2024-04-24T15:31:41.300 INFO:teuthology.orchestra.run.smithi096.stdout:[client.1] 2024-04-24T15:31:41.300 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQC+JSlmK1dcMBAAdDLbRvWmI/tlXSG2pesd/g== 2024-04-24T15:31:41.320 INFO:teuthology.orchestra.run.smithi151.stdout:[mgr.y] 2024-04-24T15:31:41.321 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQC+JSlmM+2wJRAA5lAUJs+sIbWx9bICRvh4Yg== 2024-04-24T15:31:41.321 INFO:teuthology.orchestra.run.smithi151.stdout:[osd.0] 2024-04-24T15:31:41.321 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQDBJSlmFKEFNxAANDfyexoX4aAgvuS9DWovTA== 2024-04-24T15:31:41.321 INFO:teuthology.orchestra.run.smithi151.stdout:[osd.1] 2024-04-24T15:31:41.321 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQDEJSlm6Av+NBAA1jLoJ920bMYDcYTdVwRRzg== 2024-04-24T15:31:41.321 INFO:teuthology.orchestra.run.smithi151.stdout:[osd.2] 2024-04-24T15:31:41.321 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQDHJSlmt0lqMxAASzbO6YbuaHzIGDOFBXoFlA== 2024-04-24T15:31:41.321 INFO:teuthology.orchestra.run.smithi151.stdout:[osd.3] 2024-04-24T15:31:41.321 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQDKJSlmrXWjMRAAxPR4FDyqgtJG1iB+w3PjNg== 2024-04-24T15:31:41.321 INFO:teuthology.orchestra.run.smithi151.stdout:[mgr.x] 2024-04-24T15:31:41.321 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQC+JSlmETjEKhAA9wh+5cmx1SihtNg6fOhaKQ== 2024-04-24T15:31:41.321 INFO:teuthology.orchestra.run.smithi151.stdout:[osd.4] 2024-04-24T15:31:41.321 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQDQJSlm+KmeLhAA0/gnYzjUM6Zib8nJh8TgyA== 2024-04-24T15:31:41.321 INFO:teuthology.orchestra.run.smithi151.stdout:[osd.5] 2024-04-24T15:31:41.322 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQDTJSlmkoxILRAA1uFjsxz9+Q1NHiNabtSB+Q== 2024-04-24T15:31:41.322 INFO:teuthology.orchestra.run.smithi151.stdout:[osd.6] 2024-04-24T15:31:41.322 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQDWJSlmC4WnKxAA/hOFdcUdVWsLuUj8FYBxiQ== 2024-04-24T15:31:41.322 INFO:teuthology.orchestra.run.smithi151.stdout:[osd.7] 2024-04-24T15:31:41.322 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQDZJSlm6Wa4KRAADju1ApyiijXgFAOOIW4qLg== 2024-04-24T15:31:41.322 INFO:teuthology.orchestra.run.smithi151.stdout:[client.0] 2024-04-24T15:31:41.322 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQC+JSlmdmBILRAAo6q6vJhbd+CfoVFSgb18dw== 2024-04-24T15:31:41.322 INFO:teuthology.orchestra.run.smithi151.stdout:[client.1] 2024-04-24T15:31:41.322 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQC+JSlmK1dcMBAAdDLbRvWmI/tlXSG2pesd/g== 2024-04-24T15:31:41.323 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-24T15:31:41.343 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-24T15:31:41.413 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T15:31:41.416 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T15:31:41.466 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T15:31:41.468 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T15:31:41.517 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T15:31:41.519 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T15:31:41.568 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T15:31:41.570 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T15:31:41.619 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-24T15:31:41.621 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-24T15:31:41.670 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T15:31:41.672 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T15:31:41.722 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T15:31:41.725 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T15:31:41.774 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T15:31:41.776 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T15:31:41.825 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T15:31:41.828 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T15:31:41.877 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-24T15:31:41.880 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-24T15:31:41.932 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-24T15:31:41.934 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-24T15:31:41.986 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-24T15:31:41.987 DEBUG:teuthology.orchestra.run.smithi096:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-24T15:31:42.013 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-24T15:31:42.106 DEBUG:teuthology.orchestra.run.smithi096:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-24T15:31:42.133 DEBUG:teuthology.orchestra.run.smithi096:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-24T15:31:42.200 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-24T15:31:42.293 DEBUG:teuthology.orchestra.run.smithi096:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-24T15:31:42.320 DEBUG:teuthology.orchestra.run.smithi151:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-24T15:31:42.349 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-24T15:31:42.443 DEBUG:teuthology.orchestra.run.smithi151:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-24T15:31:42.470 DEBUG:teuthology.orchestra.run.smithi096:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-24T15:31:42.472 DEBUG:teuthology.orchestra.run.smithi151:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-24T15:31:42.528 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-24T15:31:42.528 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-24T15:31:42.528 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-24T15:31:42.530 INFO:tasks.ceph.mon.a:Started 2024-04-24T15:31:42.530 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-24T15:31:42.530 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-24T15:31:42.533 INFO:tasks.ceph.mon.c:Started 2024-04-24T15:31:42.533 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-24T15:31:42.533 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-24T15:31:42.570 INFO:tasks.ceph.mon.b:Started 2024-04-24T15:31:42.570 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-24T15:31:42.570 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-24T15:31:42.571 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-24T15:31:42.573 INFO:tasks.ceph.mgr.y:Started 2024-04-24T15:31:42.573 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-24T15:31:42.573 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-24T15:31:42.575 INFO:tasks.ceph.mgr.x:Started 2024-04-24T15:31:42.575 INFO:tasks.ceph:Setting crush tunables to default 2024-04-24T15:31:42.575 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --cluster ceph osd crush tunables default 2024-04-24T15:31:48.224 INFO:tasks.ceph.mgr.y.smithi096.stderr:2024-04-24T15:31:48.225+0000 7f1e8451f1c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T15:31:48.396 INFO:tasks.ceph.mgr.y.smithi096.stderr:2024-04-24T15:31:48.396+0000 7f1e8451f1c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T15:31:48.749 INFO:tasks.ceph.mgr.y.smithi096.stderr:2024-04-24T15:31:48.750+0000 7f1e8451f1c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T15:31:48.806 INFO:teuthology.orchestra.run.smithi096.stderr:adjusted tunables profile to default 2024-04-24T15:31:48.817 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-24T15:31:48.817 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-24T15:31:48.817 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-24T15:31:48.843 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-24T15:31:48.843 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-24T15:31:48.909 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-24T15:31:48.909 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-24T15:31:48.976 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-24T15:31:48.976 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-24T15:31:48.976 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-04-24T15:31:48.976+0000 7f1bf5dc51c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T15:31:49.042 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-24T15:31:49.043 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-24T15:31:49.074 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-24T15:31:49.074 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-24T15:31:49.155 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-04-24T15:31:49.155+0000 7f1bf5dc51c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T15:31:49.156 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-24T15:31:49.157 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-24T15:31:49.224 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-24T15:31:49.224 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-24T15:31:49.264 INFO:tasks.ceph.mgr.y.smithi096.stderr:2024-04-24T15:31:49.265+0000 7f1e8451f1c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T15:31:49.290 DEBUG:teuthology.orchestra.run.smithi151:> sudo ceph --cluster ceph osd new 92941fb9-96ee-4b2b-8147-3c1d61192f94 0 2024-04-24T15:31:49.402 INFO:tasks.ceph.mgr.y.smithi096.stderr:2024-04-24T15:31:49.403+0000 7f1e8451f1c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T15:31:49.545 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-04-24T15:31:49.545+0000 7f1bf5dc51c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T15:31:49.561 INFO:teuthology.orchestra.run.smithi151.stdout:0 2024-04-24T15:31:49.574 DEBUG:teuthology.orchestra.run.smithi151:> sudo ceph --cluster ceph osd new 628e868a-9884-42e6-9331-caef97cb6967 1 2024-04-24T15:31:49.746 INFO:tasks.ceph.mgr.y.smithi096.stderr:2024-04-24T15:31:49.747+0000 7f1e8451f1c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T15:31:49.787 INFO:teuthology.orchestra.run.smithi151.stdout:1 2024-04-24T15:31:49.800 DEBUG:teuthology.orchestra.run.smithi151:> sudo ceph --cluster ceph osd new 46de3f65-87a8-460c-b57a-4468d9c89a57 2 2024-04-24T15:31:50.009 INFO:teuthology.orchestra.run.smithi151.stdout:2 2024-04-24T15:31:50.022 DEBUG:teuthology.orchestra.run.smithi151:> sudo ceph --cluster ceph osd new 5507e063-10bc-4e32-b1ec-00aa3e47d993 3 2024-04-24T15:31:50.081 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-04-24T15:31:50.082+0000 7f1bf5dc51c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T15:31:50.225 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-04-24T15:31:50.225+0000 7f1bf5dc51c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T15:31:50.234 INFO:teuthology.orchestra.run.smithi151.stdout:3 2024-04-24T15:31:50.247 DEBUG:teuthology.orchestra.run.smithi151:> sudo ceph --cluster ceph osd new 6a74d453-1048-44ef-a5b0-4277583d76c2 4 2024-04-24T15:31:50.460 INFO:teuthology.orchestra.run.smithi151.stdout:4 2024-04-24T15:31:50.463 INFO:tasks.ceph.mgr.y.smithi096.stderr:2024-04-24T15:31:50.464+0000 7f1e8451f1c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T15:31:50.472 DEBUG:teuthology.orchestra.run.smithi151:> sudo ceph --cluster ceph osd new 3645ef18-5574-49f8-ab81-b40651208309 5 2024-04-24T15:31:50.533 INFO:tasks.ceph.mgr.y.smithi096.stderr:2024-04-24T15:31:50.534+0000 7f1e8451f1c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T15:31:50.597 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-04-24T15:31:50.597+0000 7f1bf5dc51c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T15:31:50.665 INFO:tasks.ceph.mgr.y.smithi096.stderr:2024-04-24T15:31:50.665+0000 7f1e8451f1c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T15:31:50.681 INFO:teuthology.orchestra.run.smithi151.stdout:5 2024-04-24T15:31:50.693 DEBUG:teuthology.orchestra.run.smithi151:> sudo ceph --cluster ceph osd new 6b8b0b36-9246-4241-a197-c69fa48a6202 6 2024-04-24T15:31:50.731 INFO:tasks.ceph.mgr.y.smithi096.stderr:2024-04-24T15:31:50.732+0000 7f1e8451f1c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T15:31:50.904 INFO:teuthology.orchestra.run.smithi151.stdout:6 2024-04-24T15:31:50.908 INFO:tasks.ceph.mgr.y.smithi096.stderr:2024-04-24T15:31:50.909+0000 7f1e8451f1c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T15:31:50.917 DEBUG:teuthology.orchestra.run.smithi151:> sudo ceph --cluster ceph osd new 4501cd9f-e437-4ddd-af3d-129fdf1ee61a 7 2024-04-24T15:31:50.984 INFO:tasks.ceph.mgr.y.smithi096.stderr:2024-04-24T15:31:50.985+0000 7f1e8451f1c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T15:31:51.058 INFO:tasks.ceph.mgr.y.smithi096.stderr:2024-04-24T15:31:51.059+0000 7f1e8451f1c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T15:31:51.127 INFO:teuthology.orchestra.run.smithi151.stdout:7 2024-04-24T15:31:51.140 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-24T15:31:51.140 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-24T15:31:51.142 INFO:tasks.ceph.osd.0:Started 2024-04-24T15:31:51.142 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-24T15:31:51.142 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-24T15:31:51.144 INFO:tasks.ceph.osd.1:Started 2024-04-24T15:31:51.144 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-24T15:31:51.144 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-24T15:31:51.146 INFO:tasks.ceph.osd.2:Started 2024-04-24T15:31:51.146 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-24T15:31:51.146 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-24T15:31:51.148 INFO:tasks.ceph.osd.3:Started 2024-04-24T15:31:51.148 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-24T15:31:51.149 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-24T15:31:51.149 INFO:tasks.ceph.mgr.y.smithi096.stderr:2024-04-24T15:31:51.149+0000 7f1e8451f1c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T15:31:51.151 INFO:tasks.ceph.osd.4:Started 2024-04-24T15:31:51.151 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-24T15:31:51.151 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-24T15:31:51.153 INFO:tasks.ceph.osd.5:Started 2024-04-24T15:31:51.153 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-24T15:31:51.153 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-24T15:31:51.155 INFO:tasks.ceph.osd.6:Started 2024-04-24T15:31:51.155 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-24T15:31:51.155 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-24T15:31:51.157 INFO:tasks.ceph.osd.7:Started 2024-04-24T15:31:51.158 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T15:31:51.378 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-04-24T15:31:51.378+0000 7f1bf5dc51c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T15:31:51.380 INFO:tasks.ceph.mgr.y.smithi096.stderr:2024-04-24T15:31:51.381+0000 7f1e8451f1c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T15:31:51.398 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:31:51.398 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":10,"fsid":"253d426c-62cb-4194-8514-6cd970fd85c7","created":"2024-04-24T15:31:47.870327+0000","modified":"2024-04-24T15:31:51.124910+0000","last_up_change":"0.000000","last_in_change":"2024-04-24T15:31:51.124910+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"92941fb9-96ee-4b2b-8147-3c1d61192f94","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"628e868a-9884-42e6-9331-caef97cb6967","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"46de3f65-87a8-460c-b57a-4468d9c89a57","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"5507e063-10bc-4e32-b1ec-00aa3e47d993","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"6a74d453-1048-44ef-a5b0-4277583d76c2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"3645ef18-5574-49f8-ab81-b40651208309","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"6b8b0b36-9246-4241-a197-c69fa48a6202","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"4501cd9f-e437-4ddd-af3d-129fdf1ee61a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T15:31:51.411 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-24T15:31:51.411 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-24T15:31:51.449 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-04-24T15:31:51.449+0000 7f1bf5dc51c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T15:31:51.462 INFO:tasks.ceph.mgr.y.smithi096.stderr:2024-04-24T15:31:51.462+0000 7f1e8451f1c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T15:31:51.582 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-04-24T15:31:51.582+0000 7f1bf5dc51c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T15:31:51.650 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-04-24T15:31:51.650+0000 7f1bf5dc51c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T15:31:51.746 INFO:tasks.ceph.mgr.y.smithi096.stderr:2024-04-24T15:31:51.747+0000 7f1e8451f1c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T15:31:51.749 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:31:51.750+0000 7f4f9b2b4540 -1 Falling back to public interface 2024-04-24T15:31:51.750 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:31:51.750+0000 7fc8ac79f540 -1 Falling back to public interface 2024-04-24T15:31:51.752 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:31:51.753+0000 7f80839a7540 -1 Falling back to public interface 2024-04-24T15:31:51.755 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:31:51.755+0000 7f7e373ad540 -1 Falling back to public interface 2024-04-24T15:31:51.765 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:31:51.765+0000 7f27d7d47540 -1 Falling back to public interface 2024-04-24T15:31:51.766 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:31:51.767+0000 7f296cbe6540 -1 Falling back to public interface 2024-04-24T15:31:51.778 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:31:51.779+0000 7faf159b4540 -1 Falling back to public interface 2024-04-24T15:31:51.783 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:31:51.783+0000 7f23b40ef540 -1 Falling back to public interface 2024-04-24T15:31:51.819 INFO:tasks.ceph.mgr.y.smithi096.stderr:2024-04-24T15:31:51.820+0000 7f1e8451f1c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T15:31:51.834 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-04-24T15:31:51.834+0000 7f1bf5dc51c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T15:31:51.912 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-04-24T15:31:51.912+0000 7f1bf5dc51c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T15:31:51.987 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-04-24T15:31:51.987+0000 7f1bf5dc51c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T15:31:52.080 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-04-24T15:31:52.080+0000 7f1bf5dc51c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T15:31:52.217 DEBUG:teuthology.orchestra.run.smithi096:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-24T15:31:52.258 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-04-24T15:31:52.258+0000 7f1bf5dc51c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T15:31:52.326 INFO:tasks.ceph.mgr.y.smithi096.stderr:2024-04-24T15:31:52.327+0000 7f1e8451f1c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T15:31:52.340 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-04-24T15:31:52.340+0000 7f1bf5dc51c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T15:31:52.406 INFO:teuthology.misc.health.smithi096.stdout: 2024-04-24T15:31:52.406 INFO:teuthology.misc.health.smithi096.stdout:{"epoch":10,"fsid":"253d426c-62cb-4194-8514-6cd970fd85c7","created":"2024-04-24T15:31:47.870327+0000","modified":"2024-04-24T15:31:51.124910+0000","last_up_change":"0.000000","last_in_change":"2024-04-24T15:31:51.124910+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"92941fb9-96ee-4b2b-8147-3c1d61192f94","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"628e868a-9884-42e6-9331-caef97cb6967","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"46de3f65-87a8-460c-b57a-4468d9c89a57","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"5507e063-10bc-4e32-b1ec-00aa3e47d993","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"6a74d453-1048-44ef-a5b0-4277583d76c2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"3645ef18-5574-49f8-ab81-b40651208309","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"6b8b0b36-9246-4241-a197-c69fa48a6202","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"4501cd9f-e437-4ddd-af3d-129fdf1ee61a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T15:31:52.417 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-24T15:31:52.629 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-04-24T15:31:52.630+0000 7f1bf5dc51c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T15:31:52.645 INFO:tasks.ceph.mgr.y.smithi096.stderr:2024-04-24T15:31:52.646+0000 7f1e8451f1c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T15:31:52.700 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-04-24T15:31:52.700+0000 7f1bf5dc51c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T15:31:52.784 INFO:tasks.ceph.mgr.y.smithi096.stderr:2024-04-24T15:31:52.785+0000 7f1e8451f1c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T15:31:53.140 INFO:tasks.ceph.mgr.y.smithi096.stderr:2024-04-24T15:31:53.141+0000 7f1e8451f1c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T15:31:53.209 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-04-24T15:31:53.209+0000 7f1bf5dc51c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T15:31:53.281 INFO:tasks.ceph.mgr.y.smithi096.stderr:2024-04-24T15:31:53.282+0000 7f1e8451f1c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T15:31:53.429 INFO:tasks.ceph.mgr.y.smithi096.stderr:2024-04-24T15:31:53.430+0000 7f1e8451f1c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T15:31:53.526 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-04-24T15:31:53.526+0000 7f1bf5dc51c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T15:31:53.527 INFO:tasks.ceph.mgr.y.smithi096.stderr:2024-04-24T15:31:53.528+0000 7f1e8451f1c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T15:31:53.672 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-04-24T15:31:53.672+0000 7f1bf5dc51c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T15:31:54.033 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-04-24T15:31:54.033+0000 7f1bf5dc51c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T15:31:54.177 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-04-24T15:31:54.178+0000 7f1bf5dc51c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T15:31:54.335 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-04-24T15:31:54.335+0000 7f1bf5dc51c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T15:31:54.436 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-04-24T15:31:54.436+0000 7f1bf5dc51c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T15:31:54.562 INFO:tasks.ceph.mgr.y.smithi096.stderr:2024-04-24T15:31:54.563+0000 7f1e48c23700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-24T15:31:54.566 INFO:tasks.ceph.mgr.y.smithi096.stderr:2024-04-24T15:31:54.567+0000 7f1e48c23700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-24T15:31:55.214 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:31:55.215+0000 7f80839a7540 -1 osd.2 0 log_to_monitors true 2024-04-24T15:31:55.238 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:31:55.238+0000 7fc8ac79f540 -1 osd.5 0 log_to_monitors true 2024-04-24T15:31:55.248 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:31:55.249+0000 7f4f9b2b4540 -1 osd.1 0 log_to_monitors true 2024-04-24T15:31:55.274 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:31:55.274+0000 7f7e373ad540 -1 osd.7 0 log_to_monitors true 2024-04-24T15:31:55.277 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:31:55.278+0000 7f296cbe6540 -1 osd.0 0 log_to_monitors true 2024-04-24T15:31:55.280 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:31:55.280+0000 7f23b40ef540 -1 osd.6 0 log_to_monitors true 2024-04-24T15:31:55.296 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:31:55.296+0000 7faf159b4540 -1 osd.3 0 log_to_monitors true 2024-04-24T15:31:55.297 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:31:55.298+0000 7f27d7d47540 -1 osd.4 0 log_to_monitors true 2024-04-24T15:31:57.833 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:31:57.833+0000 7faf0c5b9700 -1 osd.3 0 waiting for initial osdmap 2024-04-24T15:31:57.837 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:31:57.837+0000 7faf053b0700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T15:31:57.882 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:31:57.882+0000 7f29637eb700 -1 osd.0 0 waiting for initial osdmap 2024-04-24T15:31:57.889 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:31:57.889+0000 7f295c5e2700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T15:31:57.907 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:31:57.907+0000 7f4f91eb9700 -1 osd.1 0 waiting for initial osdmap 2024-04-24T15:31:57.908 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:31:57.909+0000 7f7e2dfb2700 -1 osd.7 0 waiting for initial osdmap 2024-04-24T15:31:57.909 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:31:57.909+0000 7f23aacf4700 -1 osd.6 0 waiting for initial osdmap 2024-04-24T15:31:57.913 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:31:57.913+0000 7f27ce94c700 -1 osd.4 0 waiting for initial osdmap 2024-04-24T15:31:57.913 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:31:57.913+0000 7f7e26da9700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T15:31:57.913 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:31:57.913+0000 7f4f8acb0700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T15:31:57.916 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:31:57.917+0000 7f23a3aeb700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T15:31:57.917 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:31:57.917+0000 7f27c7743700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T15:31:57.939 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:31:57.939+0000 7fc8a33a4700 -1 osd.5 0 waiting for initial osdmap 2024-04-24T15:31:57.942 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:31:57.942+0000 7fc89c19b700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T15:31:58.204 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:31:58.204+0000 7f807a5ac700 -1 osd.2 0 waiting for initial osdmap 2024-04-24T15:31:58.207 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:31:58.207+0000 7f80733a3700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T15:31:59.224 DEBUG:teuthology.orchestra.run.smithi096:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-24T15:31:59.536 INFO:teuthology.misc.health.smithi096.stdout: 2024-04-24T15:31:59.640 INFO:teuthology.misc.health.smithi096.stdout:{"epoch":13,"fsid":"253d426c-62cb-4194-8514-6cd970fd85c7","created":"2024-04-24T15:31:47.870327+0000","modified":"2024-04-24T15:31:58.839279+0000","last_up_change":"2024-04-24T15:31:58.839279+0000","last_in_change":"2024-04-24T15:31:51.124910+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"92941fb9-96ee-4b2b-8147-3c1d61192f94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":44106},{"type":"v1","addr":"172.21.15.96:6817","nonce":44106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":44106},{"type":"v1","addr":"172.21.15.96:6819","nonce":44106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":44106},{"type":"v1","addr":"172.21.15.96:6823","nonce":44106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":44106},{"type":"v1","addr":"172.21.15.96:6821","nonce":44106}]},"public_addr":"172.21.15.96:6817/44106","cluster_addr":"172.21.15.96:6819/44106","heartbeat_back_addr":"172.21.15.96:6823/44106","heartbeat_front_addr":"172.21.15.96:6821/44106","state":["exists","up"]},{"osd":1,"uuid":"628e868a-9884-42e6-9331-caef97cb6967","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":44089},{"type":"v1","addr":"172.21.15.96:6801","nonce":44089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":44089},{"type":"v1","addr":"172.21.15.96:6803","nonce":44089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":44089},{"type":"v1","addr":"172.21.15.96:6807","nonce":44089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":44089},{"type":"v1","addr":"172.21.15.96:6805","nonce":44089}]},"public_addr":"172.21.15.96:6801/44089","cluster_addr":"172.21.15.96:6803/44089","heartbeat_back_addr":"172.21.15.96:6807/44089","heartbeat_front_addr":"172.21.15.96:6805/44089","state":["exists","up"]},{"osd":2,"uuid":"46de3f65-87a8-460c-b57a-4468d9c89a57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":44090},{"type":"v1","addr":"172.21.15.96:6809","nonce":44090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":44090},{"type":"v1","addr":"172.21.15.96:6811","nonce":44090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":44090},{"type":"v1","addr":"172.21.15.96:6815","nonce":44090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":44090},{"type":"v1","addr":"172.21.15.96:6813","nonce":44090}]},"public_addr":"172.21.15.96:6809/44090","cluster_addr":"172.21.15.96:6811/44090","heartbeat_back_addr":"172.21.15.96:6815/44090","heartbeat_front_addr":"172.21.15.96:6813/44090","state":["exists","up"]},{"osd":3,"uuid":"5507e063-10bc-4e32-b1ec-00aa3e47d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":44121},{"type":"v1","addr":"172.21.15.96:6825","nonce":44121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":44121},{"type":"v1","addr":"172.21.15.96:6827","nonce":44121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":44121},{"type":"v1","addr":"172.21.15.96:6831","nonce":44121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":44121},{"type":"v1","addr":"172.21.15.96:6829","nonce":44121}]},"public_addr":"172.21.15.96:6825/44121","cluster_addr":"172.21.15.96:6827/44121","heartbeat_back_addr":"172.21.15.96:6831/44121","heartbeat_front_addr":"172.21.15.96:6829/44121","state":["exists","up"]},{"osd":4,"uuid":"6a74d453-1048-44ef-a5b0-4277583d76c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":44025},{"type":"v1","addr":"172.21.15.151:6817","nonce":44025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":44025},{"type":"v1","addr":"172.21.15.151:6819","nonce":44025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":44025},{"type":"v1","addr":"172.21.15.151:6823","nonce":44025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":44025},{"type":"v1","addr":"172.21.15.151:6821","nonce":44025}]},"public_addr":"172.21.15.151:6817/44025","cluster_addr":"172.21.15.151:6819/44025","heartbeat_back_addr":"172.21.15.151:6823/44025","heartbeat_front_addr":"172.21.15.151:6821/44025","state":["exists","up"]},{"osd":5,"uuid":"3645ef18-5574-49f8-ab81-b40651208309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":44023},{"type":"v1","addr":"172.21.15.151:6801","nonce":44023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":44023},{"type":"v1","addr":"172.21.15.151:6803","nonce":44023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":44023},{"type":"v1","addr":"172.21.15.151:6807","nonce":44023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":44023},{"type":"v1","addr":"172.21.15.151:6805","nonce":44023}]},"public_addr":"172.21.15.151:6801/44023","cluster_addr":"172.21.15.151:6803/44023","heartbeat_back_addr":"172.21.15.151:6807/44023","heartbeat_front_addr":"172.21.15.151:6805/44023","state":["exists","up"]},{"osd":6,"uuid":"6b8b0b36-9246-4241-a197-c69fa48a6202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":44057},{"type":"v1","addr":"172.21.15.151:6825","nonce":44057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":44057},{"type":"v1","addr":"172.21.15.151:6827","nonce":44057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":44057},{"type":"v1","addr":"172.21.15.151:6831","nonce":44057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":44057},{"type":"v1","addr":"172.21.15.151:6829","nonce":44057}]},"public_addr":"172.21.15.151:6825/44057","cluster_addr":"172.21.15.151:6827/44057","heartbeat_back_addr":"172.21.15.151:6831/44057","heartbeat_front_addr":"172.21.15.151:6829/44057","state":["exists","up"]},{"osd":7,"uuid":"4501cd9f-e437-4ddd-af3d-129fdf1ee61a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":44024},{"type":"v1","addr":"172.21.15.151:6809","nonce":44024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":44024},{"type":"v1","addr":"172.21.15.151:6811","nonce":44024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":44024},{"type":"v1","addr":"172.21.15.151:6815","nonce":44024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":44024},{"type":"v1","addr":"172.21.15.151:6813","nonce":44024}]},"public_addr":"172.21.15.151:6809/44024","cluster_addr":"172.21.15.151:6811/44024","heartbeat_back_addr":"172.21.15.151:6815/44024","heartbeat_front_addr":"172.21.15.151:6813/44024","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T15:31:59.642 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-24T15:31:59.642 INFO:tasks.ceph:Creating RBD pool 2024-04-24T15:31:59.642 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --cluster ceph osd pool create rbd 8 2024-04-24T15:32:00.949 INFO:teuthology.orchestra.run.smithi096.stderr:pool 'rbd' created 2024-04-24T15:32:00.964 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2024-04-24T15:32:01.953 INFO:teuthology.orchestra.run.smithi096.stderr:enabled application 'rbd' on pool 'rbd' 2024-04-24T15:32:01.961 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-24T15:32:01.961 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-24T15:32:01.962 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-24T15:32:01.962 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-24T15:32:01.962 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-24T15:32:02.333 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:32:02.345 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":4,"flags":0,"active_gid":4100,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":43785},{"type":"v1","addr":"172.21.15.96:6833","nonce":43785}]},"active_addr":"172.21.15.96:6833/43785","active_change":"2024-04-24T15:31:53.541966+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":4111,"name":"x","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.96:0","nonce":1884602651}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.96:0","nonce":3699403259}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.96:0","nonce":175180183}]}]} 2024-04-24T15:32:02.351 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-24T15:32:02.351 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-24T15:32:02.351 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T15:32:02.668 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:32:02.668 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":16,"fsid":"253d426c-62cb-4194-8514-6cd970fd85c7","created":"2024-04-24T15:31:47.870327+0000","modified":"2024-04-24T15:32:01.940168+0000","last_up_change":"2024-04-24T15:31:58.839279+0000","last_in_change":"2024-04-24T15:31:51.124910+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T15:31:59.589762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-24T15:31:59.971601+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"92941fb9-96ee-4b2b-8147-3c1d61192f94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":44106},{"type":"v1","addr":"172.21.15.96:6817","nonce":44106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":44106},{"type":"v1","addr":"172.21.15.96:6819","nonce":44106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":44106},{"type":"v1","addr":"172.21.15.96:6823","nonce":44106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":44106},{"type":"v1","addr":"172.21.15.96:6821","nonce":44106}]},"public_addr":"172.21.15.96:6817/44106","cluster_addr":"172.21.15.96:6819/44106","heartbeat_back_addr":"172.21.15.96:6823/44106","heartbeat_front_addr":"172.21.15.96:6821/44106","state":["exists","up"]},{"osd":1,"uuid":"628e868a-9884-42e6-9331-caef97cb6967","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":44089},{"type":"v1","addr":"172.21.15.96:6801","nonce":44089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":44089},{"type":"v1","addr":"172.21.15.96:6803","nonce":44089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":44089},{"type":"v1","addr":"172.21.15.96:6807","nonce":44089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":44089},{"type":"v1","addr":"172.21.15.96:6805","nonce":44089}]},"public_addr":"172.21.15.96:6801/44089","cluster_addr":"172.21.15.96:6803/44089","heartbeat_back_addr":"172.21.15.96:6807/44089","heartbeat_front_addr":"172.21.15.96:6805/44089","state":["exists","up"]},{"osd":2,"uuid":"46de3f65-87a8-460c-b57a-4468d9c89a57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":44090},{"type":"v1","addr":"172.21.15.96:6809","nonce":44090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":44090},{"type":"v1","addr":"172.21.15.96:6811","nonce":44090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":44090},{"type":"v1","addr":"172.21.15.96:6815","nonce":44090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":44090},{"type":"v1","addr":"172.21.15.96:6813","nonce":44090}]},"public_addr":"172.21.15.96:6809/44090","cluster_addr":"172.21.15.96:6811/44090","heartbeat_back_addr":"172.21.15.96:6815/44090","heartbeat_front_addr":"172.21.15.96:6813/44090","state":["exists","up"]},{"osd":3,"uuid":"5507e063-10bc-4e32-b1ec-00aa3e47d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":44121},{"type":"v1","addr":"172.21.15.96:6825","nonce":44121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":44121},{"type":"v1","addr":"172.21.15.96:6827","nonce":44121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":44121},{"type":"v1","addr":"172.21.15.96:6831","nonce":44121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":44121},{"type":"v1","addr":"172.21.15.96:6829","nonce":44121}]},"public_addr":"172.21.15.96:6825/44121","cluster_addr":"172.21.15.96:6827/44121","heartbeat_back_addr":"172.21.15.96:6831/44121","heartbeat_front_addr":"172.21.15.96:6829/44121","state":["exists","up"]},{"osd":4,"uuid":"6a74d453-1048-44ef-a5b0-4277583d76c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":44025},{"type":"v1","addr":"172.21.15.151:6817","nonce":44025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":44025},{"type":"v1","addr":"172.21.15.151:6819","nonce":44025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":44025},{"type":"v1","addr":"172.21.15.151:6823","nonce":44025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":44025},{"type":"v1","addr":"172.21.15.151:6821","nonce":44025}]},"public_addr":"172.21.15.151:6817/44025","cluster_addr":"172.21.15.151:6819/44025","heartbeat_back_addr":"172.21.15.151:6823/44025","heartbeat_front_addr":"172.21.15.151:6821/44025","state":["exists","up"]},{"osd":5,"uuid":"3645ef18-5574-49f8-ab81-b40651208309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":44023},{"type":"v1","addr":"172.21.15.151:6801","nonce":44023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":44023},{"type":"v1","addr":"172.21.15.151:6803","nonce":44023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":44023},{"type":"v1","addr":"172.21.15.151:6807","nonce":44023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":44023},{"type":"v1","addr":"172.21.15.151:6805","nonce":44023}]},"public_addr":"172.21.15.151:6801/44023","cluster_addr":"172.21.15.151:6803/44023","heartbeat_back_addr":"172.21.15.151:6807/44023","heartbeat_front_addr":"172.21.15.151:6805/44023","state":["exists","up"]},{"osd":6,"uuid":"6b8b0b36-9246-4241-a197-c69fa48a6202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":44057},{"type":"v1","addr":"172.21.15.151:6825","nonce":44057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":44057},{"type":"v1","addr":"172.21.15.151:6827","nonce":44057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":44057},{"type":"v1","addr":"172.21.15.151:6831","nonce":44057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":44057},{"type":"v1","addr":"172.21.15.151:6829","nonce":44057}]},"public_addr":"172.21.15.151:6825/44057","cluster_addr":"172.21.15.151:6827/44057","heartbeat_back_addr":"172.21.15.151:6831/44057","heartbeat_front_addr":"172.21.15.151:6829/44057","state":["exists","up"]},{"osd":7,"uuid":"4501cd9f-e437-4ddd-af3d-129fdf1ee61a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":44024},{"type":"v1","addr":"172.21.15.151:6809","nonce":44024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":44024},{"type":"v1","addr":"172.21.15.151:6811","nonce":44024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":44024},{"type":"v1","addr":"172.21.15.151:6815","nonce":44024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":44024},{"type":"v1","addr":"172.21.15.151:6813","nonce":44024}]},"public_addr":"172.21.15.151:6809/44024","cluster_addr":"172.21.15.151:6811/44024","heartbeat_back_addr":"172.21.15.151:6815/44024","heartbeat_front_addr":"172.21.15.151:6813/44024","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T15:31:56.300910+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T15:31:56.202064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T15:31:56.198961+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T15:31:56.346777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T15:31:56.282585+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T15:31:56.257943+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T15:31:56.260479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T15:31:56.242093+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T15:32:02.681 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-24T15:32:02.682 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T15:32:02.999 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:32:03.000 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":17,"fsid":"253d426c-62cb-4194-8514-6cd970fd85c7","created":"2024-04-24T15:31:47.870327+0000","modified":"2024-04-24T15:32:02.948192+0000","last_up_change":"2024-04-24T15:31:58.839279+0000","last_in_change":"2024-04-24T15:31:51.124910+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T15:31:59.589762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-24T15:31:59.971601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"92941fb9-96ee-4b2b-8147-3c1d61192f94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":44106},{"type":"v1","addr":"172.21.15.96:6817","nonce":44106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":44106},{"type":"v1","addr":"172.21.15.96:6819","nonce":44106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":44106},{"type":"v1","addr":"172.21.15.96:6823","nonce":44106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":44106},{"type":"v1","addr":"172.21.15.96:6821","nonce":44106}]},"public_addr":"172.21.15.96:6817/44106","cluster_addr":"172.21.15.96:6819/44106","heartbeat_back_addr":"172.21.15.96:6823/44106","heartbeat_front_addr":"172.21.15.96:6821/44106","state":["exists","up"]},{"osd":1,"uuid":"628e868a-9884-42e6-9331-caef97cb6967","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":44089},{"type":"v1","addr":"172.21.15.96:6801","nonce":44089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":44089},{"type":"v1","addr":"172.21.15.96:6803","nonce":44089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":44089},{"type":"v1","addr":"172.21.15.96:6807","nonce":44089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":44089},{"type":"v1","addr":"172.21.15.96:6805","nonce":44089}]},"public_addr":"172.21.15.96:6801/44089","cluster_addr":"172.21.15.96:6803/44089","heartbeat_back_addr":"172.21.15.96:6807/44089","heartbeat_front_addr":"172.21.15.96:6805/44089","state":["exists","up"]},{"osd":2,"uuid":"46de3f65-87a8-460c-b57a-4468d9c89a57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":44090},{"type":"v1","addr":"172.21.15.96:6809","nonce":44090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":44090},{"type":"v1","addr":"172.21.15.96:6811","nonce":44090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":44090},{"type":"v1","addr":"172.21.15.96:6815","nonce":44090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":44090},{"type":"v1","addr":"172.21.15.96:6813","nonce":44090}]},"public_addr":"172.21.15.96:6809/44090","cluster_addr":"172.21.15.96:6811/44090","heartbeat_back_addr":"172.21.15.96:6815/44090","heartbeat_front_addr":"172.21.15.96:6813/44090","state":["exists","up"]},{"osd":3,"uuid":"5507e063-10bc-4e32-b1ec-00aa3e47d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":44121},{"type":"v1","addr":"172.21.15.96:6825","nonce":44121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":44121},{"type":"v1","addr":"172.21.15.96:6827","nonce":44121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":44121},{"type":"v1","addr":"172.21.15.96:6831","nonce":44121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":44121},{"type":"v1","addr":"172.21.15.96:6829","nonce":44121}]},"public_addr":"172.21.15.96:6825/44121","cluster_addr":"172.21.15.96:6827/44121","heartbeat_back_addr":"172.21.15.96:6831/44121","heartbeat_front_addr":"172.21.15.96:6829/44121","state":["exists","up"]},{"osd":4,"uuid":"6a74d453-1048-44ef-a5b0-4277583d76c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":44025},{"type":"v1","addr":"172.21.15.151:6817","nonce":44025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":44025},{"type":"v1","addr":"172.21.15.151:6819","nonce":44025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":44025},{"type":"v1","addr":"172.21.15.151:6823","nonce":44025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":44025},{"type":"v1","addr":"172.21.15.151:6821","nonce":44025}]},"public_addr":"172.21.15.151:6817/44025","cluster_addr":"172.21.15.151:6819/44025","heartbeat_back_addr":"172.21.15.151:6823/44025","heartbeat_front_addr":"172.21.15.151:6821/44025","state":["exists","up"]},{"osd":5,"uuid":"3645ef18-5574-49f8-ab81-b40651208309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":44023},{"type":"v1","addr":"172.21.15.151:6801","nonce":44023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":44023},{"type":"v1","addr":"172.21.15.151:6803","nonce":44023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":44023},{"type":"v1","addr":"172.21.15.151:6807","nonce":44023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":44023},{"type":"v1","addr":"172.21.15.151:6805","nonce":44023}]},"public_addr":"172.21.15.151:6801/44023","cluster_addr":"172.21.15.151:6803/44023","heartbeat_back_addr":"172.21.15.151:6807/44023","heartbeat_front_addr":"172.21.15.151:6805/44023","state":["exists","up"]},{"osd":6,"uuid":"6b8b0b36-9246-4241-a197-c69fa48a6202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":44057},{"type":"v1","addr":"172.21.15.151:6825","nonce":44057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":44057},{"type":"v1","addr":"172.21.15.151:6827","nonce":44057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":44057},{"type":"v1","addr":"172.21.15.151:6831","nonce":44057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":44057},{"type":"v1","addr":"172.21.15.151:6829","nonce":44057}]},"public_addr":"172.21.15.151:6825/44057","cluster_addr":"172.21.15.151:6827/44057","heartbeat_back_addr":"172.21.15.151:6831/44057","heartbeat_front_addr":"172.21.15.151:6829/44057","state":["exists","up"]},{"osd":7,"uuid":"4501cd9f-e437-4ddd-af3d-129fdf1ee61a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":44024},{"type":"v1","addr":"172.21.15.151:6809","nonce":44024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":44024},{"type":"v1","addr":"172.21.15.151:6811","nonce":44024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":44024},{"type":"v1","addr":"172.21.15.151:6815","nonce":44024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":44024},{"type":"v1","addr":"172.21.15.151:6813","nonce":44024}]},"public_addr":"172.21.15.151:6809/44024","cluster_addr":"172.21.15.151:6811/44024","heartbeat_back_addr":"172.21.15.151:6815/44024","heartbeat_front_addr":"172.21.15.151:6813/44024","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T15:31:56.300910+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T15:31:56.202064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T15:31:56.198961+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T15:31:56.346777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T15:31:56.282585+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T15:31:56.257943+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T15:31:56.260479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T15:31:56.242093+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T15:32:03.013 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-24T15:32:03.014 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-24T15:32:03.014 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-24T15:32:03.015 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-24T15:32:03.015 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-24T15:32:03.016 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-24T15:32:03.016 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-24T15:32:03.017 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-24T15:32:03.275 INFO:teuthology.orchestra.run.smithi096.stdout:55834574850 2024-04-24T15:32:03.275 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-24T15:32:03.299 INFO:teuthology.orchestra.run.smithi096.stdout:55834574851 2024-04-24T15:32:03.299 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-24T15:32:03.301 INFO:teuthology.orchestra.run.smithi096.stdout:55834574852 2024-04-24T15:32:03.301 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-24T15:32:03.302 INFO:teuthology.orchestra.run.smithi096.stdout:55834574851 2024-04-24T15:32:03.303 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-24T15:32:03.304 INFO:teuthology.orchestra.run.smithi096.stdout:55834574851 2024-04-24T15:32:03.304 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-24T15:32:03.308 INFO:teuthology.orchestra.run.smithi096.stdout:55834574851 2024-04-24T15:32:03.308 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-24T15:32:03.313 INFO:teuthology.orchestra.run.smithi096.stdout:55834574851 2024-04-24T15:32:03.313 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-24T15:32:03.525 INFO:teuthology.orchestra.run.smithi096.stdout:55834574851 2024-04-24T15:32:03.526 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-24T15:32:03.850 INFO:teuthology.orchestra.run.smithi096.stdout:55834574850 2024-04-24T15:32:03.877 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.6 2024-04-24T15:32:03.896 INFO:teuthology.orchestra.run.smithi096.stdout:55834574851 2024-04-24T15:32:03.905 INFO:teuthology.orchestra.run.smithi096.stdout:55834574849 2024-04-24T15:32:03.905 INFO:teuthology.orchestra.run.smithi096.stdout:55834574850 2024-04-24T15:32:03.921 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.4 2024-04-24T15:32:03.922 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574849 for osd.1 2024-04-24T15:32:03.928 INFO:teuthology.orchestra.run.smithi096.stdout:55834574850 2024-04-24T15:32:03.930 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.7 2024-04-24T15:32:03.933 INFO:teuthology.orchestra.run.smithi096.stdout:55834574850 2024-04-24T15:32:03.949 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.5 2024-04-24T15:32:03.951 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.0 2024-04-24T15:32:03.975 INFO:teuthology.orchestra.run.smithi096.stdout:55834574851 2024-04-24T15:32:03.989 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.2 2024-04-24T15:32:03.989 DEBUG:teuthology.parallel:result is None 2024-04-24T15:32:04.058 INFO:teuthology.orchestra.run.smithi096.stdout:55834574851 2024-04-24T15:32:04.072 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.3 2024-04-24T15:32:04.072 DEBUG:teuthology.parallel:result is None 2024-04-24T15:32:04.877 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-24T15:32:04.923 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-24T15:32:04.923 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-24T15:32:04.931 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-24T15:32:04.950 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-24T15:32:04.952 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-24T15:32:05.317 INFO:teuthology.orchestra.run.smithi096.stdout:55834574852 2024-04-24T15:32:05.332 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.4 2024-04-24T15:32:05.332 DEBUG:teuthology.parallel:result is None 2024-04-24T15:32:05.369 INFO:teuthology.orchestra.run.smithi096.stdout:55834574851 2024-04-24T15:32:05.371 INFO:teuthology.orchestra.run.smithi096.stdout:55834574851 2024-04-24T15:32:05.386 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.7 2024-04-24T15:32:05.387 DEBUG:teuthology.parallel:result is None 2024-04-24T15:32:05.387 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.6 2024-04-24T15:32:05.387 DEBUG:teuthology.parallel:result is None 2024-04-24T15:32:05.451 INFO:teuthology.orchestra.run.smithi096.stdout:55834574851 2024-04-24T15:32:05.465 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.0 2024-04-24T15:32:05.465 DEBUG:teuthology.parallel:result is None 2024-04-24T15:32:05.468 INFO:teuthology.orchestra.run.smithi096.stdout:55834574850 2024-04-24T15:32:05.482 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574850 for osd.1 2024-04-24T15:32:05.482 DEBUG:teuthology.parallel:result is None 2024-04-24T15:32:05.493 INFO:teuthology.orchestra.run.smithi096.stdout:55834574851 2024-04-24T15:32:05.506 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.5 2024-04-24T15:32:05.506 DEBUG:teuthology.parallel:result is None 2024-04-24T15:32:05.507 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-24T15:32:05.507 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T15:32:05.859 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:32:05.861 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-04-24T15:32:05.872 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":14,"stamp":"2024-04-24T15:32:05.550440+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":13,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2380276,"kb_used_data":1352,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":752594444,"statfs":{"total":773094113280,"available":770656710656,"internally_reserved":0,"allocated":1384448,"data_stored":1570836,"data_compressed":9684,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000459"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":8,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-24T15:32:01.952204+0000","last_change":"2024-04-24T15:32:01.952204+0000","last_active":"2024-04-24T15:32:01.952204+0000","last_peered":"2024-04-24T15:32:01.952204+0000","last_clean":"2024-04-24T15:32:01.952204+0000","last_became_active":"2024-04-24T15:32:01.951968+0000","last_became_peered":"2024-04-24T15:32:01.951968+0000","last_unstale":"2024-04-24T15:32:01.952204+0000","last_undegraded":"2024-04-24T15:32:01.952204+0000","last_fullsized":"2024-04-24T15:32:01.952204+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:40:51.884922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":8,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-24T15:32:01.950055+0000","last_change":"2024-04-24T15:32:01.950055+0000","last_active":"2024-04-24T15:32:01.950055+0000","last_peered":"2024-04-24T15:32:01.950055+0000","last_clean":"2024-04-24T15:32:01.950055+0000","last_became_active":"2024-04-24T15:32:01.949783+0000","last_became_peered":"2024-04-24T15:32:01.949783+0000","last_unstale":"2024-04-24T15:32:01.950055+0000","last_undegraded":"2024-04-24T15:32:01.950055+0000","last_fullsized":"2024-04-24T15:32:01.950055+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:25:05.065671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-24T15:32:01.956639+0000","last_change":"2024-04-24T15:32:01.953602+0000","last_active":"2024-04-24T15:32:01.956639+0000","last_peered":"2024-04-24T15:32:01.956639+0000","last_clean":"2024-04-24T15:32:01.956639+0000","last_became_active":"2024-04-24T15:32:01.953388+0000","last_became_peered":"2024-04-24T15:32:01.953388+0000","last_unstale":"2024-04-24T15:32:01.956639+0000","last_undegraded":"2024-04-24T15:32:01.956639+0000","last_fullsized":"2024-04-24T15:32:01.956639+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:17:16.496379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-24T15:32:01.956596+0000","last_change":"2024-04-24T15:32:01.951950+0000","last_active":"2024-04-24T15:32:01.956596+0000","last_peered":"2024-04-24T15:32:01.956596+0000","last_clean":"2024-04-24T15:32:01.956596+0000","last_became_active":"2024-04-24T15:32:01.951704+0000","last_became_peered":"2024-04-24T15:32:01.951704+0000","last_unstale":"2024-04-24T15:32:01.956596+0000","last_undegraded":"2024-04-24T15:32:01.956596+0000","last_fullsized":"2024-04-24T15:32:01.956596+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:54:03.505685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":160,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-24T15:32:01.954219+0000","last_change":"2024-04-24T15:32:00.950217+0000","last_active":"2024-04-24T15:32:01.954219+0000","last_peered":"2024-04-24T15:32:01.954219+0000","last_clean":"2024-04-24T15:32:01.954219+0000","last_became_active":"2024-04-24T15:32:00.949910+0000","last_became_peered":"2024-04-24T15:32:00.949910+0000","last_unstale":"2024-04-24T15:32:01.954219+0000","last_undegraded":"2024-04-24T15:32:01.954219+0000","last_fullsized":"2024-04-24T15:32:01.954219+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:31:59.935287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:31:59.935287+0000","last_clean_scrub_stamp":"2024-04-24T15:31:59.935287+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:07:39.423442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":8,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-24T15:32:01.950716+0000","last_change":"2024-04-24T15:32:01.950716+0000","last_active":"2024-04-24T15:32:01.950716+0000","last_peered":"2024-04-24T15:32:01.950716+0000","last_clean":"2024-04-24T15:32:01.950716+0000","last_became_active":"2024-04-24T15:32:01.950485+0000","last_became_peered":"2024-04-24T15:32:01.950485+0000","last_unstale":"2024-04-24T15:32:01.950716+0000","last_undegraded":"2024-04-24T15:32:01.950716+0000","last_fullsized":"2024-04-24T15:32:01.950716+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:05:53.978876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-24T15:32:01.951741+0000","last_change":"2024-04-24T15:32:01.950612+0000","last_active":"2024-04-24T15:32:01.951741+0000","last_peered":"2024-04-24T15:32:01.951741+0000","last_clean":"2024-04-24T15:32:01.951741+0000","last_became_active":"2024-04-24T15:32:01.950350+0000","last_became_peered":"2024-04-24T15:32:01.950350+0000","last_unstale":"2024-04-24T15:32:01.951741+0000","last_undegraded":"2024-04-24T15:32:01.951741+0000","last_fullsized":"2024-04-24T15:32:01.951741+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:26:41.550351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":8,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-24T15:32:01.948167+0000","last_change":"2024-04-24T15:32:01.948167+0000","last_active":"2024-04-24T15:32:01.948167+0000","last_peered":"2024-04-24T15:32:01.948167+0000","last_clean":"2024-04-24T15:32:01.948167+0000","last_became_active":"2024-04-24T15:32:01.947995+0000","last_became_peered":"2024-04-24T15:32:01.947995+0000","last_unstale":"2024-04-24T15:32:01.948167+0000","last_undegraded":"2024-04-24T15:32:01.948167+0000","last_fullsized":"2024-04-24T15:32:01.948167+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:06:00.698624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":8,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-24T15:32:01.953602+0000","last_change":"2024-04-24T15:32:01.953602+0000","last_active":"2024-04-24T15:32:01.953602+0000","last_peered":"2024-04-24T15:32:01.953602+0000","last_clean":"2024-04-24T15:32:01.953602+0000","last_became_active":"2024-04-24T15:32:01.953338+0000","last_became_peered":"2024-04-24T15:32:01.953338+0000","last_unstale":"2024-04-24T15:32:01.953602+0000","last_undegraded":"2024-04-24T15:32:01.953602+0000","last_fullsized":"2024-04-24T15:32:01.953602+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:02:00.211754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9684,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297576,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074264,"statfs":{"total":96636764160,"available":96332046336,"internally_reserved":0,"allocated":225280,"data_stored":643074,"data_compressed":4842,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297512,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074328,"statfs":{"total":96636764160,"available":96332111872,"internally_reserved":0,"allocated":155648,"data_stored":46970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297512,"kb_used_data":148,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074328,"statfs":{"total":96636764160,"available":96332111872,"internally_reserved":0,"allocated":151552,"data_stored":46014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97599999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297604,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074236,"statfs":{"total":96636764160,"available":96332017664,"internally_reserved":0,"allocated":212992,"data_stored":637338,"data_compressed":4842,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.7890000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.9950000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2030000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.151}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.2509999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297524,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074316,"statfs":{"total":96636764160,"available":96332099584,"internally_reserved":0,"allocated":163840,"data_stored":51750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29799999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297512,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074328,"statfs":{"total":96636764160,"available":96332111872,"internally_reserved":0,"allocated":155648,"data_stored":46970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.254}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.307}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78000000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79100000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297524,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074316,"statfs":{"total":96636764160,"available":96332099584,"internally_reserved":0,"allocated":163840,"data_stored":51750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.317}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33100000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297512,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074328,"statfs":{"total":96636764160,"available":96332111872,"internally_reserved":0,"allocated":155648,"data_stored":46970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27700000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4842,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4842,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T15:32:05.873 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T15:32:06.184 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:32:06.186 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-04-24T15:32:06.197 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":14,"stamp":"2024-04-24T15:32:05.550440+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":13,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2380276,"kb_used_data":1352,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":752594444,"statfs":{"total":773094113280,"available":770656710656,"internally_reserved":0,"allocated":1384448,"data_stored":1570836,"data_compressed":9684,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000459"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":8,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-24T15:32:01.952204+0000","last_change":"2024-04-24T15:32:01.952204+0000","last_active":"2024-04-24T15:32:01.952204+0000","last_peered":"2024-04-24T15:32:01.952204+0000","last_clean":"2024-04-24T15:32:01.952204+0000","last_became_active":"2024-04-24T15:32:01.951968+0000","last_became_peered":"2024-04-24T15:32:01.951968+0000","last_unstale":"2024-04-24T15:32:01.952204+0000","last_undegraded":"2024-04-24T15:32:01.952204+0000","last_fullsized":"2024-04-24T15:32:01.952204+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:40:51.884922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":8,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-24T15:32:01.950055+0000","last_change":"2024-04-24T15:32:01.950055+0000","last_active":"2024-04-24T15:32:01.950055+0000","last_peered":"2024-04-24T15:32:01.950055+0000","last_clean":"2024-04-24T15:32:01.950055+0000","last_became_active":"2024-04-24T15:32:01.949783+0000","last_became_peered":"2024-04-24T15:32:01.949783+0000","last_unstale":"2024-04-24T15:32:01.950055+0000","last_undegraded":"2024-04-24T15:32:01.950055+0000","last_fullsized":"2024-04-24T15:32:01.950055+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:25:05.065671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-24T15:32:01.956639+0000","last_change":"2024-04-24T15:32:01.953602+0000","last_active":"2024-04-24T15:32:01.956639+0000","last_peered":"2024-04-24T15:32:01.956639+0000","last_clean":"2024-04-24T15:32:01.956639+0000","last_became_active":"2024-04-24T15:32:01.953388+0000","last_became_peered":"2024-04-24T15:32:01.953388+0000","last_unstale":"2024-04-24T15:32:01.956639+0000","last_undegraded":"2024-04-24T15:32:01.956639+0000","last_fullsized":"2024-04-24T15:32:01.956639+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:17:16.496379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-24T15:32:01.956596+0000","last_change":"2024-04-24T15:32:01.951950+0000","last_active":"2024-04-24T15:32:01.956596+0000","last_peered":"2024-04-24T15:32:01.956596+0000","last_clean":"2024-04-24T15:32:01.956596+0000","last_became_active":"2024-04-24T15:32:01.951704+0000","last_became_peered":"2024-04-24T15:32:01.951704+0000","last_unstale":"2024-04-24T15:32:01.956596+0000","last_undegraded":"2024-04-24T15:32:01.956596+0000","last_fullsized":"2024-04-24T15:32:01.956596+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:54:03.505685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":160,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-24T15:32:01.954219+0000","last_change":"2024-04-24T15:32:00.950217+0000","last_active":"2024-04-24T15:32:01.954219+0000","last_peered":"2024-04-24T15:32:01.954219+0000","last_clean":"2024-04-24T15:32:01.954219+0000","last_became_active":"2024-04-24T15:32:00.949910+0000","last_became_peered":"2024-04-24T15:32:00.949910+0000","last_unstale":"2024-04-24T15:32:01.954219+0000","last_undegraded":"2024-04-24T15:32:01.954219+0000","last_fullsized":"2024-04-24T15:32:01.954219+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:31:59.935287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:31:59.935287+0000","last_clean_scrub_stamp":"2024-04-24T15:31:59.935287+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:07:39.423442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":8,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-24T15:32:01.950716+0000","last_change":"2024-04-24T15:32:01.950716+0000","last_active":"2024-04-24T15:32:01.950716+0000","last_peered":"2024-04-24T15:32:01.950716+0000","last_clean":"2024-04-24T15:32:01.950716+0000","last_became_active":"2024-04-24T15:32:01.950485+0000","last_became_peered":"2024-04-24T15:32:01.950485+0000","last_unstale":"2024-04-24T15:32:01.950716+0000","last_undegraded":"2024-04-24T15:32:01.950716+0000","last_fullsized":"2024-04-24T15:32:01.950716+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:05:53.978876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-24T15:32:01.951741+0000","last_change":"2024-04-24T15:32:01.950612+0000","last_active":"2024-04-24T15:32:01.951741+0000","last_peered":"2024-04-24T15:32:01.951741+0000","last_clean":"2024-04-24T15:32:01.951741+0000","last_became_active":"2024-04-24T15:32:01.950350+0000","last_became_peered":"2024-04-24T15:32:01.950350+0000","last_unstale":"2024-04-24T15:32:01.951741+0000","last_undegraded":"2024-04-24T15:32:01.951741+0000","last_fullsized":"2024-04-24T15:32:01.951741+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:26:41.550351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":8,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-24T15:32:01.948167+0000","last_change":"2024-04-24T15:32:01.948167+0000","last_active":"2024-04-24T15:32:01.948167+0000","last_peered":"2024-04-24T15:32:01.948167+0000","last_clean":"2024-04-24T15:32:01.948167+0000","last_became_active":"2024-04-24T15:32:01.947995+0000","last_became_peered":"2024-04-24T15:32:01.947995+0000","last_unstale":"2024-04-24T15:32:01.948167+0000","last_undegraded":"2024-04-24T15:32:01.948167+0000","last_fullsized":"2024-04-24T15:32:01.948167+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:06:00.698624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":8,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-24T15:32:01.953602+0000","last_change":"2024-04-24T15:32:01.953602+0000","last_active":"2024-04-24T15:32:01.953602+0000","last_peered":"2024-04-24T15:32:01.953602+0000","last_clean":"2024-04-24T15:32:01.953602+0000","last_became_active":"2024-04-24T15:32:01.953338+0000","last_became_peered":"2024-04-24T15:32:01.953338+0000","last_unstale":"2024-04-24T15:32:01.953602+0000","last_undegraded":"2024-04-24T15:32:01.953602+0000","last_fullsized":"2024-04-24T15:32:01.953602+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:02:00.211754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9684,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297576,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074264,"statfs":{"total":96636764160,"available":96332046336,"internally_reserved":0,"allocated":225280,"data_stored":643074,"data_compressed":4842,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297512,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074328,"statfs":{"total":96636764160,"available":96332111872,"internally_reserved":0,"allocated":155648,"data_stored":46970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297512,"kb_used_data":148,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074328,"statfs":{"total":96636764160,"available":96332111872,"internally_reserved":0,"allocated":151552,"data_stored":46014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97599999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297604,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074236,"statfs":{"total":96636764160,"available":96332017664,"internally_reserved":0,"allocated":212992,"data_stored":637338,"data_compressed":4842,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.7890000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.9950000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2030000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.151}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.2509999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297524,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074316,"statfs":{"total":96636764160,"available":96332099584,"internally_reserved":0,"allocated":163840,"data_stored":51750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29799999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297512,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074328,"statfs":{"total":96636764160,"available":96332111872,"internally_reserved":0,"allocated":155648,"data_stored":46970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.254}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.307}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78000000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79100000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297524,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074316,"statfs":{"total":96636764160,"available":96332099584,"internally_reserved":0,"allocated":163840,"data_stored":51750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.317}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33100000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297512,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074328,"statfs":{"total":96636764160,"available":96332111872,"internally_reserved":0,"allocated":155648,"data_stored":46970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27700000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4842,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4842,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T15:32:06.198 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-24T15:32:06.198 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-24T15:32:06.198 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-24T15:32:06.198 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-24T15:32:06.548 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:32:06.549 INFO:teuthology.orchestra.run.smithi096.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-24T15:32:06.562 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-24T15:32:06.563 INFO:teuthology.run_tasks:Running task exec... 2024-04-24T15:32:06.577 INFO:teuthology.task.exec:Executing custom commands... 2024-04-24T15:32:06.577 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi096.front.sepia.ceph.com 2024-04-24T15:32:06.577 DEBUG:teuthology.orchestra.run.smithi096:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd pool create cache 4' 2024-04-24T15:32:06.985 INFO:teuthology.orchestra.run.smithi096.stderr:pool 'cache' created 2024-04-24T15:32:06.997 DEBUG:teuthology.orchestra.run.smithi096:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd tier add rbd cache' 2024-04-24T15:32:07.994 INFO:teuthology.orchestra.run.smithi096.stderr:pool 'cache' is now (or already was) a tier of 'rbd' 2024-04-24T15:32:08.008 DEBUG:teuthology.orchestra.run.smithi096:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd tier cache-mode cache writeback' 2024-04-24T15:32:08.998 INFO:teuthology.orchestra.run.smithi096.stderr:set cache-mode for pool 'cache' to writeback 2024-04-24T15:32:09.012 DEBUG:teuthology.orchestra.run.smithi096:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd tier set-overlay rbd cache' 2024-04-24T15:32:09.999 INFO:teuthology.orchestra.run.smithi096.stderr:overlay for 'rbd' is now (or already was) 'cache' 2024-04-24T15:32:10.011 DEBUG:teuthology.orchestra.run.smithi096:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd pool set cache hit_set_type bloom' 2024-04-24T15:32:11.001 INFO:teuthology.orchestra.run.smithi096.stderr:set pool 3 hit_set_type to bloom 2024-04-24T15:32:11.014 DEBUG:teuthology.orchestra.run.smithi096:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd pool set cache hit_set_count 8' 2024-04-24T15:32:12.013 INFO:teuthology.orchestra.run.smithi096.stderr:set pool 3 hit_set_count to 8 2024-04-24T15:32:12.027 DEBUG:teuthology.orchestra.run.smithi096:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd pool set cache hit_set_period 60' 2024-04-24T15:32:13.015 INFO:teuthology.orchestra.run.smithi096.stderr:set pool 3 hit_set_period to 60 2024-04-24T15:32:13.029 DEBUG:teuthology.orchestra.run.smithi096:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd pool set cache target_max_objects 250' 2024-04-24T15:32:14.025 INFO:teuthology.orchestra.run.smithi096.stderr:set pool 3 target_max_objects to 250 2024-04-24T15:32:14.039 INFO:teuthology.run_tasks:Running task thrashosds... 2024-04-24T15:32:14.050 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-04-24T15:32:14.050 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'wip-yuriw4-testing-20240423.151325-quincy'}, 'ceph': {'conf': {'client': {'rbd qos iops burst': 100, 'rbd qos iops limit': 50, 'rbd qos schedule tick min': 100}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(CACHE_POOL_NO_HIT_SET\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request'], 'sha1': '330067dbce47655d1f762ef14d546d2fde16a6a9'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '330067dbce47655d1f762ef14d546d2fde16a6a9'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuriw4-testing-20240423.151325-quincy', 'sha1': '330067dbce47655d1f762ef14d546d2fde16a6a9'}} 2024-04-24T15:32:14.050 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-04-24T15:32:14.050 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-04-24T15:32:14.391 INFO:tasks.thrashosds:Beginning thrashosds... 2024-04-24T15:32:14.391 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-24T15:32:14.391 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T15:32:14.702 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:32:14.704 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-04-24T15:32:14.715 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-04-24T15:32:14.020480+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":25,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2380876,"kb_used_data":2012,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":752593844,"statfs":{"total":773094113280,"available":770656096256,"internally_reserved":0,"allocated":2060288,"data_stored":1911250,"data_compressed":9684,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.023356"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T15:32:12.008621+0000","last_change":"2024-04-24T15:32:01.952204+0000","last_active":"2024-04-24T15:32:12.008621+0000","last_peered":"2024-04-24T15:32:12.008621+0000","last_clean":"2024-04-24T15:32:12.008621+0000","last_became_active":"2024-04-24T15:32:01.951968+0000","last_became_peered":"2024-04-24T15:32:01.951968+0000","last_unstale":"2024-04-24T15:32:12.008621+0000","last_undegraded":"2024-04-24T15:32:12.008621+0000","last_fullsized":"2024-04-24T15:32:12.008621+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:40:51.884922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-24T15:32:11.455135+0000","last_change":"2024-04-24T15:32:01.950055+0000","last_active":"2024-04-24T15:32:11.455135+0000","last_peered":"2024-04-24T15:32:11.455135+0000","last_clean":"2024-04-24T15:32:11.455135+0000","last_became_active":"2024-04-24T15:32:01.949783+0000","last_became_peered":"2024-04-24T15:32:01.949783+0000","last_unstale":"2024-04-24T15:32:11.455135+0000","last_undegraded":"2024-04-24T15:32:11.455135+0000","last_fullsized":"2024-04-24T15:32:11.455135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:25:05.065671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T15:32:10.380743+0000","last_change":"2024-04-24T15:32:01.953602+0000","last_active":"2024-04-24T15:32:10.380743+0000","last_peered":"2024-04-24T15:32:10.380743+0000","last_clean":"2024-04-24T15:32:10.380743+0000","last_became_active":"2024-04-24T15:32:01.953388+0000","last_became_peered":"2024-04-24T15:32:01.953388+0000","last_unstale":"2024-04-24T15:32:10.380743+0000","last_undegraded":"2024-04-24T15:32:10.380743+0000","last_fullsized":"2024-04-24T15:32:10.380743+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:17:16.496379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T15:32:12.105469+0000","last_change":"2024-04-24T15:32:07.997117+0000","last_active":"2024-04-24T15:32:12.105469+0000","last_peered":"2024-04-24T15:32:12.105469+0000","last_clean":"2024-04-24T15:32:12.105469+0000","last_became_active":"2024-04-24T15:32:07.996832+0000","last_became_peered":"2024-04-24T15:32:07.996832+0000","last_unstale":"2024-04-24T15:32:12.105469+0000","last_undegraded":"2024-04-24T15:32:12.105469+0000","last_fullsized":"2024-04-24T15:32:12.105469+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_clean_scrub_stamp":"2024-04-24T15:32:06.972928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:15:29.644726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T15:32:10.380730+0000","last_change":"2024-04-24T15:32:01.951950+0000","last_active":"2024-04-24T15:32:10.380730+0000","last_peered":"2024-04-24T15:32:10.380730+0000","last_clean":"2024-04-24T15:32:10.380730+0000","last_became_active":"2024-04-24T15:32:01.951704+0000","last_became_peered":"2024-04-24T15:32:01.951704+0000","last_unstale":"2024-04-24T15:32:10.380730+0000","last_undegraded":"2024-04-24T15:32:10.380730+0000","last_fullsized":"2024-04-24T15:32:10.380730+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:54:03.505685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T15:32:12.104690+0000","last_change":"2024-04-24T15:32:08.145022+0000","last_active":"2024-04-24T15:32:12.104690+0000","last_peered":"2024-04-24T15:32:12.104690+0000","last_clean":"2024-04-24T15:32:12.104690+0000","last_became_active":"2024-04-24T15:32:08.144753+0000","last_became_peered":"2024-04-24T15:32:08.144753+0000","last_unstale":"2024-04-24T15:32:12.104690+0000","last_undegraded":"2024-04-24T15:32:12.104690+0000","last_fullsized":"2024-04-24T15:32:12.104690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_clean_scrub_stamp":"2024-04-24T15:32:06.972928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:59:20.855396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":165,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T15:32:10.380733+0000","last_change":"2024-04-24T15:32:00.950217+0000","last_active":"2024-04-24T15:32:10.380733+0000","last_peered":"2024-04-24T15:32:10.380733+0000","last_clean":"2024-04-24T15:32:10.380733+0000","last_became_active":"2024-04-24T15:32:00.949910+0000","last_became_peered":"2024-04-24T15:32:00.949910+0000","last_unstale":"2024-04-24T15:32:10.380733+0000","last_undegraded":"2024-04-24T15:32:10.380733+0000","last_fullsized":"2024-04-24T15:32:10.380733+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:31:59.935287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:31:59.935287+0000","last_clean_scrub_stamp":"2024-04-24T15:31:59.935287+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:07:39.423442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T15:32:09.120318+0000","last_change":"2024-04-24T15:32:01.950716+0000","last_active":"2024-04-24T15:32:09.120318+0000","last_peered":"2024-04-24T15:32:09.120318+0000","last_clean":"2024-04-24T15:32:09.120318+0000","last_became_active":"2024-04-24T15:32:01.950485+0000","last_became_peered":"2024-04-24T15:32:01.950485+0000","last_unstale":"2024-04-24T15:32:09.120318+0000","last_undegraded":"2024-04-24T15:32:09.120318+0000","last_fullsized":"2024-04-24T15:32:09.120318+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:05:53.978876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-24T15:32:11.455122+0000","last_change":"2024-04-24T15:32:07.997527+0000","last_active":"2024-04-24T15:32:11.455122+0000","last_peered":"2024-04-24T15:32:11.455122+0000","last_clean":"2024-04-24T15:32:11.455122+0000","last_became_active":"2024-04-24T15:32:07.997226+0000","last_became_peered":"2024-04-24T15:32:07.997226+0000","last_unstale":"2024-04-24T15:32:11.455122+0000","last_undegraded":"2024-04-24T15:32:11.455122+0000","last_fullsized":"2024-04-24T15:32:11.455122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_clean_scrub_stamp":"2024-04-24T15:32:06.972928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:59:43.225408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-24T15:32:11.000070+0000","last_change":"2024-04-24T15:32:01.950612+0000","last_active":"2024-04-24T15:32:11.000070+0000","last_peered":"2024-04-24T15:32:11.000070+0000","last_clean":"2024-04-24T15:32:11.000070+0000","last_became_active":"2024-04-24T15:32:01.950350+0000","last_became_peered":"2024-04-24T15:32:01.950350+0000","last_unstale":"2024-04-24T15:32:11.000070+0000","last_undegraded":"2024-04-24T15:32:11.000070+0000","last_fullsized":"2024-04-24T15:32:11.000070+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:26:41.550351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T15:32:12.008623+0000","last_change":"2024-04-24T15:32:07.997874+0000","last_active":"2024-04-24T15:32:12.008623+0000","last_peered":"2024-04-24T15:32:12.008623+0000","last_clean":"2024-04-24T15:32:12.008623+0000","last_became_active":"2024-04-24T15:32:07.997609+0000","last_became_peered":"2024-04-24T15:32:07.997609+0000","last_unstale":"2024-04-24T15:32:12.008623+0000","last_undegraded":"2024-04-24T15:32:12.008623+0000","last_fullsized":"2024-04-24T15:32:12.008623+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_clean_scrub_stamp":"2024-04-24T15:32:06.972928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:44:05.129107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T15:32:09.120328+0000","last_change":"2024-04-24T15:32:01.948167+0000","last_active":"2024-04-24T15:32:09.120328+0000","last_peered":"2024-04-24T15:32:09.120328+0000","last_clean":"2024-04-24T15:32:09.120328+0000","last_became_active":"2024-04-24T15:32:01.947995+0000","last_became_peered":"2024-04-24T15:32:01.947995+0000","last_unstale":"2024-04-24T15:32:09.120328+0000","last_undegraded":"2024-04-24T15:32:09.120328+0000","last_fullsized":"2024-04-24T15:32:09.120328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:06:00.698624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-24T15:32:11.455116+0000","last_change":"2024-04-24T15:32:01.953602+0000","last_active":"2024-04-24T15:32:11.455116+0000","last_peered":"2024-04-24T15:32:11.455116+0000","last_clean":"2024-04-24T15:32:11.455116+0000","last_became_active":"2024-04-24T15:32:01.953338+0000","last_became_peered":"2024-04-24T15:32:01.953338+0000","last_unstale":"2024-04-24T15:32:11.455116+0000","last_undegraded":"2024-04-24T15:32:11.455116+0000","last_fullsized":"2024-04-24T15:32:11.455116+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:02:00.211754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":8,"acting":8,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9684,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297648,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074192,"statfs":{"total":96636764160,"available":96331972608,"internally_reserved":0,"allocated":299008,"data_stored":680503,"data_compressed":4842,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297604,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074236,"statfs":{"total":96636764160,"available":96332017664,"internally_reserved":0,"allocated":253952,"data_stored":96286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42999999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297592,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074248,"statfs":{"total":96636764160,"available":96332029952,"internally_reserved":0,"allocated":237568,"data_stored":89179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.251}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297648,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074192,"statfs":{"total":96636764160,"available":96331972608,"internally_reserved":0,"allocated":299008,"data_stored":680503,"data_compressed":4842,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.245}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297596,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074244,"statfs":{"total":96636764160,"available":96332025856,"internally_reserved":0,"allocated":237568,"data_stored":89179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.28}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297596,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074244,"statfs":{"total":96636764160,"available":96332025856,"internally_reserved":0,"allocated":241664,"data_stored":90135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38300000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297600,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074240,"statfs":{"total":96636764160,"available":96332021760,"internally_reserved":0,"allocated":249856,"data_stored":95330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73499999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297592,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074248,"statfs":{"total":96636764160,"available":96332029952,"internally_reserved":0,"allocated":241664,"data_stored":90135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4842,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4842,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T15:32:14.716 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T15:32:15.028 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:32:15.030 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-04-24T15:32:15.041 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-04-24T15:32:14.020480+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":25,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2380876,"kb_used_data":2012,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":752593844,"statfs":{"total":773094113280,"available":770656096256,"internally_reserved":0,"allocated":2060288,"data_stored":1911250,"data_compressed":9684,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.023356"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T15:32:12.008621+0000","last_change":"2024-04-24T15:32:01.952204+0000","last_active":"2024-04-24T15:32:12.008621+0000","last_peered":"2024-04-24T15:32:12.008621+0000","last_clean":"2024-04-24T15:32:12.008621+0000","last_became_active":"2024-04-24T15:32:01.951968+0000","last_became_peered":"2024-04-24T15:32:01.951968+0000","last_unstale":"2024-04-24T15:32:12.008621+0000","last_undegraded":"2024-04-24T15:32:12.008621+0000","last_fullsized":"2024-04-24T15:32:12.008621+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:40:51.884922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-24T15:32:11.455135+0000","last_change":"2024-04-24T15:32:01.950055+0000","last_active":"2024-04-24T15:32:11.455135+0000","last_peered":"2024-04-24T15:32:11.455135+0000","last_clean":"2024-04-24T15:32:11.455135+0000","last_became_active":"2024-04-24T15:32:01.949783+0000","last_became_peered":"2024-04-24T15:32:01.949783+0000","last_unstale":"2024-04-24T15:32:11.455135+0000","last_undegraded":"2024-04-24T15:32:11.455135+0000","last_fullsized":"2024-04-24T15:32:11.455135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:25:05.065671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T15:32:10.380743+0000","last_change":"2024-04-24T15:32:01.953602+0000","last_active":"2024-04-24T15:32:10.380743+0000","last_peered":"2024-04-24T15:32:10.380743+0000","last_clean":"2024-04-24T15:32:10.380743+0000","last_became_active":"2024-04-24T15:32:01.953388+0000","last_became_peered":"2024-04-24T15:32:01.953388+0000","last_unstale":"2024-04-24T15:32:10.380743+0000","last_undegraded":"2024-04-24T15:32:10.380743+0000","last_fullsized":"2024-04-24T15:32:10.380743+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:17:16.496379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T15:32:12.105469+0000","last_change":"2024-04-24T15:32:07.997117+0000","last_active":"2024-04-24T15:32:12.105469+0000","last_peered":"2024-04-24T15:32:12.105469+0000","last_clean":"2024-04-24T15:32:12.105469+0000","last_became_active":"2024-04-24T15:32:07.996832+0000","last_became_peered":"2024-04-24T15:32:07.996832+0000","last_unstale":"2024-04-24T15:32:12.105469+0000","last_undegraded":"2024-04-24T15:32:12.105469+0000","last_fullsized":"2024-04-24T15:32:12.105469+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_clean_scrub_stamp":"2024-04-24T15:32:06.972928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:15:29.644726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T15:32:10.380730+0000","last_change":"2024-04-24T15:32:01.951950+0000","last_active":"2024-04-24T15:32:10.380730+0000","last_peered":"2024-04-24T15:32:10.380730+0000","last_clean":"2024-04-24T15:32:10.380730+0000","last_became_active":"2024-04-24T15:32:01.951704+0000","last_became_peered":"2024-04-24T15:32:01.951704+0000","last_unstale":"2024-04-24T15:32:10.380730+0000","last_undegraded":"2024-04-24T15:32:10.380730+0000","last_fullsized":"2024-04-24T15:32:10.380730+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:54:03.505685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T15:32:12.104690+0000","last_change":"2024-04-24T15:32:08.145022+0000","last_active":"2024-04-24T15:32:12.104690+0000","last_peered":"2024-04-24T15:32:12.104690+0000","last_clean":"2024-04-24T15:32:12.104690+0000","last_became_active":"2024-04-24T15:32:08.144753+0000","last_became_peered":"2024-04-24T15:32:08.144753+0000","last_unstale":"2024-04-24T15:32:12.104690+0000","last_undegraded":"2024-04-24T15:32:12.104690+0000","last_fullsized":"2024-04-24T15:32:12.104690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_clean_scrub_stamp":"2024-04-24T15:32:06.972928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:59:20.855396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":165,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T15:32:10.380733+0000","last_change":"2024-04-24T15:32:00.950217+0000","last_active":"2024-04-24T15:32:10.380733+0000","last_peered":"2024-04-24T15:32:10.380733+0000","last_clean":"2024-04-24T15:32:10.380733+0000","last_became_active":"2024-04-24T15:32:00.949910+0000","last_became_peered":"2024-04-24T15:32:00.949910+0000","last_unstale":"2024-04-24T15:32:10.380733+0000","last_undegraded":"2024-04-24T15:32:10.380733+0000","last_fullsized":"2024-04-24T15:32:10.380733+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:31:59.935287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:31:59.935287+0000","last_clean_scrub_stamp":"2024-04-24T15:31:59.935287+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:07:39.423442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T15:32:09.120318+0000","last_change":"2024-04-24T15:32:01.950716+0000","last_active":"2024-04-24T15:32:09.120318+0000","last_peered":"2024-04-24T15:32:09.120318+0000","last_clean":"2024-04-24T15:32:09.120318+0000","last_became_active":"2024-04-24T15:32:01.950485+0000","last_became_peered":"2024-04-24T15:32:01.950485+0000","last_unstale":"2024-04-24T15:32:09.120318+0000","last_undegraded":"2024-04-24T15:32:09.120318+0000","last_fullsized":"2024-04-24T15:32:09.120318+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:05:53.978876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-24T15:32:11.455122+0000","last_change":"2024-04-24T15:32:07.997527+0000","last_active":"2024-04-24T15:32:11.455122+0000","last_peered":"2024-04-24T15:32:11.455122+0000","last_clean":"2024-04-24T15:32:11.455122+0000","last_became_active":"2024-04-24T15:32:07.997226+0000","last_became_peered":"2024-04-24T15:32:07.997226+0000","last_unstale":"2024-04-24T15:32:11.455122+0000","last_undegraded":"2024-04-24T15:32:11.455122+0000","last_fullsized":"2024-04-24T15:32:11.455122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_clean_scrub_stamp":"2024-04-24T15:32:06.972928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:59:43.225408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-24T15:32:11.000070+0000","last_change":"2024-04-24T15:32:01.950612+0000","last_active":"2024-04-24T15:32:11.000070+0000","last_peered":"2024-04-24T15:32:11.000070+0000","last_clean":"2024-04-24T15:32:11.000070+0000","last_became_active":"2024-04-24T15:32:01.950350+0000","last_became_peered":"2024-04-24T15:32:01.950350+0000","last_unstale":"2024-04-24T15:32:11.000070+0000","last_undegraded":"2024-04-24T15:32:11.000070+0000","last_fullsized":"2024-04-24T15:32:11.000070+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:26:41.550351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T15:32:12.008623+0000","last_change":"2024-04-24T15:32:07.997874+0000","last_active":"2024-04-24T15:32:12.008623+0000","last_peered":"2024-04-24T15:32:12.008623+0000","last_clean":"2024-04-24T15:32:12.008623+0000","last_became_active":"2024-04-24T15:32:07.997609+0000","last_became_peered":"2024-04-24T15:32:07.997609+0000","last_unstale":"2024-04-24T15:32:12.008623+0000","last_undegraded":"2024-04-24T15:32:12.008623+0000","last_fullsized":"2024-04-24T15:32:12.008623+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_clean_scrub_stamp":"2024-04-24T15:32:06.972928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:44:05.129107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T15:32:09.120328+0000","last_change":"2024-04-24T15:32:01.948167+0000","last_active":"2024-04-24T15:32:09.120328+0000","last_peered":"2024-04-24T15:32:09.120328+0000","last_clean":"2024-04-24T15:32:09.120328+0000","last_became_active":"2024-04-24T15:32:01.947995+0000","last_became_peered":"2024-04-24T15:32:01.947995+0000","last_unstale":"2024-04-24T15:32:09.120328+0000","last_undegraded":"2024-04-24T15:32:09.120328+0000","last_fullsized":"2024-04-24T15:32:09.120328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:06:00.698624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-24T15:32:11.455116+0000","last_change":"2024-04-24T15:32:01.953602+0000","last_active":"2024-04-24T15:32:11.455116+0000","last_peered":"2024-04-24T15:32:11.455116+0000","last_clean":"2024-04-24T15:32:11.455116+0000","last_became_active":"2024-04-24T15:32:01.953338+0000","last_became_peered":"2024-04-24T15:32:01.953338+0000","last_unstale":"2024-04-24T15:32:11.455116+0000","last_undegraded":"2024-04-24T15:32:11.455116+0000","last_fullsized":"2024-04-24T15:32:11.455116+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:02:00.211754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":8,"acting":8,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9684,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297648,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074192,"statfs":{"total":96636764160,"available":96331972608,"internally_reserved":0,"allocated":299008,"data_stored":680503,"data_compressed":4842,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297604,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074236,"statfs":{"total":96636764160,"available":96332017664,"internally_reserved":0,"allocated":253952,"data_stored":96286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42999999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297592,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074248,"statfs":{"total":96636764160,"available":96332029952,"internally_reserved":0,"allocated":237568,"data_stored":89179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.251}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297648,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074192,"statfs":{"total":96636764160,"available":96331972608,"internally_reserved":0,"allocated":299008,"data_stored":680503,"data_compressed":4842,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.245}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297596,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074244,"statfs":{"total":96636764160,"available":96332025856,"internally_reserved":0,"allocated":237568,"data_stored":89179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.28}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297596,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074244,"statfs":{"total":96636764160,"available":96332025856,"internally_reserved":0,"allocated":241664,"data_stored":90135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38300000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297600,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074240,"statfs":{"total":96636764160,"available":96332021760,"internally_reserved":0,"allocated":249856,"data_stored":95330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73499999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297592,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074248,"statfs":{"total":96636764160,"available":96332029952,"internally_reserved":0,"allocated":241664,"data_stored":90135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4842,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4842,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T15:32:15.042 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-24T15:32:15.043 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-04-24T15:32:15.354 INFO:teuthology.orchestra.run.smithi096.stdout:epoch 25 2024-04-24T15:32:15.354 INFO:teuthology.orchestra.run.smithi096.stdout:fsid 253d426c-62cb-4194-8514-6cd970fd85c7 2024-04-24T15:32:15.354 INFO:teuthology.orchestra.run.smithi096.stdout:created 2024-04-24T15:31:47.870327+0000 2024-04-24T15:32:15.354 INFO:teuthology.orchestra.run.smithi096.stdout:modified 2024-04-24T15:32:14.014281+0000 2024-04-24T15:32:15.355 INFO:teuthology.orchestra.run.smithi096.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-24T15:32:15.355 INFO:teuthology.orchestra.run.smithi096.stdout:crush_version 4 2024-04-24T15:32:15.355 INFO:teuthology.orchestra.run.smithi096.stdout:full_ratio 0.95 2024-04-24T15:32:15.355 INFO:teuthology.orchestra.run.smithi096.stdout:backfillfull_ratio 0.9 2024-04-24T15:32:15.355 INFO:teuthology.orchestra.run.smithi096.stdout:nearfull_ratio 0.85 2024-04-24T15:32:15.355 INFO:teuthology.orchestra.run.smithi096.stdout:require_min_compat_client luminous 2024-04-24T15:32:15.355 INFO:teuthology.orchestra.run.smithi096.stdout:min_compat_client jewel 2024-04-24T15:32:15.355 INFO:teuthology.orchestra.run.smithi096.stdout:require_osd_release quincy 2024-04-24T15:32:15.355 INFO:teuthology.orchestra.run.smithi096.stdout:stretch_mode_enabled false 2024-04-24T15:32:15.355 INFO:teuthology.orchestra.run.smithi096.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 16 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr 2024-04-24T15:32:15.355 INFO:teuthology.orchestra.run.smithi096.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 21 lfor 21/21/21 flags hashpspool tiers 3 read_tier 3 write_tier 3 stripe_width 0 application rbd 2024-04-24T15:32:15.355 INFO:teuthology.orchestra.run.smithi096.stdout:pool 3 'cache' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 4 pgp_num 4 autoscale_mode off last_change 25 lfor 21/21/21 flags hashpspool,incomplete_clones tier_of 2 cache_mode writeback target_objects 250 hit_set bloom{false_positive_probability: 0.05, target_size: 0, seed: 0} 60s x8 decay_rate 0 search_last_n 0 stripe_width 0 application rbd 2024-04-24T15:32:15.355 INFO:teuthology.orchestra.run.smithi096.stdout:max_osd 8 2024-04-24T15:32:15.355 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.96:6816/44106,v1:172.21.15.96:6817/44106] [v2:172.21.15.96:6818/44106,v1:172.21.15.96:6819/44106] exists,up 92941fb9-96ee-4b2b-8147-3c1d61192f94 2024-04-24T15:32:15.355 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.96:6800/44089,v1:172.21.15.96:6801/44089] [v2:172.21.15.96:6802/44089,v1:172.21.15.96:6803/44089] exists,up 628e868a-9884-42e6-9331-caef97cb6967 2024-04-24T15:32:15.355 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.96:6808/44090,v1:172.21.15.96:6809/44090] [v2:172.21.15.96:6810/44090,v1:172.21.15.96:6811/44090] exists,up 46de3f65-87a8-460c-b57a-4468d9c89a57 2024-04-24T15:32:15.356 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.96:6824/44121,v1:172.21.15.96:6825/44121] [v2:172.21.15.96:6826/44121,v1:172.21.15.96:6827/44121] exists,up 5507e063-10bc-4e32-b1ec-00aa3e47d993 2024-04-24T15:32:15.356 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.151:6816/44025,v1:172.21.15.151:6817/44025] [v2:172.21.15.151:6818/44025,v1:172.21.15.151:6819/44025] exists,up 6a74d453-1048-44ef-a5b0-4277583d76c2 2024-04-24T15:32:15.356 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.151:6800/44023,v1:172.21.15.151:6801/44023] [v2:172.21.15.151:6802/44023,v1:172.21.15.151:6803/44023] exists,up 3645ef18-5574-49f8-ab81-b40651208309 2024-04-24T15:32:15.356 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.151:6824/44057,v1:172.21.15.151:6825/44057] [v2:172.21.15.151:6826/44057,v1:172.21.15.151:6827/44057] exists,up 6b8b0b36-9246-4241-a197-c69fa48a6202 2024-04-24T15:32:15.356 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.151:6808/44024,v1:172.21.15.151:6809/44024] [v2:172.21.15.151:6810/44024,v1:172.21.15.151:6811/44024] exists,up 4501cd9f-e437-4ddd-af3d-129fdf1ee61a 2024-04-24T15:32:15.367 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.96:6816/44106,v1:172.21.15.96:6817/44106] [v2:172.21.15.96:6818/44106,v1:172.21.15.96:6819/44106] exists,up 92941fb9-96ee-4b2b-8147-3c1d61192f94', 'osd.1 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.96:6800/44089,v1:172.21.15.96:6801/44089] [v2:172.21.15.96:6802/44089,v1:172.21.15.96:6803/44089] exists,up 628e868a-9884-42e6-9331-caef97cb6967', 'osd.2 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.96:6808/44090,v1:172.21.15.96:6809/44090] [v2:172.21.15.96:6810/44090,v1:172.21.15.96:6811/44090] exists,up 46de3f65-87a8-460c-b57a-4468d9c89a57', 'osd.3 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.96:6824/44121,v1:172.21.15.96:6825/44121] [v2:172.21.15.96:6826/44121,v1:172.21.15.96:6827/44121] exists,up 5507e063-10bc-4e32-b1ec-00aa3e47d993', 'osd.4 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.151:6816/44025,v1:172.21.15.151:6817/44025] [v2:172.21.15.151:6818/44025,v1:172.21.15.151:6819/44025] exists,up 6a74d453-1048-44ef-a5b0-4277583d76c2', 'osd.5 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.151:6800/44023,v1:172.21.15.151:6801/44023] [v2:172.21.15.151:6802/44023,v1:172.21.15.151:6803/44023] exists,up 3645ef18-5574-49f8-ab81-b40651208309', 'osd.6 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.151:6824/44057,v1:172.21.15.151:6825/44057] [v2:172.21.15.151:6826/44057,v1:172.21.15.151:6827/44057] exists,up 6b8b0b36-9246-4241-a197-c69fa48a6202', 'osd.7 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.151:6808/44024,v1:172.21.15.151:6809/44024] [v2:172.21.15.151:6810/44024,v1:172.21.15.151:6811/44024] exists,up 4501cd9f-e437-4ddd-af3d-129fdf1ee61a'] 2024-04-24T15:32:15.368 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-04-24T15:32:15.485 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:32:15.485 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mon.a", 2024-04-24T15:32:15.485 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster": "ceph", 2024-04-24T15:32:15.485 INFO:teuthology.orchestra.run.smithi096.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-04-24T15:32:15.485 INFO:teuthology.orchestra.run.smithi096.stdout: "admin_socket_mode": "", 2024-04-24T15:32:15.486 INFO:teuthology.orchestra.run.smithi096.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-04-24T15:32:15.486 INFO:teuthology.orchestra.run.smithi096.stdout: "auth_client_required": "cephx, none", 2024-04-24T15:32:15.486 INFO:teuthology.orchestra.run.smithi096.stdout: "auth_cluster_required": "cephx", 2024-04-24T15:32:15.486 INFO:teuthology.orchestra.run.smithi096.stdout: "auth_debug": "true", 2024-04-24T15:32:15.486 INFO:teuthology.orchestra.run.smithi096.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-04-24T15:32:15.486 INFO:teuthology.orchestra.run.smithi096.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-04-24T15:32:15.486 INFO:teuthology.orchestra.run.smithi096.stdout: "auth_service_required": "cephx", 2024-04-24T15:32:15.486 INFO:teuthology.orchestra.run.smithi096.stdout: "auth_service_ticket_ttl": "240.000000", 2024-04-24T15:32:15.486 INFO:teuthology.orchestra.run.smithi096.stdout: "auth_supported": "cephx", 2024-04-24T15:32:15.486 INFO:teuthology.orchestra.run.smithi096.stdout: "bdev_aio": "true", 2024-04-24T15:32:15.486 INFO:teuthology.orchestra.run.smithi096.stdout: "bdev_aio_max_queue_depth": "1024", 2024-04-24T15:32:15.486 INFO:teuthology.orchestra.run.smithi096.stdout: "bdev_aio_poll_ms": "250", 2024-04-24T15:32:15.486 INFO:teuthology.orchestra.run.smithi096.stdout: "bdev_aio_reap_max": "16", 2024-04-24T15:32:15.486 INFO:teuthology.orchestra.run.smithi096.stdout: "bdev_async_discard": "false", 2024-04-24T15:32:15.487 INFO:teuthology.orchestra.run.smithi096.stdout: "bdev_block_size": "4096", 2024-04-24T15:32:15.487 INFO:teuthology.orchestra.run.smithi096.stdout: "bdev_debug_aio": "false", 2024-04-24T15:32:15.487 INFO:teuthology.orchestra.run.smithi096.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-04-24T15:32:15.487 INFO:teuthology.orchestra.run.smithi096.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-04-24T15:32:15.487 INFO:teuthology.orchestra.run.smithi096.stdout: "bdev_debug_inflight_ios": "false", 2024-04-24T15:32:15.487 INFO:teuthology.orchestra.run.smithi096.stdout: "bdev_enable_discard": "false", 2024-04-24T15:32:15.487 INFO:teuthology.orchestra.run.smithi096.stdout: "bdev_flock_retry": "3", 2024-04-24T15:32:15.487 INFO:teuthology.orchestra.run.smithi096.stdout: "bdev_flock_retry_interval": "0.100000", 2024-04-24T15:32:15.487 INFO:teuthology.orchestra.run.smithi096.stdout: "bdev_inject_crash": "0", 2024-04-24T15:32:15.487 INFO:teuthology.orchestra.run.smithi096.stdout: "bdev_inject_crash_flush_delay": "2", 2024-04-24T15:32:15.487 INFO:teuthology.orchestra.run.smithi096.stdout: "bdev_ioring": "false", 2024-04-24T15:32:15.487 INFO:teuthology.orchestra.run.smithi096.stdout: "bdev_ioring_hipri": "false", 2024-04-24T15:32:15.487 INFO:teuthology.orchestra.run.smithi096.stdout: "bdev_ioring_sqthread_poll": "false", 2024-04-24T15:32:15.487 INFO:teuthology.orchestra.run.smithi096.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-04-24T15:32:15.487 INFO:teuthology.orchestra.run.smithi096.stdout: "bdev_read_buffer_alignment": "4096", 2024-04-24T15:32:15.488 INFO:teuthology.orchestra.run.smithi096.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-04-24T15:32:15.488 INFO:teuthology.orchestra.run.smithi096.stdout: "bdev_type": "", 2024-04-24T15:32:15.488 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_alloc_size": "1048576", 2024-04-24T15:32:15.488 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_allocator": "hybrid", 2024-04-24T15:32:15.488 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_buffered_io": "true", 2024-04-24T15:32:15.488 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_check_for_zeros": "false", 2024-04-24T15:32:15.488 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_check_volume_selector_often": "false", 2024-04-24T15:32:15.488 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-04-24T15:32:15.488 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_compact_log_sync": "false", 2024-04-24T15:32:15.488 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-04-24T15:32:15.488 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-04-24T15:32:15.488 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_log_compact_min_size": "16777216", 2024-04-24T15:32:15.488 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_log_replay_check_allocations": "true", 2024-04-24T15:32:15.489 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_max_log_runway": "4194304", 2024-04-24T15:32:15.489 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_max_prefetch": "1048576", 2024-04-24T15:32:15.489 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_min_flush_size": "524288", 2024-04-24T15:32:15.489 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_min_log_runway": "1048576", 2024-04-24T15:32:15.489 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_replay_recovery": "false", 2024-04-24T15:32:15.489 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-04-24T15:32:15.489 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_shared_alloc_size": "65536", 2024-04-24T15:32:15.489 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_sync_write": "false", 2024-04-24T15:32:15.489 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-04-24T15:32:15.489 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-04-24T15:32:15.489 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-04-24T15:32:15.489 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_allocation_from_file": "true", 2024-04-24T15:32:15.489 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_allocator": "hybrid", 2024-04-24T15:32:15.489 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-04-24T15:32:15.490 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-04-24T15:32:15.490 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-04-24T15:32:15.490 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-04-24T15:32:15.490 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-04-24T15:32:15.490 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-04-24T15:32:15.490 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blobid_prealloc": "10240", 2024-04-24T15:32:15.490 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_block_create": "true", 2024-04-24T15:32:15.490 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_block_db_create": "false", 2024-04-24T15:32:15.490 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_block_db_path": "", 2024-04-24T15:32:15.490 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_block_db_size": "0", 2024-04-24T15:32:15.490 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_block_path": "", 2024-04-24T15:32:15.490 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_block_preallocate_file": "false", 2024-04-24T15:32:15.490 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_block_size": "107374182400", 2024-04-24T15:32:15.491 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_block_wal_create": "false", 2024-04-24T15:32:15.491 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_block_wal_path": "", 2024-04-24T15:32:15.491 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_block_wal_size": "100663296", 2024-04-24T15:32:15.491 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_bluefs": "true", 2024-04-24T15:32:15.491 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-04-24T15:32:15.491 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_bluefs_env_mirror": "false", 2024-04-24T15:32:15.491 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-04-24T15:32:15.491 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-04-24T15:32:15.491 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-04-24T15:32:15.491 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-04-24T15:32:15.491 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-04-24T15:32:15.491 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-04-24T15:32:15.491 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_autotune": "true", 2024-04-24T15:32:15.491 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-04-24T15:32:15.492 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-04-24T15:32:15.492 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-04-24T15:32:15.492 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-04-24T15:32:15.492 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_size": "0", 2024-04-24T15:32:15.492 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-04-24T15:32:15.492 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-04-24T15:32:15.492 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-04-24T15:32:15.492 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-04-24T15:32:15.492 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_type": "2q", 2024-04-24T15:32:15.492 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-04-24T15:32:15.492 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_clone_cow": "true", 2024-04-24T15:32:15.492 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_compression_algorithm": "snappy", 2024-04-24T15:32:15.492 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_compression_max_blob_size": "0", 2024-04-24T15:32:15.493 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-04-24T15:32:15.493 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-04-24T15:32:15.493 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_compression_min_blob_size": "0", 2024-04-24T15:32:15.493 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-04-24T15:32:15.493 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-04-24T15:32:15.493 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_compression_mode": "none", 2024-04-24T15:32:15.493 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-04-24T15:32:15.493 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_csum_type": "crc32c", 2024-04-24T15:32:15.493 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_debug_enforce_settings": "default", 2024-04-24T15:32:15.493 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_debug_freelist": "false", 2024-04-24T15:32:15.493 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_debug_fsck_abort": "false", 2024-04-24T15:32:15.493 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-04-24T15:32:15.493 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_debug_inject_bug21040": "false", 2024-04-24T15:32:15.493 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-04-24T15:32:15.494 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_debug_inject_read_err": "false", 2024-04-24T15:32:15.494 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_debug_legacy_omap": "false", 2024-04-24T15:32:15.494 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-04-24T15:32:15.494 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-04-24T15:32:15.494 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-04-24T15:32:15.494 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-04-24T15:32:15.494 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_debug_prefill": "0.000000", 2024-04-24T15:32:15.494 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-04-24T15:32:15.494 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-04-24T15:32:15.494 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-04-24T15:32:15.494 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_debug_small_allocations": "0", 2024-04-24T15:32:15.494 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-04-24T15:32:15.494 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_default_buffered_read": "true", 2024-04-24T15:32:15.494 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_default_buffered_write": "false", 2024-04-24T15:32:15.495 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_deferred_batch_ops": "0", 2024-04-24T15:32:15.495 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-04-24T15:32:15.495 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-04-24T15:32:15.495 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-04-24T15:32:15.495 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-04-24T15:32:15.495 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-04-24T15:32:15.495 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-04-24T15:32:15.495 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-04-24T15:32:15.495 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fail_eio": "false", 2024-04-24T15:32:15.495 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-04-24T15:32:15.495 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-04-24T15:32:15.495 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-04-24T15:32:15.495 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-04-24T15:32:15.495 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_on_mkfs": "true", 2024-04-24T15:32:15.496 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-04-24T15:32:15.496 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_on_mount": "false", 2024-04-24T15:32:15.496 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-04-24T15:32:15.496 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_on_umount": "false", 2024-04-24T15:32:15.496 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-04-24T15:32:15.496 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-04-24T15:32:15.496 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-04-24T15:32:15.496 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-04-24T15:32:15.496 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-04-24T15:32:15.496 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-04-24T15:32:15.496 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-04-24T15:32:15.496 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-04-24T15:32:15.496 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_ignore_data_csum": "false", 2024-04-24T15:32:15.496 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-04-24T15:32:15.496 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_kvbackend": "rocksdb", 2024-04-24T15:32:15.497 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-04-24T15:32:15.497 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-04-24T15:32:15.497 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_log_op_age": "5.000000", 2024-04-24T15:32:15.497 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_max_alloc_size": "0", 2024-04-24T15:32:15.497 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_max_blob_size": "0", 2024-04-24T15:32:15.497 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-04-24T15:32:15.497 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-04-24T15:32:15.497 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_max_defer_interval": "3.000000", 2024-04-24T15:32:15.497 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_max_deferred_txc": "32", 2024-04-24T15:32:15.497 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_min_alloc_size": "0", 2024-04-24T15:32:15.497 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-04-24T15:32:15.497 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-04-24T15:32:15.497 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_nid_prealloc": "1024", 2024-04-24T15:32:15.497 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_prefer_deferred_size": "0", 2024-04-24T15:32:15.497 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-04-24T15:32:15.497 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-04-24T15:32:15.498 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_qfsck_on_mount": "true", 2024-04-24T15:32:15.498 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_retry_disk_reads": "3", 2024-04-24T15:32:15.498 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_rocksdb_cf": "true", 2024-04-24T15:32:15.498 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L P", 2024-04-24T15:32:15.498 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_rocksdb_options": "compression=kNoCompression,max_write_buffer_number=4,min_write_buffer_number_to_merge=1,recycle_log_file_num=4,write_buffer_size=268435456,writable_file_max_buffer_size=0,compaction_readahead_size=2097152,max_background_compactions=2,max_total_wal_size=1073741824", 2024-04-24T15:32:15.498 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_rocksdb_options_annex": "", 2024-04-24T15:32:15.498 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_spdk_coremask": "0x1", 2024-04-24T15:32:15.498 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_spdk_io_sleep": "5", 2024-04-24T15:32:15.498 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_spdk_max_io_completion": "0", 2024-04-24T15:32:15.498 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_spdk_mem": "512", 2024-04-24T15:32:15.498 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_sync_submit_transaction": "false", 2024-04-24T15:32:15.498 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_throttle_bytes": "67108864", 2024-04-24T15:32:15.498 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_throttle_cost_per_io": "0", 2024-04-24T15:32:15.498 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-04-24T15:32:15.499 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-04-24T15:32:15.499 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-04-24T15:32:15.499 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-04-24T15:32:15.499 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_tracing": "false", 2024-04-24T15:32:15.499 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-04-24T15:32:15.499 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-04-24T15:32:15.499 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_volume_selection_reserved": "0", 2024-04-24T15:32:15.499 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-04-24T15:32:15.499 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-04-24T15:32:15.499 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-04-24T15:32:15.499 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-04-24T15:32:15.499 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-04-24T15:32:15.499 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-04-24T15:32:15.499 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_zero_block_detection": "false", 2024-04-24T15:32:15.499 INFO:teuthology.orchestra.run.smithi096.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-04-24T15:32:15.500 INFO:teuthology.orchestra.run.smithi096.stdout: "cephfs_mirror_action_update_interval": "2", 2024-04-24T15:32:15.500 INFO:teuthology.orchestra.run.smithi096.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-04-24T15:32:15.500 INFO:teuthology.orchestra.run.smithi096.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-04-24T15:32:15.500 INFO:teuthology.orchestra.run.smithi096.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-04-24T15:32:15.500 INFO:teuthology.orchestra.run.smithi096.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-04-24T15:32:15.500 INFO:teuthology.orchestra.run.smithi096.stdout: "cephfs_mirror_mount_timeout": "10", 2024-04-24T15:32:15.500 INFO:teuthology.orchestra.run.smithi096.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-04-24T15:32:15.500 INFO:teuthology.orchestra.run.smithi096.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-04-24T15:32:15.500 INFO:teuthology.orchestra.run.smithi096.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-04-24T15:32:15.500 INFO:teuthology.orchestra.run.smithi096.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-04-24T15:32:15.500 INFO:teuthology.orchestra.run.smithi096.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-04-24T15:32:15.500 INFO:teuthology.orchestra.run.smithi096.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-04-24T15:32:15.500 INFO:teuthology.orchestra.run.smithi096.stdout: "cephx_cluster_require_signatures": "false", 2024-04-24T15:32:15.500 INFO:teuthology.orchestra.run.smithi096.stdout: "cephx_cluster_require_version": "2", 2024-04-24T15:32:15.501 INFO:teuthology.orchestra.run.smithi096.stdout: "cephx_require_signatures": "false", 2024-04-24T15:32:15.501 INFO:teuthology.orchestra.run.smithi096.stdout: "cephx_require_version": "2", 2024-04-24T15:32:15.501 INFO:teuthology.orchestra.run.smithi096.stdout: "cephx_service_require_signatures": "false", 2024-04-24T15:32:15.501 INFO:teuthology.orchestra.run.smithi096.stdout: "cephx_service_require_version": "2", 2024-04-24T15:32:15.501 INFO:teuthology.orchestra.run.smithi096.stdout: "cephx_sign_messages": "true", 2024-04-24T15:32:15.501 INFO:teuthology.orchestra.run.smithi096.stdout: "chdir": "", 2024-04-24T15:32:15.501 INFO:teuthology.orchestra.run.smithi096.stdout: "client_acl_type": "", 2024-04-24T15:32:15.501 INFO:teuthology.orchestra.run.smithi096.stdout: "client_asio_thread_count": "2", 2024-04-24T15:32:15.501 INFO:teuthology.orchestra.run.smithi096.stdout: "client_cache_mid": "0.750000", 2024-04-24T15:32:15.501 INFO:teuthology.orchestra.run.smithi096.stdout: "client_cache_size": "16384", 2024-04-24T15:32:15.501 INFO:teuthology.orchestra.run.smithi096.stdout: "client_caps_release_delay": "5", 2024-04-24T15:32:15.501 INFO:teuthology.orchestra.run.smithi096.stdout: "client_check_pool_perm": "true", 2024-04-24T15:32:15.501 INFO:teuthology.orchestra.run.smithi096.stdout: "client_collect_and_send_global_metrics": "false", 2024-04-24T15:32:15.502 INFO:teuthology.orchestra.run.smithi096.stdout: "client_debug_force_sync_read": "false", 2024-04-24T15:32:15.502 INFO:teuthology.orchestra.run.smithi096.stdout: "client_debug_getattr_caps": "false", 2024-04-24T15:32:15.502 INFO:teuthology.orchestra.run.smithi096.stdout: "client_debug_inject_tick_delay": "0", 2024-04-24T15:32:15.502 INFO:teuthology.orchestra.run.smithi096.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-04-24T15:32:15.502 INFO:teuthology.orchestra.run.smithi096.stdout: "client_die_on_failed_remount": "false", 2024-04-24T15:32:15.502 INFO:teuthology.orchestra.run.smithi096.stdout: "client_dirsize_rbytes": "true", 2024-04-24T15:32:15.502 INFO:teuthology.orchestra.run.smithi096.stdout: "client_force_lazyio": "false", 2024-04-24T15:32:15.502 INFO:teuthology.orchestra.run.smithi096.stdout: "client_fs": "", 2024-04-24T15:32:15.502 INFO:teuthology.orchestra.run.smithi096.stdout: "client_inject_fixed_oldest_tid": "false", 2024-04-24T15:32:15.502 INFO:teuthology.orchestra.run.smithi096.stdout: "client_inject_release_failure": "false", 2024-04-24T15:32:15.502 INFO:teuthology.orchestra.run.smithi096.stdout: "client_max_inline_size": "4096", 2024-04-24T15:32:15.502 INFO:teuthology.orchestra.run.smithi096.stdout: "client_max_retries_on_remount_failure": "5", 2024-04-24T15:32:15.502 INFO:teuthology.orchestra.run.smithi096.stdout: "client_mds_namespace": "", 2024-04-24T15:32:15.502 INFO:teuthology.orchestra.run.smithi096.stdout: "client_metadata": "", 2024-04-24T15:32:15.503 INFO:teuthology.orchestra.run.smithi096.stdout: "client_mount_gid": "-1", 2024-04-24T15:32:15.503 INFO:teuthology.orchestra.run.smithi096.stdout: "client_mount_timeout": "300", 2024-04-24T15:32:15.503 INFO:teuthology.orchestra.run.smithi096.stdout: "client_mount_uid": "-1", 2024-04-24T15:32:15.503 INFO:teuthology.orchestra.run.smithi096.stdout: "client_mountpoint": "/", 2024-04-24T15:32:15.503 INFO:teuthology.orchestra.run.smithi096.stdout: "client_notify_timeout": "10", 2024-04-24T15:32:15.503 INFO:teuthology.orchestra.run.smithi096.stdout: "client_oc": "true", 2024-04-24T15:32:15.503 INFO:teuthology.orchestra.run.smithi096.stdout: "client_oc_max_dirty": "104857600", 2024-04-24T15:32:15.503 INFO:teuthology.orchestra.run.smithi096.stdout: "client_oc_max_dirty_age": "5.000000", 2024-04-24T15:32:15.503 INFO:teuthology.orchestra.run.smithi096.stdout: "client_oc_max_objects": "1000", 2024-04-24T15:32:15.503 INFO:teuthology.orchestra.run.smithi096.stdout: "client_oc_size": "209715200", 2024-04-24T15:32:15.503 INFO:teuthology.orchestra.run.smithi096.stdout: "client_oc_target_dirty": "8388608", 2024-04-24T15:32:15.503 INFO:teuthology.orchestra.run.smithi096.stdout: "client_permissions": "true", 2024-04-24T15:32:15.503 INFO:teuthology.orchestra.run.smithi096.stdout: "client_quota": "true", 2024-04-24T15:32:15.503 INFO:teuthology.orchestra.run.smithi096.stdout: "client_quota_df": "true", 2024-04-24T15:32:15.504 INFO:teuthology.orchestra.run.smithi096.stdout: "client_readahead_max_bytes": "0", 2024-04-24T15:32:15.504 INFO:teuthology.orchestra.run.smithi096.stdout: "client_readahead_max_periods": "4", 2024-04-24T15:32:15.504 INFO:teuthology.orchestra.run.smithi096.stdout: "client_readahead_min": "131072", 2024-04-24T15:32:15.504 INFO:teuthology.orchestra.run.smithi096.stdout: "client_reconnect_stale": "false", 2024-04-24T15:32:15.504 INFO:teuthology.orchestra.run.smithi096.stdout: "client_shutdown_timeout": "30", 2024-04-24T15:32:15.504 INFO:teuthology.orchestra.run.smithi096.stdout: "client_snapdir": ".snap", 2024-04-24T15:32:15.504 INFO:teuthology.orchestra.run.smithi096.stdout: "client_tick_interval": "1", 2024-04-24T15:32:15.504 INFO:teuthology.orchestra.run.smithi096.stdout: "client_trace": "", 2024-04-24T15:32:15.504 INFO:teuthology.orchestra.run.smithi096.stdout: "client_try_dentry_invalidate": "false", 2024-04-24T15:32:15.504 INFO:teuthology.orchestra.run.smithi096.stdout: "client_use_faked_inos": "false", 2024-04-24T15:32:15.504 INFO:teuthology.orchestra.run.smithi096.stdout: "client_use_random_mds": "false", 2024-04-24T15:32:15.504 INFO:teuthology.orchestra.run.smithi096.stdout: "clog_to_graylog": "false", 2024-04-24T15:32:15.504 INFO:teuthology.orchestra.run.smithi096.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-04-24T15:32:15.504 INFO:teuthology.orchestra.run.smithi096.stdout: "clog_to_graylog_port": "12201", 2024-04-24T15:32:15.505 INFO:teuthology.orchestra.run.smithi096.stdout: "clog_to_monitors": "default=true", 2024-04-24T15:32:15.505 INFO:teuthology.orchestra.run.smithi096.stdout: "clog_to_syslog": "false", 2024-04-24T15:32:15.505 INFO:teuthology.orchestra.run.smithi096.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-04-24T15:32:15.505 INFO:teuthology.orchestra.run.smithi096.stdout: "clog_to_syslog_level": "info", 2024-04-24T15:32:15.505 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addr": "-", 2024-04-24T15:32:15.505 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_network": "", 2024-04-24T15:32:15.505 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_network_interface": "", 2024-04-24T15:32:15.505 INFO:teuthology.orchestra.run.smithi096.stdout: "compressor_zlib_isal": "false", 2024-04-24T15:32:15.505 INFO:teuthology.orchestra.run.smithi096.stdout: "compressor_zlib_level": "5", 2024-04-24T15:32:15.505 INFO:teuthology.orchestra.run.smithi096.stdout: "compressor_zlib_winsize": "-15", 2024-04-24T15:32:15.505 INFO:teuthology.orchestra.run.smithi096.stdout: "compressor_zstd_level": "1", 2024-04-24T15:32:15.505 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-04-24T15:32:15.505 INFO:teuthology.orchestra.run.smithi096.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-04-24T15:32:15.505 INFO:teuthology.orchestra.run.smithi096.stdout: "crimson_alien_op_num_threads": "6", 2024-04-24T15:32:15.505 INFO:teuthology.orchestra.run.smithi096.stdout: "crimson_alien_thread_cpu_cores": "", 2024-04-24T15:32:15.506 INFO:teuthology.orchestra.run.smithi096.stdout: "crimson_osd_obc_lru_size": "10", 2024-04-24T15:32:15.506 INFO:teuthology.orchestra.run.smithi096.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-04-24T15:32:15.506 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_location": "", 2024-04-24T15:32:15.506 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_location_hook": "", 2024-04-24T15:32:15.506 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_location_hook_timeout": "10", 2024-04-24T15:32:15.506 INFO:teuthology.orchestra.run.smithi096.stdout: "daemonize": "false", 2024-04-24T15:32:15.506 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_alienstore": "0/5", 2024-04-24T15:32:15.506 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_allow_any_pool_priority": "false", 2024-04-24T15:32:15.506 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_asok": "1/5", 2024-04-24T15:32:15.506 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_asok_assert_abort": "false", 2024-04-24T15:32:15.506 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_asserts_on_shutdown": "true", 2024-04-24T15:32:15.506 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_auth": "20/20", 2024-04-24T15:32:15.506 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_bdev": "1/3", 2024-04-24T15:32:15.506 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_bluefs": "1/5", 2024-04-24T15:32:15.507 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_bluestore": "1/5", 2024-04-24T15:32:15.507 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_buffer": "0/1", 2024-04-24T15:32:15.507 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_ceph_exporter": "1/5", 2024-04-24T15:32:15.507 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_cephfs_mirror": "0/5", 2024-04-24T15:32:15.507 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_cephsqlite": "0/5", 2024-04-24T15:32:15.507 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_civetweb": "1/10", 2024-04-24T15:32:15.507 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_client": "0/5", 2024-04-24T15:32:15.507 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_compressor": "1/5", 2024-04-24T15:32:15.507 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_context": "0/1", 2024-04-24T15:32:15.507 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_crush": "1/1", 2024-04-24T15:32:15.507 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_crypto": "1/5", 2024-04-24T15:32:15.507 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_deliberately_leak_memory": "false", 2024-04-24T15:32:15.507 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_disable_randomized_ping": "false", 2024-04-24T15:32:15.507 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_dpdk": "1/5", 2024-04-24T15:32:15.507 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_eventtrace": "1/5", 2024-04-24T15:32:15.507 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_filer": "0/1", 2024-04-24T15:32:15.508 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_filestore": "1/3", 2024-04-24T15:32:15.508 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_finisher": "1/1", 2024-04-24T15:32:15.508 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_fuse": "1/5", 2024-04-24T15:32:15.508 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_heartbeat_testing_span": "0", 2024-04-24T15:32:15.508 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_heartbeatmap": "1/5", 2024-04-24T15:32:15.508 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_immutable_obj_cache": "0/5", 2024-04-24T15:32:15.508 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_javaclient": "1/5", 2024-04-24T15:32:15.508 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_journal": "1/3", 2024-04-24T15:32:15.508 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_journaler": "0/5", 2024-04-24T15:32:15.508 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_kstore": "1/5", 2024-04-24T15:32:15.508 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_leveldb": "4/5", 2024-04-24T15:32:15.508 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_lockdep": "0/1", 2024-04-24T15:32:15.508 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_mclock": "1/5", 2024-04-24T15:32:15.509 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_mds": "1/5", 2024-04-24T15:32:15.509 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_mds_balancer": "1/5", 2024-04-24T15:32:15.509 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_mds_locker": "1/5", 2024-04-24T15:32:15.509 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_mds_log": "1/5", 2024-04-24T15:32:15.509 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_mds_log_expire": "1/5", 2024-04-24T15:32:15.509 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_mds_migrator": "1/5", 2024-04-24T15:32:15.509 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_memdb": "4/5", 2024-04-24T15:32:15.509 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_mgr": "2/5", 2024-04-24T15:32:15.509 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_mgrc": "1/5", 2024-04-24T15:32:15.509 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_mon": "20/20", 2024-04-24T15:32:15.509 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_monc": "0/10", 2024-04-24T15:32:15.509 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_ms": "1/1", 2024-04-24T15:32:15.509 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_none": "0/5", 2024-04-24T15:32:15.509 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_objclass": "0/5", 2024-04-24T15:32:15.510 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_objectcacher": "0/5", 2024-04-24T15:32:15.510 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_objecter": "0/1", 2024-04-24T15:32:15.510 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_optracker": "0/5", 2024-04-24T15:32:15.510 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_osd": "1/5", 2024-04-24T15:32:15.510 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_paxos": "20/20", 2024-04-24T15:32:15.510 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_perfcounter": "1/5", 2024-04-24T15:32:15.510 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_prioritycache": "1/5", 2024-04-24T15:32:15.510 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_rados": "0/5", 2024-04-24T15:32:15.510 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_rbd": "0/5", 2024-04-24T15:32:15.510 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_rbd_mirror": "0/5", 2024-04-24T15:32:15.510 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_rbd_pwl": "0/5", 2024-04-24T15:32:15.510 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_rbd_replay": "0/5", 2024-04-24T15:32:15.511 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_refs": "0/0", 2024-04-24T15:32:15.511 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_reserver": "1/1", 2024-04-24T15:32:15.511 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_rgw": "1/5", 2024-04-24T15:32:15.511 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_rgw_datacache": "1/5", 2024-04-24T15:32:15.511 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_rgw_sync": "1/5", 2024-04-24T15:32:15.511 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_rocksdb": "4/5", 2024-04-24T15:32:15.511 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_seastore": "0/5", 2024-04-24T15:32:15.511 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_seastore_cache": "0/5", 2024-04-24T15:32:15.511 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_seastore_cleaner": "0/5", 2024-04-24T15:32:15.511 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_seastore_device": "0/5", 2024-04-24T15:32:15.511 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_seastore_journal": "0/5", 2024-04-24T15:32:15.511 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_seastore_lba": "0/5", 2024-04-24T15:32:15.511 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_seastore_odata": "0/5", 2024-04-24T15:32:15.511 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_seastore_omap": "0/5", 2024-04-24T15:32:15.512 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_seastore_onode": "0/5", 2024-04-24T15:32:15.512 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_seastore_tm": "0/5", 2024-04-24T15:32:15.512 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_striper": "0/1", 2024-04-24T15:32:15.512 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_test": "0/5", 2024-04-24T15:32:15.512 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_throttle": "1/1", 2024-04-24T15:32:15.512 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_timer": "0/1", 2024-04-24T15:32:15.512 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_tp": "0/5", 2024-04-24T15:32:15.512 INFO:teuthology.orchestra.run.smithi096.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-04-24T15:32:15.512 INFO:teuthology.orchestra.run.smithi096.stdout: "device_failure_prediction_mode": "none", 2024-04-24T15:32:15.512 INFO:teuthology.orchestra.run.smithi096.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-04-24T15:32:15.512 INFO:teuthology.orchestra.run.smithi096.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2024-04-24T15:32:15.512 INFO:teuthology.orchestra.run.smithi096.stdout: "err_to_graylog": "false", 2024-04-24T15:32:15.512 INFO:teuthology.orchestra.run.smithi096.stdout: "err_to_journald": "false", 2024-04-24T15:32:15.512 INFO:teuthology.orchestra.run.smithi096.stdout: "err_to_stderr": "true", 2024-04-24T15:32:15.513 INFO:teuthology.orchestra.run.smithi096.stdout: "err_to_syslog": "false", 2024-04-24T15:32:15.513 INFO:teuthology.orchestra.run.smithi096.stdout: "event_tracing": "false", 2024-04-24T15:32:15.513 INFO:teuthology.orchestra.run.smithi096.stdout: "exporter_addr": "0.0.0.0", 2024-04-24T15:32:15.513 INFO:teuthology.orchestra.run.smithi096.stdout: "exporter_http_port": "9926", 2024-04-24T15:32:15.513 INFO:teuthology.orchestra.run.smithi096.stdout: "exporter_prio_limit": "5", 2024-04-24T15:32:15.513 INFO:teuthology.orchestra.run.smithi096.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-04-24T15:32:15.513 INFO:teuthology.orchestra.run.smithi096.stdout: "exporter_sort_metrics": "true", 2024-04-24T15:32:15.513 INFO:teuthology.orchestra.run.smithi096.stdout: "exporter_stats_period": "5", 2024-04-24T15:32:15.513 INFO:teuthology.orchestra.run.smithi096.stdout: "fake_statfs_for_testing": "0", 2024-04-24T15:32:15.513 INFO:teuthology.orchestra.run.smithi096.stdout: "fatal_signal_handlers": "true", 2024-04-24T15:32:15.513 INFO:teuthology.orchestra.run.smithi096.stdout: "filer_max_purge_ops": "10", 2024-04-24T15:32:15.513 INFO:teuthology.orchestra.run.smithi096.stdout: "filer_max_truncate_ops": "128", 2024-04-24T15:32:15.513 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_apply_finisher_threads": "1", 2024-04-24T15:32:15.513 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_blackhole": "false", 2024-04-24T15:32:15.513 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_btrfs_clone_range": "true", 2024-04-24T15:32:15.514 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_btrfs_snap": "true", 2024-04-24T15:32:15.514 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_caller_concurrency": "10", 2024-04-24T15:32:15.514 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_collect_device_partition_information": "true", 2024-04-24T15:32:15.514 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_commit_timeout": "600.000000", 2024-04-24T15:32:15.514 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_debug_inject_read_err": "false", 2024-04-24T15:32:15.514 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_debug_omap_check": "false", 2024-04-24T15:32:15.514 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_debug_random_read_err": "0.000000", 2024-04-24T15:32:15.514 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_debug_verify_split": "false", 2024-04-24T15:32:15.514 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_dump_file": "", 2024-04-24T15:32:15.514 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-04-24T15:32:15.514 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-04-24T15:32:15.514 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_fadvise": "true", 2024-04-24T15:32:15.514 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_fail_eio": "true", 2024-04-24T15:32:15.514 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_fd_cache_shards": "16", 2024-04-24T15:32:15.515 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_fd_cache_size": "128", 2024-04-24T15:32:15.515 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_fiemap": "false", 2024-04-24T15:32:15.515 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_fiemap_threshold": "4096", 2024-04-24T15:32:15.515 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-04-24T15:32:15.515 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_index_retry_probability": "0.000000", 2024-04-24T15:32:15.515 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_inject_stall": "0", 2024-04-24T15:32:15.515 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_journal_parallel": "false", 2024-04-24T15:32:15.515 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_journal_trailing": "false", 2024-04-24T15:32:15.515 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_journal_writeahead": "false", 2024-04-24T15:32:15.515 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_kill_at": "0", 2024-04-24T15:32:15.515 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-04-24T15:32:15.515 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_max_inline_xattr_size": "0", 2024-04-24T15:32:15.515 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-04-24T15:32:15.516 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-04-24T15:32:15.516 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-04-24T15:32:15.516 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_max_inline_xattrs": "0", 2024-04-24T15:32:15.516 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-04-24T15:32:15.516 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_max_inline_xattrs_other": "2", 2024-04-24T15:32:15.516 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-04-24T15:32:15.516 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_max_sync_interval": "5.000000", 2024-04-24T15:32:15.516 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_max_xattr_value_size": "0", 2024-04-24T15:32:15.516 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-04-24T15:32:15.516 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-04-24T15:32:15.516 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-04-24T15:32:15.516 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_merge_threshold": "-10", 2024-04-24T15:32:15.516 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_min_sync_interval": "0.010000", 2024-04-24T15:32:15.517 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_odsync_write": "false", 2024-04-24T15:32:15.517 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_omap_backend": "rocksdb", 2024-04-24T15:32:15.517 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_omap_backend_path": "", 2024-04-24T15:32:15.517 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_omap_header_cache_size": "1024", 2024-04-24T15:32:15.517 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_ondisk_finisher_threads": "1", 2024-04-24T15:32:15.517 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-04-24T15:32:15.517 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_op_thread_timeout": "60", 2024-04-24T15:32:15.517 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_op_threads": "2", 2024-04-24T15:32:15.517 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_punch_hole": "false", 2024-04-24T15:32:15.517 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-04-24T15:32:15.517 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-04-24T15:32:15.517 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-04-24T15:32:15.517 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-04-24T15:32:15.517 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-04-24T15:32:15.518 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_queue_max_bytes": "104857600", 2024-04-24T15:32:15.518 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-04-24T15:32:15.518 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-04-24T15:32:15.518 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-04-24T15:32:15.518 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_queue_max_ops": "50", 2024-04-24T15:32:15.518 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-04-24T15:32:15.518 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_seek_data_hole": "false", 2024-04-24T15:32:15.518 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_sloppy_crc": "false", 2024-04-24T15:32:15.518 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-04-24T15:32:15.518 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_splice": "false", 2024-04-24T15:32:15.518 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_split_multiple": "2", 2024-04-24T15:32:15.518 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_split_rand_factor": "20", 2024-04-24T15:32:15.519 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_update_to": "1000", 2024-04-24T15:32:15.519 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-04-24T15:32:15.519 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-04-24T15:32:15.519 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-04-24T15:32:15.519 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-04-24T15:32:15.519 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-04-24T15:32:15.519 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-04-24T15:32:15.519 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_wbthrottle_enable": "true", 2024-04-24T15:32:15.519 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-04-24T15:32:15.519 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-04-24T15:32:15.519 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-04-24T15:32:15.519 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-04-24T15:32:15.519 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-04-24T15:32:15.520 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-04-24T15:32:15.520 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_xfs_extsize": "false", 2024-04-24T15:32:15.520 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_zfs_snap": "false", 2024-04-24T15:32:15.520 INFO:teuthology.orchestra.run.smithi096.stdout: "fio_dir": "/tmp/fio", 2024-04-24T15:32:15.520 INFO:teuthology.orchestra.run.smithi096.stdout: "fsid": "253d426c-62cb-4194-8514-6cd970fd85c7", 2024-04-24T15:32:15.520 INFO:teuthology.orchestra.run.smithi096.stdout: "fuse_allow_other": "true", 2024-04-24T15:32:15.520 INFO:teuthology.orchestra.run.smithi096.stdout: "fuse_atomic_o_trunc": "true", 2024-04-24T15:32:15.520 INFO:teuthology.orchestra.run.smithi096.stdout: "fuse_big_writes": "true", 2024-04-24T15:32:15.520 INFO:teuthology.orchestra.run.smithi096.stdout: "fuse_debug": "false", 2024-04-24T15:32:15.520 INFO:teuthology.orchestra.run.smithi096.stdout: "fuse_default_permissions": "false", 2024-04-24T15:32:15.520 INFO:teuthology.orchestra.run.smithi096.stdout: "fuse_disable_pagecache": "false", 2024-04-24T15:32:15.520 INFO:teuthology.orchestra.run.smithi096.stdout: "fuse_max_write": "0", 2024-04-24T15:32:15.520 INFO:teuthology.orchestra.run.smithi096.stdout: "fuse_multithreaded": "true", 2024-04-24T15:32:15.520 INFO:teuthology.orchestra.run.smithi096.stdout: "fuse_require_active_mds": "true", 2024-04-24T15:32:15.521 INFO:teuthology.orchestra.run.smithi096.stdout: "fuse_set_user_groups": "true", 2024-04-24T15:32:15.521 INFO:teuthology.orchestra.run.smithi096.stdout: "fuse_splice_move": "true", 2024-04-24T15:32:15.521 INFO:teuthology.orchestra.run.smithi096.stdout: "fuse_splice_read": "true", 2024-04-24T15:32:15.521 INFO:teuthology.orchestra.run.smithi096.stdout: "fuse_splice_write": "true", 2024-04-24T15:32:15.521 INFO:teuthology.orchestra.run.smithi096.stdout: "fuse_syncfs_on_mksnap": "true", 2024-04-24T15:32:15.521 INFO:teuthology.orchestra.run.smithi096.stdout: "fuse_use_invalidate_cb": "true", 2024-04-24T15:32:15.521 INFO:teuthology.orchestra.run.smithi096.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-04-24T15:32:15.521 INFO:teuthology.orchestra.run.smithi096.stdout: "gss_target_name": "ceph", 2024-04-24T15:32:15.521 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_file": "", 2024-04-24T15:32:15.521 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_inject_failure": "0", 2024-04-24T15:32:15.521 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_interval": "5", 2024-04-24T15:32:15.521 INFO:teuthology.orchestra.run.smithi096.stdout: "host": "smithi096", 2024-04-24T15:32:15.521 INFO:teuthology.orchestra.run.smithi096.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-04-24T15:32:15.521 INFO:teuthology.orchestra.run.smithi096.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-04-24T15:32:15.522 INFO:teuthology.orchestra.run.smithi096.stdout: "immutable_object_cache_max_size": "1073741824", 2024-04-24T15:32:15.522 INFO:teuthology.orchestra.run.smithi096.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-04-24T15:32:15.522 INFO:teuthology.orchestra.run.smithi096.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-04-24T15:32:15.522 INFO:teuthology.orchestra.run.smithi096.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-04-24T15:32:15.522 INFO:teuthology.orchestra.run.smithi096.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-04-24T15:32:15.522 INFO:teuthology.orchestra.run.smithi096.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-04-24T15:32:15.522 INFO:teuthology.orchestra.run.smithi096.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-04-24T15:32:15.522 INFO:teuthology.orchestra.run.smithi096.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-04-24T15:32:15.522 INFO:teuthology.orchestra.run.smithi096.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-04-24T15:32:15.522 INFO:teuthology.orchestra.run.smithi096.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-04-24T15:32:15.522 INFO:teuthology.orchestra.run.smithi096.stdout: "immutable_object_cache_watermark": "0.900000", 2024-04-24T15:32:15.522 INFO:teuthology.orchestra.run.smithi096.stdout: "inject_early_sigterm": "false", 2024-04-24T15:32:15.522 INFO:teuthology.orchestra.run.smithi096.stdout: "jaeger_tracing_enable": "false", 2024-04-24T15:32:15.523 INFO:teuthology.orchestra.run.smithi096.stdout: "journal_aio": "true", 2024-04-24T15:32:15.523 INFO:teuthology.orchestra.run.smithi096.stdout: "journal_align_min_size": "65536", 2024-04-24T15:32:15.523 INFO:teuthology.orchestra.run.smithi096.stdout: "journal_block_align": "true", 2024-04-24T15:32:15.523 INFO:teuthology.orchestra.run.smithi096.stdout: "journal_block_size": "4096", 2024-04-24T15:32:15.523 INFO:teuthology.orchestra.run.smithi096.stdout: "journal_dio": "true", 2024-04-24T15:32:15.523 INFO:teuthology.orchestra.run.smithi096.stdout: "journal_discard": "false", 2024-04-24T15:32:15.523 INFO:teuthology.orchestra.run.smithi096.stdout: "journal_force_aio": "false", 2024-04-24T15:32:15.523 INFO:teuthology.orchestra.run.smithi096.stdout: "journal_ignore_corruption": "false", 2024-04-24T15:32:15.523 INFO:teuthology.orchestra.run.smithi096.stdout: "journal_max_write_bytes": "10485760", 2024-04-24T15:32:15.523 INFO:teuthology.orchestra.run.smithi096.stdout: "journal_max_write_entries": "100", 2024-04-24T15:32:15.523 INFO:teuthology.orchestra.run.smithi096.stdout: "journal_replay_from": "0", 2024-04-24T15:32:15.523 INFO:teuthology.orchestra.run.smithi096.stdout: "journal_throttle_high_multiple": "0.000000", 2024-04-24T15:32:15.523 INFO:teuthology.orchestra.run.smithi096.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-04-24T15:32:15.523 INFO:teuthology.orchestra.run.smithi096.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-04-24T15:32:15.524 INFO:teuthology.orchestra.run.smithi096.stdout: "journal_throttle_max_multiple": "0.000000", 2024-04-24T15:32:15.524 INFO:teuthology.orchestra.run.smithi096.stdout: "journal_write_header_frequency": "0", 2024-04-24T15:32:15.524 INFO:teuthology.orchestra.run.smithi096.stdout: "journal_zero_on_create": "false", 2024-04-24T15:32:15.524 INFO:teuthology.orchestra.run.smithi096.stdout: "journaler_prefetch_periods": "10", 2024-04-24T15:32:15.524 INFO:teuthology.orchestra.run.smithi096.stdout: "journaler_prezero_periods": "5", 2024-04-24T15:32:15.524 INFO:teuthology.orchestra.run.smithi096.stdout: "journaler_write_head_interval": "15", 2024-04-24T15:32:15.524 INFO:teuthology.orchestra.run.smithi096.stdout: "key": "", 2024-04-24T15:32:15.524 INFO:teuthology.orchestra.run.smithi096.stdout: "keyfile": "", 2024-04-24T15:32:15.524 INFO:teuthology.orchestra.run.smithi096.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-04-24T15:32:15.524 INFO:teuthology.orchestra.run.smithi096.stdout: "kstore_backend": "rocksdb", 2024-04-24T15:32:15.524 INFO:teuthology.orchestra.run.smithi096.stdout: "kstore_default_stripe_size": "65536", 2024-04-24T15:32:15.524 INFO:teuthology.orchestra.run.smithi096.stdout: "kstore_fsck_on_mount": "false", 2024-04-24T15:32:15.524 INFO:teuthology.orchestra.run.smithi096.stdout: "kstore_fsck_on_mount_deep": "true", 2024-04-24T15:32:15.524 INFO:teuthology.orchestra.run.smithi096.stdout: "kstore_max_bytes": "67108864", 2024-04-24T15:32:15.524 INFO:teuthology.orchestra.run.smithi096.stdout: "kstore_max_ops": "512", 2024-04-24T15:32:15.525 INFO:teuthology.orchestra.run.smithi096.stdout: "kstore_nid_prealloc": "1024", 2024-04-24T15:32:15.525 INFO:teuthology.orchestra.run.smithi096.stdout: "kstore_onode_map_size": "1024", 2024-04-24T15:32:15.525 INFO:teuthology.orchestra.run.smithi096.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-04-24T15:32:15.525 INFO:teuthology.orchestra.run.smithi096.stdout: "kstore_sync_submit_transaction": "false", 2024-04-24T15:32:15.525 INFO:teuthology.orchestra.run.smithi096.stdout: "kstore_sync_transaction": "false", 2024-04-24T15:32:15.525 INFO:teuthology.orchestra.run.smithi096.stdout: "leveldb_block_size": "65536", 2024-04-24T15:32:15.525 INFO:teuthology.orchestra.run.smithi096.stdout: "leveldb_bloom_size": "0", 2024-04-24T15:32:15.525 INFO:teuthology.orchestra.run.smithi096.stdout: "leveldb_cache_size": "536870912", 2024-04-24T15:32:15.525 INFO:teuthology.orchestra.run.smithi096.stdout: "leveldb_compact_on_mount": "false", 2024-04-24T15:32:15.525 INFO:teuthology.orchestra.run.smithi096.stdout: "leveldb_compression": "false", 2024-04-24T15:32:15.525 INFO:teuthology.orchestra.run.smithi096.stdout: "leveldb_log": "", 2024-04-24T15:32:15.525 INFO:teuthology.orchestra.run.smithi096.stdout: "leveldb_log_to_ceph_log": "true", 2024-04-24T15:32:15.525 INFO:teuthology.orchestra.run.smithi096.stdout: "leveldb_max_open_files": "0", 2024-04-24T15:32:15.526 INFO:teuthology.orchestra.run.smithi096.stdout: "leveldb_paranoid": "false", 2024-04-24T15:32:15.526 INFO:teuthology.orchestra.run.smithi096.stdout: "leveldb_write_buffer_size": "33554432", 2024-04-24T15:32:15.526 INFO:teuthology.orchestra.run.smithi096.stdout: "librados_thread_count": "2", 2024-04-24T15:32:15.526 INFO:teuthology.orchestra.run.smithi096.stdout: "lockdep": "false", 2024-04-24T15:32:15.526 INFO:teuthology.orchestra.run.smithi096.stdout: "lockdep_force_backtrace": "false", 2024-04-24T15:32:15.526 INFO:teuthology.orchestra.run.smithi096.stdout: "log_coarse_timestamps": "true", 2024-04-24T15:32:15.526 INFO:teuthology.orchestra.run.smithi096.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-04-24T15:32:15.526 INFO:teuthology.orchestra.run.smithi096.stdout: "log_flush_on_exit": "false", 2024-04-24T15:32:15.526 INFO:teuthology.orchestra.run.smithi096.stdout: "log_graylog_host": "127.0.0.1", 2024-04-24T15:32:15.526 INFO:teuthology.orchestra.run.smithi096.stdout: "log_graylog_port": "12201", 2024-04-24T15:32:15.526 INFO:teuthology.orchestra.run.smithi096.stdout: "log_max_new": "1000", 2024-04-24T15:32:15.526 INFO:teuthology.orchestra.run.smithi096.stdout: "log_max_recent": "10000", 2024-04-24T15:32:15.526 INFO:teuthology.orchestra.run.smithi096.stdout: "log_stderr_prefix": "", 2024-04-24T15:32:15.527 INFO:teuthology.orchestra.run.smithi096.stdout: "log_stop_at_utilization": "0.970000", 2024-04-24T15:32:15.527 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": "true", 2024-04-24T15:32:15.527 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_graylog": "false", 2024-04-24T15:32:15.527 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_journald": "false", 2024-04-24T15:32:15.527 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_stderr": "false", 2024-04-24T15:32:15.527 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_syslog": "false", 2024-04-24T15:32:15.527 INFO:teuthology.orchestra.run.smithi096.stdout: "max_rotating_auth_attempts": "10", 2024-04-24T15:32:15.527 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-04-24T15:32:15.527 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_action_on_write_error": "1", 2024-04-24T15:32:15.527 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_alternate_name_max": "8192", 2024-04-24T15:32:15.527 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_asio_thread_count": "2", 2024-04-24T15:32:15.527 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_bal_export_pin": "true", 2024-04-24T15:32:15.527 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_bal_fragment_dirs": "true", 2024-04-24T15:32:15.527 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-04-24T15:32:15.528 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_bal_fragment_interval": "5", 2024-04-24T15:32:15.528 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_bal_fragment_size_max": "100000", 2024-04-24T15:32:15.528 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_bal_idle_threshold": "0.000000", 2024-04-24T15:32:15.528 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_bal_interval": "10", 2024-04-24T15:32:15.528 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_bal_max": "-1", 2024-04-24T15:32:15.528 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_bal_max_until": "-1", 2024-04-24T15:32:15.528 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_bal_merge_size": "50", 2024-04-24T15:32:15.528 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_bal_midchunk": "0.300000", 2024-04-24T15:32:15.528 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_bal_min_rebalance": "0.100000", 2024-04-24T15:32:15.528 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_bal_min_start": "0.200000", 2024-04-24T15:32:15.528 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_bal_minchunk": "0.001000", 2024-04-24T15:32:15.528 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_bal_mode": "0", 2024-04-24T15:32:15.528 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_bal_need_max": "1.200000", 2024-04-24T15:32:15.528 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_bal_need_min": "0.800000", 2024-04-24T15:32:15.528 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-04-24T15:32:15.528 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_bal_sample_interval": "3.000000", 2024-04-24T15:32:15.529 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_bal_split_bits": "3", 2024-04-24T15:32:15.529 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_bal_split_rd": "25000.000000", 2024-04-24T15:32:15.529 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_bal_split_size": "10000", 2024-04-24T15:32:15.529 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_bal_split_wr": "10000.000000", 2024-04-24T15:32:15.529 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_bal_target_decay": "10.000000", 2024-04-24T15:32:15.529 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-04-24T15:32:15.529 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_beacon_grace": "15.000000", 2024-04-24T15:32:15.529 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_beacon_interval": "4.000000", 2024-04-24T15:32:15.529 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_beacon_mon_down_grace": "60", 2024-04-24T15:32:15.529 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_cache_memory_limit": "4294967296", 2024-04-24T15:32:15.529 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_cache_mid": "0.700000", 2024-04-24T15:32:15.529 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_cache_release_free_interval": "10", 2024-04-24T15:32:15.529 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_cache_reservation": "0.050000", 2024-04-24T15:32:15.529 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-04-24T15:32:15.529 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_cache_trim_interval": "1", 2024-04-24T15:32:15.529 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_cache_trim_threshold": "262144", 2024-04-24T15:32:15.530 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-04-24T15:32:15.530 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-04-24T15:32:15.530 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_client_delegate_inos_pct": "50", 2024-04-24T15:32:15.530 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_client_prealloc_inos": "1000", 2024-04-24T15:32:15.530 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-04-24T15:32:15.530 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_connect_bootstrapping": "false", 2024-04-24T15:32:15.530 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_damage_table_max_entries": "10000", 2024-04-24T15:32:15.530 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-04-24T15:32:15.530 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_debug_auth_pins": "false", 2024-04-24T15:32:15.530 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_debug_frag": "false", 2024-04-24T15:32:15.530 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_debug_scatterstat": "false", 2024-04-24T15:32:15.530 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_debug_subtrees": "false", 2024-04-24T15:32:15.530 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_decay_halflife": "5.000000", 2024-04-24T15:32:15.530 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_default_dir_hash": "2", 2024-04-24T15:32:15.531 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_defer_session_stale": "true", 2024-04-24T15:32:15.531 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_deny_all_reconnect": "false", 2024-04-24T15:32:15.531 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_dir_keys_per_op": "16384", 2024-04-24T15:32:15.531 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_dir_max_commit_size": "10", 2024-04-24T15:32:15.531 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_dir_max_entries": "0", 2024-04-24T15:32:15.531 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_dirstat_min_interval": "1.000000", 2024-04-24T15:32:15.531 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_dump_cache_after_rejoin": "false", 2024-04-24T15:32:15.531 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_dump_cache_on_map": "false", 2024-04-24T15:32:15.531 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_dump_cache_threshold_file": "0", 2024-04-24T15:32:15.531 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-04-24T15:32:15.531 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_early_reply": "true", 2024-04-24T15:32:15.531 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_enable_op_tracker": "true", 2024-04-24T15:32:15.531 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_enforce_unique_name": "true", 2024-04-24T15:32:15.531 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_export_ephemeral_distributed": "true", 2024-04-24T15:32:15.531 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-04-24T15:32:15.531 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_export_ephemeral_random": "true", 2024-04-24T15:32:15.532 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-04-24T15:32:15.532 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_forward_all_requests_to_auth": "false", 2024-04-24T15:32:15.532 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-04-24T15:32:15.532 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-04-24T15:32:15.532 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-04-24T15:32:15.532 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_health_cache_threshold": "1.500000", 2024-04-24T15:32:15.532 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_health_summarize_threshold": "10", 2024-04-24T15:32:15.532 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_heartbeat_grace": "15.000000", 2024-04-24T15:32:15.532 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_heartbeat_reset_grace": "1000", 2024-04-24T15:32:15.532 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_inject_health_dummy": "false", 2024-04-24T15:32:15.532 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-04-24T15:32:15.532 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_inject_migrator_session_race": "false", 2024-04-24T15:32:15.532 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-04-24T15:32:15.532 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-04-24T15:32:15.532 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-04-24T15:32:15.532 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_join_fs": "", 2024-04-24T15:32:15.533 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_journal_format": "1", 2024-04-24T15:32:15.533 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_kill_create_at": "0", 2024-04-24T15:32:15.533 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_kill_export_at": "0", 2024-04-24T15:32:15.533 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_kill_import_at": "0", 2024-04-24T15:32:15.533 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_kill_journal_at": "0", 2024-04-24T15:32:15.533 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_kill_journal_expire_at": "0", 2024-04-24T15:32:15.533 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_kill_journal_replay_at": "0", 2024-04-24T15:32:15.533 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_kill_link_at": "0", 2024-04-24T15:32:15.533 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_kill_mdstable_at": "0", 2024-04-24T15:32:15.533 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_kill_openc_at": "0", 2024-04-24T15:32:15.533 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_kill_rename_at": "0", 2024-04-24T15:32:15.533 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_kill_skip_replaying_inotable": "false", 2024-04-24T15:32:15.533 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_log_events_per_segment": "1024", 2024-04-24T15:32:15.533 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_log_max_events": "-1", 2024-04-24T15:32:15.533 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_log_max_segments": "128", 2024-04-24T15:32:15.533 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_log_pause": "false", 2024-04-24T15:32:15.534 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_log_segment_size": "0", 2024-04-24T15:32:15.534 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_log_skip_corrupt_events": "false", 2024-04-24T15:32:15.534 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_log_warn_factor": "2.000000", 2024-04-24T15:32:15.534 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_max_caps_per_client": "1048576", 2024-04-24T15:32:15.534 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_max_completed_flushes": "100000", 2024-04-24T15:32:15.534 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_max_completed_requests": "100000", 2024-04-24T15:32:15.534 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_max_export_size": "20971520", 2024-04-24T15:32:15.534 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_max_file_recover": "32", 2024-04-24T15:32:15.534 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_max_purge_files": "64", 2024-04-24T15:32:15.534 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_max_purge_ops": "8192", 2024-04-24T15:32:15.534 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-04-24T15:32:15.534 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-04-24T15:32:15.534 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_max_snaps_per_dir": "100", 2024-04-24T15:32:15.534 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_max_xattr_pairs_size": "65536", 2024-04-24T15:32:15.534 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_metrics_update_interval": "2", 2024-04-24T15:32:15.535 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_min_caps_per_client": "100", 2024-04-24T15:32:15.535 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_min_caps_working_set": "10000", 2024-04-24T15:32:15.535 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-04-24T15:32:15.535 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_numa_node": "-1", 2024-04-24T15:32:15.535 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-04-24T15:32:15.535 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_op_complaint_time": "30.000000", 2024-04-24T15:32:15.535 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_op_history_duration": "600", 2024-04-24T15:32:15.535 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_op_history_size": "20", 2024-04-24T15:32:15.535 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_op_history_slow_op_size": "20", 2024-04-24T15:32:15.535 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_op_history_slow_op_threshold": "10", 2024-04-24T15:32:15.535 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_op_log_threshold": "5", 2024-04-24T15:32:15.535 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_ping_grace": "15", 2024-04-24T15:32:15.535 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_ping_interval": "5", 2024-04-24T15:32:15.535 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-04-24T15:32:15.535 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-04-24T15:32:15.536 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_recall_max_caps": "30000", 2024-04-24T15:32:15.536 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-04-24T15:32:15.536 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_recall_max_decay_threshold": "131072", 2024-04-24T15:32:15.536 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-04-24T15:32:15.536 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_recall_warning_threshold": "262144", 2024-04-24T15:32:15.536 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_reconnect_timeout": "45.000000", 2024-04-24T15:32:15.536 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_replay_interval": "1.000000", 2024-04-24T15:32:15.536 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-04-24T15:32:15.536 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-04-24T15:32:15.536 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_root_ino_gid": "0", 2024-04-24T15:32:15.536 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_root_ino_uid": "0", 2024-04-24T15:32:15.536 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-04-24T15:32:15.536 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_session_blocklist_on_evict": "true", 2024-04-24T15:32:15.536 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_session_blocklist_on_timeout": "true", 2024-04-24T15:32:15.536 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-04-24T15:32:15.536 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-04-24T15:32:15.537 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-04-24T15:32:15.537 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-04-24T15:32:15.537 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-04-24T15:32:15.537 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_session_metadata_threshold": "16777216", 2024-04-24T15:32:15.537 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-04-24T15:32:15.537 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_shutdown_check": "0", 2024-04-24T15:32:15.537 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_skip_ino": "0", 2024-04-24T15:32:15.537 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_sleep_rank_change": "0.000000", 2024-04-24T15:32:15.537 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_snap_max_uid": "4294967294", 2024-04-24T15:32:15.537 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_snap_min_uid": "0", 2024-04-24T15:32:15.537 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_snap_rstat": "false", 2024-04-24T15:32:15.537 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_standby_replay_damaged": "false", 2024-04-24T15:32:15.537 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_symlink_recovery": "true", 2024-04-24T15:32:15.537 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_task_status_update_interval": "2.000000", 2024-04-24T15:32:15.537 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_thrash_exports": "0", 2024-04-24T15:32:15.537 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_thrash_fragments": "0", 2024-04-24T15:32:15.538 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_tick_interval": "5.000000", 2024-04-24T15:32:15.538 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_valgrind_exit": "false", 2024-04-24T15:32:15.538 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_verify_backtrace": "1", 2024-04-24T15:32:15.538 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_verify_scatter": "false", 2024-04-24T15:32:15.538 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_wipe_ino_prealloc": "false", 2024-04-24T15:32:15.538 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_wipe_sessions": "false", 2024-04-24T15:32:15.538 INFO:teuthology.orchestra.run.smithi096.stdout: "mempool_debug": "false", 2024-04-24T15:32:15.538 INFO:teuthology.orchestra.run.smithi096.stdout: "memstore_debug_omit_block_device_write": "false", 2024-04-24T15:32:15.538 INFO:teuthology.orchestra.run.smithi096.stdout: "memstore_device_bytes": "1073741824", 2024-04-24T15:32:15.538 INFO:teuthology.orchestra.run.smithi096.stdout: "memstore_page_set": "false", 2024-04-24T15:32:15.538 INFO:teuthology.orchestra.run.smithi096.stdout: "memstore_page_size": "65536", 2024-04-24T15:32:15.538 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr_client_bytes": "134217728", 2024-04-24T15:32:15.538 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr_client_messages": "512", 2024-04-24T15:32:15.538 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-04-24T15:32:15.539 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr_connect_retry_interval": "1.000000", 2024-04-24T15:32:15.539 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-04-24T15:32:15.539 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-04-24T15:32:15.539 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr_disabled_modules": "", 2024-04-24T15:32:15.539 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-04-24T15:32:15.539 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr_max_pg_num_change": "128", 2024-04-24T15:32:15.539 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr_mds_bytes": "134217728", 2024-04-24T15:32:15.539 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr_mds_messages": "128", 2024-04-24T15:32:15.539 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-04-24T15:32:15.539 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr_mon_bytes": "134217728", 2024-04-24T15:32:15.539 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr_mon_messages": "128", 2024-04-24T15:32:15.539 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr_osd_bytes": "536870912", 2024-04-24T15:32:15.539 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr_osd_messages": "8192", 2024-04-24T15:32:15.539 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr_pool": "true", 2024-04-24T15:32:15.539 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr_service_beacon_grace": "60.000000", 2024-04-24T15:32:15.539 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr_standby_modules": "true", 2024-04-24T15:32:15.540 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr_stats_period": "5", 2024-04-24T15:32:15.540 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr_stats_threshold": "5", 2024-04-24T15:32:15.540 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr_tick_period": "2", 2024-04-24T15:32:15.540 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-04-24T15:32:15.540 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_accept_timeout_factor": "2.000000", 2024-04-24T15:32:15.540 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_allow_pool_delete": "true", 2024-04-24T15:32:15.540 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_allow_pool_size_one": "true", 2024-04-24T15:32:15.540 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_auth_validate_all_caps": "true", 2024-04-24T15:32:15.540 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-04-24T15:32:15.540 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-04-24T15:32:15.540 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_client_bytes": "104857600", 2024-04-24T15:32:15.540 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_client_directed_command_retry": "5", 2024-04-24T15:32:15.540 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_client_hunt_interval": "3.000000", 2024-04-24T15:32:15.540 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-04-24T15:32:15.540 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-04-24T15:32:15.540 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-04-24T15:32:15.541 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_client_hunt_parallel": "3", 2024-04-24T15:32:15.541 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_client_log_interval": "1.000000", 2024-04-24T15:32:15.541 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-04-24T15:32:15.541 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_client_ping_interval": "10.000000", 2024-04-24T15:32:15.541 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_client_ping_timeout": "30.000000", 2024-04-24T15:32:15.541 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_clock_drift_allowed": "1.000000", 2024-04-24T15:32:15.541 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-04-24T15:32:15.541 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-04-24T15:32:15.541 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_cluster_log_file_level": "debug", 2024-04-24T15:32:15.541 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_cluster_log_to_file": "true", 2024-04-24T15:32:15.541 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_cluster_log_to_graylog": "false", 2024-04-24T15:32:15.541 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-04-24T15:32:15.541 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-04-24T15:32:15.541 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_cluster_log_to_journald": "false", 2024-04-24T15:32:15.541 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_cluster_log_to_stderr": "false", 2024-04-24T15:32:15.541 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-04-24T15:32:15.542 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-04-24T15:32:15.542 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_cluster_log_to_syslog_level": "info", 2024-04-24T15:32:15.542 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_compact_on_bootstrap": "false", 2024-04-24T15:32:15.542 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_compact_on_start": "false", 2024-04-24T15:32:15.542 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_compact_on_trim": "true", 2024-04-24T15:32:15.542 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_con_tracker_persist_interval": "10", 2024-04-24T15:32:15.542 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_con_tracker_score_halflife": "43200", 2024-04-24T15:32:15.542 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_config_key_max_entry_size": "65536", 2024-04-24T15:32:15.542 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_cpu_threads": "4", 2024-04-24T15:32:15.542 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_crush_min_required_version": "hammer", 2024-04-24T15:32:15.542 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_daemon_bytes": "419430400", 2024-04-24T15:32:15.542 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-04-24T15:32:15.542 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_data_avail_crit": "5", 2024-04-24T15:32:15.542 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_data_avail_warn": "5", 2024-04-24T15:32:15.543 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_data_size_warn": "16106127360", 2024-04-24T15:32:15.543 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_debug_block_osdmap_trim": "false", 2024-04-24T15:32:15.543 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-04-24T15:32:15.543 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_debug_dump_json": "false", 2024-04-24T15:32:15.543 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-04-24T15:32:15.543 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_debug_dump_transactions": "false", 2024-04-24T15:32:15.543 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_debug_extra_checks": "false", 2024-04-24T15:32:15.543 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-04-24T15:32:15.543 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-04-24T15:32:15.543 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_debug_no_require_pacific": "false", 2024-04-24T15:32:15.543 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_debug_no_require_quincy": "false", 2024-04-24T15:32:15.543 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-04-24T15:32:15.543 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_delta_reset_interval": "10.000000", 2024-04-24T15:32:15.543 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_dns_srv_name": "ceph-mon", 2024-04-24T15:32:15.543 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_down_mkfs_grace": "120", 2024-04-24T15:32:15.543 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_election_default_strategy": "1", 2024-04-24T15:32:15.544 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_election_timeout": "5.000000", 2024-04-24T15:32:15.544 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-04-24T15:32:15.544 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_elector_ping_divisor": "2", 2024-04-24T15:32:15.544 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_elector_ping_timeout": "2.000000", 2024-04-24T15:32:15.544 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_enable_op_tracker": "true", 2024-04-24T15:32:15.544 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_fake_pool_delete": "false", 2024-04-24T15:32:15.544 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_force_quorum_join": "false", 2024-04-24T15:32:15.544 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_globalid_prealloc": "10000", 2024-04-24T15:32:15.544 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_health_detail_to_clog": "false", 2024-04-24T15:32:15.544 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_health_log_update_period": "5", 2024-04-24T15:32:15.544 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_health_max_detail": "50", 2024-04-24T15:32:15.544 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_health_to_clog": "true", 2024-04-24T15:32:15.544 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_health_to_clog_interval": "600", 2024-04-24T15:32:15.544 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-04-24T15:32:15.544 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_host": "172.21.15.96,[v2:172.21.15.96:3301,v1:172.21.15.96:6790],172.21.15.151", 2024-04-24T15:32:15.545 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_host_override": "", 2024-04-24T15:32:15.545 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_initial_members": "", 2024-04-24T15:32:15.545 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-04-24T15:32:15.545 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-04-24T15:32:15.545 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-04-24T15:32:15.545 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-04-24T15:32:15.545 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_keyvaluedb": "rocksdb", 2024-04-24T15:32:15.545 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_lease": "5.000000", 2024-04-24T15:32:15.545 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-04-24T15:32:15.545 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-04-24T15:32:15.545 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_log_full_interval": "50", 2024-04-24T15:32:15.545 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_log_max": "10000", 2024-04-24T15:32:15.545 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_log_max_summary": "50", 2024-04-24T15:32:15.545 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_max_log_entries_per_event": "4096", 2024-04-24T15:32:15.546 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_max_log_epochs": "500", 2024-04-24T15:32:15.546 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_max_mdsmap_epochs": "500", 2024-04-24T15:32:15.546 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_max_mgrmap_epochs": "500", 2024-04-24T15:32:15.546 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_max_osd": "10000", 2024-04-24T15:32:15.546 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_max_pg_per_osd": "10000", 2024-04-24T15:32:15.546 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_max_pool_pg_num": "65536", 2024-04-24T15:32:15.546 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-04-24T15:32:15.546 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-04-24T15:32:15.546 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_mds_force_trim_to": "0", 2024-04-24T15:32:15.546 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_mds_skip_sanity": "false", 2024-04-24T15:32:15.546 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_memory_autotune": "true", 2024-04-24T15:32:15.546 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_memory_target": "2147483648", 2024-04-24T15:32:15.547 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_mgr_beacon_grace": "30", 2024-04-24T15:32:15.547 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-04-24T15:32:15.547 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_mgr_digest_period": "5", 2024-04-24T15:32:15.547 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_mgr_inactive_grace": "60", 2024-04-24T15:32:15.547 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_mgr_mkfs_grace": "240", 2024-04-24T15:32:15.547 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-04-24T15:32:15.547 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_min_osdmap_epochs": "500", 2024-04-24T15:32:15.547 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_op_complaint_time": "30", 2024-04-24T15:32:15.547 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_op_history_duration": "600", 2024-04-24T15:32:15.547 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_op_history_size": "20", 2024-04-24T15:32:15.547 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_op_history_slow_op_size": "20", 2024-04-24T15:32:15.547 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_op_history_slow_op_threshold": "10", 2024-04-24T15:32:15.547 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_op_log_threshold": "5", 2024-04-24T15:32:15.547 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-04-24T15:32:15.547 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-04-24T15:32:15.548 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-04-24T15:32:15.548 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osd_auto_mark_in": "false", 2024-04-24T15:32:15.548 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osd_auto_mark_new_in": "true", 2024-04-24T15:32:15.548 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-04-24T15:32:15.548 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-04-24T15:32:15.548 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osd_cache_size": "500", 2024-04-24T15:32:15.548 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osd_cache_size_min": "134217728", 2024-04-24T15:32:15.548 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osd_crush_smoke_test": "true", 2024-04-24T15:32:15.548 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osd_destroyed_out_interval": "600", 2024-04-24T15:32:15.548 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osd_down_out_interval": "600", 2024-04-24T15:32:15.548 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-04-24T15:32:15.548 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-04-24T15:32:15.548 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osd_force_trim_to": "0", 2024-04-24T15:32:15.548 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osd_full_ratio": "0.950000", 2024-04-24T15:32:15.548 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-04-24T15:32:15.549 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osd_laggy_halflife": "3600", 2024-04-24T15:32:15.549 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osd_laggy_max_interval": "300", 2024-04-24T15:32:15.549 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osd_laggy_weight": "0.300000", 2024-04-24T15:32:15.549 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-04-24T15:32:15.549 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osd_max_creating_pgs": "1024", 2024-04-24T15:32:15.549 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osd_max_initial_pgs": "1024", 2024-04-24T15:32:15.549 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osd_min_down_reporters": "2", 2024-04-24T15:32:15.549 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-04-24T15:32:15.549 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-04-24T15:32:15.549 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-04-24T15:32:15.549 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osd_prime_pg_temp": "true", 2024-04-24T15:32:15.549 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-04-24T15:32:15.549 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-04-24T15:32:15.549 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osd_report_timeout": "900", 2024-04-24T15:32:15.550 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-04-24T15:32:15.550 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-04-24T15:32:15.550 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osd_warn_num_repaired": "10", 2024-04-24T15:32:15.550 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osd_warn_op_age": "32.000000", 2024-04-24T15:32:15.550 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-04-24T15:32:15.550 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osdmap_full_prune_interval": "10", 2024-04-24T15:32:15.550 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osdmap_full_prune_min": "10000", 2024-04-24T15:32:15.550 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-04-24T15:32:15.550 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-04-24T15:32:15.550 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_pg_stuck_threshold": "60", 2024-04-24T15:32:15.550 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-04-24T15:32:15.550 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_pg_warn_min_objects": "10000", 2024-04-24T15:32:15.550 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_pg_warn_min_per_osd": "0", 2024-04-24T15:32:15.550 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-04-24T15:32:15.551 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_pool_quota_crit_threshold": "0", 2024-04-24T15:32:15.551 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_pool_quota_warn_threshold": "0", 2024-04-24T15:32:15.551 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_probe_timeout": "2.000000", 2024-04-24T15:32:15.551 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_reweight_max_change": "0.050000", 2024-04-24T15:32:15.551 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_reweight_max_osds": "4", 2024-04-24T15:32:15.551 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-04-24T15:32:15.551 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-04-24T15:32:15.551 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-04-24T15:32:15.551 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-04-24T15:32:15.551 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-04-24T15:32:15.551 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_scrub_interval": "86400", 2024-04-24T15:32:15.551 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_scrub_max_keys": "100", 2024-04-24T15:32:15.551 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_scrub_timeout": "300", 2024-04-24T15:32:15.552 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_session_timeout": "300", 2024-04-24T15:32:15.552 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_smart_report_timeout": "5", 2024-04-24T15:32:15.552 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_stat_smooth_intervals": "6", 2024-04-24T15:32:15.552 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-04-24T15:32:15.552 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_stretch_pool_min_size": "2", 2024-04-24T15:32:15.552 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_stretch_pool_size": "4", 2024-04-24T15:32:15.552 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-04-24T15:32:15.552 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_subscribe_interval": "86400.000000", 2024-04-24T15:32:15.552 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_sync_debug": "false", 2024-04-24T15:32:15.552 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_sync_max_payload_keys": "2000", 2024-04-24T15:32:15.552 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_sync_max_payload_size": "1048576", 2024-04-24T15:32:15.552 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_sync_provider_kill_at": "0", 2024-04-24T15:32:15.552 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_sync_requester_kill_at": "0", 2024-04-24T15:32:15.552 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_sync_timeout": "60.000000", 2024-04-24T15:32:15.552 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_target_pg_per_osd": "100", 2024-04-24T15:32:15.553 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_tick_interval": "5", 2024-04-24T15:32:15.553 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_timecheck_interval": "300.000000", 2024-04-24T15:32:15.553 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-04-24T15:32:15.553 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_warn_older_version_delay": "604800", 2024-04-24T15:32:15.553 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-04-24T15:32:15.553 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-04-24T15:32:15.553 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-04-24T15:32:15.553 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_warn_on_filestore_osds": "false", 2024-04-24T15:32:15.553 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-04-24T15:32:15.553 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-04-24T15:32:15.553 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-04-24T15:32:15.553 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_warn_on_misplaced": "false", 2024-04-24T15:32:15.553 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-04-24T15:32:15.553 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_warn_on_older_version": "true", 2024-04-24T15:32:15.554 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-04-24T15:32:15.554 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_warn_on_pool_no_app": "false", 2024-04-24T15:32:15.554 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-04-24T15:32:15.554 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-04-24T15:32:15.554 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-04-24T15:32:15.554 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-04-24T15:32:15.554 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_warn_on_too_few_osds": "false", 2024-04-24T15:32:15.554 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-04-24T15:32:15.554 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-04-24T15:32:15.554 INFO:teuthology.orchestra.run.smithi096.stdout: "monmap": "", 2024-04-24T15:32:15.554 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_async_op_threads": "3", 2024-04-24T15:32:15.554 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_async_rdma_buffer_size": "131072", 2024-04-24T15:32:15.554 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_async_rdma_cm": "false", 2024-04-24T15:32:15.555 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_async_rdma_device_name": "", 2024-04-24T15:32:15.555 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_async_rdma_dscp": "96", 2024-04-24T15:32:15.555 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-04-24T15:32:15.555 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_async_rdma_gid_idx": "0", 2024-04-24T15:32:15.555 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_async_rdma_local_gid": "", 2024-04-24T15:32:15.555 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_async_rdma_polling_us": "1000", 2024-04-24T15:32:15.555 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_async_rdma_port_num": "1", 2024-04-24T15:32:15.555 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-04-24T15:32:15.555 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-04-24T15:32:15.555 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_async_rdma_roce_ver": "1", 2024-04-24T15:32:15.555 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_async_rdma_send_buffers": "1024", 2024-04-24T15:32:15.555 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_async_rdma_sl": "3", 2024-04-24T15:32:15.555 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_async_rdma_support_srq": "true", 2024-04-24T15:32:15.556 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_async_rdma_type": "ib", 2024-04-24T15:32:15.556 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_async_reap_threshold": "5", 2024-04-24T15:32:15.556 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_bind_before_connect": "false", 2024-04-24T15:32:15.556 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_bind_ipv4": "true", 2024-04-24T15:32:15.556 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_bind_ipv6": "false", 2024-04-24T15:32:15.556 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_bind_msgr1": "true", 2024-04-24T15:32:15.556 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_bind_msgr2": "true", 2024-04-24T15:32:15.556 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_bind_port_max": "7568", 2024-04-24T15:32:15.556 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_bind_port_min": "6800", 2024-04-24T15:32:15.556 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_bind_prefer_ipv4": "false", 2024-04-24T15:32:15.556 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_bind_retry_count": "3", 2024-04-24T15:32:15.556 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_bind_retry_delay": "5", 2024-04-24T15:32:15.556 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_blackhole_client": "false", 2024-04-24T15:32:15.556 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_blackhole_mds": "false", 2024-04-24T15:32:15.557 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_blackhole_mgr": "false", 2024-04-24T15:32:15.557 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_blackhole_mon": "false", 2024-04-24T15:32:15.557 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_blackhole_osd": "false", 2024-04-24T15:32:15.557 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_client_mode": "crc secure", 2024-04-24T15:32:15.557 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_cluster_mode": "crc secure", 2024-04-24T15:32:15.557 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_cluster_type": "", 2024-04-24T15:32:15.557 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_compress_secure": "false", 2024-04-24T15:32:15.557 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_connection_idle_timeout": "900", 2024-04-24T15:32:15.557 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_connection_ready_timeout": "10", 2024-04-24T15:32:15.557 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_crc_data": "true", 2024-04-24T15:32:15.557 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_crc_header": "true", 2024-04-24T15:32:15.557 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_die_on_bad_msg": "false", 2024-04-24T15:32:15.557 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_die_on_bug": "true", 2024-04-24T15:32:15.558 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_die_on_old_message": "true", 2024-04-24T15:32:15.558 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_die_on_skipped_message": "false", 2024-04-24T15:32:15.558 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_die_on_unhandled_msg": "false", 2024-04-24T15:32:15.558 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-04-24T15:32:15.558 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_dpdk_coremask": "0xF", 2024-04-24T15:32:15.558 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-04-24T15:32:15.558 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_dpdk_devs_allowlist": "", 2024-04-24T15:32:15.558 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_dpdk_enable_tso": "true", 2024-04-24T15:32:15.558 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-04-24T15:32:15.558 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-04-24T15:32:15.558 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_dpdk_hugepages": "", 2024-04-24T15:32:15.558 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_dpdk_hw_flow_control": "true", 2024-04-24T15:32:15.558 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-04-24T15:32:15.558 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_dpdk_lro": "true", 2024-04-24T15:32:15.559 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_dpdk_memory_channel": "4", 2024-04-24T15:32:15.559 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-04-24T15:32:15.559 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_dpdk_pmd": "", 2024-04-24T15:32:15.559 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_dpdk_port_id": "0", 2024-04-24T15:32:15.559 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-04-24T15:32:15.559 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_dump_corrupt_message_level": "1", 2024-04-24T15:32:15.559 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_dump_on_send": "false", 2024-04-24T15:32:15.559 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_initial_backoff": "0.200000", 2024-04-24T15:32:15.559 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_inject_delay_max": "1.000000", 2024-04-24T15:32:15.559 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_inject_delay_probability": "0.000000", 2024-04-24T15:32:15.559 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_inject_delay_type": "", 2024-04-24T15:32:15.559 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_inject_internal_delays": "0.000000", 2024-04-24T15:32:15.559 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_inject_socket_failures": "5000", 2024-04-24T15:32:15.560 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_learn_addr_from_peer": "true", 2024-04-24T15:32:15.560 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_max_accept_failures": "4", 2024-04-24T15:32:15.560 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_max_backoff": "15.000000", 2024-04-24T15:32:15.560 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_mon_client_mode": "secure crc", 2024-04-24T15:32:15.560 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_mon_cluster_mode": "secure crc", 2024-04-24T15:32:15.560 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_mon_service_mode": "secure crc", 2024-04-24T15:32:15.560 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_osd_compress_min_size": "1024", 2024-04-24T15:32:15.560 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_osd_compress_mode": "none", 2024-04-24T15:32:15.560 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_osd_compression_algorithm": "snappy zlib zstd lz4", 2024-04-24T15:32:15.560 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-04-24T15:32:15.560 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_pq_min_cost": "65536", 2024-04-24T15:32:15.560 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_public_type": "", 2024-04-24T15:32:15.560 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_service_mode": "crc secure", 2024-04-24T15:32:15.560 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_tcp_listen_backlog": "512", 2024-04-24T15:32:15.561 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_tcp_nodelay": "true", 2024-04-24T15:32:15.561 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-04-24T15:32:15.561 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_tcp_rcvbuf": "0", 2024-04-24T15:32:15.561 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_type": "async+posix", 2024-04-24T15:32:15.561 INFO:teuthology.orchestra.run.smithi096.stdout: "no_config_file": "false", 2024-04-24T15:32:15.561 INFO:teuthology.orchestra.run.smithi096.stdout: "objecter_completion_locks_per_session": "32", 2024-04-24T15:32:15.561 INFO:teuthology.orchestra.run.smithi096.stdout: "objecter_debug_inject_relock_delay": "false", 2024-04-24T15:32:15.561 INFO:teuthology.orchestra.run.smithi096.stdout: "objecter_inflight_op_bytes": "104857600", 2024-04-24T15:32:15.561 INFO:teuthology.orchestra.run.smithi096.stdout: "objecter_inflight_ops": "1024", 2024-04-24T15:32:15.561 INFO:teuthology.orchestra.run.smithi096.stdout: "objecter_inject_no_watch_ping": "false", 2024-04-24T15:32:15.561 INFO:teuthology.orchestra.run.smithi096.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-04-24T15:32:15.561 INFO:teuthology.orchestra.run.smithi096.stdout: "objecter_tick_interval": "5.000000", 2024-04-24T15:32:15.561 INFO:teuthology.orchestra.run.smithi096.stdout: "objecter_timeout": "10.000000", 2024-04-24T15:32:15.561 INFO:teuthology.orchestra.run.smithi096.stdout: "objectstore_blackhole": "false", 2024-04-24T15:32:15.562 INFO:teuthology.orchestra.run.smithi096.stdout: "openssl_engine_opts": "", 2024-04-24T15:32:15.562 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_agent_delay_time": "5.000000", 2024-04-24T15:32:15.562 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_agent_hist_halflife": "1000", 2024-04-24T15:32:15.562 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_agent_max_low_ops": "2", 2024-04-24T15:32:15.562 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_agent_max_ops": "4", 2024-04-24T15:32:15.562 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-04-24T15:32:15.562 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_agent_quantize_effort": "0.100000", 2024-04-24T15:32:15.562 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_agent_slop": "0.020000", 2024-04-24T15:32:15.562 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-04-24T15:32:15.562 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_allow_recovery_below_min_size": "true", 2024-04-24T15:32:15.562 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_asio_thread_count": "2", 2024-04-24T15:32:15.562 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_async_recovery_min_cost": "100", 2024-04-24T15:32:15.562 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_auto_mark_unfound_lost": "false", 2024-04-24T15:32:15.563 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_backfill_retry_interval": "30.000000", 2024-04-24T15:32:15.563 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_backfill_scan_max": "512", 2024-04-24T15:32:15.563 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_backfill_scan_min": "64", 2024-04-24T15:32:15.563 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_backoff_on_degraded": "false", 2024-04-24T15:32:15.563 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_backoff_on_peering": "false", 2024-04-24T15:32:15.563 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_backoff_on_unfound": "true", 2024-04-24T15:32:15.563 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_beacon_report_interval": "300", 2024-04-24T15:32:15.563 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_bench_duration": "30", 2024-04-24T15:32:15.563 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-04-24T15:32:15.563 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_bench_max_block_size": "67108864", 2024-04-24T15:32:15.563 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_bench_small_size_max_iops": "100", 2024-04-24T15:32:15.563 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_blkin_trace_all": "false", 2024-04-24T15:32:15.563 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-04-24T15:32:15.564 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-04-24T15:32:15.564 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_check_for_log_corruption": "false", 2024-04-24T15:32:15.564 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-04-24T15:32:15.564 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-04-24T15:32:15.564 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2024-04-24T15:32:15.564 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-04-24T15:32:15.564 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_class_update_on_start": "true", 2024-04-24T15:32:15.564 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_client_message_cap": "256", 2024-04-24T15:32:15.564 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_client_message_size_cap": "524288000", 2024-04-24T15:32:15.564 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_client_op_priority": "63", 2024-04-24T15:32:15.564 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_client_watch_timeout": "30", 2024-04-24T15:32:15.564 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_command_max_records": "256", 2024-04-24T15:32:15.564 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_compact_on_start": "false", 2024-04-24T15:32:15.564 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-04-24T15:32:15.565 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_crush_chooseleaf_type": "0", 2024-04-24T15:32:15.565 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_crush_initial_weight": "-1.000000", 2024-04-24T15:32:15.565 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_crush_update_on_start": "true", 2024-04-24T15:32:15.565 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_crush_update_weight_set": "true", 2024-04-24T15:32:15.565 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-04-24T15:32:15.565 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-04-24T15:32:15.565 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-04-24T15:32:15.565 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_debug_drop_ping_duration": "0", 2024-04-24T15:32:15.565 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-04-24T15:32:15.565 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_debug_feed_pullee": "-1", 2024-04-24T15:32:15.565 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-04-24T15:32:15.565 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-04-24T15:32:15.565 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-04-24T15:32:15.566 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_debug_misdirected_ops": "false", 2024-04-24T15:32:15.566 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_debug_no_acting_change": "false", 2024-04-24T15:32:15.566 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_debug_no_purge_strays": "false", 2024-04-24T15:32:15.566 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_debug_op_order": "false", 2024-04-24T15:32:15.566 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_debug_pg_log_writeout": "false", 2024-04-24T15:32:15.566 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_debug_pretend_recovery_active": "false", 2024-04-24T15:32:15.566 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-04-24T15:32:15.566 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-04-24T15:32:15.566 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_debug_shutdown": "false", 2024-04-24T15:32:15.566 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-04-24T15:32:15.566 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-04-24T15:32:15.566 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_debug_verify_cached_snaps": "false", 2024-04-24T15:32:15.566 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_debug_verify_missing_on_start": "false", 2024-04-24T15:32:15.566 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_debug_verify_snaps": "false", 2024-04-24T15:32:15.567 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-04-24T15:32:15.567 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-04-24T15:32:15.567 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_deep_scrub_keys": "1024", 2024-04-24T15:32:15.567 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-04-24T15:32:15.567 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-04-24T15:32:15.567 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-04-24T15:32:15.567 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_deep_scrub_stride": "524288", 2024-04-24T15:32:15.567 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-04-24T15:32:15.567 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_default_data_pool_replay_window": "5", 2024-04-24T15:32:15.567 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_default_notify_timeout": "30", 2024-04-24T15:32:15.567 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_delete_sleep": "0.000000", 2024-04-24T15:32:15.567 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-04-24T15:32:15.567 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-04-24T15:32:15.567 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-04-24T15:32:15.568 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_discard_disconnected_ops": "true", 2024-04-24T15:32:15.568 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_enable_op_tracker": "true", 2024-04-24T15:32:15.568 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-04-24T15:32:15.568 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-04-24T15:32:15.568 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-04-24T15:32:15.568 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_fast_info": "true", 2024-04-24T15:32:15.568 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_fast_shutdown": "true", 2024-04-24T15:32:15.568 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-04-24T15:32:15.568 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_fast_shutdown_timeout": "15", 2024-04-24T15:32:15.568 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-04-24T15:32:15.568 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-04-24T15:32:15.568 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-04-24T15:32:15.568 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_function_tracing": "false", 2024-04-24T15:32:15.568 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_heartbeat_grace": "20", 2024-04-24T15:32:15.568 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_heartbeat_interval": "6", 2024-04-24T15:32:15.569 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-04-24T15:32:15.569 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_heartbeat_min_peers": "10", 2024-04-24T15:32:15.569 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_heartbeat_min_size": "2000", 2024-04-24T15:32:15.569 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_heartbeat_stale": "600", 2024-04-24T15:32:15.569 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-04-24T15:32:15.569 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_hit_set_max_size": "100000", 2024-04-24T15:32:15.569 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_hit_set_min_size": "1000", 2024-04-24T15:32:15.569 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-04-24T15:32:15.569 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-04-24T15:32:15.569 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-04-24T15:32:15.569 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-04-24T15:32:15.569 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-04-24T15:32:15.569 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_journal_flush_on_shutdown": "true", 2024-04-24T15:32:15.569 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_journal_size": "5120", 2024-04-24T15:32:15.569 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_kill_backfill_at": "0", 2024-04-24T15:32:15.570 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_loop_before_reset_tphandle": "64", 2024-04-24T15:32:15.570 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_map_cache_size": "50", 2024-04-24T15:32:15.570 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_map_dedup": "true", 2024-04-24T15:32:15.570 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_map_message_max": "40", 2024-04-24T15:32:15.570 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_map_message_max_bytes": "10485760", 2024-04-24T15:32:15.570 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_map_share_max_epochs": "40", 2024-04-24T15:32:15.570 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_max_attr_name_len": "100", 2024-04-24T15:32:15.570 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_max_attr_size": "0", 2024-04-24T15:32:15.570 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_max_backfills": "1", 2024-04-24T15:32:15.570 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_max_markdown_count": "5", 2024-04-24T15:32:15.570 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_max_markdown_period": "600", 2024-04-24T15:32:15.570 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_max_object_name_len": "2048", 2024-04-24T15:32:15.570 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_max_object_namespace_len": "256", 2024-04-24T15:32:15.570 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_max_object_size": "134217728", 2024-04-24T15:32:15.571 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-04-24T15:32:15.571 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_max_omap_entries_per_request": "1024", 2024-04-24T15:32:15.571 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_max_pg_blocked_by": "16", 2024-04-24T15:32:15.571 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_max_pg_log_entries": "10000", 2024-04-24T15:32:15.571 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-04-24T15:32:15.571 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_max_pgls": "1024", 2024-04-24T15:32:15.571 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_max_push_cost": "8388608", 2024-04-24T15:32:15.571 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_max_push_objects": "10", 2024-04-24T15:32:15.571 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_max_scrubs": "3", 2024-04-24T15:32:15.571 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-04-24T15:32:15.571 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_max_trimming_pgs": "2", 2024-04-24T15:32:15.571 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_max_write_op_reply_len": "64", 2024-04-24T15:32:15.571 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_max_write_size": "90", 2024-04-24T15:32:15.572 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-04-24T15:32:15.572 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-04-24T15:32:15.572 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-04-24T15:32:15.572 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-04-24T15:32:15.572 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-04-24T15:32:15.572 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-04-24T15:32:15.572 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-04-24T15:32:15.572 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mclock_override_recovery_settings": "false", 2024-04-24T15:32:15.572 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mclock_profile": "balanced", 2024-04-24T15:32:15.572 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-04-24T15:32:15.572 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-04-24T15:32:15.572 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-04-24T15:32:15.572 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-04-24T15:32:15.573 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-04-24T15:32:15.573 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-04-24T15:32:15.573 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-04-24T15:32:15.573 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-04-24T15:32:15.573 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-04-24T15:32:15.573 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-04-24T15:32:15.573 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mclock_skip_benchmark": "false", 2024-04-24T15:32:15.573 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_memory_base": "805306368", 2024-04-24T15:32:15.573 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_memory_cache_min": "134217728", 2024-04-24T15:32:15.573 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-04-24T15:32:15.573 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-04-24T15:32:15.573 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_memory_target": "4294967296", 2024-04-24T15:32:15.573 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_memory_target_autotune": "false", 2024-04-24T15:32:15.573 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-04-24T15:32:15.573 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_min_pg_log_entries": "250", 2024-04-24T15:32:15.574 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_min_recovery_priority": "0", 2024-04-24T15:32:15.574 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mon_heartbeat_interval": "30", 2024-04-24T15:32:15.574 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-04-24T15:32:15.574 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mon_report_interval": "5", 2024-04-24T15:32:15.574 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mon_report_max_in_flight": "2", 2024-04-24T15:32:15.574 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-04-24T15:32:15.574 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_num_cache_shards": "32", 2024-04-24T15:32:15.574 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_num_op_tracker_shard": "32", 2024-04-24T15:32:15.574 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_numa_auto_affinity": "true", 2024-04-24T15:32:15.574 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_numa_node": "-1", 2024-04-24T15:32:15.574 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_numa_prefer_iface": "true", 2024-04-24T15:32:15.574 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-04-24T15:32:15.574 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_objecter_finishers": "1", 2024-04-24T15:32:15.574 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_objectstore": "bluestore", 2024-04-24T15:32:15.575 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_objectstore_fuse": "false", 2024-04-24T15:32:15.575 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_objectstore_tracing": "false", 2024-04-24T15:32:15.575 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_op_complaint_time": "30.000000", 2024-04-24T15:32:15.575 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_op_history_duration": "600", 2024-04-24T15:32:15.575 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_op_history_size": "20", 2024-04-24T15:32:15.575 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_op_history_slow_op_size": "20", 2024-04-24T15:32:15.575 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-04-24T15:32:15.575 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_op_log_threshold": "5", 2024-04-24T15:32:15.575 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_op_num_shards": "0", 2024-04-24T15:32:15.575 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_op_num_shards_hdd": "5", 2024-04-24T15:32:15.575 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_op_num_shards_ssd": "8", 2024-04-24T15:32:15.575 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_op_num_threads_per_shard": "0", 2024-04-24T15:32:15.575 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-04-24T15:32:15.575 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-04-24T15:32:15.576 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-04-24T15:32:15.576 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_op_pq_min_cost": "65536", 2024-04-24T15:32:15.576 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_op_queue": "mclock_scheduler", 2024-04-24T15:32:15.576 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_op_queue_cut_off": "high", 2024-04-24T15:32:15.576 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_op_thread_suicide_timeout": "150", 2024-04-24T15:32:15.576 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_op_thread_timeout": "15", 2024-04-24T15:32:15.576 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_open_classes_on_start": "true", 2024-04-24T15:32:15.576 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_os_flags": "0", 2024-04-24T15:32:15.576 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_peering_op_priority": "255", 2024-04-24T15:32:15.576 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pg_delete_cost": "1048576", 2024-04-24T15:32:15.576 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pg_delete_priority": "5", 2024-04-24T15:32:15.576 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-04-24T15:32:15.576 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-04-24T15:32:15.576 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pg_log_dups_tracked": "3000", 2024-04-24T15:32:15.576 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pg_log_trim_max": "10000", 2024-04-24T15:32:15.576 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pg_log_trim_min": "100", 2024-04-24T15:32:15.577 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-04-24T15:32:15.577 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pg_object_context_cache_count": "64", 2024-04-24T15:32:15.577 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pg_stat_report_interval_max": "500", 2024-04-24T15:32:15.577 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-04-24T15:32:15.577 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-04-24T15:32:15.577 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-04-24T15:32:15.577 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-04-24T15:32:15.577 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-04-24T15:32:15.577 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-04-24T15:32:15.577 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pool_default_crush_rule": "-1", 2024-04-24T15:32:15.577 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pool_default_ec_fast_read": "false", 2024-04-24T15:32:15.577 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-04-24T15:32:15.577 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pool_default_flag_bulk": "false", 2024-04-24T15:32:15.577 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-04-24T15:32:15.577 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pool_default_flag_nodelete": "false", 2024-04-24T15:32:15.578 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-04-24T15:32:15.578 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-04-24T15:32:15.578 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pool_default_flags": "0", 2024-04-24T15:32:15.578 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-04-24T15:32:15.578 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pool_default_min_size": "0", 2024-04-24T15:32:15.578 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-04-24T15:32:15.578 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pool_default_pg_num": "32", 2024-04-24T15:32:15.578 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pool_default_pgp_num": "0", 2024-04-24T15:32:15.578 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-04-24T15:32:15.578 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pool_default_size": "2", 2024-04-24T15:32:15.578 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pool_default_type": "replicated", 2024-04-24T15:32:15.578 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-04-24T15:32:15.578 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pool_use_gmt_hitset": "true", 2024-04-24T15:32:15.578 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_push_per_object_cost": "1000", 2024-04-24T15:32:15.579 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_read_ec_check_for_errors": "false", 2024-04-24T15:32:15.579 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_recover_clone_overlap": "true", 2024-04-24T15:32:15.579 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_recover_clone_overlap_limit": "10", 2024-04-24T15:32:15.579 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_recovery_cost": "20971520", 2024-04-24T15:32:15.579 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_recovery_delay_start": "0.000000", 2024-04-24T15:32:15.579 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_recovery_max_active": "0", 2024-04-24T15:32:15.579 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_recovery_max_active_hdd": "3", 2024-04-24T15:32:15.579 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_recovery_max_active_ssd": "10", 2024-04-24T15:32:15.579 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_recovery_max_chunk": "8388608", 2024-04-24T15:32:15.579 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-04-24T15:32:15.579 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_recovery_max_single_start": "1", 2024-04-24T15:32:15.579 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_recovery_op_priority": "3", 2024-04-24T15:32:15.579 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_recovery_op_warn_multiple": "16", 2024-04-24T15:32:15.579 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_recovery_priority": "5", 2024-04-24T15:32:15.580 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_recovery_retry_interval": "30.000000", 2024-04-24T15:32:15.580 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_recovery_sleep": "0.000000", 2024-04-24T15:32:15.580 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-04-24T15:32:15.580 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-04-24T15:32:15.580 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-04-24T15:32:15.580 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_repair_during_recovery": "false", 2024-04-24T15:32:15.580 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_requested_scrub_priority": "120", 2024-04-24T15:32:15.580 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-04-24T15:32:15.580 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_rollback_to_cluster_snap": "", 2024-04-24T15:32:15.580 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_scrub_auto_repair": "false", 2024-04-24T15:32:15.580 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-04-24T15:32:15.580 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-04-24T15:32:15.580 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_scrub_begin_hour": "0", 2024-04-24T15:32:15.580 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_scrub_begin_week_day": "0", 2024-04-24T15:32:15.580 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_scrub_chunk_max": "25", 2024-04-24T15:32:15.581 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_scrub_chunk_min": "5", 2024-04-24T15:32:15.581 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_scrub_cost": "52428800", 2024-04-24T15:32:15.581 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_scrub_during_recovery": "false", 2024-04-24T15:32:15.581 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_scrub_end_hour": "0", 2024-04-24T15:32:15.581 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_scrub_end_week_day": "0", 2024-04-24T15:32:15.581 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_scrub_event_cost": "4096", 2024-04-24T15:32:15.581 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-04-24T15:32:15.581 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-04-24T15:32:15.581 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_scrub_invalid_stats": "true", 2024-04-24T15:32:15.581 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_scrub_load_threshold": "0.500000", 2024-04-24T15:32:15.581 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_scrub_max_interval": "604800.000000", 2024-04-24T15:32:15.581 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_scrub_max_preemptions": "5", 2024-04-24T15:32:15.581 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_scrub_min_interval": "86400.000000", 2024-04-24T15:32:15.582 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_scrub_priority": "5", 2024-04-24T15:32:15.582 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_scrub_sleep": "0.000000", 2024-04-24T15:32:15.582 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_shutdown_pgref_assert": "false", 2024-04-24T15:32:15.582 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_skip_data_digest": "false", 2024-04-24T15:32:15.582 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_smart_report_timeout": "5", 2024-04-24T15:32:15.582 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_snap_trim_cost": "1048576", 2024-04-24T15:32:15.582 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_snap_trim_priority": "5", 2024-04-24T15:32:15.582 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_snap_trim_sleep": "0.000000", 2024-04-24T15:32:15.582 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-04-24T15:32:15.582 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-04-24T15:32:15.582 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-04-24T15:32:15.582 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-04-24T15:32:15.582 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_target_transaction_size": "30", 2024-04-24T15:32:15.583 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-04-24T15:32:15.583 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-04-24T15:32:15.583 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-04-24T15:32:15.583 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-04-24T15:32:15.583 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-04-24T15:32:15.583 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-04-24T15:32:15.583 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-04-24T15:32:15.583 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_tier_default_cache_mode": "writeback", 2024-04-24T15:32:15.583 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-04-24T15:32:15.583 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-04-24T15:32:15.583 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_tracing": "false", 2024-04-24T15:32:15.583 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_use_stale_snap": "false", 2024-04-24T15:32:15.583 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T15:32:15.583 INFO:teuthology.orchestra.run.smithi096.stdout: "osdc_blkin_trace_all": "false", 2024-04-24T15:32:15.584 INFO:teuthology.orchestra.run.smithi096.stdout: "paxos_kill_at": "0", 2024-04-24T15:32:15.584 INFO:teuthology.orchestra.run.smithi096.stdout: "paxos_max_join_drift": "10", 2024-04-24T15:32:15.584 INFO:teuthology.orchestra.run.smithi096.stdout: "paxos_min": "500", 2024-04-24T15:32:15.584 INFO:teuthology.orchestra.run.smithi096.stdout: "paxos_min_wait": "0.050000", 2024-04-24T15:32:15.584 INFO:teuthology.orchestra.run.smithi096.stdout: "paxos_propose_interval": "1.000000", 2024-04-24T15:32:15.584 INFO:teuthology.orchestra.run.smithi096.stdout: "paxos_service_trim_max": "500", 2024-04-24T15:32:15.584 INFO:teuthology.orchestra.run.smithi096.stdout: "paxos_service_trim_max_multiplier": "20", 2024-04-24T15:32:15.584 INFO:teuthology.orchestra.run.smithi096.stdout: "paxos_service_trim_min": "250", 2024-04-24T15:32:15.584 INFO:teuthology.orchestra.run.smithi096.stdout: "paxos_stash_full_interval": "25", 2024-04-24T15:32:15.584 INFO:teuthology.orchestra.run.smithi096.stdout: "paxos_trim_max": "500", 2024-04-24T15:32:15.584 INFO:teuthology.orchestra.run.smithi096.stdout: "paxos_trim_min": "250", 2024-04-24T15:32:15.584 INFO:teuthology.orchestra.run.smithi096.stdout: "perf": "true", 2024-04-24T15:32:15.584 INFO:teuthology.orchestra.run.smithi096.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-04-24T15:32:15.584 INFO:teuthology.orchestra.run.smithi096.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-04-24T15:32:15.585 INFO:teuthology.orchestra.run.smithi096.stdout: "plugin_dir": "/usr/lib64/ceph", 2024-04-24T15:32:15.585 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addr": "-", 2024-04-24T15:32:15.585 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addrv": "", 2024-04-24T15:32:15.585 INFO:teuthology.orchestra.run.smithi096.stdout: "public_bind_addr": "-", 2024-04-24T15:32:15.585 INFO:teuthology.orchestra.run.smithi096.stdout: "public_network": "", 2024-04-24T15:32:15.585 INFO:teuthology.orchestra.run.smithi096.stdout: "public_network_interface": "", 2024-04-24T15:32:15.585 INFO:teuthology.orchestra.run.smithi096.stdout: "qat_compressor_enabled": "false", 2024-04-24T15:32:15.585 INFO:teuthology.orchestra.run.smithi096.stdout: "rados_mon_op_timeout": "0", 2024-04-24T15:32:15.585 INFO:teuthology.orchestra.run.smithi096.stdout: "rados_osd_op_timeout": "0", 2024-04-24T15:32:15.585 INFO:teuthology.orchestra.run.smithi096.stdout: "rados_tracing": "false", 2024-04-24T15:32:15.585 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_atime_update_interval": "60", 2024-04-24T15:32:15.585 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-04-24T15:32:15.585 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_balance_parent_reads": "false", 2024-04-24T15:32:15.586 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_balance_snap_reads": "false", 2024-04-24T15:32:15.586 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_blkin_trace_all": "false", 2024-04-24T15:32:15.586 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_blocklist_expire_seconds": "0", 2024-04-24T15:32:15.586 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_blocklist_on_break_lock": "true", 2024-04-24T15:32:15.586 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_cache": "true", 2024-04-24T15:32:15.586 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_cache_block_writes_upfront": "false", 2024-04-24T15:32:15.586 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_cache_max_dirty": "25165824", 2024-04-24T15:32:15.586 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-04-24T15:32:15.586 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_cache_max_dirty_object": "0", 2024-04-24T15:32:15.586 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_cache_policy": "writearound", 2024-04-24T15:32:15.586 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_cache_size": "33554432", 2024-04-24T15:32:15.586 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_cache_target_dirty": "16777216", 2024-04-24T15:32:15.586 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-04-24T15:32:15.586 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_clone_copy_on_read": "false", 2024-04-24T15:32:15.587 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_compression_hint": "none", 2024-04-24T15:32:15.587 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_concurrent_management_ops": "10", 2024-04-24T15:32:15.587 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-04-24T15:32:15.587 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_default_clone_format": "auto", 2024-04-24T15:32:15.587 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_default_data_pool": "", 2024-04-24T15:32:15.587 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_default_features": "61", 2024-04-24T15:32:15.587 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_default_format": "2", 2024-04-24T15:32:15.587 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_default_map_options": "", 2024-04-24T15:32:15.587 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_default_order": "22", 2024-04-24T15:32:15.587 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_default_pool": "rbd", 2024-04-24T15:32:15.587 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-04-24T15:32:15.587 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_default_stripe_count": "0", 2024-04-24T15:32:15.587 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_default_stripe_unit": "0", 2024-04-24T15:32:15.587 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_disable_zero_copy_writes": "true", 2024-04-24T15:32:15.588 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_discard_granularity_bytes": "65536", 2024-04-24T15:32:15.588 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-04-24T15:32:15.588 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_enable_alloc_hint": "true", 2024-04-24T15:32:15.588 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-04-24T15:32:15.588 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_io_scheduler": "simple", 2024-04-24T15:32:15.588 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-04-24T15:32:15.588 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_journal_commit_age": "5.000000", 2024-04-24T15:32:15.588 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-04-24T15:32:15.588 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-04-24T15:32:15.588 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-04-24T15:32:15.588 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-04-24T15:32:15.588 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_journal_object_flush_interval": "0", 2024-04-24T15:32:15.589 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-04-24T15:32:15.589 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-04-24T15:32:15.589 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_journal_order": "24", 2024-04-24T15:32:15.589 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_journal_pool": "", 2024-04-24T15:32:15.589 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_journal_splay_width": "4", 2024-04-24T15:32:15.589 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_localize_parent_reads": "false", 2024-04-24T15:32:15.589 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_localize_snap_reads": "false", 2024-04-24T15:32:15.589 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-04-24T15:32:15.589 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-04-24T15:32:15.589 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-04-24T15:32:15.589 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-04-24T15:32:15.589 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-04-24T15:32:15.589 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-04-24T15:32:15.590 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_mirror_image_policy_type": "simple", 2024-04-24T15:32:15.590 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-04-24T15:32:15.590 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-04-24T15:32:15.590 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-04-24T15:32:15.590 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-04-24T15:32:15.590 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-04-24T15:32:15.590 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-04-24T15:32:15.590 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-04-24T15:32:15.590 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_mirror_memory_autotune": "true", 2024-04-24T15:32:15.590 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_mirror_memory_base": "805306368", 2024-04-24T15:32:15.590 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-04-24T15:32:15.590 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-04-24T15:32:15.590 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-04-24T15:32:15.591 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-04-24T15:32:15.591 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_mirror_memory_target": "4294967296", 2024-04-24T15:32:15.591 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-04-24T15:32:15.591 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-04-24T15:32:15.591 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-04-24T15:32:15.591 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_mirroring_delete_delay": "0", 2024-04-24T15:32:15.591 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-04-24T15:32:15.591 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_mirroring_replay_delay": "0", 2024-04-24T15:32:15.591 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-04-24T15:32:15.591 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-04-24T15:32:15.591 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_move_to_trash_on_remove": "false", 2024-04-24T15:32:15.591 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-04-24T15:32:15.591 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_mtime_update_interval": "60", 2024-04-24T15:32:15.591 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_non_blocking_aio": "true", 2024-04-24T15:32:15.592 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_op_thread_timeout": "60", 2024-04-24T15:32:15.592 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_op_threads": "1", 2024-04-24T15:32:15.592 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_parent_cache_enabled": "false", 2024-04-24T15:32:15.592 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_persistent_cache_mode": "disabled", 2024-04-24T15:32:15.592 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_persistent_cache_path": "/tmp", 2024-04-24T15:32:15.592 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_persistent_cache_size": "1073741824", 2024-04-24T15:32:15.592 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_plugins": "", 2024-04-24T15:32:15.592 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_qos_bps_burst": "0", 2024-04-24T15:32:15.592 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-04-24T15:32:15.592 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_qos_bps_limit": "0", 2024-04-24T15:32:15.592 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_qos_exclude_ops": "0", 2024-04-24T15:32:15.592 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_qos_iops_burst": "0", 2024-04-24T15:32:15.592 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-04-24T15:32:15.592 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_qos_iops_limit": "0", 2024-04-24T15:32:15.592 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_qos_read_bps_burst": "0", 2024-04-24T15:32:15.593 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-04-24T15:32:15.593 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_qos_read_bps_limit": "0", 2024-04-24T15:32:15.593 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_qos_read_iops_burst": "0", 2024-04-24T15:32:15.593 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-04-24T15:32:15.593 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_qos_read_iops_limit": "0", 2024-04-24T15:32:15.593 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_qos_schedule_tick_min": "50", 2024-04-24T15:32:15.593 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_qos_write_bps_burst": "0", 2024-04-24T15:32:15.593 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-04-24T15:32:15.593 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_qos_write_bps_limit": "0", 2024-04-24T15:32:15.593 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_qos_write_iops_burst": "0", 2024-04-24T15:32:15.593 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-04-24T15:32:15.593 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_qos_write_iops_limit": "0", 2024-04-24T15:32:15.593 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_quiesce_notification_attempts": "10", 2024-04-24T15:32:15.593 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_read_from_replica_policy": "default", 2024-04-24T15:32:15.593 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-04-24T15:32:15.594 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_readahead_max_bytes": "524288", 2024-04-24T15:32:15.594 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_readahead_trigger_requests": "10", 2024-04-24T15:32:15.594 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_request_timed_out_seconds": "30", 2024-04-24T15:32:15.594 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_skip_partial_discard": "true", 2024-04-24T15:32:15.594 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-04-24T15:32:15.594 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_tracing": "false", 2024-04-24T15:32:15.594 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_validate_names": "true", 2024-04-24T15:32:15.594 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_validate_pool": "true", 2024-04-24T15:32:15.594 INFO:teuthology.orchestra.run.smithi096.stdout: "restapi_base_url": "", 2024-04-24T15:32:15.594 INFO:teuthology.orchestra.run.smithi096.stdout: "restapi_log_level": "", 2024-04-24T15:32:15.594 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_acl_grants_max_num": "100", 2024-04-24T15:32:15.594 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_admin_entry": "admin", 2024-04-24T15:32:15.594 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_backend_store": "rados", 2024-04-24T15:32:15.595 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_barbican_url": "", 2024-04-24T15:32:15.595 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_beast_enable_async": "true", 2024-04-24T15:32:15.595 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-04-24T15:32:15.595 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-04-24T15:32:15.595 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_bucket_index_max_aio": "128", 2024-04-24T15:32:15.595 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-04-24T15:32:15.595 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_bucket_quota_ttl": "600", 2024-04-24T15:32:15.595 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_cache_enabled": "true", 2024-04-24T15:32:15.595 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_cache_expiry_interval": "900", 2024-04-24T15:32:15.595 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_cache_lru_size": "10000", 2024-04-24T15:32:15.595 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_content_length_compat": "false", 2024-04-24T15:32:15.595 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_copy_obj_progress": "true", 2024-04-24T15:32:15.595 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-04-24T15:32:15.595 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_cors_rules_max_num": "100", 2024-04-24T15:32:15.595 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_cross_domain_policy": "", 2024-04-24T15:32:15.595 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_crypt_default_encryption_key": "", 2024-04-24T15:32:15.596 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_crypt_kmip_addr": "", 2024-04-24T15:32:15.596 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_crypt_kmip_ca_path": "", 2024-04-24T15:32:15.596 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_crypt_kmip_client_cert": "", 2024-04-24T15:32:15.596 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_crypt_kmip_client_key": "", 2024-04-24T15:32:15.596 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-04-24T15:32:15.596 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_crypt_kmip_password": "", 2024-04-24T15:32:15.596 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-04-24T15:32:15.596 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_crypt_kmip_username": "", 2024-04-24T15:32:15.596 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_crypt_require_ssl": "true", 2024-04-24T15:32:15.596 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-04-24T15:32:15.596 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-04-24T15:32:15.596 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-04-24T15:32:15.596 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-04-24T15:32:15.596 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-04-24T15:32:15.597 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-04-24T15:32:15.597 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-04-24T15:32:15.597 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-04-24T15:32:15.597 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-04-24T15:32:15.597 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-04-24T15:32:15.597 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-04-24T15:32:15.597 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-04-24T15:32:15.597 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-04-24T15:32:15.597 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-04-24T15:32:15.597 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_crypt_suppress_logs": "true", 2024-04-24T15:32:15.597 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_crypt_vault_addr": "", 2024-04-24T15:32:15.597 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_crypt_vault_auth": "token", 2024-04-24T15:32:15.597 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_crypt_vault_namespace": "", 2024-04-24T15:32:15.597 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_crypt_vault_prefix": "", 2024-04-24T15:32:15.597 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-04-24T15:32:15.598 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-04-24T15:32:15.598 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-04-24T15:32:15.598 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-04-24T15:32:15.598 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_crypt_vault_token_file": "", 2024-04-24T15:32:15.598 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-04-24T15:32:15.598 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_curl_buffersize": "524288", 2024-04-24T15:32:15.598 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_curl_low_speed_limit": "1024", 2024-04-24T15:32:15.598 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_curl_low_speed_time": "300", 2024-04-24T15:32:15.598 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-04-24T15:32:15.598 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-04-24T15:32:15.598 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-04-24T15:32:15.598 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-04-24T15:32:15.598 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-04-24T15:32:15.599 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_d3n_l1_fadvise": "4", 2024-04-24T15:32:15.599 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-04-24T15:32:15.599 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-04-24T15:32:15.599 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-04-24T15:32:15.599 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-04-24T15:32:15.599 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_data_log_changes_size": "1000", 2024-04-24T15:32:15.599 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_data_log_num_shards": "128", 2024-04-24T15:32:15.599 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-04-24T15:32:15.599 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_data_log_window": "30", 2024-04-24T15:32:15.599 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_data_notify_interval_msec": "200", 2024-04-24T15:32:15.599 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-04-24T15:32:15.599 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-04-24T15:32:15.599 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_default_data_log_backing": "fifo", 2024-04-24T15:32:15.600 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-04-24T15:32:15.600 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_default_region_info_oid": "default.region", 2024-04-24T15:32:15.600 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-04-24T15:32:15.600 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-04-24T15:32:15.600 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_defer_to_bucket_acls": "", 2024-04-24T15:32:15.600 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-04-24T15:32:15.600 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-04-24T15:32:15.600 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-04-24T15:32:15.600 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-04-24T15:32:15.600 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-04-24T15:32:15.600 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-04-24T15:32:15.600 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-04-24T15:32:15.600 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-04-24T15:32:15.600 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_dmclock_data_res": "500.000000", 2024-04-24T15:32:15.601 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-04-24T15:32:15.601 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-04-24T15:32:15.601 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-04-24T15:32:15.601 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-04-24T15:32:15.601 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_dns_name": "", 2024-04-24T15:32:15.601 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_dns_s3website_name": "", 2024-04-24T15:32:15.601 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_dynamic_resharding": "true", 2024-04-24T15:32:15.601 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-04-24T15:32:15.601 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_enable_gc_threads": "true", 2024-04-24T15:32:15.601 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_enable_lc_threads": "true", 2024-04-24T15:32:15.601 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_enable_ops_log": "false", 2024-04-24T15:32:15.601 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_enable_quota_threads": "true", 2024-04-24T15:32:15.601 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_enable_static_website": "false", 2024-04-24T15:32:15.602 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_enable_usage_log": "false", 2024-04-24T15:32:15.602 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_enforce_swift_acls": "true", 2024-04-24T15:32:15.602 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_exit_timeout_secs": "120", 2024-04-24T15:32:15.602 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_expose_bucket": "false", 2024-04-24T15:32:15.602 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_extended_http_attrs": "", 2024-04-24T15:32:15.602 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-04-24T15:32:15.602 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_frontends": "beast port=7480", 2024-04-24T15:32:15.602 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_gc_max_concurrent_io": "10", 2024-04-24T15:32:15.602 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_gc_max_deferred": "50", 2024-04-24T15:32:15.602 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-04-24T15:32:15.602 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_gc_max_objs": "32", 2024-04-24T15:32:15.602 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_gc_max_queue_size": "134213632", 2024-04-24T15:32:15.602 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_gc_max_trim_chunk": "16", 2024-04-24T15:32:15.603 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_gc_obj_min_wait": "7200", 2024-04-24T15:32:15.603 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_gc_processor_max_time": "3600", 2024-04-24T15:32:15.603 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_gc_processor_period": "3600", 2024-04-24T15:32:15.603 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-04-24T15:32:15.603 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_get_obj_window_size": "16777216", 2024-04-24T15:32:15.603 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_healthcheck_disabling_path": "", 2024-04-24T15:32:15.603 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_ignore_get_invalid_range": "false", 2024-04-24T15:32:15.603 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_init_timeout": "300", 2024-04-24T15:32:15.603 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-04-24T15:32:15.603 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-04-24T15:32:15.603 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-04-24T15:32:15.603 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_keystone_admin_domain": "", 2024-04-24T15:32:15.603 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_keystone_admin_password": "", 2024-04-24T15:32:15.603 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_keystone_admin_password_path": "", 2024-04-24T15:32:15.604 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_keystone_admin_project": "", 2024-04-24T15:32:15.604 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_keystone_admin_tenant": "", 2024-04-24T15:32:15.604 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_keystone_admin_token": "", 2024-04-24T15:32:15.604 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_keystone_admin_token_path": "", 2024-04-24T15:32:15.604 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_keystone_admin_user": "", 2024-04-24T15:32:15.604 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_keystone_api_version": "2", 2024-04-24T15:32:15.604 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_keystone_barbican_domain": "", 2024-04-24T15:32:15.604 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_keystone_barbican_password": "", 2024-04-24T15:32:15.604 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_keystone_barbican_project": "", 2024-04-24T15:32:15.604 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_keystone_barbican_tenant": "", 2024-04-24T15:32:15.604 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_keystone_barbican_user": "", 2024-04-24T15:32:15.604 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-04-24T15:32:15.604 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_keystone_implicit_tenants": "false", 2024-04-24T15:32:15.605 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-04-24T15:32:15.605 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_keystone_service_token_enabled": "false", 2024-04-24T15:32:15.605 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_keystone_token_cache_size": "10000", 2024-04-24T15:32:15.605 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_keystone_url": "", 2024-04-24T15:32:15.605 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_keystone_verify_ssl": "true", 2024-04-24T15:32:15.605 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_lc_debug_interval": "-1", 2024-04-24T15:32:15.605 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_lc_lock_max_time": "90", 2024-04-24T15:32:15.605 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_lc_max_objs": "32", 2024-04-24T15:32:15.605 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_lc_max_rules": "1000", 2024-04-24T15:32:15.605 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_lc_max_worker": "3", 2024-04-24T15:32:15.605 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_lc_max_wp_worker": "3", 2024-04-24T15:32:15.605 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_lc_thread_delay": "0", 2024-04-24T15:32:15.605 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-04-24T15:32:15.605 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_ldap_dnattr": "uid", 2024-04-24T15:32:15.606 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-04-24T15:32:15.606 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_ldap_searchfilter": "", 2024-04-24T15:32:15.606 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-04-24T15:32:15.606 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_ldap_uri": "", 2024-04-24T15:32:15.606 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-04-24T15:32:15.606 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-04-24T15:32:15.606 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-04-24T15:32:15.606 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_log_http_headers": "", 2024-04-24T15:32:15.606 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_log_nonexistent_bucket": "false", 2024-04-24T15:32:15.606 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-04-24T15:32:15.606 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_log_object_name_utc": "false", 2024-04-24T15:32:15.606 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_luarocks_location": "/tmp/luarocks", 2024-04-24T15:32:15.606 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_max_attr_name_len": "0", 2024-04-24T15:32:15.607 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_max_attr_size": "0", 2024-04-24T15:32:15.607 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_max_attrs_num_in_req": "0", 2024-04-24T15:32:15.607 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_max_chunk_size": "4194304", 2024-04-24T15:32:15.607 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_max_concurrent_requests": "1024", 2024-04-24T15:32:15.607 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_max_dynamic_shards": "1999", 2024-04-24T15:32:15.607 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_max_listing_results": "1000", 2024-04-24T15:32:15.607 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_max_notify_retries": "10", 2024-04-24T15:32:15.607 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_max_objs_per_shard": "100000", 2024-04-24T15:32:15.607 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_max_put_param_size": "1048576", 2024-04-24T15:32:15.607 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_max_put_size": "5368709120", 2024-04-24T15:32:15.607 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_max_slo_entries": "1000", 2024-04-24T15:32:15.607 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_md_log_max_shards": "64", 2024-04-24T15:32:15.607 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_md_notify_interval_msec": "200", 2024-04-24T15:32:15.607 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-04-24T15:32:15.607 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_mp_lock_max_time": "600", 2024-04-24T15:32:15.608 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-04-24T15:32:15.608 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_multipart_min_part_size": "5242880", 2024-04-24T15:32:15.608 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-04-24T15:32:15.608 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-04-24T15:32:15.608 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_nfs_fhcache_size": "2017", 2024-04-24T15:32:15.608 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-04-24T15:32:15.608 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_nfs_lru_lanes": "5", 2024-04-24T15:32:15.608 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_nfs_max_gc": "300", 2024-04-24T15:32:15.608 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-04-24T15:32:15.608 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_nfs_run_gc_threads": "false", 2024-04-24T15:32:15.608 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_nfs_run_lc_threads": "false", 2024-04-24T15:32:15.608 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_nfs_run_quota_threads": "false", 2024-04-24T15:32:15.608 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_nfs_run_sync_thread": "false", 2024-04-24T15:32:15.608 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-04-24T15:32:15.608 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-04-24T15:32:15.609 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_num_async_rados_threads": "32", 2024-04-24T15:32:15.609 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_num_control_oids": "8", 2024-04-24T15:32:15.609 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_numa_node": "-1", 2024-04-24T15:32:15.609 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_obj_stripe_size": "4194304", 2024-04-24T15:32:15.609 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-04-24T15:32:15.609 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_objexp_chunk_size": "100", 2024-04-24T15:32:15.609 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_objexp_gc_interval": "600", 2024-04-24T15:32:15.609 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_objexp_hints_num_shards": "127", 2024-04-24T15:32:15.609 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-04-24T15:32:15.609 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-04-24T15:32:15.609 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_op_thread_timeout": "600", 2024-04-24T15:32:15.609 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_op_tracing": "false", 2024-04-24T15:32:15.609 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_opa_token": "", 2024-04-24T15:32:15.609 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_opa_url": "", 2024-04-24T15:32:15.609 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_opa_verify_ssl": "true", 2024-04-24T15:32:15.610 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-04-24T15:32:15.610 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_ops_log_file_path": "", 2024-04-24T15:32:15.610 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_ops_log_rados": "true", 2024-04-24T15:32:15.610 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_ops_log_socket_path": "", 2024-04-24T15:32:15.610 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-04-24T15:32:15.610 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-04-24T15:32:15.610 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_period_push_interval": "2.000000", 2024-04-24T15:32:15.610 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_period_push_interval_max": "30.000000", 2024-04-24T15:32:15.610 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_period_root_pool": ".rgw.root", 2024-04-24T15:32:15.610 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_print_continue": "true", 2024-04-24T15:32:15.610 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_print_prohibited_content_length": "false", 2024-04-24T15:32:15.610 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-04-24T15:32:15.610 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-04-24T15:32:15.611 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-04-24T15:32:15.611 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-04-24T15:32:15.611 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_rados_tracing": "false", 2024-04-24T15:32:15.611 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_realm": "", 2024-04-24T15:32:15.611 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_realm_id": "", 2024-04-24T15:32:15.611 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-04-24T15:32:15.611 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_region": "", 2024-04-24T15:32:15.611 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_region_root_pool": ".rgw.root", 2024-04-24T15:32:15.611 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_relaxed_region_enforcement": "false", 2024-04-24T15:32:15.611 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-04-24T15:32:15.611 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-04-24T15:32:15.611 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_request_uri": "", 2024-04-24T15:32:15.611 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_reshard_batch_size": "64", 2024-04-24T15:32:15.611 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-04-24T15:32:15.611 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_reshard_max_aio": "128", 2024-04-24T15:32:15.611 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_reshard_num_logs": "16", 2024-04-24T15:32:15.612 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_reshard_thread_interval": "600", 2024-04-24T15:32:15.612 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_resolve_cname": "false", 2024-04-24T15:32:15.612 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_rest_getusage_op_compat": "false", 2024-04-24T15:32:15.612 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_run_sync_thread": "true", 2024-04-24T15:32:15.612 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-04-24T15:32:15.612 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-04-24T15:32:15.612 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_s3_auth_use_keystone": "false", 2024-04-24T15:32:15.612 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_s3_auth_use_ldap": "false", 2024-04-24T15:32:15.612 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_s3_auth_use_rados": "true", 2024-04-24T15:32:15.612 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_s3_auth_use_sts": "false", 2024-04-24T15:32:15.612 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-04-24T15:32:15.612 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_s3_success_create_obj_status": "0", 2024-04-24T15:32:15.612 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-04-24T15:32:15.612 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_scheduler_type": "throttler", 2024-04-24T15:32:15.613 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_script_uri": "", 2024-04-24T15:32:15.613 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_service_provider_name": "", 2024-04-24T15:32:15.613 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-04-24T15:32:15.613 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_sts_client_id": "", 2024-04-24T15:32:15.613 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_sts_client_secret": "", 2024-04-24T15:32:15.613 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_sts_entry": "sts", 2024-04-24T15:32:15.613 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_sts_key": "sts", 2024-04-24T15:32:15.613 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_sts_max_session_duration": "43200", 2024-04-24T15:32:15.613 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_sts_min_session_duration": "900", 2024-04-24T15:32:15.613 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_sts_token_introspection_url": "", 2024-04-24T15:32:15.613 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_swift_account_in_url": "false", 2024-04-24T15:32:15.613 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_swift_auth_entry": "auth", 2024-04-24T15:32:15.613 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_swift_auth_url": "", 2024-04-24T15:32:15.613 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_swift_custom_header": "", 2024-04-24T15:32:15.614 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_swift_enforce_content_length": "false", 2024-04-24T15:32:15.614 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_swift_need_stats": "true", 2024-04-24T15:32:15.614 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_swift_tenant_name": "", 2024-04-24T15:32:15.614 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_swift_token_expiration": "86400", 2024-04-24T15:32:15.614 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_swift_url": "", 2024-04-24T15:32:15.614 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_swift_url_prefix": "swift", 2024-04-24T15:32:15.614 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_swift_versioning_enabled": "false", 2024-04-24T15:32:15.614 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-04-24T15:32:15.614 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_sync_lease_period": "120", 2024-04-24T15:32:15.614 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-04-24T15:32:15.614 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_sync_log_trim_interval": "1200", 2024-04-24T15:32:15.614 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-04-24T15:32:15.614 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-04-24T15:32:15.614 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-04-24T15:32:15.615 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_sync_obj_etag_verify": "false", 2024-04-24T15:32:15.615 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_sync_trace_history_size": "4096", 2024-04-24T15:32:15.615 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-04-24T15:32:15.615 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-04-24T15:32:15.615 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_thread_pool_size": "512", 2024-04-24T15:32:15.615 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_torrent_comment": "", 2024-04-24T15:32:15.615 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_torrent_createby": "", 2024-04-24T15:32:15.615 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_torrent_encoding": "", 2024-04-24T15:32:15.615 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_torrent_flag": "false", 2024-04-24T15:32:15.615 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_torrent_origin": "", 2024-04-24T15:32:15.615 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_torrent_sha_unit": "524288", 2024-04-24T15:32:15.615 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_torrent_tracker": "", 2024-04-24T15:32:15.615 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_trust_forwarded_https": "false", 2024-04-24T15:32:15.615 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-04-24T15:32:15.615 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_usage_log_tick_interval": "30", 2024-04-24T15:32:15.616 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_usage_max_shards": "32", 2024-04-24T15:32:15.616 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_usage_max_user_shards": "1", 2024-04-24T15:32:15.616 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_use_opa_authz": "false", 2024-04-24T15:32:15.616 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-04-24T15:32:15.616 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_user_default_quota_max_size": "-1", 2024-04-24T15:32:15.616 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_user_max_buckets": "1000", 2024-04-24T15:32:15.616 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-04-24T15:32:15.616 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-04-24T15:32:15.616 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_user_quota_sync_interval": "86400", 2024-04-24T15:32:15.616 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-04-24T15:32:15.616 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_user_unique_email": "true", 2024-04-24T15:32:15.616 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_verify_ssl": "true", 2024-04-24T15:32:15.616 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_website_routing_rules_max_num": "50", 2024-04-24T15:32:15.616 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_zone": "", 2024-04-24T15:32:15.616 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_zone_id": "", 2024-04-24T15:32:15.616 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-04-24T15:32:15.617 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_zonegroup": "", 2024-04-24T15:32:15.617 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_zonegroup_id": "", 2024-04-24T15:32:15.617 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-04-24T15:32:15.617 INFO:teuthology.orchestra.run.smithi096.stdout: "rocksdb_block_size": "4096", 2024-04-24T15:32:15.617 INFO:teuthology.orchestra.run.smithi096.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-04-24T15:32:15.617 INFO:teuthology.orchestra.run.smithi096.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-04-24T15:32:15.617 INFO:teuthology.orchestra.run.smithi096.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-04-24T15:32:15.617 INFO:teuthology.orchestra.run.smithi096.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-04-24T15:32:15.617 INFO:teuthology.orchestra.run.smithi096.stdout: "rocksdb_cache_shard_bits": "4", 2024-04-24T15:32:15.617 INFO:teuthology.orchestra.run.smithi096.stdout: "rocksdb_cache_size": "536870912", 2024-04-24T15:32:15.617 INFO:teuthology.orchestra.run.smithi096.stdout: "rocksdb_cache_type": "binned_lru", 2024-04-24T15:32:15.617 INFO:teuthology.orchestra.run.smithi096.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-04-24T15:32:15.617 INFO:teuthology.orchestra.run.smithi096.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-04-24T15:32:15.617 INFO:teuthology.orchestra.run.smithi096.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-04-24T15:32:15.617 INFO:teuthology.orchestra.run.smithi096.stdout: "rocksdb_collect_compaction_stats": "false", 2024-04-24T15:32:15.617 INFO:teuthology.orchestra.run.smithi096.stdout: "rocksdb_collect_extended_stats": "false", 2024-04-24T15:32:15.617 INFO:teuthology.orchestra.run.smithi096.stdout: "rocksdb_collect_memory_stats": "false", 2024-04-24T15:32:15.618 INFO:teuthology.orchestra.run.smithi096.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-04-24T15:32:15.618 INFO:teuthology.orchestra.run.smithi096.stdout: "rocksdb_index_type": "binary_search", 2024-04-24T15:32:15.618 INFO:teuthology.orchestra.run.smithi096.stdout: "rocksdb_log_to_ceph_log": "true", 2024-04-24T15:32:15.618 INFO:teuthology.orchestra.run.smithi096.stdout: "rocksdb_metadata_block_size": "4096", 2024-04-24T15:32:15.618 INFO:teuthology.orchestra.run.smithi096.stdout: "rocksdb_partition_filters": "false", 2024-04-24T15:32:15.618 INFO:teuthology.orchestra.run.smithi096.stdout: "rocksdb_perf": "false", 2024-04-24T15:32:15.618 INFO:teuthology.orchestra.run.smithi096.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-04-24T15:32:15.618 INFO:teuthology.orchestra.run.smithi096.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-04-24T15:32:15.618 INFO:teuthology.orchestra.run.smithi096.stdout: "rotating_keys_renewal_timeout": "10", 2024-04-24T15:32:15.618 INFO:teuthology.orchestra.run.smithi096.stdout: "run_dir": "/var/run/ceph", 2024-04-24T15:32:15.618 INFO:teuthology.orchestra.run.smithi096.stdout: "seastore_block_create": "true", 2024-04-24T15:32:15.618 INFO:teuthology.orchestra.run.smithi096.stdout: "seastore_cache_lru_size": "67108864", 2024-04-24T15:32:15.618 INFO:teuthology.orchestra.run.smithi096.stdout: "seastore_default_max_object_size": "16777216", 2024-04-24T15:32:15.618 INFO:teuthology.orchestra.run.smithi096.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-04-24T15:32:15.619 INFO:teuthology.orchestra.run.smithi096.stdout: "seastore_device_size": "107374182400", 2024-04-24T15:32:15.619 INFO:teuthology.orchestra.run.smithi096.stdout: "seastore_init_rewrite_segments_num_per_device": "6", 2024-04-24T15:32:15.619 INFO:teuthology.orchestra.run.smithi096.stdout: "seastore_journal_batch_capacity": "16", 2024-04-24T15:32:15.619 INFO:teuthology.orchestra.run.smithi096.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-04-24T15:32:15.619 INFO:teuthology.orchestra.run.smithi096.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-04-24T15:32:15.619 INFO:teuthology.orchestra.run.smithi096.stdout: "seastore_journal_iodepth_limit": "5", 2024-04-24T15:32:15.619 INFO:teuthology.orchestra.run.smithi096.stdout: "seastore_segment_size": "67108864", 2024-04-24T15:32:15.619 INFO:teuthology.orchestra.run.smithi096.stdout: "setgroup": "", 2024-04-24T15:32:15.619 INFO:teuthology.orchestra.run.smithi096.stdout: "setuser": "", 2024-04-24T15:32:15.619 INFO:teuthology.orchestra.run.smithi096.stdout: "setuser_match_path": "", 2024-04-24T15:32:15.619 INFO:teuthology.orchestra.run.smithi096.stdout: "target_max_misplaced_ratio": "0.050000", 2024-04-24T15:32:15.619 INFO:teuthology.orchestra.run.smithi096.stdout: "thp": "false", 2024-04-24T15:32:15.619 INFO:teuthology.orchestra.run.smithi096.stdout: "threadpool_default_timeout": "60", 2024-04-24T15:32:15.619 INFO:teuthology.orchestra.run.smithi096.stdout: "threadpool_empty_queue_max_wait": "2", 2024-04-24T15:32:15.619 INFO:teuthology.orchestra.run.smithi096.stdout: "throttler_perf_counter": "true" 2024-04-24T15:32:15.619 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:32:15.622 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-04-24T15:32:15.752 INFO:teuthology.orchestra.run.smithi096.stdout:mon.a: {} 2024-04-24T15:32:15.763 INFO:teuthology.orchestra.run.smithi096.stdout:mon.b: {} 2024-04-24T15:32:15.772 INFO:teuthology.orchestra.run.smithi096.stdout:mon.c: {} 2024-04-24T15:32:15.773 INFO:teuthology.orchestra.run.smithi096.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-04-24T15:32:15.773 INFO:teuthology.orchestra.run.smithi096.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-04-24T15:32:15.773 INFO:teuthology.orchestra.run.smithi096.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-04-24T15:32:15.784 DEBUG:teuthology.orchestra.run.smithi151:> type ceph-objectstore-tool 2024-04-24T15:32:15.804 INFO:teuthology.orchestra.run.smithi151.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-04-24T15:32:15.804 DEBUG:teuthology.orchestra.run.smithi096:> type ceph-objectstore-tool 2024-04-24T15:32:15.821 INFO:teuthology.orchestra.run.smithi096.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-04-24T15:32:15.821 INFO:teuthology.run_tasks:Running task rbd_fsx... 2024-04-24T15:32:15.831 INFO:tasks.rbd_fsx:starting rbd_fsx... 2024-04-24T15:32:15.831 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-04-24T15:32:15.831 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-04-24T15:32:15.831 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-24T15:32:15.831 INFO:tasks.thrashosds.thrasher:Removing osd 1, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7] 2024-04-24T15:32:15.831 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 1 2024-04-24T15:32:15.832 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-04-24T15:32:15.832 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-04-24T15:32:15.832 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-24T15:32:15.832 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-04-24T15:32:15.832 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-24T15:32:15.833 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-04-24T15:32:15.833 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-24T15:32:15.833 DEBUG:teuthology.orchestra.run.smithi096:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 1 -w 1 -t 1 -h 1 -l 250000000 -S 0 -N 6000 pool_client.0 image_client.0 2024-04-24T15:32:15.846 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:32:15.846+0000 7f23abcf6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 44021) UID: 0 2024-04-24T15:32:15.897 INFO:teuthology.orchestra.run.smithi096.stdout:mapped writes DISABLED 2024-04-24T15:32:15.897 INFO:teuthology.orchestra.run.smithi096.stdout:mapped reads DISABLED 2024-04-24T15:32:15.897 INFO:teuthology.orchestra.run.smithi096.stdout:Seed set to 6094 2024-04-24T15:32:15.934 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:32:15.934+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:32:16.001 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:32:16.001 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:32:16.001 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-04-24T15:32:16.001 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:32:16.022 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-24T15:32:16.033 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-04-24T15:32:16.034 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:32:16.034+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:32:16.062 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-04-24T15:32:16.082 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-04-24T15:32:16.102 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-04-24T15:32:16.135 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:32:16.136+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:32:16.135 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-04-24T15:32:16.154 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-04-24T15:32:16.164 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:16.164 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:16.173 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-04-24T15:32:16.175 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-24T15:32:16.192 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-04-24T15:32:16.194 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T15:32:16.194 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T15:32:16.194 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T15:32:16.194 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T15:32:16.194 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T15:32:16.194 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T15:32:16.194 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T15:32:16.194 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T15:32:16.250 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:32:16.250+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:32:16.251 INFO:tasks.ceph.mon.a.smithi096.stderr:2024-04-24T15:32:16.251+0000 7fa88ddc8700 -1 mon.a@0(leader).osd e26 definitely_dead 0 2024-04-24T15:32:16.315 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:16.316 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:16.325 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-24T15:32:16.337 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:32:16.338+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:32:16.438 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:32:16.438+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:32:16.489 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:16.490 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:16.499 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-24T15:32:16.541 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:32:16.542+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:32:16.640 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:32:16.640+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:32:16.664 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:16.664 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:16.674 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-24T15:32:16.758 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:32:16.758+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:32:16.837 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:16.837 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:16.842 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:32:16.843+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:32:16.847 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-24T15:32:16.943 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:32:16.943+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:32:17.011 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:17.011 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:17.020 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-24T15:32:17.044 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:32:17.044+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:32:17.048 INFO:teuthology.orchestra.run.smithi096.stderr:marked out osd.1. 2024-04-24T15:32:17.049 INFO:teuthology.orchestra.run.smithi096.stderr:noscrub is set 2024-04-24T15:32:17.145 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:32:17.145+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:32:17.172 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:17.172 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:17.182 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-24T15:32:17.246 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:32:17.246+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:32:17.307 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:17.307 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:17.317 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-24T15:32:17.347 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:32:17.348+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:32:17.449 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:32:17.450+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:32:17.480 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:17.480 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:17.489 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-24T15:32:17.550 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:32:17.550+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:32:17.613 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:17.613 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:17.623 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-24T15:32:17.667 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:32:17.667+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:32:17.746 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:17.746 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:17.752 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:32:17.752+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:32:17.756 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-24T15:32:17.854 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:32:17.854+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:32:17.892 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:17.892 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:17.902 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-24T15:32:17.987 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:32:17.987+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:32:18.058 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:32:18.059+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:32:18.072 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:18.072 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:18.082 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-24T15:32:18.167 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:32:18.167+0000 7f23abcf6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 44021) UID: 0 2024-04-24T15:32:18.210 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-24T15:32:18.245 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:18.245 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:18.255 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-24T15:32:18.297 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:32:18.297+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:32:18.348 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-04-24T15:32:18.360 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:32:18.360+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:32:18.366 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-04-24T15:32:18.383 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-04-24T15:32:18.401 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-04-24T15:32:18.420 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:18.420 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:18.420 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-04-24T15:32:18.430 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-24T15:32:18.437 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-04-24T15:32:18.456 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-04-24T15:32:18.461 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:32:18.462+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:32:18.474 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-04-24T15:32:18.475 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T15:32:18.475 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T15:32:18.475 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T15:32:18.475 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T15:32:18.476 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T15:32:18.476 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T15:32:18.476 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T15:32:18.476 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T15:32:18.553 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:32:18.553 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:32:18.553 INFO:teuthology.orchestra.run.smithi151.stdout: "complaint_time": 30, 2024-04-24T15:32:18.554 INFO:teuthology.orchestra.run.smithi151.stdout: "num_blocked_ops": 0 2024-04-24T15:32:18.554 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:32:18.562 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:32:18.562+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:32:18.563 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-24T15:32:18.662 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:32:18.663+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:32:18.729 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:32:18.729 INFO:teuthology.orchestra.run.smithi151.stdout: "size": 20, 2024-04-24T15:32:18.729 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 600, 2024-04-24T15:32:18.729 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [] 2024-04-24T15:32:18.729 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:32:18.739 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-24T15:32:18.780 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:32:18.780+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:32:18.864 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:32:18.865+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:32:18.902 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:32:18.902 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:32:18.902 INFO:teuthology.orchestra.run.smithi151.stdout: "num_ops": 0 2024-04-24T15:32:18.902 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:32:18.912 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-24T15:32:18.965 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:32:18.966+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:32:19.035 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:32:19.035 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:32:19.035 INFO:teuthology.orchestra.run.smithi151.stdout: "complaint_time": 30, 2024-04-24T15:32:19.035 INFO:teuthology.orchestra.run.smithi151.stdout: "num_blocked_ops": 0 2024-04-24T15:32:19.035 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:32:19.046 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-24T15:32:19.063 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-24T15:32:19.065 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:32:19.066+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:32:19.166 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:32:19.167+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:32:19.171 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:32:19.171 INFO:teuthology.orchestra.run.smithi151.stdout: "size": 20, 2024-04-24T15:32:19.171 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 600, 2024-04-24T15:32:19.171 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [] 2024-04-24T15:32:19.171 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:32:19.181 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-24T15:32:19.267 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:32:19.268+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:32:19.345 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:32:19.346 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:32:19.346 INFO:teuthology.orchestra.run.smithi151.stdout: "num_ops": 0 2024-04-24T15:32:19.346 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:32:19.355 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-24T15:32:19.368 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:32:19.369+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:32:19.470 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:32:19.471+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:32:19.517 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:32:19.517 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:32:19.518 INFO:teuthology.orchestra.run.smithi151.stdout: "complaint_time": 30, 2024-04-24T15:32:19.518 INFO:teuthology.orchestra.run.smithi151.stdout: "num_blocked_ops": 0 2024-04-24T15:32:19.518 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:32:19.527 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-24T15:32:19.570 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:32:19.571+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:32:19.653 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:32:19.654 INFO:teuthology.orchestra.run.smithi151.stdout: "size": 20, 2024-04-24T15:32:19.654 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 600, 2024-04-24T15:32:19.654 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [ 2024-04-24T15:32:19.654 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.654 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4184.0:18 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-04-24T15:32:19.654 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:00.973237+0000", 2024-04-24T15:32:19.654 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 18.679449305999999, 2024-04-24T15:32:19.654 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0075407699999999996, 2024-04-24T15:32:19.654 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:19.654 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:19.654 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:19.654 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4184", 2024-04-24T15:32:19.655 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/1884602651", 2024-04-24T15:32:19.655 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18 2024-04-24T15:32:19.655 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.655 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:19.655 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.655 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:19.655 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.973237+0000", 2024-04-24T15:32:19.655 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:19.655 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.655 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.655 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:19.655 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.973237+0000", 2024-04-24T15:32:19.655 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:19.655 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.655 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.656 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:19.656 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.973237+0000", 2024-04-24T15:32:19.656 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:19.656 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.656 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.656 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:19.656 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.973241+0000", 2024-04-24T15:32:19.656 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9750000000000001e-06 2024-04-24T15:32:19.656 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.656 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.656 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:19.656 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.973246+0000", 2024-04-24T15:32:19.656 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.5670000000000002e-06 2024-04-24T15:32:19.656 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.657 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.657 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:19.657 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.973258+0000", 2024-04-24T15:32:19.657 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.1707e-05 2024-04-24T15:32:19.657 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.657 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.657 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:19.657 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.973684+0000", 2024-04-24T15:32:19.657 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00042587099999999998 2024-04-24T15:32:19.657 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.657 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.657 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:19.657 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.973745+0000", 2024-04-24T15:32:19.657 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.0976e-05 2024-04-24T15:32:19.657 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.658 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.658 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:19.658 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.973849+0000", 2024-04-24T15:32:19.658 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000103999 2024-04-24T15:32:19.658 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.658 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.658 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:19.658 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.976929+0000", 2024-04-24T15:32:19.658 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0030801689999999998 2024-04-24T15:32:19.658 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.658 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.658 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:19.658 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.980645+0000", 2024-04-24T15:32:19.658 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0037165060000000001 2024-04-24T15:32:19.659 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.659 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.659 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:19.659 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.980743+0000", 2024-04-24T15:32:19.659 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.7728999999999999e-05 2024-04-24T15:32:19.659 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.659 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.659 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:19.659 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.980778+0000", 2024-04-24T15:32:19.659 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.485e-05 2024-04-24T15:32:19.659 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:19.659 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:19.659 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:19.659 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.659 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.660 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4184.0:21 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 65536~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-04-24T15:32:19.660 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:00.985588+0000", 2024-04-24T15:32:19.660 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 18.667098975999998, 2024-04-24T15:32:19.660 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0073858090000000001, 2024-04-24T15:32:19.660 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:19.660 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:19.660 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:19.660 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4184", 2024-04-24T15:32:19.660 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/1884602651", 2024-04-24T15:32:19.660 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 21 2024-04-24T15:32:19.660 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.660 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:19.660 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.660 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:19.661 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.985588+0000", 2024-04-24T15:32:19.661 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:19.661 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.661 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.661 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:19.661 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.985588+0000", 2024-04-24T15:32:19.661 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:19.661 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.661 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.661 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:19.661 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.985587+0000", 2024-04-24T15:32:19.661 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:19.661 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.661 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.661 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:19.662 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.985653+0000", 2024-04-24T15:32:19.662 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.6086000000000004e-05 2024-04-24T15:32:19.662 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.662 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.662 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:19.662 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.985660+0000", 2024-04-24T15:32:19.662 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.3790000000000003e-06 2024-04-24T15:32:19.662 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.662 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.662 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:19.662 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.985664+0000", 2024-04-24T15:32:19.662 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.053e-06 2024-04-24T15:32:19.662 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.663 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.663 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:19.663 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.986266+0000", 2024-04-24T15:32:19.663 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00060216800000000004 2024-04-24T15:32:19.663 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.663 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.663 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:19.663 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.986321+0000", 2024-04-24T15:32:19.663 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.5275999999999997e-05 2024-04-24T15:32:19.663 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.663 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.663 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:19.663 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.986410+0000", 2024-04-24T15:32:19.663 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.9137999999999999e-05 2024-04-24T15:32:19.663 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.663 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.664 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:19.664 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.989564+0000", 2024-04-24T15:32:19.664 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0031535209999999998 2024-04-24T15:32:19.664 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.664 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.664 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:19.664 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.992882+0000", 2024-04-24T15:32:19.664 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0033180190000000002 2024-04-24T15:32:19.664 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.664 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.664 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:19.664 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.992934+0000", 2024-04-24T15:32:19.664 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.24e-05 2024-04-24T15:32:19.664 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.664 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.665 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:19.665 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.992973+0000", 2024-04-24T15:32:19.665 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9274999999999999e-05 2024-04-24T15:32:19.665 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:19.665 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:19.665 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:19.665 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.665 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.665 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4184.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-04-24T15:32:19.665 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:00.985670+0000", 2024-04-24T15:32:19.665 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 18.667016135000001, 2024-04-24T15:32:19.665 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0074023860000000004, 2024-04-24T15:32:19.665 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:19.665 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:19.665 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:19.666 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4184", 2024-04-24T15:32:19.666 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/1884602651", 2024-04-24T15:32:19.666 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 22 2024-04-24T15:32:19.666 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.666 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:19.666 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.666 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:19.666 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.985670+0000", 2024-04-24T15:32:19.666 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:19.666 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.666 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.666 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:19.666 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.985670+0000", 2024-04-24T15:32:19.667 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:19.667 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.667 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.667 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:19.667 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.985670+0000", 2024-04-24T15:32:19.667 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:19.667 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.667 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.667 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:19.667 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.985673+0000", 2024-04-24T15:32:19.667 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.1939999999999998e-06 2024-04-24T15:32:19.667 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.667 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.668 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:19.668 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.985677+0000", 2024-04-24T15:32:19.668 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.7079999999999999e-06 2024-04-24T15:32:19.668 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.668 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.668 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:19.668 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.985679+0000", 2024-04-24T15:32:19.668 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.3470000000000001e-06 2024-04-24T15:32:19.668 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.668 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.668 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:19.668 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.986801+0000", 2024-04-24T15:32:19.668 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0011212609999999999 2024-04-24T15:32:19.668 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.669 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.669 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:19.669 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.986887+0000", 2024-04-24T15:32:19.669 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.6877000000000002e-05 2024-04-24T15:32:19.669 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.669 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.669 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:19.669 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.987022+0000", 2024-04-24T15:32:19.669 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000134991 2024-04-24T15:32:19.669 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.669 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.669 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:19.669 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.992467+0000", 2024-04-24T15:32:19.669 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0054450219999999999 2024-04-24T15:32:19.670 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.670 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.670 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:19.670 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.993004+0000", 2024-04-24T15:32:19.670 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00053681400000000004 2024-04-24T15:32:19.670 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.670 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.670 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:19.670 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.993049+0000", 2024-04-24T15:32:19.671 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4709999999999997e-05 2024-04-24T15:32:19.671 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.671 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.671 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:19.671 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.993073+0000", 2024-04-24T15:32:19.671 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.3886000000000001e-05 2024-04-24T15:32:19.671 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:19.671 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:19.671 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:19.671 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.671 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.671 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4184.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-04-24T15:32:19.671 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:01.007690+0000", 2024-04-24T15:32:19.671 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 18.644996902999999, 2024-04-24T15:32:19.671 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0065139869999999997, 2024-04-24T15:32:19.672 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:19.672 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:19.672 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:19.672 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4184", 2024-04-24T15:32:19.672 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/1884602651", 2024-04-24T15:32:19.672 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 32 2024-04-24T15:32:19.672 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.672 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:19.672 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.672 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:19.672 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007690+0000", 2024-04-24T15:32:19.672 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:19.672 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.672 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.673 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:19.673 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007690+0000", 2024-04-24T15:32:19.673 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:19.673 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.673 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.673 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:19.673 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007689+0000", 2024-04-24T15:32:19.673 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:19.673 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.673 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.673 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:19.673 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007762+0000", 2024-04-24T15:32:19.673 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2523999999999999e-05 2024-04-24T15:32:19.673 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.673 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.673 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:19.674 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007771+0000", 2024-04-24T15:32:19.674 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.0159999999999997e-06 2024-04-24T15:32:19.674 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.674 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.674 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:19.674 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007775+0000", 2024-04-24T15:32:19.674 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2359999999999998e-06 2024-04-24T15:32:19.674 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.674 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.674 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:19.674 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.008537+0000", 2024-04-24T15:32:19.674 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00076218599999999998 2024-04-24T15:32:19.674 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.674 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.675 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:19.675 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.008619+0000", 2024-04-24T15:32:19.675 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.2136000000000001e-05 2024-04-24T15:32:19.675 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.675 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.675 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:19.675 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.008760+0000", 2024-04-24T15:32:19.675 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00014038500000000001 2024-04-24T15:32:19.675 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.675 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.675 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:19.675 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.010709+0000", 2024-04-24T15:32:19.675 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0019494029999999999 2024-04-24T15:32:19.675 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.675 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.676 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:19.676 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.014140+0000", 2024-04-24T15:32:19.676 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0034304790000000002 2024-04-24T15:32:19.676 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.676 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.676 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:19.676 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.014178+0000", 2024-04-24T15:32:19.676 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8612000000000002e-05 2024-04-24T15:32:19.676 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.676 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.676 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:19.676 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.014204+0000", 2024-04-24T15:32:19.676 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.5548e-05 2024-04-24T15:32:19.676 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:19.676 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:19.676 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:19.677 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.677 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.677 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4184.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-04-24T15:32:19.677 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:01.007780+0000", 2024-04-24T15:32:19.677 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 18.644907063000002, 2024-04-24T15:32:19.677 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0064758460000000004, 2024-04-24T15:32:19.677 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:19.677 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:19.677 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:19.677 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4184", 2024-04-24T15:32:19.677 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/1884602651", 2024-04-24T15:32:19.677 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 33 2024-04-24T15:32:19.677 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.677 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:19.677 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.678 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:19.678 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007780+0000", 2024-04-24T15:32:19.678 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:19.678 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.678 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.678 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:19.678 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007780+0000", 2024-04-24T15:32:19.678 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:19.678 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.678 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.678 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:19.678 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007779+0000", 2024-04-24T15:32:19.679 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:19.679 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.679 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.679 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:19.679 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007782+0000", 2024-04-24T15:32:19.679 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.4930000000000001e-06 2024-04-24T15:32:19.679 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.679 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.679 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:19.679 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007785+0000", 2024-04-24T15:32:19.679 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.535e-06 2024-04-24T15:32:19.679 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.679 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.679 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:19.679 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007789+0000", 2024-04-24T15:32:19.680 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5590000000000001e-06 2024-04-24T15:32:19.680 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.680 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.680 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:19.680 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.009175+0000", 2024-04-24T15:32:19.680 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.001386417 2024-04-24T15:32:19.680 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.680 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.680 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:19.680 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.009233+0000", 2024-04-24T15:32:19.680 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.7414000000000003e-05 2024-04-24T15:32:19.680 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.680 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.680 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:19.681 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.009314+0000", 2024-04-24T15:32:19.681 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.1800999999999998e-05 2024-04-24T15:32:19.681 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.681 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.681 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:19.681 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.010745+0000", 2024-04-24T15:32:19.681 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.001430965 2024-04-24T15:32:19.681 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.681 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.681 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:19.681 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.014210+0000", 2024-04-24T15:32:19.681 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0034649540000000001 2024-04-24T15:32:19.681 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.681 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.682 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:19.682 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.014237+0000", 2024-04-24T15:32:19.682 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7109000000000001e-05 2024-04-24T15:32:19.682 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.682 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.682 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:19.682 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.014255+0000", 2024-04-24T15:32:19.682 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.7946000000000001e-05 2024-04-24T15:32:19.682 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:19.682 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:19.682 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:19.682 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.682 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.683 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4184.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-04-24T15:32:19.683 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:01.007792+0000", 2024-04-24T15:32:19.683 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 18.644894106999999, 2024-04-24T15:32:19.683 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0065062920000000003, 2024-04-24T15:32:19.683 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:19.683 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:19.683 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:19.683 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4184", 2024-04-24T15:32:19.683 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/1884602651", 2024-04-24T15:32:19.683 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 34 2024-04-24T15:32:19.683 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.683 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:19.683 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.683 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:19.683 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007792+0000", 2024-04-24T15:32:19.683 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:19.684 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.684 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.684 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:19.684 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007792+0000", 2024-04-24T15:32:19.684 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:19.684 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.684 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.684 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:19.684 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007792+0000", 2024-04-24T15:32:19.684 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:19.684 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.684 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.684 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:19.684 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007795+0000", 2024-04-24T15:32:19.685 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.396e-06 2024-04-24T15:32:19.685 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.685 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.685 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:19.685 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007798+0000", 2024-04-24T15:32:19.685 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.2899999999999998e-06 2024-04-24T15:32:19.685 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.685 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.685 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:19.685 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007801+0000", 2024-04-24T15:32:19.685 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5080000000000001e-06 2024-04-24T15:32:19.685 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.685 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.685 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:19.685 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.009688+0000", 2024-04-24T15:32:19.686 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.001886956 2024-04-24T15:32:19.686 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.686 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.686 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:19.686 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.009765+0000", 2024-04-24T15:32:19.686 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.6483000000000006e-05 2024-04-24T15:32:19.686 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.686 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.686 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:19.686 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.009904+0000", 2024-04-24T15:32:19.686 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00013925100000000001 2024-04-24T15:32:19.686 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.686 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.686 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:19.687 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.011742+0000", 2024-04-24T15:32:19.687 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.001837878 2024-04-24T15:32:19.687 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.687 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.687 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:19.687 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.014260+0000", 2024-04-24T15:32:19.687 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0025179149999999999 2024-04-24T15:32:19.687 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.687 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.687 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:19.687 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.014282+0000", 2024-04-24T15:32:19.687 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.1923999999999999e-05 2024-04-24T15:32:19.688 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.688 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.688 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:19.688 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.014299+0000", 2024-04-24T15:32:19.688 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.6931000000000001e-05 2024-04-24T15:32:19.688 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:19.688 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:19.688 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:19.688 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.688 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.688 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4184.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-04-24T15:32:19.688 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:01.007879+0000", 2024-04-24T15:32:19.688 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 18.644807524000001, 2024-04-24T15:32:19.688 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0064617579999999997, 2024-04-24T15:32:19.688 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:19.689 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:19.689 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:19.689 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4184", 2024-04-24T15:32:19.689 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/1884602651", 2024-04-24T15:32:19.689 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 35 2024-04-24T15:32:19.689 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.689 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:19.689 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.689 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:19.689 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007879+0000", 2024-04-24T15:32:19.689 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:19.689 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.689 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.689 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:19.689 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007879+0000", 2024-04-24T15:32:19.689 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:19.690 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.690 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.690 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:19.690 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007878+0000", 2024-04-24T15:32:19.690 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:19.690 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.690 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.690 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:19.690 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007949+0000", 2024-04-24T15:32:19.690 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.0714000000000001e-05 2024-04-24T15:32:19.690 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.690 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.690 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:19.690 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007961+0000", 2024-04-24T15:32:19.690 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.1969e-05 2024-04-24T15:32:19.691 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.691 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.691 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:19.691 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007965+0000", 2024-04-24T15:32:19.691 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.4620000000000001e-06 2024-04-24T15:32:19.691 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.691 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.691 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:19.691 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.010182+0000", 2024-04-24T15:32:19.691 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0022175599999999999 2024-04-24T15:32:19.691 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.691 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.691 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:19.691 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.010238+0000", 2024-04-24T15:32:19.691 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.5583e-05 2024-04-24T15:32:19.692 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.692 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.692 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:19.692 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.010321+0000", 2024-04-24T15:32:19.692 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3515000000000003e-05 2024-04-24T15:32:19.692 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.692 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.692 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:19.692 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.012956+0000", 2024-04-24T15:32:19.692 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.002634426 2024-04-24T15:32:19.692 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.692 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.692 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:19.692 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.014303+0000", 2024-04-24T15:32:19.692 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.001347507 2024-04-24T15:32:19.692 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.693 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.693 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:19.693 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.014325+0000", 2024-04-24T15:32:19.693 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.1577999999999999e-05 2024-04-24T15:32:19.693 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.693 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.693 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:19.693 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.014341+0000", 2024-04-24T15:32:19.693 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.6121999999999999e-05 2024-04-24T15:32:19.693 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:19.693 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:19.693 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:19.693 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.693 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.693 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4184.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-04-24T15:32:19.693 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:01.007969+0000", 2024-04-24T15:32:19.694 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 18.644717505999999, 2024-04-24T15:32:19.694 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0090800020000000002, 2024-04-24T15:32:19.694 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:19.694 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:19.694 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:19.694 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4184", 2024-04-24T15:32:19.694 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/1884602651", 2024-04-24T15:32:19.694 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 36 2024-04-24T15:32:19.694 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.694 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:19.694 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.694 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:19.695 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007969+0000", 2024-04-24T15:32:19.695 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:19.695 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.695 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.695 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:19.695 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007969+0000", 2024-04-24T15:32:19.695 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:19.695 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.695 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.696 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:19.696 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007969+0000", 2024-04-24T15:32:19.696 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:19.696 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.696 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.696 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:19.696 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007971+0000", 2024-04-24T15:32:19.696 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7219999999999999e-06 2024-04-24T15:32:19.696 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.696 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.696 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:19.696 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007975+0000", 2024-04-24T15:32:19.696 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.7019999999999999e-06 2024-04-24T15:32:19.696 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.697 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.697 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:19.697 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007978+0000", 2024-04-24T15:32:19.697 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.4059999999999999e-06 2024-04-24T15:32:19.697 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.697 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.697 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:19.697 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.010779+0000", 2024-04-24T15:32:19.697 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0028010610000000001 2024-04-24T15:32:19.697 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.697 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.697 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:19.697 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.010887+0000", 2024-04-24T15:32:19.697 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00010871599999999999 2024-04-24T15:32:19.698 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.698 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.698 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:19.698 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.011026+0000", 2024-04-24T15:32:19.698 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00013850200000000001 2024-04-24T15:32:19.698 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.698 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.698 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:19.698 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.015873+0000", 2024-04-24T15:32:19.698 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0048467680000000004 2024-04-24T15:32:19.698 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.698 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.698 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:19.699 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.016993+0000", 2024-04-24T15:32:19.699 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0011203809999999999 2024-04-24T15:32:19.699 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.699 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.699 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:19.699 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.017027+0000", 2024-04-24T15:32:19.699 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.3637000000000003e-05 2024-04-24T15:32:19.699 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.699 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.699 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:19.701 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.017049+0000", 2024-04-24T15:32:19.701 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.2509000000000001e-05 2024-04-24T15:32:19.701 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:19.701 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:19.701 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:19.701 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.702 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.702 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4184.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-04-24T15:32:19.702 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:01.007981+0000", 2024-04-24T15:32:19.702 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 18.644705312999999, 2024-04-24T15:32:19.702 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0091181590000000007, 2024-04-24T15:32:19.702 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:19.702 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:19.702 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:19.702 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4184", 2024-04-24T15:32:19.702 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/1884602651", 2024-04-24T15:32:19.702 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 37 2024-04-24T15:32:19.702 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.702 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:19.703 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.703 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:19.703 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007981+0000", 2024-04-24T15:32:19.703 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:19.703 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.703 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.703 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:19.703 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007981+0000", 2024-04-24T15:32:19.703 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:19.703 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.703 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.703 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:19.703 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007981+0000", 2024-04-24T15:32:19.703 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:32:19.703 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.703 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.704 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:19.704 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007984+0000", 2024-04-24T15:32:19.704 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.3469999999999999e-06 2024-04-24T15:32:19.704 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.704 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.704 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:19.704 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007988+0000", 2024-04-24T15:32:19.704 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.4479999999999999e-06 2024-04-24T15:32:19.704 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.704 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.704 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:19.704 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007990+0000", 2024-04-24T15:32:19.704 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.26e-06 2024-04-24T15:32:19.704 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.705 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.705 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:19.705 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.011341+0000", 2024-04-24T15:32:19.705 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0033510459999999999 2024-04-24T15:32:19.705 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.705 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.705 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:19.705 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.011400+0000", 2024-04-24T15:32:19.705 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.9206000000000002e-05 2024-04-24T15:32:19.705 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.705 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.705 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:19.705 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.011488+0000", 2024-04-24T15:32:19.705 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.7301999999999999e-05 2024-04-24T15:32:19.705 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.706 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.706 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:19.706 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.015901+0000", 2024-04-24T15:32:19.706 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0044129549999999997 2024-04-24T15:32:19.706 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.706 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.706 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:19.706 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.017057+0000", 2024-04-24T15:32:19.706 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0011566790000000001 2024-04-24T15:32:19.706 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.706 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.706 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:19.707 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.017081+0000", 2024-04-24T15:32:19.707 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.3790000000000001e-05 2024-04-24T15:32:19.707 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.707 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.707 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:19.707 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.017099+0000", 2024-04-24T15:32:19.707 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.8346e-05 2024-04-24T15:32:19.707 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:19.707 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:19.707 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:19.707 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.707 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.707 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4184.0:69 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-04-24T15:32:19.707 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:01.057223+0000", 2024-04-24T15:32:19.708 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 18.595463289000001, 2024-04-24T15:32:19.708 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0080831549999999999, 2024-04-24T15:32:19.708 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:19.708 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:19.708 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:19.708 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4184", 2024-04-24T15:32:19.708 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/1884602651", 2024-04-24T15:32:19.708 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 69 2024-04-24T15:32:19.708 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.708 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:19.708 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.708 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:19.708 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.057223+0000", 2024-04-24T15:32:19.708 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:19.709 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.709 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.709 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:19.709 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.057223+0000", 2024-04-24T15:32:19.709 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:19.709 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.709 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.709 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:19.709 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.057223+0000", 2024-04-24T15:32:19.709 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:19.709 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.709 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.709 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:19.709 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.057261+0000", 2024-04-24T15:32:19.710 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.7905999999999999e-05 2024-04-24T15:32:19.710 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.710 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.710 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:19.710 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.057267+0000", 2024-04-24T15:32:19.710 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.4389999999999997e-06 2024-04-24T15:32:19.710 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.710 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.710 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:19.710 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.057270+0000", 2024-04-24T15:32:19.710 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.0489999999999999e-06 2024-04-24T15:32:19.710 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.710 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.711 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:19.711 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.058232+0000", 2024-04-24T15:32:19.711 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00096145599999999996 2024-04-24T15:32:19.711 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.711 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.711 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:19.711 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.058284+0000", 2024-04-24T15:32:19.711 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.2213999999999998e-05 2024-04-24T15:32:19.711 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.711 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.711 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:19.711 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.058364+0000", 2024-04-24T15:32:19.711 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0096e-05 2024-04-24T15:32:19.711 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.712 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.712 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:19.712 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.059686+0000", 2024-04-24T15:32:19.712 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0013213739999999999 2024-04-24T15:32:19.712 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.712 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.712 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:19.712 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.065225+0000", 2024-04-24T15:32:19.712 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.005539675 2024-04-24T15:32:19.712 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.712 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.712 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:19.712 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.065272+0000", 2024-04-24T15:32:19.712 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6369999999999998e-05 2024-04-24T15:32:19.713 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.713 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.713 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:19.713 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.065306+0000", 2024-04-24T15:32:19.713 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.4875000000000001e-05 2024-04-24T15:32:19.713 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:19.713 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:19.713 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:19.713 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.713 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.713 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4184.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-04-24T15:32:19.713 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:01.057275+0000", 2024-04-24T15:32:19.713 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 18.595411853999998, 2024-04-24T15:32:19.713 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0081317479999999994, 2024-04-24T15:32:19.713 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:19.714 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:19.714 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:19.714 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4184", 2024-04-24T15:32:19.714 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/1884602651", 2024-04-24T15:32:19.714 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 70 2024-04-24T15:32:19.714 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.714 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:19.714 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.714 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:19.714 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.057275+0000", 2024-04-24T15:32:19.714 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:19.714 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.714 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.715 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:19.715 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.057275+0000", 2024-04-24T15:32:19.715 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:19.715 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.715 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.715 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:19.715 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.057274+0000", 2024-04-24T15:32:19.715 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:19.715 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.715 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.715 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:19.715 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.057277+0000", 2024-04-24T15:32:19.715 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.9790000000000002e-06 2024-04-24T15:32:19.715 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.715 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.716 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:19.716 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.057281+0000", 2024-04-24T15:32:19.716 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5109999999999999e-06 2024-04-24T15:32:19.716 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.716 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.716 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:19.716 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.057289+0000", 2024-04-24T15:32:19.716 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.4300000000000006e-06 2024-04-24T15:32:19.716 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.716 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.716 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:19.716 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.058763+0000", 2024-04-24T15:32:19.716 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.001473238 2024-04-24T15:32:19.716 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.717 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.717 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:19.717 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.058850+0000", 2024-04-24T15:32:19.717 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.7600000000000002e-05 2024-04-24T15:32:19.717 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.717 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.717 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:19.717 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.058972+0000", 2024-04-24T15:32:19.717 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00012228099999999999 2024-04-24T15:32:19.717 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.717 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.717 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:19.717 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.060449+0000", 2024-04-24T15:32:19.717 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.001476705 2024-04-24T15:32:19.717 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.718 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.718 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:19.718 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.065340+0000", 2024-04-24T15:32:19.718 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0048906929999999998 2024-04-24T15:32:19.718 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.718 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.718 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:19.718 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.065370+0000", 2024-04-24T15:32:19.718 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.9887999999999999e-05 2024-04-24T15:32:19.718 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.718 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.718 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:19.718 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.065406+0000", 2024-04-24T15:32:19.719 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.6755999999999998e-05 2024-04-24T15:32:19.719 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:19.719 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:19.719 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:19.719 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.719 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.719 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4184.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-04-24T15:32:19.719 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:01.074168+0000", 2024-04-24T15:32:19.719 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 18.578518238000001, 2024-04-24T15:32:19.719 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0068309110000000003, 2024-04-24T15:32:19.719 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:19.719 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:19.719 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:19.719 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4184", 2024-04-24T15:32:19.720 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/1884602651", 2024-04-24T15:32:19.720 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 88 2024-04-24T15:32:19.720 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.720 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:19.720 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.720 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:19.720 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.074168+0000", 2024-04-24T15:32:19.720 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:19.720 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.720 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.720 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:19.720 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.074168+0000", 2024-04-24T15:32:19.720 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:19.720 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.721 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.721 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:19.721 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.074162+0000", 2024-04-24T15:32:19.721 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999943 2024-04-24T15:32:19.721 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.721 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.721 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:19.721 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.074171+0000", 2024-04-24T15:32:19.721 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.2230000000000007e-06 2024-04-24T15:32:19.721 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.721 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.721 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:19.721 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.074175+0000", 2024-04-24T15:32:19.721 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8800000000000001e-06 2024-04-24T15:32:19.721 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.722 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.722 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:19.722 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.074178+0000", 2024-04-24T15:32:19.722 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.1879999999999998e-06 2024-04-24T15:32:19.722 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.722 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.722 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:19.722 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.076404+0000", 2024-04-24T15:32:19.722 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0022259179999999999 2024-04-24T15:32:19.722 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.722 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.722 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:19.722 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.076447+0000", 2024-04-24T15:32:19.722 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2978000000000001e-05 2024-04-24T15:32:19.723 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.723 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.723 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:19.723 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.076529+0000", 2024-04-24T15:32:19.723 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.1940000000000005e-05 2024-04-24T15:32:19.723 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.723 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.723 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:19.723 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.077197+0000", 2024-04-24T15:32:19.723 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00066798300000000003 2024-04-24T15:32:19.723 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.723 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.723 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:19.723 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.080952+0000", 2024-04-24T15:32:19.724 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0037559410000000001 2024-04-24T15:32:19.724 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.724 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.724 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:19.724 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.080980+0000", 2024-04-24T15:32:19.724 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7134000000000001e-05 2024-04-24T15:32:19.724 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.724 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.724 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:19.724 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.080999+0000", 2024-04-24T15:32:19.724 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.9646000000000001e-05 2024-04-24T15:32:19.724 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:19.724 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:19.724 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:19.725 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.725 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.725 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4184.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-04-24T15:32:19.725 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:01.180426+0000", 2024-04-24T15:32:19.725 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 18.472260631000001, 2024-04-24T15:32:19.725 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0063214070000000002, 2024-04-24T15:32:19.725 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:19.725 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:19.725 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:19.725 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4184", 2024-04-24T15:32:19.725 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/1884602651", 2024-04-24T15:32:19.725 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 116 2024-04-24T15:32:19.725 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.725 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:19.726 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.726 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:19.726 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.180426+0000", 2024-04-24T15:32:19.726 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:19.726 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.726 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.726 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:19.726 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.180426+0000", 2024-04-24T15:32:19.726 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:19.726 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.726 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.726 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:19.726 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.180425+0000", 2024-04-24T15:32:19.726 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:19.727 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.727 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.727 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:19.727 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.180430+0000", 2024-04-24T15:32:19.727 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4569999999999998e-06 2024-04-24T15:32:19.727 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.727 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.727 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:19.727 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.180436+0000", 2024-04-24T15:32:19.727 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.4060000000000002e-06 2024-04-24T15:32:19.727 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.727 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.727 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:19.727 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.180442+0000", 2024-04-24T15:32:19.727 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.5809999999999996e-06 2024-04-24T15:32:19.728 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.728 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.728 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:19.728 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.185273+0000", 2024-04-24T15:32:19.728 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0048306069999999998 2024-04-24T15:32:19.728 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.728 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.728 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:19.728 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.185348+0000", 2024-04-24T15:32:19.728 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.538e-05 2024-04-24T15:32:19.728 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.728 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.728 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:19.728 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.185470+0000", 2024-04-24T15:32:19.729 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00012215300000000001 2024-04-24T15:32:19.729 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.729 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.729 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:19.729 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.186159+0000", 2024-04-24T15:32:19.729 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00068886399999999997 2024-04-24T15:32:19.729 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.729 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.729 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:19.729 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.186672+0000", 2024-04-24T15:32:19.729 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00051336100000000005 2024-04-24T15:32:19.729 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.729 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.729 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:19.730 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.186713+0000", 2024-04-24T15:32:19.730 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.104e-05 2024-04-24T15:32:19.730 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.730 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.730 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:19.730 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.186747+0000", 2024-04-24T15:32:19.730 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.4063000000000002e-05 2024-04-24T15:32:19.730 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:19.730 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:19.730 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:19.730 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.730 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.730 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4184.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-04-24T15:32:19.730 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:01.180448+0000", 2024-04-24T15:32:19.731 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 18.472238600000001, 2024-04-24T15:32:19.731 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0063607250000000002, 2024-04-24T15:32:19.731 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:19.731 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:19.731 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:19.731 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4184", 2024-04-24T15:32:19.731 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/1884602651", 2024-04-24T15:32:19.731 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 117 2024-04-24T15:32:19.731 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.731 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:19.731 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.731 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:19.731 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.180448+0000", 2024-04-24T15:32:19.731 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:19.732 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.732 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.732 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:19.732 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.180448+0000", 2024-04-24T15:32:19.732 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:19.732 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.732 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.732 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:19.732 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.180448+0000", 2024-04-24T15:32:19.732 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:19.732 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.732 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.732 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:19.732 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.180452+0000", 2024-04-24T15:32:19.733 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.997e-06 2024-04-24T15:32:19.733 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.733 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.733 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:19.733 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.180459+0000", 2024-04-24T15:32:19.733 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2389999999999996e-06 2024-04-24T15:32:19.733 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.733 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.733 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:19.733 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.180463+0000", 2024-04-24T15:32:19.733 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.7189999999999999e-06 2024-04-24T15:32:19.733 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.733 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.733 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:19.734 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.185758+0000", 2024-04-24T15:32:19.734 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0052953050000000002 2024-04-24T15:32:19.734 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.734 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.734 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:19.734 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.185805+0000", 2024-04-24T15:32:19.734 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.7490000000000001e-05 2024-04-24T15:32:19.734 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.734 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.734 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:19.734 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.185883+0000", 2024-04-24T15:32:19.734 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7824999999999994e-05 2024-04-24T15:32:19.734 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.735 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.735 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:19.735 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.186252+0000", 2024-04-24T15:32:19.735 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00036880799999999998 2024-04-24T15:32:19.735 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.735 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.735 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:19.735 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.186770+0000", 2024-04-24T15:32:19.735 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00051828499999999995 2024-04-24T15:32:19.735 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.735 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.735 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:19.735 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.186793+0000", 2024-04-24T15:32:19.735 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.2549999999999999e-05 2024-04-24T15:32:19.736 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.736 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.736 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:19.736 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.186809+0000", 2024-04-24T15:32:19.736 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.5883e-05 2024-04-24T15:32:19.736 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:19.736 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:19.736 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:19.736 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.736 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.736 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4184.0:154 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-04-24T15:32:19.736 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:01.928400+0000", 2024-04-24T15:32:19.736 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 17.724286843000002, 2024-04-24T15:32:19.737 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0077033919999999999, 2024-04-24T15:32:19.737 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:19.737 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:19.737 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:19.737 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4184", 2024-04-24T15:32:19.737 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/1884602651", 2024-04-24T15:32:19.737 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 154 2024-04-24T15:32:19.737 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.737 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:19.737 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.737 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:19.737 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928400+0000", 2024-04-24T15:32:19.737 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:19.737 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.738 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.738 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:19.738 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928400+0000", 2024-04-24T15:32:19.738 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:19.738 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.738 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.738 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:19.738 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928399+0000", 2024-04-24T15:32:19.738 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:19.738 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.738 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.738 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:19.739 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928402+0000", 2024-04-24T15:32:19.739 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.497e-06 2024-04-24T15:32:19.739 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.739 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.739 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:19.739 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928407+0000", 2024-04-24T15:32:19.739 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.1950000000000002e-06 2024-04-24T15:32:19.739 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.739 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.739 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:19.739 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928410+0000", 2024-04-24T15:32:19.739 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.5000000000000002e-06 2024-04-24T15:32:19.739 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.739 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.740 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:19.740 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928822+0000", 2024-04-24T15:32:19.740 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00041253900000000002 2024-04-24T15:32:19.740 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.740 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.740 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:19.740 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928865+0000", 2024-04-24T15:32:19.740 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2570000000000001e-05 2024-04-24T15:32:19.740 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.740 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.740 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:19.740 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928944+0000", 2024-04-24T15:32:19.740 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.9634000000000004e-05 2024-04-24T15:32:19.740 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.741 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.741 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:19.741 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.932122+0000", 2024-04-24T15:32:19.741 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.003177862 2024-04-24T15:32:19.741 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.741 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.741 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:19.741 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.936054+0000", 2024-04-24T15:32:19.741 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0039315569999999996 2024-04-24T15:32:19.741 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.741 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.741 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:19.741 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.936083+0000", 2024-04-24T15:32:19.741 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.9043e-05 2024-04-24T15:32:19.742 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.742 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.742 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:19.742 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.936103+0000", 2024-04-24T15:32:19.742 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.0292000000000002e-05 2024-04-24T15:32:19.742 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:19.742 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:19.742 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:19.742 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.742 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.742 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4184.0:155 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-04-24T15:32:19.742 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:01.928434+0000", 2024-04-24T15:32:19.742 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 17.7242523, 2024-04-24T15:32:19.743 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.011257120000000001, 2024-04-24T15:32:19.743 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:19.743 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:19.743 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:19.743 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4184", 2024-04-24T15:32:19.743 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/1884602651", 2024-04-24T15:32:19.743 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 155 2024-04-24T15:32:19.743 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.743 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:19.743 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.743 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:19.743 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928434+0000", 2024-04-24T15:32:19.743 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:19.743 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.744 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.744 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:19.744 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928434+0000", 2024-04-24T15:32:19.744 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:19.744 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.744 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.744 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:19.744 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928434+0000", 2024-04-24T15:32:19.744 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:19.744 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.744 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.744 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:19.744 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928562+0000", 2024-04-24T15:32:19.744 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00012848100000000001 2024-04-24T15:32:19.745 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.745 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.745 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:19.745 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928568+0000", 2024-04-24T15:32:19.745 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.4500000000000003e-06 2024-04-24T15:32:19.745 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.745 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.745 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:19.745 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928572+0000", 2024-04-24T15:32:19.745 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.1880000000000004e-06 2024-04-24T15:32:19.745 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.745 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.745 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:19.745 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.929192+0000", 2024-04-24T15:32:19.746 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00062016399999999998 2024-04-24T15:32:19.746 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.746 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.746 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:19.746 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.929240+0000", 2024-04-24T15:32:19.746 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8112999999999998e-05 2024-04-24T15:32:19.746 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.746 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.746 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:19.746 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.929316+0000", 2024-04-24T15:32:19.746 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.5486000000000002e-05 2024-04-24T15:32:19.746 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.746 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.747 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:19.747 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.935151+0000", 2024-04-24T15:32:19.747 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00583498 2024-04-24T15:32:19.747 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.747 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.747 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:19.747 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.939589+0000", 2024-04-24T15:32:19.747 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0044387840000000003 2024-04-24T15:32:19.747 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.747 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.747 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:19.747 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.939670+0000", 2024-04-24T15:32:19.747 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0389000000000006e-05 2024-04-24T15:32:19.747 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.747 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.748 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:19.748 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.939691+0000", 2024-04-24T15:32:19.748 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.1543999999999999e-05 2024-04-24T15:32:19.748 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:19.748 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:19.748 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:19.748 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.748 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.748 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4184.0:156 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-04-24T15:32:19.748 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:01.928576+0000", 2024-04-24T15:32:19.748 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 17.72411056, 2024-04-24T15:32:19.748 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.011200128, 2024-04-24T15:32:19.748 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:19.748 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:19.749 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:19.749 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4184", 2024-04-24T15:32:19.749 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/1884602651", 2024-04-24T15:32:19.749 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 156 2024-04-24T15:32:19.749 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.749 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:19.749 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.749 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:19.749 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928576+0000", 2024-04-24T15:32:19.749 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:19.749 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.749 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.749 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:19.749 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928576+0000", 2024-04-24T15:32:19.750 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:19.750 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.750 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.750 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:19.750 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928576+0000", 2024-04-24T15:32:19.750 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:19.750 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.750 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.750 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:19.750 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928578+0000", 2024-04-24T15:32:19.750 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.3070000000000001e-06 2024-04-24T15:32:19.750 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.750 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.751 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:19.751 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928581+0000", 2024-04-24T15:32:19.751 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.3419999999999999e-06 2024-04-24T15:32:19.751 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.751 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.751 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:19.751 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928584+0000", 2024-04-24T15:32:19.751 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.1780000000000002e-06 2024-04-24T15:32:19.751 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.751 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.751 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:19.751 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.929860+0000", 2024-04-24T15:32:19.751 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.001276578 2024-04-24T15:32:19.751 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.752 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.752 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:19.752 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.929907+0000", 2024-04-24T15:32:19.752 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.693e-05 2024-04-24T15:32:19.752 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.752 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.752 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:19.752 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.929985+0000", 2024-04-24T15:32:19.752 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7565999999999996e-05 2024-04-24T15:32:19.752 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.752 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.752 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:19.752 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.935155+0000", 2024-04-24T15:32:19.752 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0051699110000000001 2024-04-24T15:32:19.752 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.752 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.753 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:19.753 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.939724+0000", 2024-04-24T15:32:19.753 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0045697189999999999 2024-04-24T15:32:19.753 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.753 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.753 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:19.753 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.939753+0000", 2024-04-24T15:32:19.753 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8547e-05 2024-04-24T15:32:19.753 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.753 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.753 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:19.753 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.939776+0000", 2024-04-24T15:32:19.753 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.3348000000000001e-05 2024-04-24T15:32:19.753 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:19.753 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:19.754 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:19.754 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.754 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.754 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4184.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-04-24T15:32:19.754 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:01.928586+0000", 2024-04-24T15:32:19.754 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 17.724100250999999, 2024-04-24T15:32:19.754 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.011256990999999999, 2024-04-24T15:32:19.754 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:19.754 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:19.754 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:19.754 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4184", 2024-04-24T15:32:19.754 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/1884602651", 2024-04-24T15:32:19.754 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 157 2024-04-24T15:32:19.754 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.755 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:19.755 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.755 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:19.755 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928586+0000", 2024-04-24T15:32:19.755 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:19.755 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.755 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.755 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:19.755 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928586+0000", 2024-04-24T15:32:19.755 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:19.755 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.755 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.755 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:19.755 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928586+0000", 2024-04-24T15:32:19.755 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:32:19.756 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.756 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.756 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:19.756 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928592+0000", 2024-04-24T15:32:19.756 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6400000000000002e-06 2024-04-24T15:32:19.756 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.756 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.756 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:19.756 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928596+0000", 2024-04-24T15:32:19.756 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2169999999999996e-06 2024-04-24T15:32:19.756 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.756 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.756 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:19.756 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928605+0000", 2024-04-24T15:32:19.756 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.1260000000000004e-06 2024-04-24T15:32:19.757 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.757 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.757 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:19.757 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.930310+0000", 2024-04-24T15:32:19.757 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0017047200000000001 2024-04-24T15:32:19.757 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.757 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.757 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:19.757 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.930366+0000", 2024-04-24T15:32:19.757 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6357999999999998e-05 2024-04-24T15:32:19.757 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.757 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.757 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:19.757 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.930438+0000", 2024-04-24T15:32:19.757 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1469000000000003e-05 2024-04-24T15:32:19.758 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.758 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.758 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:19.758 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.935159+0000", 2024-04-24T15:32:19.758 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0047212809999999999 2024-04-24T15:32:19.758 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.758 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.758 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:19.758 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.939803+0000", 2024-04-24T15:32:19.758 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0046441920000000001 2024-04-24T15:32:19.758 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.758 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.758 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:19.758 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.939826+0000", 2024-04-24T15:32:19.759 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.2949e-05 2024-04-24T15:32:19.759 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.759 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.759 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:19.759 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.939843+0000", 2024-04-24T15:32:19.759 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.7266999999999999e-05 2024-04-24T15:32:19.759 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:19.759 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:19.759 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:19.759 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.759 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.759 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4184.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-04-24T15:32:19.759 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:01.928608+0000", 2024-04-24T15:32:19.759 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 17.724078106, 2024-04-24T15:32:19.759 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.011304719, 2024-04-24T15:32:19.760 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:19.760 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:19.760 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:19.760 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4184", 2024-04-24T15:32:19.760 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/1884602651", 2024-04-24T15:32:19.760 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 158 2024-04-24T15:32:19.760 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.760 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:19.760 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.760 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:19.760 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928608+0000", 2024-04-24T15:32:19.760 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:19.760 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.760 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.760 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:19.761 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928608+0000", 2024-04-24T15:32:19.761 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:19.761 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.761 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.761 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:19.761 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928608+0000", 2024-04-24T15:32:19.761 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:19.761 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.761 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.761 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:19.761 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928652+0000", 2024-04-24T15:32:19.761 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3856999999999997e-05 2024-04-24T15:32:19.761 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.761 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.761 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:19.762 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928660+0000", 2024-04-24T15:32:19.762 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.9619999999999993e-06 2024-04-24T15:32:19.762 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.762 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.762 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:19.762 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928667+0000", 2024-04-24T15:32:19.762 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.029e-06 2024-04-24T15:32:19.762 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.762 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.762 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:19.762 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.930668+0000", 2024-04-24T15:32:19.762 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0020010219999999999 2024-04-24T15:32:19.762 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.763 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.763 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:19.763 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.930712+0000", 2024-04-24T15:32:19.763 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3897999999999999e-05 2024-04-24T15:32:19.763 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.763 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.763 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:19.763 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.930784+0000", 2024-04-24T15:32:19.763 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1898999999999997e-05 2024-04-24T15:32:19.763 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.763 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.763 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:19.763 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.935162+0000", 2024-04-24T15:32:19.763 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.004378364 2024-04-24T15:32:19.763 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.764 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.764 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:19.764 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.939870+0000", 2024-04-24T15:32:19.764 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0047080439999999998 2024-04-24T15:32:19.764 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.764 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.764 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:19.764 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.939893+0000", 2024-04-24T15:32:19.764 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.2989e-05 2024-04-24T15:32:19.764 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.764 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.764 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:19.764 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.939913+0000", 2024-04-24T15:32:19.764 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.9925999999999999e-05 2024-04-24T15:32:19.765 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:19.765 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:19.765 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:19.765 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.765 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.765 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4184.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-04-24T15:32:19.765 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:01.928671+0000", 2024-04-24T15:32:19.765 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 17.724015579, 2024-04-24T15:32:19.765 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.011298967, 2024-04-24T15:32:19.765 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:19.765 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:19.765 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:19.765 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4184", 2024-04-24T15:32:19.766 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/1884602651", 2024-04-24T15:32:19.766 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 159 2024-04-24T15:32:19.766 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.766 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:19.766 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.766 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:19.766 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928671+0000", 2024-04-24T15:32:19.766 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:19.766 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.766 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.766 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:19.766 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928671+0000", 2024-04-24T15:32:19.766 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:19.766 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.767 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.767 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:19.767 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928671+0000", 2024-04-24T15:32:19.767 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:19.767 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.767 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.767 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:19.767 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928674+0000", 2024-04-24T15:32:19.767 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.715e-06 2024-04-24T15:32:19.767 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.767 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.767 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:19.767 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928680+0000", 2024-04-24T15:32:19.767 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.0939999999999998e-06 2024-04-24T15:32:19.768 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.768 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.768 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:19.768 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928683+0000", 2024-04-24T15:32:19.768 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.1360000000000001e-06 2024-04-24T15:32:19.768 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.768 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.768 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:19.768 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.931109+0000", 2024-04-24T15:32:19.768 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0024267379999999999 2024-04-24T15:32:19.768 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.768 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.768 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:19.769 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.931153+0000", 2024-04-24T15:32:19.769 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3970000000000001e-05 2024-04-24T15:32:19.769 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.769 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.769 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:19.769 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.931231+0000", 2024-04-24T15:32:19.769 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.8053999999999995e-05 2024-04-24T15:32:19.769 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.769 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.769 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:19.769 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.935166+0000", 2024-04-24T15:32:19.769 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0039344159999999996 2024-04-24T15:32:19.769 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.769 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.770 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:19.770 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.939932+0000", 2024-04-24T15:32:19.770 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0047658980000000002 2024-04-24T15:32:19.770 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.770 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.770 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:19.770 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.939953+0000", 2024-04-24T15:32:19.770 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.1107e-05 2024-04-24T15:32:19.770 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:19.770 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:19.770 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:19.770 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.939970+0000", 2024-04-24T15:32:19.770 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.7110999999999999e-05 2024-04-24T15:32:19.771 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:19.771 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:19.771 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:19.771 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:19.771 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:19.771 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:32:19.772 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:32:19.773+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:32:19.773 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-24T15:32:19.873 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:32:19.874+0000 7f23abcf6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 44021) UID: 0 2024-04-24T15:32:19.898 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:32:19.898 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:32:19.898 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-04-24T15:32:19.898 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:32:19.910 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-24T15:32:19.974 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:32:19.975+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:32:20.036 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:32:20.037 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:32:20.037 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-04-24T15:32:20.037 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-04-24T15:32:20.037 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:32:20.047 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-24T15:32:20.061 INFO:teuthology.orchestra.run.smithi096.stderr:nodeep-scrub is set 2024-04-24T15:32:20.114 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:32:20.115+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:32:20.176 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:32:20.177+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:32:20.190 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:32:20.190 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-04-24T15:32:20.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-04-24T15:32:20.190 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-04-24T15:32:20.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.191 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:18 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'8, mlcod=15'5)", 2024-04-24T15:32:20.191 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:00.973932+0000", 2024-04-24T15:32:20.191 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.216296994, 2024-04-24T15:32:20.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0064286839999999996, 2024-04-24T15:32:20.191 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:20.191 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:20.191 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:20.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:20.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.973932+0000", 2024-04-24T15:32:20.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:20.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:20.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.973932+0000", 2024-04-24T15:32:20.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:20.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:20.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.973931+0000", 2024-04-24T15:32:20.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:20.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:20.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.973938+0000", 2024-04-24T15:32:20.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.029e-06 2024-04-24T15:32:20.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:20.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.973948+0000", 2024-04-24T15:32:20.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9129999999999998e-06 2024-04-24T15:32:20.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:20.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.973952+0000", 2024-04-24T15:32:20.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3069999999999996e-06 2024-04-24T15:32:20.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:20.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.973986+0000", 2024-04-24T15:32:20.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3643000000000003e-05 2024-04-24T15:32:20.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:20.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.974006+0000", 2024-04-24T15:32:20.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9658999999999999e-05 2024-04-24T15:32:20.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:20.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.980330+0000", 2024-04-24T15:32:20.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0063243479999999996 2024-04-24T15:32:20.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:20.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.980361+0000", 2024-04-24T15:32:20.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0527000000000001e-05 2024-04-24T15:32:20.195 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:20.195 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:20.195 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:20.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.195 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:19 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'9, mlcod=15'7)", 2024-04-24T15:32:20.195 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:00.981269+0000", 2024-04-24T15:32:20.195 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.208959637, 2024-04-24T15:32:20.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035371999999999999, 2024-04-24T15:32:20.195 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:20.195 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:20.195 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:20.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:20.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.981269+0000", 2024-04-24T15:32:20.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:20.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:20.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.981269+0000", 2024-04-24T15:32:20.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:20.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:20.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.981268+0000", 2024-04-24T15:32:20.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:20.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:20.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.981292+0000", 2024-04-24T15:32:20.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4011000000000001e-05 2024-04-24T15:32:20.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:20.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.981300+0000", 2024-04-24T15:32:20.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5079999999999999e-06 2024-04-24T15:32:20.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:20.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.981305+0000", 2024-04-24T15:32:20.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.904e-06 2024-04-24T15:32:20.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.198 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:20.198 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.981342+0000", 2024-04-24T15:32:20.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6659000000000003e-05 2024-04-24T15:32:20.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.198 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:20.198 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.981361+0000", 2024-04-24T15:32:20.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9247000000000001e-05 2024-04-24T15:32:20.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.198 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:20.198 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.984777+0000", 2024-04-24T15:32:20.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003415778 2024-04-24T15:32:20.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:20.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.984806+0000", 2024-04-24T15:32:20.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9916e-05 2024-04-24T15:32:20.199 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:20.199 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:20.199 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:20.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.199 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:21 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'11, mlcod=15'8)", 2024-04-24T15:32:20.199 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:00.986502+0000", 2024-04-24T15:32:20.199 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.203727060999999, 2024-04-24T15:32:20.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0062008819999999996, 2024-04-24T15:32:20.200 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:20.200 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:20.200 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:20.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:20.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.986502+0000", 2024-04-24T15:32:20.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:20.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:20.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.986502+0000", 2024-04-24T15:32:20.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:20.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:20.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.986501+0000", 2024-04-24T15:32:20.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:20.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:20.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.986639+0000", 2024-04-24T15:32:20.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013731400000000001 2024-04-24T15:32:20.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:20.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.986649+0000", 2024-04-24T15:32:20.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.041e-05 2024-04-24T15:32:20.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:20.202 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.986654+0000", 2024-04-24T15:32:20.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2000000000000002e-06 2024-04-24T15:32:20.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:20.202 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.986678+0000", 2024-04-24T15:32:20.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3731e-05 2024-04-24T15:32:20.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:20.202 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.986691+0000", 2024-04-24T15:32:20.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3078000000000001e-05 2024-04-24T15:32:20.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:20.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.992666+0000", 2024-04-24T15:32:20.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0059751650000000002 2024-04-24T15:32:20.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:20.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.992703+0000", 2024-04-24T15:32:20.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6622000000000003e-05 2024-04-24T15:32:20.203 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:20.203 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:20.203 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:20.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.204 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:22 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'12, mlcod=15'8)", 2024-04-24T15:32:20.204 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:00.987102+0000", 2024-04-24T15:32:20.204 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.203127214999999, 2024-04-24T15:32:20.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0056291650000000002, 2024-04-24T15:32:20.204 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:20.204 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:20.204 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:20.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:20.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.987102+0000", 2024-04-24T15:32:20.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:20.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.205 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:20.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.987102+0000", 2024-04-24T15:32:20.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:20.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.205 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:20.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.987101+0000", 2024-04-24T15:32:20.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:20.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.205 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:20.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.987106+0000", 2024-04-24T15:32:20.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0969999999999996e-06 2024-04-24T15:32:20.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.206 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.206 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:20.206 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.987113+0000", 2024-04-24T15:32:20.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6789999999999999e-06 2024-04-24T15:32:20.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.206 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.206 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:20.206 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.987117+0000", 2024-04-24T15:32:20.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6519999999999999e-06 2024-04-24T15:32:20.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.206 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.206 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:20.206 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.987143+0000", 2024-04-24T15:32:20.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6098999999999999e-05 2024-04-24T15:32:20.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.207 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:20.207 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.987157+0000", 2024-04-24T15:32:20.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4100999999999999e-05 2024-04-24T15:32:20.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.207 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:20.207 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.992706+0000", 2024-04-24T15:32:20.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0055495630000000004 2024-04-24T15:32:20.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:20.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.992731+0000", 2024-04-24T15:32:20.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4519e-05 2024-04-24T15:32:20.208 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:20.208 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:20.208 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:20.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.208 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:23 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'13, mlcod=15'11)", 2024-04-24T15:32:20.208 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:00.993573+0000", 2024-04-24T15:32:20.208 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.196656277999999, 2024-04-24T15:32:20.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0037599529999999999, 2024-04-24T15:32:20.208 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:20.208 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:20.209 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:20.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:20.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.993573+0000", 2024-04-24T15:32:20.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:20.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:20.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.993573+0000", 2024-04-24T15:32:20.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:20.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:20.210 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.993571+0000", 2024-04-24T15:32:20.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999986 2024-04-24T15:32:20.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.210 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:20.210 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.993591+0000", 2024-04-24T15:32:20.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9738e-05 2024-04-24T15:32:20.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.210 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:20.210 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.993605+0000", 2024-04-24T15:32:20.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3516000000000001e-05 2024-04-24T15:32:20.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.211 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:20.211 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.993611+0000", 2024-04-24T15:32:20.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4930000000000003e-06 2024-04-24T15:32:20.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.211 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:20.211 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.993646+0000", 2024-04-24T15:32:20.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5067999999999997e-05 2024-04-24T15:32:20.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.211 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:20.211 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.993667+0000", 2024-04-24T15:32:20.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0517999999999999e-05 2024-04-24T15:32:20.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:20.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.997297+0000", 2024-04-24T15:32:20.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0036301549999999999 2024-04-24T15:32:20.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:20.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.997333+0000", 2024-04-24T15:32:20.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5682000000000002e-05 2024-04-24T15:32:20.212 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:20.212 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:20.212 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:20.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.213 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:36 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'22, mlcod=15'15)", 2024-04-24T15:32:20.213 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.011092+0000", 2024-04-24T15:32:20.213 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.17913682, 2024-04-24T15:32:20.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004602887, 2024-04-24T15:32:20.213 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:20.213 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:20.213 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:20.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:20.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.011092+0000", 2024-04-24T15:32:20.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:20.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:20.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.011092+0000", 2024-04-24T15:32:20.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:20.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:20.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.011092+0000", 2024-04-24T15:32:20.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:20.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:20.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.011099+0000", 2024-04-24T15:32:20.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4900000000000003e-06 2024-04-24T15:32:20.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:20.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.011107+0000", 2024-04-24T15:32:20.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7370000000000006e-06 2024-04-24T15:32:20.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:20.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.011111+0000", 2024-04-24T15:32:20.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.822e-06 2024-04-24T15:32:20.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:20.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.011137+0000", 2024-04-24T15:32:20.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6092999999999999e-05 2024-04-24T15:32:20.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:20.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.011149+0000", 2024-04-24T15:32:20.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1727e-05 2024-04-24T15:32:20.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:20.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.015673+0000", 2024-04-24T15:32:20.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004524686 2024-04-24T15:32:20.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:20.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.015695+0000", 2024-04-24T15:32:20.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.179e-05 2024-04-24T15:32:20.217 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:20.217 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:20.217 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:20.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.217 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:37 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'23, mlcod=15'15)", 2024-04-24T15:32:20.217 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.011567+0000", 2024-04-24T15:32:20.217 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.178661786999999, 2024-04-24T15:32:20.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004149865, 2024-04-24T15:32:20.217 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:20.217 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:20.217 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:20.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:20.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.011567+0000", 2024-04-24T15:32:20.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:20.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:20.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.011567+0000", 2024-04-24T15:32:20.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:20.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:20.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.011567+0000", 2024-04-24T15:32:20.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:20.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:20.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.011572+0000", 2024-04-24T15:32:20.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0710000000000002e-06 2024-04-24T15:32:20.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:20.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.011579+0000", 2024-04-24T15:32:20.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7519999999999996e-06 2024-04-24T15:32:20.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:20.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.011583+0000", 2024-04-24T15:32:20.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.579e-06 2024-04-24T15:32:20.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:20.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.011614+0000", 2024-04-24T15:32:20.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1188e-05 2024-04-24T15:32:20.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:20.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.011627+0000", 2024-04-24T15:32:20.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.257e-05 2024-04-24T15:32:20.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:20.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.015697+0000", 2024-04-24T15:32:20.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0040705649999999999 2024-04-24T15:32:20.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:20.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.015717+0000", 2024-04-24T15:32:20.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9721e-05 2024-04-24T15:32:20.221 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:20.221 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:20.221 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:20.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.221 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:67 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'41, mlcod=15'39)", 2024-04-24T15:32:20.221 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.057501+0000", 2024-04-24T15:32:20.221 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.132727725999999, 2024-04-24T15:32:20.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0042378980000000004, 2024-04-24T15:32:20.222 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:20.222 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:20.222 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:20.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:20.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.057501+0000", 2024-04-24T15:32:20.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:20.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:20.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.057501+0000", 2024-04-24T15:32:20.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:20.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:20.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.057500+0000", 2024-04-24T15:32:20.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:20.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:20.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.057509+0000", 2024-04-24T15:32:20.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9660000000000002e-06 2024-04-24T15:32:20.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:20.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.057523+0000", 2024-04-24T15:32:20.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3634e-05 2024-04-24T15:32:20.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:20.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.057529+0000", 2024-04-24T15:32:20.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6390000000000003e-06 2024-04-24T15:32:20.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:20.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.057565+0000", 2024-04-24T15:32:20.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.54e-05 2024-04-24T15:32:20.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:20.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.057581+0000", 2024-04-24T15:32:20.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6268000000000001e-05 2024-04-24T15:32:20.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:20.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.061711+0000", 2024-04-24T15:32:20.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0041296859999999996 2024-04-24T15:32:20.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:20.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.061739+0000", 2024-04-24T15:32:20.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8394e-05 2024-04-24T15:32:20.225 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:20.225 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:20.226 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:20.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.226 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:68 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'42, mlcod=15'39)", 2024-04-24T15:32:20.226 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.058047+0000", 2024-04-24T15:32:20.226 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.132182104999998, 2024-04-24T15:32:20.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004741706, 2024-04-24T15:32:20.226 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:20.226 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:20.226 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:20.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:20.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.058047+0000", 2024-04-24T15:32:20.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:20.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:20.227 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.058047+0000", 2024-04-24T15:32:20.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:20.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:20.227 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.058046+0000", 2024-04-24T15:32:20.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:20.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.228 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:20.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.058053+0000", 2024-04-24T15:32:20.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5660000000000003e-06 2024-04-24T15:32:20.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.228 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:20.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.058063+0000", 2024-04-24T15:32:20.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9350000000000006e-06 2024-04-24T15:32:20.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.228 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:20.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.058067+0000", 2024-04-24T15:32:20.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3200000000000001e-06 2024-04-24T15:32:20.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:20.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.060633+0000", 2024-04-24T15:32:20.229 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0025664189999999999 2024-04-24T15:32:20.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:20.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.060655+0000", 2024-04-24T15:32:20.229 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1492e-05 2024-04-24T15:32:20.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:20.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.062761+0000", 2024-04-24T15:32:20.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002105797 2024-04-24T15:32:20.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.230 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:20.230 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.062789+0000", 2024-04-24T15:32:20.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7847e-05 2024-04-24T15:32:20.230 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:20.230 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:20.230 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:20.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.230 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:69 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'43, mlcod=15'39)", 2024-04-24T15:32:20.230 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.058504+0000", 2024-04-24T15:32:20.231 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.131724740999999, 2024-04-24T15:32:20.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.006494606, 2024-04-24T15:32:20.231 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:20.231 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:20.231 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:20.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.231 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:20.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.058504+0000", 2024-04-24T15:32:20.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:20.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.231 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:20.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.058504+0000", 2024-04-24T15:32:20.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:20.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.232 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:20.232 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.058503+0000", 2024-04-24T15:32:20.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:20.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.232 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:20.232 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.058588+0000", 2024-04-24T15:32:20.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4087999999999999e-05 2024-04-24T15:32:20.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.232 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:20.232 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.058597+0000", 2024-04-24T15:32:20.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7459999999999995e-06 2024-04-24T15:32:20.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:20.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.058602+0000", 2024-04-24T15:32:20.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6929999999999998e-06 2024-04-24T15:32:20.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:20.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.060983+0000", 2024-04-24T15:32:20.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023812690000000001 2024-04-24T15:32:20.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:20.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.060996+0000", 2024-04-24T15:32:20.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2826999999999999e-05 2024-04-24T15:32:20.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:20.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.064972+0000", 2024-04-24T15:32:20.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0039757890000000004 2024-04-24T15:32:20.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:20.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.064999+0000", 2024-04-24T15:32:20.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6917999999999999e-05 2024-04-24T15:32:20.234 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:20.234 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:20.235 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:20.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.235 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:70 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'44, mlcod=15'39)", 2024-04-24T15:32:20.235 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.059180+0000", 2024-04-24T15:32:20.235 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.131048989, 2024-04-24T15:32:20.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0058429709999999998, 2024-04-24T15:32:20.235 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:20.235 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:20.235 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:20.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:20.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.059180+0000", 2024-04-24T15:32:20.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:20.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.236 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:20.236 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.059180+0000", 2024-04-24T15:32:20.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:20.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.236 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:20.236 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.059179+0000", 2024-04-24T15:32:20.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:20.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.236 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:20.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.059184+0000", 2024-04-24T15:32:20.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.074e-06 2024-04-24T15:32:20.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:20.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.059194+0000", 2024-04-24T15:32:20.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6050000000000004e-06 2024-04-24T15:32:20.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:20.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.059199+0000", 2024-04-24T15:32:20.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2639999999999999e-06 2024-04-24T15:32:20.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.238 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:20.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.061424+0000", 2024-04-24T15:32:20.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0022244880000000002 2024-04-24T15:32:20.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.238 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:20.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.061444+0000", 2024-04-24T15:32:20.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0633e-05 2024-04-24T15:32:20.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.238 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:20.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.065001+0000", 2024-04-24T15:32:20.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035567670000000002 2024-04-24T15:32:20.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:20.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.065023+0000", 2024-04-24T15:32:20.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1724000000000001e-05 2024-04-24T15:32:20.239 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:20.239 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:20.239 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:20.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.239 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:88 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'58, mlcod=15'51)", 2024-04-24T15:32:20.239 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.076587+0000", 2024-04-24T15:32:20.239 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.113641831999999, 2024-04-24T15:32:20.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0042421120000000001, 2024-04-24T15:32:20.240 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:20.240 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:20.240 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:20.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:20.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.076587+0000", 2024-04-24T15:32:20.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:20.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:20.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.076587+0000", 2024-04-24T15:32:20.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:20.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:20.241 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.076587+0000", 2024-04-24T15:32:20.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:20.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:20.241 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.076591+0000", 2024-04-24T15:32:20.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6990000000000002e-06 2024-04-24T15:32:20.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:20.241 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.076599+0000", 2024-04-24T15:32:20.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.464e-06 2024-04-24T15:32:20.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:20.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.076602+0000", 2024-04-24T15:32:20.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6689999999999999e-06 2024-04-24T15:32:20.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:20.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.077691+0000", 2024-04-24T15:32:20.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001088405 2024-04-24T15:32:20.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:20.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.077708+0000", 2024-04-24T15:32:20.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7142e-05 2024-04-24T15:32:20.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:20.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.080806+0000", 2024-04-24T15:32:20.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003097982 2024-04-24T15:32:20.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:20.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.080829+0000", 2024-04-24T15:32:20.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3280000000000001e-05 2024-04-24T15:32:20.243 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:20.243 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:20.244 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:20.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.244 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:90 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'60, mlcod=15'57)", 2024-04-24T15:32:20.244 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.082088+0000", 2024-04-24T15:32:20.244 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.108141404000001, 2024-04-24T15:32:20.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0040776370000000003, 2024-04-24T15:32:20.244 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:20.244 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:20.244 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:20.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:20.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.082088+0000", 2024-04-24T15:32:20.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:20.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:20.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.082088+0000", 2024-04-24T15:32:20.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:20.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:20.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.082087+0000", 2024-04-24T15:32:20.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:20.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:20.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.082136+0000", 2024-04-24T15:32:20.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8655000000000001e-05 2024-04-24T15:32:20.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.246 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:20.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.082143+0000", 2024-04-24T15:32:20.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.43e-06 2024-04-24T15:32:20.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.246 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:20.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.082148+0000", 2024-04-24T15:32:20.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.549e-06 2024-04-24T15:32:20.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:20.247 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.082171+0000", 2024-04-24T15:32:20.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3363999999999999e-05 2024-04-24T15:32:20.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:20.247 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.082182+0000", 2024-04-24T15:32:20.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1136e-05 2024-04-24T15:32:20.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:20.247 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.086139+0000", 2024-04-24T15:32:20.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0039572369999999997 2024-04-24T15:32:20.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:20.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.086165+0000", 2024-04-24T15:32:20.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5863999999999998e-05 2024-04-24T15:32:20.248 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:20.248 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:20.248 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:20.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.248 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:122 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'84, mlcod=15'82)", 2024-04-24T15:32:20.248 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.191452+0000", 2024-04-24T15:32:20.248 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.998776600999999, 2024-04-24T15:32:20.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0034105450000000001, 2024-04-24T15:32:20.249 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:20.249 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:20.249 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:20.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:20.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.191452+0000", 2024-04-24T15:32:20.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:20.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:20.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.191452+0000", 2024-04-24T15:32:20.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:20.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:20.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.191452+0000", 2024-04-24T15:32:20.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:20.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:20.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.191458+0000", 2024-04-24T15:32:20.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5289999999999997e-06 2024-04-24T15:32:20.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:20.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.191467+0000", 2024-04-24T15:32:20.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3610000000000001e-06 2024-04-24T15:32:20.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:20.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.191471+0000", 2024-04-24T15:32:20.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9169999999999999e-06 2024-04-24T15:32:20.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:20.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.191498+0000", 2024-04-24T15:32:20.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7328999999999999e-05 2024-04-24T15:32:20.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:20.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.191514+0000", 2024-04-24T15:32:20.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5718000000000001e-05 2024-04-24T15:32:20.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:20.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.194837+0000", 2024-04-24T15:32:20.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003323431 2024-04-24T15:32:20.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:20.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.194863+0000", 2024-04-24T15:32:20.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5806e-05 2024-04-24T15:32:20.252 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:20.252 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:20.253 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:20.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.253 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:154 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'108, mlcod=15'105)", 2024-04-24T15:32:20.253 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.929033+0000", 2024-04-24T15:32:20.253 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.261195608000001, 2024-04-24T15:32:20.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0068887990000000001, 2024-04-24T15:32:20.253 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:20.253 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:20.253 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:20.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:20.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929033+0000", 2024-04-24T15:32:20.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:20.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:20.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929033+0000", 2024-04-24T15:32:20.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:20.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:20.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929033+0000", 2024-04-24T15:32:20.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:20.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:20.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929042+0000", 2024-04-24T15:32:20.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5440000000000004e-06 2024-04-24T15:32:20.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:20.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929051+0000", 2024-04-24T15:32:20.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0580000000000004e-06 2024-04-24T15:32:20.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:20.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929055+0000", 2024-04-24T15:32:20.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.337e-06 2024-04-24T15:32:20.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:20.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929074+0000", 2024-04-24T15:32:20.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9724999999999998e-05 2024-04-24T15:32:20.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:20.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929084+0000", 2024-04-24T15:32:20.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7049999999999994e-06 2024-04-24T15:32:20.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:20.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.935895+0000", 2024-04-24T15:32:20.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0068104359999999996 2024-04-24T15:32:20.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:20.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.935922+0000", 2024-04-24T15:32:20.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7538999999999998e-05 2024-04-24T15:32:20.257 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:20.257 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:20.257 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:20.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.257 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:155 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'109, mlcod=15'105)", 2024-04-24T15:32:20.257 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.929398+0000", 2024-04-24T15:32:20.257 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.260830627000001, 2024-04-24T15:32:20.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0098754700000000008, 2024-04-24T15:32:20.257 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:20.257 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:20.257 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:20.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:20.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929398+0000", 2024-04-24T15:32:20.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:20.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:20.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929398+0000", 2024-04-24T15:32:20.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:20.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:20.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929398+0000", 2024-04-24T15:32:20.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:20.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.259 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:20.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929571+0000", 2024-04-24T15:32:20.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017281999999999999 2024-04-24T15:32:20.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.259 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:20.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929578+0000", 2024-04-24T15:32:20.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8929999999999999e-06 2024-04-24T15:32:20.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.259 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:20.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929581+0000", 2024-04-24T15:32:20.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1709999999999998e-06 2024-04-24T15:32:20.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:20.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929605+0000", 2024-04-24T15:32:20.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3938e-05 2024-04-24T15:32:20.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:20.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929618+0000", 2024-04-24T15:32:20.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3653e-05 2024-04-24T15:32:20.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:20.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939186+0000", 2024-04-24T15:32:20.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0095674260000000004 2024-04-24T15:32:20.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:20.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939274+0000", 2024-04-24T15:32:20.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7928000000000002e-05 2024-04-24T15:32:20.261 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:20.261 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:20.261 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:20.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.261 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:156 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'110, mlcod=15'105)", 2024-04-24T15:32:20.261 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.930050+0000", 2024-04-24T15:32:20.262 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.260179311000002, 2024-04-24T15:32:20.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0092911650000000005, 2024-04-24T15:32:20.262 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:20.262 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:20.262 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:20.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:20.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930050+0000", 2024-04-24T15:32:20.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:20.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:20.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930050+0000", 2024-04-24T15:32:20.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:20.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:20.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930049+0000", 2024-04-24T15:32:20.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:20.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:20.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930053+0000", 2024-04-24T15:32:20.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.98e-06 2024-04-24T15:32:20.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:20.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930058+0000", 2024-04-24T15:32:20.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3530000000000002e-06 2024-04-24T15:32:20.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:20.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930061+0000", 2024-04-24T15:32:20.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6970000000000002e-06 2024-04-24T15:32:20.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:20.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930082+0000", 2024-04-24T15:32:20.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0815e-05 2024-04-24T15:32:20.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.265 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:20.265 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930093+0000", 2024-04-24T15:32:20.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0699999999999999e-05 2024-04-24T15:32:20.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.265 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:20.265 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939280+0000", 2024-04-24T15:32:20.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0091874880000000006 2024-04-24T15:32:20.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.265 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:20.265 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939341+0000", 2024-04-24T15:32:20.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0608e-05 2024-04-24T15:32:20.266 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:20.266 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:20.266 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:20.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.266 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:157 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'111, mlcod=15'106)", 2024-04-24T15:32:20.266 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.930506+0000", 2024-04-24T15:32:20.266 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.259722452999998, 2024-04-24T15:32:20.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.008889246, 2024-04-24T15:32:20.266 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:20.266 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:20.266 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:20.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:20.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930506+0000", 2024-04-24T15:32:20.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:20.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:20.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930506+0000", 2024-04-24T15:32:20.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:20.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:20.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930506+0000", 2024-04-24T15:32:20.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:20.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:20.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930510+0000", 2024-04-24T15:32:20.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1810000000000003e-06 2024-04-24T15:32:20.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:20.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930516+0000", 2024-04-24T15:32:20.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1920000000000002e-06 2024-04-24T15:32:20.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:20.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930520+0000", 2024-04-24T15:32:20.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.501e-06 2024-04-24T15:32:20.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.269 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.269 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:20.269 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930539+0000", 2024-04-24T15:32:20.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8929000000000001e-05 2024-04-24T15:32:20.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.269 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.269 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:20.269 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930550+0000", 2024-04-24T15:32:20.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1168999999999999e-05 2024-04-24T15:32:20.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.269 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.269 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:20.270 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939343+0000", 2024-04-24T15:32:20.270 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0087932900000000005 2024-04-24T15:32:20.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.270 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:20.270 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939396+0000", 2024-04-24T15:32:20.270 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2454999999999998e-05 2024-04-24T15:32:20.270 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:20.270 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:20.270 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:20.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.270 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:158 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'112, mlcod=15'106)", 2024-04-24T15:32:20.270 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.930877+0000", 2024-04-24T15:32:20.271 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.259351762000001, 2024-04-24T15:32:20.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.008572741, 2024-04-24T15:32:20.271 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:20.271 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:20.271 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:20.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.271 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:20.271 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930877+0000", 2024-04-24T15:32:20.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:20.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.271 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:20.271 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930877+0000", 2024-04-24T15:32:20.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:20.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:20.272 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930877+0000", 2024-04-24T15:32:20.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:20.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:20.272 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930968+0000", 2024-04-24T15:32:20.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1638999999999995e-05 2024-04-24T15:32:20.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:20.272 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930975+0000", 2024-04-24T15:32:20.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2169999999999999e-06 2024-04-24T15:32:20.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.273 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.273 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:20.273 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930977+0000", 2024-04-24T15:32:20.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.791e-06 2024-04-24T15:32:20.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.273 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.273 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:20.273 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931002+0000", 2024-04-24T15:32:20.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4491e-05 2024-04-24T15:32:20.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.273 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.273 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:20.273 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931012+0000", 2024-04-24T15:32:20.274 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0217000000000001e-05 2024-04-24T15:32:20.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.274 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.274 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:20.274 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939398+0000", 2024-04-24T15:32:20.274 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.008385366 2024-04-24T15:32:20.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.274 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.274 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:20.274 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939450+0000", 2024-04-24T15:32:20.274 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2326e-05 2024-04-24T15:32:20.274 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:20.274 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:20.275 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:20.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.275 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:159 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'113, mlcod=15'106)", 2024-04-24T15:32:20.275 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.931313+0000", 2024-04-24T15:32:20.275 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.258916410000001, 2024-04-24T15:32:20.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0081994159999999993, 2024-04-24T15:32:20.275 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:20.275 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:20.275 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:20.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.275 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:20.275 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931313+0000", 2024-04-24T15:32:20.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:20.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:20.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931313+0000", 2024-04-24T15:32:20.276 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:20.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:20.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931312+0000", 2024-04-24T15:32:20.276 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:20.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:20.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931316+0000", 2024-04-24T15:32:20.276 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0589999999999996e-06 2024-04-24T15:32:20.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.277 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.277 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:20.277 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931322+0000", 2024-04-24T15:32:20.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2020000000000001e-06 2024-04-24T15:32:20.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.277 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.277 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:20.277 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931327+0000", 2024-04-24T15:32:20.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2440000000000004e-06 2024-04-24T15:32:20.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.277 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.277 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:20.277 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931348+0000", 2024-04-24T15:32:20.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1464e-05 2024-04-24T15:32:20.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.278 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:20.278 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931359+0000", 2024-04-24T15:32:20.278 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0699e-05 2024-04-24T15:32:20.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.278 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:20.278 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939452+0000", 2024-04-24T15:32:20.278 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0080931749999999993 2024-04-24T15:32:20.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:20.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:20.278 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:20.278 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939512+0000", 2024-04-24T15:32:20.279 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0050000000000003e-05 2024-04-24T15:32:20.279 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:20.279 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:20.279 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:20.279 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:20.279 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:20.279 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:32:20.280 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-24T15:32:20.281 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:32:20.281+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:32:20.381 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:32:20.382+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:32:20.447 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:32:20.447 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:32:20.447 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-04-24T15:32:20.447 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:32:20.457 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-24T15:32:20.486 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-24T15:32:20.500 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:32:20.501+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:32:20.583 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:32:20.584+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:32:20.593 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:32:20.593 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:32:20.593 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-04-24T15:32:20.593 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-04-24T15:32:20.593 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:32:20.604 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-24T15:32:20.651 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-04-24T15:32:20.670 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-04-24T15:32:20.684 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:32:20.685+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:32:20.688 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-04-24T15:32:20.707 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-04-24T15:32:20.726 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-04-24T15:32:20.734 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:20.734 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:20.746 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-04-24T15:32:20.749 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-24T15:32:20.766 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-04-24T15:32:20.785 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:32:20.785+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:32:20.788 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-04-24T15:32:20.789 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T15:32:20.790 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T15:32:20.790 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T15:32:20.790 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T15:32:20.790 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T15:32:20.790 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T15:32:20.790 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T15:32:20.790 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T15:32:20.790 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:20.791 INFO:teuthology.orchestra.run.smithi096.stdout:skipping zero length punch hole 2024-04-24T15:32:20.796 INFO:teuthology.orchestra.run.smithi096.stdout:3 write 0x853c21 thru 0x854f4c (0x132c bytes) 2024-04-24T15:32:20.822 INFO:teuthology.orchestra.run.smithi096.stdout:4 writesame 0x87d947e thru 0x87e8137 data_size 0x9d(0xecba bytes) 2024-04-24T15:32:20.830 INFO:teuthology.orchestra.run.smithi096.stdout:5 write 0xa6de963 thru 0xa6e3911 (0x4faf bytes) 2024-04-24T15:32:20.834 INFO:teuthology.orchestra.run.smithi096.stdout:6 read 0x6528944 thru 0x65385a7 (0xfc64 bytes) 2024-04-24T15:32:20.834 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:20.843 INFO:teuthology.orchestra.run.smithi096.stdout:8 write 0xd109678 thru 0xd113161 (0x9aea bytes) 2024-04-24T15:32:20.846 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:20.846 INFO:teuthology.orchestra.run.smithi096.stdout:10 write 0x965e0fd thru 0x966853d (0xa441 bytes) 2024-04-24T15:32:20.848 INFO:teuthology.orchestra.run.smithi096.stdout:11 punch from 0x2acac31 to 0x2acb078, (0x447 bytes) 2024-04-24T15:32:20.855 INFO:teuthology.orchestra.run.smithi096.stdout:12 write 0xd98dfa0 thru 0xd99022d (0x228e bytes) 2024-04-24T15:32:20.858 INFO:teuthology.orchestra.run.smithi096.stdout:13 write 0x401a79a thru 0x40217cb (0x7032 bytes) 2024-04-24T15:32:20.860 INFO:teuthology.orchestra.run.smithi096.stdout:14 writesame 0xcf2e934 thru 0xcf39de4 data_size 0x11(0xb4b1 bytes) 2024-04-24T15:32:20.864 INFO:teuthology.orchestra.run.smithi096.stdout:truncating to largest ever: 0x18487e1 2024-04-24T15:32:20.878 INFO:teuthology.orchestra.run.smithi096.stdout:15 trunc from 0xd99022e to 0x18487e1 2024-04-24T15:32:20.886 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:32:20.886+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:32:20.911 INFO:teuthology.orchestra.run.smithi096.stdout:16 write 0xddb2cfb thru 0xddb5d9f (0x30a5 bytes) 2024-04-24T15:32:20.917 INFO:teuthology.orchestra.run.smithi096.stdout:truncating to largest ever: 0x21dc721 2024-04-24T15:32:20.918 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:20.918 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:20.931 INFO:teuthology.orchestra.run.smithi096.stdout:18 trunc from 0xddb5da0 to 0x21dc721 2024-04-24T15:32:20.936 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-24T15:32:20.944 INFO:teuthology.orchestra.run.smithi096.stdout:19 punch from 0x1d69128 to 0x1d6fd66, (0x6c3e bytes) 2024-04-24T15:32:20.944 INFO:teuthology.orchestra.run.smithi096.stdout:20 clone 1 order 19 su 32768 sc 6 2024-04-24T15:32:20.987 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:32:20.988+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:32:21.068 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:21.068 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:21.079 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-24T15:32:21.123 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:32:21.124+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:32:21.153 INFO:teuthology.orchestra.run.smithi096.stdout:21 write 0x3ed717b thru 0x3ee0bbe (0x9a44 bytes) 2024-04-24T15:32:21.173 INFO:teuthology.orchestra.run.smithi096.stdout:24 write 0x7755d1c thru 0x775def8 (0x81dd bytes) 2024-04-24T15:32:21.179 INFO:teuthology.orchestra.run.smithi096.stdout:truncating to largest ever: 0xe8424a5 2024-04-24T15:32:21.187 INFO:teuthology.orchestra.run.smithi096.stdout:25 trunc from 0x775def9 to 0xe8424a5 2024-04-24T15:32:21.188 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:32:21.189+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:32:21.194 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:21.194 INFO:teuthology.orchestra.run.smithi096.stdout:27 clone 2 order 18 su 2048 sc 9 2024-04-24T15:32:21.214 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:21.214 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:21.224 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-24T15:32:21.289 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:32:21.289+0000 7f23abcf6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 44021) UID: 0 2024-04-24T15:32:21.387 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:21.387 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:21.390 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:32:21.391+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:32:21.396 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-24T15:32:21.490 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:32:21.492+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:32:21.560 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:21.560 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:21.570 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-24T15:32:21.591 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:32:21.592+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:32:21.692 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:32:21.693+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:32:21.693 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:21.693 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:21.703 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-24T15:32:21.793 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:32:21.793+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:32:21.835 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:21.836 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:21.846 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-24T15:32:21.894 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:32:21.895+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:32:21.995 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:32:21.995+0000 7f23abcf6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 44021) UID: 0 2024-04-24T15:32:22.009 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:22.009 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:22.019 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-24T15:32:22.060 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7] out_osds: [1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-04-24T15:32:22.061 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-24T15:32:22.061 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-04-24T15:32:22.061 INFO:tasks.thrashosds.thrasher:Setting osd 0 primary_affinity to 0.000000 2024-04-24T15:32:22.061 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 0 2024-04-24T15:32:22.073 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-24T15:32:22.106 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:32:22.107+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:32:22.124 INFO:teuthology.orchestra.run.smithi096.stdout:truncating image image_client.0-clone1 from 0xe8424a5 (overlap 0x21dc721) to 0x1e7999d 2024-04-24T15:32:22.191 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:22.191 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:22.196 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:32:22.196+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:32:22.201 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-24T15:32:22.215 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #0, image image_client.0 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent1 2024-04-24T15:32:22.258 INFO:teuthology.orchestra.run.smithi096.stdout:31 read 0x61cab9f thru 0x61cd6db (0x2b3d bytes) 2024-04-24T15:32:22.258 INFO:teuthology.orchestra.run.smithi096.stdout:33 punch from 0x8b51b74 to 0x8b5710b, (0x5597 bytes) 2024-04-24T15:32:22.269 INFO:teuthology.orchestra.run.smithi096.stdout:34 trunc from 0xe8424a5 to 0x9361656 2024-04-24T15:32:22.296 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:32:22.297+0000 7f23abcf6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 44021) UID: 0 2024-04-24T15:32:22.300 INFO:teuthology.orchestra.run.smithi096.stdout:35 writesame 0xec917f3 thru 0xec9a90b data_size 0x55(0x9119 bytes) 2024-04-24T15:32:22.301 INFO:teuthology.orchestra.run.smithi096.stdout:36 writesame 0x3f1e3a2 thru 0x3f2bc84 data_size 0x1(0xd8e3 bytes) 2024-04-24T15:32:22.318 INFO:teuthology.orchestra.run.smithi096.stdout:37 read 0xb73e6de thru 0xb746063 (0x7986 bytes) 2024-04-24T15:32:22.318 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:22.318 INFO:teuthology.orchestra.run.smithi096.stdout:40 writesame 0x94e1800 thru 0x94ec2d3 data_size 0x3a(0xaad4 bytes) 2024-04-24T15:32:22.333 INFO:teuthology.orchestra.run.smithi096.stdout:41 write 0xba32ad6 thru 0xba3cddb (0xa306 bytes) 2024-04-24T15:32:22.348 INFO:teuthology.orchestra.run.smithi096.stdout:42 punch from 0x216ed01 to 0x21702a9, (0x15a8 bytes) 2024-04-24T15:32:22.348 INFO:teuthology.orchestra.run.smithi096.stdout:43 write 0xd3247b3 thru 0xd32e4e0 (0x9d2e bytes) 2024-04-24T15:32:22.362 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:22.362 INFO:teuthology.orchestra.run.smithi096.stdout:45 write 0xbbff6a0 thru 0xbc0d28b (0xdbec bytes) 2024-04-24T15:32:22.372 INFO:teuthology.orchestra.run.smithi096.stdout:46 read 0x3efc754 thru 0x3f0309a (0x6947 bytes) 2024-04-24T15:32:22.372 INFO:teuthology.orchestra.run.smithi096.stdout:47 write 0xceb6b68 thru 0xcec2582 (0xba1b bytes) 2024-04-24T15:32:22.387 INFO:teuthology.orchestra.run.smithi096.stdout:48 read 0x5b91652 thru 0x5b91691 (0x40 bytes) 2024-04-24T15:32:22.387 INFO:teuthology.orchestra.run.smithi096.stdout:49 punch from 0x19cf5de to 0x19dc338, (0xcd5a bytes) 2024-04-24T15:32:22.387 INFO:teuthology.orchestra.run.smithi096.stdout:50 punch from 0x3d6e67a to 0x3d726da, (0x4060 bytes) 2024-04-24T15:32:22.387 INFO:teuthology.orchestra.run.smithi096.stdout:51 read 0x6e57122 thru 0x6e59667 (0x2546 bytes) 2024-04-24T15:32:22.387 INFO:teuthology.orchestra.run.smithi096.stdout:54 read 0x60e224a thru 0x60e3802 (0x15b9 bytes) 2024-04-24T15:32:22.387 INFO:teuthology.orchestra.run.smithi096.stdout:56 read 0x45168cd thru 0x451ea8f (0x81c3 bytes) 2024-04-24T15:32:22.387 INFO:teuthology.orchestra.run.smithi096.stdout:57 read 0x2546d70 thru 0x25512dc (0xa56d bytes) 2024-04-24T15:32:22.387 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:22.392 INFO:teuthology.orchestra.run.smithi096.stdout:61 trunc from 0xec9a90c to 0x9cb426a 2024-04-24T15:32:22.393 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:22.393 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:22.397 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:32:22.398+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:32:22.406 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-24T15:32:22.422 INFO:teuthology.orchestra.run.smithi096.stdout:62 trunc from 0x9cb426a to 0x79a54f1 2024-04-24T15:32:22.448 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:22.448 INFO:teuthology.orchestra.run.smithi096.stdout:66 writesame 0x98acc0 thru 0x999168 data_size 0x1(0xe4a9 bytes) 2024-04-24T15:32:22.498 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:32:22.498+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:32:22.574 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:22.574 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:22.584 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-24T15:32:22.594 INFO:teuthology.orchestra.run.smithi096.stdout:67 write 0x15fcae8 thru 0x15ff3ba (0x28d3 bytes) 2024-04-24T15:32:22.598 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:32:22.600+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:32:22.623 INFO:teuthology.orchestra.run.smithi096.stdout:70 write 0xe7ae9f6 thru 0xe7b0460 (0x1a6b bytes) 2024-04-24T15:32:22.633 INFO:teuthology.orchestra.run.smithi096.stdout:71 write 0x263d796 thru 0x264b74e (0xdfb9 bytes) 2024-04-24T15:32:22.654 INFO:teuthology.orchestra.run.smithi096.stdout:73 writesame 0xa6b4f42 thru 0xa6bed88 data_size 0x1(0x9e47 bytes) 2024-04-24T15:32:22.672 INFO:teuthology.orchestra.run.smithi096.stdout:truncating to largest ever: 0xede9a58 2024-04-24T15:32:22.672 INFO:teuthology.orchestra.run.smithi096.stdout:74 trunc from 0xe7b0461 to 0xede9a58 2024-04-24T15:32:22.678 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:22.681 INFO:teuthology.orchestra.run.smithi096.stdout:78 trunc from 0xede9a58 to 0xb8efc5f 2024-04-24T15:32:22.696 INFO:teuthology.orchestra.run.smithi096.stdout:79 read 0x94d223a thru 0x94e1994 (0xf75b bytes) 2024-04-24T15:32:22.696 INFO:teuthology.orchestra.run.smithi096.stdout:80 read 0x1c321fb thru 0x1c3d895 (0xb69b bytes) 2024-04-24T15:32:22.699 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:32:22.699+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:32:22.700 INFO:teuthology.orchestra.run.smithi096.stdout:81 trunc from 0xb8efc5f to 0x7dd8eae 2024-04-24T15:32:22.715 INFO:teuthology.orchestra.run.smithi096.stdout:82 read 0xfb0f64 thru 0xfb5243 (0x42e0 bytes) 2024-04-24T15:32:22.716 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:22.725 INFO:teuthology.orchestra.run.smithi096.stdout:85 write 0xb86ab21 thru 0xb86dc6f (0x314f bytes) 2024-04-24T15:32:22.736 INFO:teuthology.orchestra.run.smithi096.stdout:87 read 0x1acae74 thru 0x1ad9008 (0xe195 bytes) 2024-04-24T15:32:22.737 INFO:teuthology.orchestra.run.smithi096.stdout:89 read 0x543837a thru 0x54420da (0x9d61 bytes) 2024-04-24T15:32:22.737 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:22.737 INFO:teuthology.orchestra.run.smithi096.stdout:93 writesame 0x93c15d5 thru 0x93c540d data_size 0x1(0x3e39 bytes) 2024-04-24T15:32:22.748 INFO:teuthology.orchestra.run.smithi096.stdout:94 read 0xc27c0c thru 0xc2ad42 (0x3137 bytes) 2024-04-24T15:32:22.755 INFO:teuthology.orchestra.run.smithi096.stdout:95 write 0xd557664 thru 0xd558737 (0x10d4 bytes) 2024-04-24T15:32:22.764 INFO:teuthology.orchestra.run.smithi096.stdout:96 write 0xa420bbd thru 0xa42aca1 (0xa0e5 bytes) 2024-04-24T15:32:22.767 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:22.768 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:22.780 INFO:teuthology.orchestra.run.smithi096.stdout:97 punch from 0xaa6cf1a to 0xaa78a6e, (0xbb54 bytes) 2024-04-24T15:32:22.780 INFO:teuthology.orchestra.run.smithi096.stdout:98 write 0x9c20623 thru 0x9c2d6a4 (0xd082 bytes) 2024-04-24T15:32:22.781 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-24T15:32:22.796 INFO:teuthology.orchestra.run.smithi096.stdout:99 read 0xc145d8e thru 0xc14970c (0x397f bytes) 2024-04-24T15:32:22.796 INFO:teuthology.orchestra.run.smithi096.stdout:100 read 0x5d49ac2 thru 0x5d58bc9 (0xf108 bytes) 2024-04-24T15:32:22.796 INFO:teuthology.orchestra.run.smithi096.stdout:101 write 0x7d39d75 thru 0x7d43463 (0x96ef bytes) 2024-04-24T15:32:22.800 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:32:22.802+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:32:22.804 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-24T15:32:22.811 INFO:teuthology.orchestra.run.smithi096.stdout:102 write 0x50a56a5 thru 0x50a9505 (0x3e61 bytes) 2024-04-24T15:32:22.823 INFO:teuthology.orchestra.run.smithi096.stdout:103 read 0x7245b2e thru 0x7245d54 (0x227 bytes) 2024-04-24T15:32:22.823 INFO:teuthology.orchestra.run.smithi096.stdout:104 read 0x1cdeb61 thru 0x1ce1905 (0x2da5 bytes) 2024-04-24T15:32:22.835 INFO:teuthology.orchestra.run.smithi096.stdout:106 trunc from 0xd558738 to 0x38fb9f3 2024-04-24T15:32:22.888 INFO:teuthology.orchestra.run.smithi096.stdout:107 writesame 0x4faad96 thru 0x4fb066b data_size 0x53(0x58d6 bytes) 2024-04-24T15:32:22.888 INFO:teuthology.orchestra.run.smithi096.stdout:108 read 0x320c8eb thru 0x32152e7 (0x89fd bytes) 2024-04-24T15:32:22.888 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:22.888 INFO:teuthology.orchestra.run.smithi096.stdout:111 punch from 0x30f6909 to 0x30fc77d, (0x5e74 bytes) 2024-04-24T15:32:22.888 INFO:teuthology.orchestra.run.smithi096.stdout:112 read 0x40012e4 thru 0x400d2e0 (0xbffd bytes) 2024-04-24T15:32:22.893 INFO:teuthology.orchestra.run.smithi096.stdout:113 writesame 0x58397f8 thru 0x583c2d9 data_size 0xb(0x2ae2 bytes) 2024-04-24T15:32:22.901 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:32:22.902+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:32:22.903 INFO:teuthology.orchestra.run.smithi096.stdout:114 trunc from 0x583c2da to 0xc771582 2024-04-24T15:32:22.908 INFO:teuthology.orchestra.run.smithi096.stdout:116 writesame 0x6bf0ab4 thru 0x6bf9f15 data_size 0x1a(0x9462 bytes) 2024-04-24T15:32:22.920 INFO:teuthology.orchestra.run.smithi096.stdout:119 punch from 0x3b2e7a9 to 0x3b32054, (0x38ab bytes) 2024-04-24T15:32:22.927 INFO:teuthology.orchestra.run.smithi096.stdout:122 write 0xd77da46 thru 0xd78d6b5 (0xfc70 bytes) 2024-04-24T15:32:22.944 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:22.945 INFO:teuthology.orchestra.run.smithi096.stdout:126 write 0x30df411 thru 0x30e46ed (0x52dd bytes) 2024-04-24T15:32:22.958 INFO:teuthology.orchestra.run.smithi096.stdout:129 punch from 0xa46670d to 0xa46a4b9, (0x3dac bytes) 2024-04-24T15:32:22.958 INFO:teuthology.orchestra.run.smithi096.stdout:130 writesame 0x78c514e thru 0x78c63ea data_size 0x1(0x129d bytes) 2024-04-24T15:32:22.965 INFO:teuthology.orchestra.run.smithi096.stdout:131 read 0x8666dab thru 0x866fb8b (0x8de1 bytes) 2024-04-24T15:32:22.965 INFO:teuthology.orchestra.run.smithi096.stdout:132 read 0xcbb3609 thru 0xcbb70c9 (0x3ac1 bytes) 2024-04-24T15:32:22.965 INFO:teuthology.orchestra.run.smithi096.stdout:133 write 0xbafc18c thru 0xbb07434 (0xb2a9 bytes) 2024-04-24T15:32:22.966 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:22.966 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:22.969 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-04-24T15:32:22.978 INFO:teuthology.orchestra.run.smithi096.stdout:135 punch from 0x60fcedf to 0x60feeea, (0x200b bytes) 2024-04-24T15:32:22.978 INFO:teuthology.orchestra.run.smithi096.stdout:136 read 0xd3e3b79 thru 0xd3e6450 (0x28d8 bytes) 2024-04-24T15:32:22.979 INFO:teuthology.orchestra.run.smithi096.stdout:137 write 0x7c82d6c thru 0x7c8efb0 (0xc245 bytes) 2024-04-24T15:32:22.980 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-24T15:32:22.990 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-04-24T15:32:22.996 INFO:teuthology.orchestra.run.smithi096.stdout:139 trunc from 0xd78d6b6 to 0xadf6450 2024-04-24T15:32:23.002 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:32:23.004+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:32:23.009 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-04-24T15:32:23.019 INFO:teuthology.orchestra.run.smithi096.stdout:140 trunc from 0xadf6450 to 0x3c213f5 2024-04-24T15:32:23.031 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-04-24T15:32:23.041 INFO:teuthology.orchestra.run.smithi096.stdout:141 write 0x2a034fd thru 0x2a0ac8e (0x7792 bytes) 2024-04-24T15:32:23.051 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-04-24T15:32:23.052 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:23.052 INFO:teuthology.orchestra.run.smithi096.stdout:143 read 0x14d3b18 thru 0x14da00f (0x64f8 bytes) 2024-04-24T15:32:23.067 INFO:teuthology.orchestra.run.smithi096.stdout:144 write 0xa87ec63 thru 0xa888f1c (0xa2ba bytes) 2024-04-24T15:32:23.074 INFO:teuthology.orchestra.run.smithi096.stderr:noscrub is unset 2024-04-24T15:32:23.074 INFO:teuthology.orchestra.run.smithi096.stderr:set osd.0 primary-affinity to 0 (802) 2024-04-24T15:32:23.074 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-04-24T15:32:23.082 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:23.082 INFO:teuthology.orchestra.run.smithi096.stdout:148 writesame 0x46b546c thru 0x46b6f73 data_size 0x14(0x1b08 bytes) 2024-04-24T15:32:23.097 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-04-24T15:32:23.103 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:32:23.104+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:32:23.116 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-04-24T15:32:23.117 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T15:32:23.117 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T15:32:23.117 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T15:32:23.117 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T15:32:23.117 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T15:32:23.117 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T15:32:23.118 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T15:32:23.118 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T15:32:23.155 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:32:23.155 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:32:23.155 INFO:teuthology.orchestra.run.smithi151.stdout: "complaint_time": 30, 2024-04-24T15:32:23.155 INFO:teuthology.orchestra.run.smithi151.stdout: "num_blocked_ops": 0 2024-04-24T15:32:23.155 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:32:23.165 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-24T15:32:23.204 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:32:23.205+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:32:23.306 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:32:23.307+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:32:23.328 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:32:23.328 INFO:teuthology.orchestra.run.smithi151.stdout: "size": 20, 2024-04-24T15:32:23.328 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 600, 2024-04-24T15:32:23.328 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [ 2024-04-24T15:32:23.328 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.328 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:11 4.1f 4:fa654f18:::rbd_id.image_client.0:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-04-24T15:32:23.328 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:19.052921+0000", 2024-04-24T15:32:23.328 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 4.276416502, 2024-04-24T15:32:23.328 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0019237950000000001, 2024-04-24T15:32:23.328 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:23.329 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:23.329 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:23.329 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:23.329 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:23.329 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 11 2024-04-24T15:32:23.329 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.329 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:23.329 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.329 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:23.329 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:19.052921+0000", 2024-04-24T15:32:23.329 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:23.329 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.329 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.329 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:23.329 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:19.052921+0000", 2024-04-24T15:32:23.330 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:23.330 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.330 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.330 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:23.330 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:19.052919+0000", 2024-04-24T15:32:23.330 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999981 2024-04-24T15:32:23.330 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.330 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.330 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:23.330 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:19.052928+0000", 2024-04-24T15:32:23.330 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.5320000000000006e-06 2024-04-24T15:32:23.330 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.330 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.331 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:23.331 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:19.052948+0000", 2024-04-24T15:32:23.331 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.0477000000000001e-05 2024-04-24T15:32:23.331 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.331 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.331 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:23.331 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:19.052956+0000", 2024-04-24T15:32:23.331 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0930000000000003e-06 2024-04-24T15:32:23.331 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.331 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.331 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:23.331 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:19.053010+0000", 2024-04-24T15:32:23.331 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.3996999999999998e-05 2024-04-24T15:32:23.331 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.331 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.332 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:23.332 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:19.053113+0000", 2024-04-24T15:32:23.332 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000103151 2024-04-24T15:32:23.332 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.332 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.332 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 5", 2024-04-24T15:32:23.332 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:19.053301+0000", 2024-04-24T15:32:23.332 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00018810499999999999 2024-04-24T15:32:23.332 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.332 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.332 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:23.333 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:19.054439+0000", 2024-04-24T15:32:23.333 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0011372380000000001 2024-04-24T15:32:23.333 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.333 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.333 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:23.333 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:19.054777+0000", 2024-04-24T15:32:23.333 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00033870800000000001 2024-04-24T15:32:23.333 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.333 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.333 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:23.333 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:19.054813+0000", 2024-04-24T15:32:23.333 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5494000000000003e-05 2024-04-24T15:32:23.333 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.333 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.333 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:23.334 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:19.054845+0000", 2024-04-24T15:32:23.334 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.1745000000000002e-05 2024-04-24T15:32:23.334 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:23.334 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:23.334 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:23.334 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.334 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.334 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_repop_reply(client.4426.0:11 4.1f e28/26 ondisk, result = 0)", 2024-04-24T15:32:23.334 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:19.054712+0000", 2024-04-24T15:32:23.334 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 4.2746253730000001, 2024-04-24T15:32:23.334 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00013924099999999999, 2024-04-24T15:32:23.334 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:23.334 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:23.334 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:23.335 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.335 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:23.335 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:19.054712+0000", 2024-04-24T15:32:23.335 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:23.335 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.335 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.335 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:23.335 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:19.054712+0000", 2024-04-24T15:32:23.335 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:23.335 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.335 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.335 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:23.335 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:19.054711+0000", 2024-04-24T15:32:23.335 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999999 2024-04-24T15:32:23.335 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.336 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.336 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:23.336 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:19.054718+0000", 2024-04-24T15:32:23.336 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.3300000000000001e-06 2024-04-24T15:32:23.336 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.336 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.336 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:23.336 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:19.054730+0000", 2024-04-24T15:32:23.336 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.2067e-05 2024-04-24T15:32:23.336 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.336 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.336 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:23.336 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:19.054737+0000", 2024-04-24T15:32:23.336 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.3559999999999998e-06 2024-04-24T15:32:23.336 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.337 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.337 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:23.337 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:19.054768+0000", 2024-04-24T15:32:23.337 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.0976000000000003e-05 2024-04-24T15:32:23.337 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.337 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.337 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:23.337 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:19.054774+0000", 2024-04-24T15:32:23.337 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.3339999999999998e-06 2024-04-24T15:32:23.337 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.337 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.337 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:23.337 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:19.054851+0000", 2024-04-24T15:32:23.337 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7119000000000005e-05 2024-04-24T15:32:23.337 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:23.338 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:23.338 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:23.338 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.338 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.338 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:14 4.1f 4:fa654f18:::rbd_id.image_client.0:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-04-24T15:32:23.338 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:19.061538+0000", 2024-04-24T15:32:23.338 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 4.2677987760000002, 2024-04-24T15:32:23.338 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00022509799999999999, 2024-04-24T15:32:23.338 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:23.338 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:23.338 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:23.338 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:23.338 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:23.338 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 14 2024-04-24T15:32:23.338 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.339 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:23.339 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.339 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:23.339 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:19.061538+0000", 2024-04-24T15:32:23.339 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:23.339 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.339 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.339 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:23.339 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:19.061538+0000", 2024-04-24T15:32:23.339 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:23.339 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.339 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.339 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:23.339 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:19.061537+0000", 2024-04-24T15:32:23.339 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999981 2024-04-24T15:32:23.340 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.340 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.340 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:23.340 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:19.061558+0000", 2024-04-24T15:32:23.340 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.1472e-05 2024-04-24T15:32:23.340 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.340 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.340 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:23.340 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:19.061573+0000", 2024-04-24T15:32:23.340 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.4479e-05 2024-04-24T15:32:23.340 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.340 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.340 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:23.340 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:19.061580+0000", 2024-04-24T15:32:23.340 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.4370000000000001e-06 2024-04-24T15:32:23.341 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.341 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.341 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:23.341 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:19.061650+0000", 2024-04-24T15:32:23.341 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.9778999999999997e-05 2024-04-24T15:32:23.341 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.341 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.341 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:23.341 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:19.061707+0000", 2024-04-24T15:32:23.341 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.7268999999999997e-05 2024-04-24T15:32:23.341 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.342 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.342 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:23.342 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:19.061764+0000", 2024-04-24T15:32:23.342 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6564000000000003e-05 2024-04-24T15:32:23.342 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:23.342 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:23.342 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:23.342 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.342 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.342 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:15 4.1f 4:fa654f18:::rbd_id.image_client.0:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-04-24T15:32:23.342 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:19.061909+0000", 2024-04-24T15:32:23.342 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 4.267428185, 2024-04-24T15:32:23.342 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00041271999999999999, 2024-04-24T15:32:23.342 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:23.343 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:23.343 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:23.343 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:23.343 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:23.343 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 15 2024-04-24T15:32:23.343 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.343 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:23.343 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.343 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:23.343 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:19.061909+0000", 2024-04-24T15:32:23.343 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:23.343 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.343 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.343 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:23.343 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:19.061909+0000", 2024-04-24T15:32:23.344 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:23.344 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.344 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.344 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:23.344 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:19.061908+0000", 2024-04-24T15:32:23.344 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999999 2024-04-24T15:32:23.344 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.344 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.344 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:23.344 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:19.061917+0000", 2024-04-24T15:32:23.344 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.7860000000000002e-06 2024-04-24T15:32:23.344 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.344 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.344 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:23.344 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:19.061928+0000", 2024-04-24T15:32:23.345 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.1257000000000001e-05 2024-04-24T15:32:23.345 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.345 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.345 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:23.345 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:19.061936+0000", 2024-04-24T15:32:23.345 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.5310000000000004e-06 2024-04-24T15:32:23.345 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.345 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.345 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:23.345 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:19.061976+0000", 2024-04-24T15:32:23.345 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9960999999999999e-05 2024-04-24T15:32:23.345 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.345 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.345 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:23.345 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:19.062057+0000", 2024-04-24T15:32:23.346 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0708e-05 2024-04-24T15:32:23.346 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.346 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.346 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:23.346 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:19.062322+0000", 2024-04-24T15:32:23.346 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00026523699999999998 2024-04-24T15:32:23.346 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:23.346 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:23.346 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:23.346 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:23.346 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:23.347 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:32:23.347 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-24T15:32:23.406 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:32:23.406+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:32:23.471 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:32:23.471 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:32:23.471 INFO:teuthology.orchestra.run.smithi151.stdout: "num_ops": 0 2024-04-24T15:32:23.471 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:32:23.481 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-24T15:32:23.507 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:32:23.508+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:32:23.608 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:32:23.608+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:32:23.645 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:32:23.645 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:32:23.645 INFO:teuthology.orchestra.run.smithi151.stdout: "complaint_time": 30, 2024-04-24T15:32:23.645 INFO:teuthology.orchestra.run.smithi151.stdout: "num_blocked_ops": 0 2024-04-24T15:32:23.645 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:32:23.655 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-24T15:32:23.709 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:32:23.710+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:32:23.810 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:32:23.811+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:32:23.820 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:32:23.820 INFO:teuthology.orchestra.run.smithi151.stdout: "size": 20, 2024-04-24T15:32:23.820 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 600, 2024-04-24T15:32:23.820 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [ 2024-04-24T15:32:23.820 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.820 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4184.0:18 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-04-24T15:32:23.820 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:00.973237+0000", 2024-04-24T15:32:23.820 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 22.846049081, 2024-04-24T15:32:23.820 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0075407699999999996, 2024-04-24T15:32:23.820 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:23.820 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:23.820 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:23.821 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4184", 2024-04-24T15:32:23.821 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/1884602651", 2024-04-24T15:32:23.821 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18 2024-04-24T15:32:23.821 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.821 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:23.821 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.821 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:23.821 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.973237+0000", 2024-04-24T15:32:23.821 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:23.821 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.821 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.821 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:23.821 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.973237+0000", 2024-04-24T15:32:23.821 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:23.822 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.822 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.822 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:23.822 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.973237+0000", 2024-04-24T15:32:23.822 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:23.822 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.822 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.822 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:23.822 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.973241+0000", 2024-04-24T15:32:23.822 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9750000000000001e-06 2024-04-24T15:32:23.822 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.822 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.822 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:23.822 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.973246+0000", 2024-04-24T15:32:23.823 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.5670000000000002e-06 2024-04-24T15:32:23.823 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.823 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.823 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:23.823 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.973258+0000", 2024-04-24T15:32:23.823 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.1707e-05 2024-04-24T15:32:23.823 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.823 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.823 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:23.823 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.973684+0000", 2024-04-24T15:32:23.823 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00042587099999999998 2024-04-24T15:32:23.823 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.823 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.823 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:23.823 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.973745+0000", 2024-04-24T15:32:23.824 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.0976e-05 2024-04-24T15:32:23.824 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.824 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.824 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:23.824 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.973849+0000", 2024-04-24T15:32:23.824 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000103999 2024-04-24T15:32:23.824 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.824 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.824 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:23.824 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.976929+0000", 2024-04-24T15:32:23.824 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0030801689999999998 2024-04-24T15:32:23.824 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.824 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.824 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:23.825 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.980645+0000", 2024-04-24T15:32:23.825 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0037165060000000001 2024-04-24T15:32:23.825 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.825 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.825 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:23.825 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.980743+0000", 2024-04-24T15:32:23.825 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.7728999999999999e-05 2024-04-24T15:32:23.825 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.825 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.825 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:23.825 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.980778+0000", 2024-04-24T15:32:23.825 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.485e-05 2024-04-24T15:32:23.825 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:23.825 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:23.825 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:23.826 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.826 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.826 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4184.0:21 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 65536~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-04-24T15:32:23.826 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:00.985588+0000", 2024-04-24T15:32:23.826 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 22.833698751, 2024-04-24T15:32:23.826 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0073858090000000001, 2024-04-24T15:32:23.826 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:23.826 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:23.826 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:23.826 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4184", 2024-04-24T15:32:23.826 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/1884602651", 2024-04-24T15:32:23.826 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 21 2024-04-24T15:32:23.826 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.826 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:23.827 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.827 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:23.827 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.985588+0000", 2024-04-24T15:32:23.827 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:23.827 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.827 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.827 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:23.827 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.985588+0000", 2024-04-24T15:32:23.827 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:23.827 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.827 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.827 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:23.827 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.985587+0000", 2024-04-24T15:32:23.827 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:23.827 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.828 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.828 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:23.828 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.985653+0000", 2024-04-24T15:32:23.828 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.6086000000000004e-05 2024-04-24T15:32:23.828 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.828 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.828 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:23.828 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.985660+0000", 2024-04-24T15:32:23.828 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.3790000000000003e-06 2024-04-24T15:32:23.828 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.828 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.828 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:23.828 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.985664+0000", 2024-04-24T15:32:23.828 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.053e-06 2024-04-24T15:32:23.828 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.828 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.829 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:23.829 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.986266+0000", 2024-04-24T15:32:23.829 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00060216800000000004 2024-04-24T15:32:23.829 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.829 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.829 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:23.829 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.986321+0000", 2024-04-24T15:32:23.829 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.5275999999999997e-05 2024-04-24T15:32:23.829 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.829 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.829 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:23.829 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.986410+0000", 2024-04-24T15:32:23.829 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.9137999999999999e-05 2024-04-24T15:32:23.829 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.829 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.830 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:23.830 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.989564+0000", 2024-04-24T15:32:23.830 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0031535209999999998 2024-04-24T15:32:23.830 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.830 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.830 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:23.830 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.992882+0000", 2024-04-24T15:32:23.830 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0033180190000000002 2024-04-24T15:32:23.830 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.830 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.830 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:23.830 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.992934+0000", 2024-04-24T15:32:23.830 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.24e-05 2024-04-24T15:32:23.830 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.831 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.831 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:23.831 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.992973+0000", 2024-04-24T15:32:23.831 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9274999999999999e-05 2024-04-24T15:32:23.831 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:23.831 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:23.831 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:23.831 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.831 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.831 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4184.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-04-24T15:32:23.831 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:00.985670+0000", 2024-04-24T15:32:23.831 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 22.833615909999999, 2024-04-24T15:32:23.831 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0074023860000000004, 2024-04-24T15:32:23.831 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:23.831 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:23.831 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:23.832 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4184", 2024-04-24T15:32:23.832 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/1884602651", 2024-04-24T15:32:23.832 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 22 2024-04-24T15:32:23.832 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.832 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:23.832 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.832 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:23.832 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.985670+0000", 2024-04-24T15:32:23.832 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:23.832 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.832 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.832 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:23.832 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.985670+0000", 2024-04-24T15:32:23.832 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:23.832 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.833 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.833 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:23.833 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.985670+0000", 2024-04-24T15:32:23.833 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:23.833 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.833 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.833 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:23.833 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.985673+0000", 2024-04-24T15:32:23.833 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.1939999999999998e-06 2024-04-24T15:32:23.833 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.833 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.833 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:23.833 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.985677+0000", 2024-04-24T15:32:23.833 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.7079999999999999e-06 2024-04-24T15:32:23.834 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.834 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.834 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:23.834 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.985679+0000", 2024-04-24T15:32:23.834 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.3470000000000001e-06 2024-04-24T15:32:23.834 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.834 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.834 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:23.834 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.986801+0000", 2024-04-24T15:32:23.834 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0011212609999999999 2024-04-24T15:32:23.834 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.834 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.834 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:23.834 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.986887+0000", 2024-04-24T15:32:23.835 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.6877000000000002e-05 2024-04-24T15:32:23.835 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.835 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.835 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:23.835 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.987022+0000", 2024-04-24T15:32:23.835 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000134991 2024-04-24T15:32:23.835 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.835 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.835 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:23.835 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.992467+0000", 2024-04-24T15:32:23.835 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0054450219999999999 2024-04-24T15:32:23.835 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.835 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.835 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:23.835 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.993004+0000", 2024-04-24T15:32:23.836 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00053681400000000004 2024-04-24T15:32:23.836 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.836 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.836 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:23.836 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.993049+0000", 2024-04-24T15:32:23.836 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4709999999999997e-05 2024-04-24T15:32:23.836 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.836 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.836 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:23.836 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.993073+0000", 2024-04-24T15:32:23.836 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.3886000000000001e-05 2024-04-24T15:32:23.836 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:23.836 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:23.836 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:23.836 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.837 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.837 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4184.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-04-24T15:32:23.837 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:01.007690+0000", 2024-04-24T15:32:23.837 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 22.811596678000001, 2024-04-24T15:32:23.837 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0065139869999999997, 2024-04-24T15:32:23.837 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:23.837 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:23.837 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:23.837 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4184", 2024-04-24T15:32:23.837 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/1884602651", 2024-04-24T15:32:23.837 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 32 2024-04-24T15:32:23.837 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.837 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:23.837 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.837 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:23.838 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007690+0000", 2024-04-24T15:32:23.838 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:23.838 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.838 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.838 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:23.838 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007690+0000", 2024-04-24T15:32:23.838 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:23.838 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.838 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.838 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:23.838 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007689+0000", 2024-04-24T15:32:23.838 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:23.838 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.838 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.838 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:23.839 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007762+0000", 2024-04-24T15:32:23.839 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2523999999999999e-05 2024-04-24T15:32:23.839 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.839 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.839 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:23.839 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007771+0000", 2024-04-24T15:32:23.839 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.0159999999999997e-06 2024-04-24T15:32:23.839 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.839 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.839 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:23.839 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007775+0000", 2024-04-24T15:32:23.839 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2359999999999998e-06 2024-04-24T15:32:23.839 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.839 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.840 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:23.840 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.008537+0000", 2024-04-24T15:32:23.840 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00076218599999999998 2024-04-24T15:32:23.840 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.840 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.840 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:23.840 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.008619+0000", 2024-04-24T15:32:23.840 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.2136000000000001e-05 2024-04-24T15:32:23.840 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.840 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.840 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:23.840 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.008760+0000", 2024-04-24T15:32:23.840 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00014038500000000001 2024-04-24T15:32:23.840 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.840 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.840 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:23.841 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.010709+0000", 2024-04-24T15:32:23.841 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0019494029999999999 2024-04-24T15:32:23.841 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.841 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.841 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:23.841 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.014140+0000", 2024-04-24T15:32:23.841 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0034304790000000002 2024-04-24T15:32:23.841 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.841 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.841 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:23.841 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.014178+0000", 2024-04-24T15:32:23.841 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8612000000000002e-05 2024-04-24T15:32:23.841 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.841 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.841 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:23.842 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.014204+0000", 2024-04-24T15:32:23.842 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.5548e-05 2024-04-24T15:32:23.842 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:23.842 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:23.842 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:23.842 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.842 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.842 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4184.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-04-24T15:32:23.842 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:01.007780+0000", 2024-04-24T15:32:23.842 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 22.811506838, 2024-04-24T15:32:23.842 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0064758460000000004, 2024-04-24T15:32:23.842 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:23.842 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:23.842 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:23.842 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4184", 2024-04-24T15:32:23.843 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/1884602651", 2024-04-24T15:32:23.843 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 33 2024-04-24T15:32:23.843 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.843 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:23.843 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.843 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:23.843 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007780+0000", 2024-04-24T15:32:23.843 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:23.843 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.843 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.843 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:23.843 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007780+0000", 2024-04-24T15:32:23.843 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:23.843 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.843 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.844 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:23.844 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007779+0000", 2024-04-24T15:32:23.844 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:23.844 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.844 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.844 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:23.844 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007782+0000", 2024-04-24T15:32:23.844 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.4930000000000001e-06 2024-04-24T15:32:23.844 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.844 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.844 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:23.844 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007785+0000", 2024-04-24T15:32:23.844 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.535e-06 2024-04-24T15:32:23.844 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.844 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.845 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:23.845 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007789+0000", 2024-04-24T15:32:23.845 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5590000000000001e-06 2024-04-24T15:32:23.845 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.845 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.845 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:23.845 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.009175+0000", 2024-04-24T15:32:23.845 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.001386417 2024-04-24T15:32:23.845 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.845 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.845 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:23.845 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.009233+0000", 2024-04-24T15:32:23.845 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.7414000000000003e-05 2024-04-24T15:32:23.845 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.845 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.846 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:23.846 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.009314+0000", 2024-04-24T15:32:23.846 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.1800999999999998e-05 2024-04-24T15:32:23.846 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.846 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.846 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:23.846 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.010745+0000", 2024-04-24T15:32:23.846 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.001430965 2024-04-24T15:32:23.846 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.846 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.846 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:23.846 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.014210+0000", 2024-04-24T15:32:23.846 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0034649540000000001 2024-04-24T15:32:23.846 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.846 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.847 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:23.847 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.014237+0000", 2024-04-24T15:32:23.847 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7109000000000001e-05 2024-04-24T15:32:23.847 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.847 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.847 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:23.847 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.014255+0000", 2024-04-24T15:32:23.847 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.7946000000000001e-05 2024-04-24T15:32:23.847 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:23.847 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:23.847 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:23.847 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.847 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.847 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4184.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-04-24T15:32:23.847 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:01.007792+0000", 2024-04-24T15:32:23.848 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 22.811493882000001, 2024-04-24T15:32:23.848 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0065062920000000003, 2024-04-24T15:32:23.848 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:23.848 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:23.848 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:23.848 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4184", 2024-04-24T15:32:23.848 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/1884602651", 2024-04-24T15:32:23.848 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 34 2024-04-24T15:32:23.848 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.848 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:23.848 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.848 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:23.848 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007792+0000", 2024-04-24T15:32:23.848 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:23.848 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.849 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.849 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:23.849 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007792+0000", 2024-04-24T15:32:23.849 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:23.849 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.849 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.849 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:23.849 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007792+0000", 2024-04-24T15:32:23.849 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:23.849 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.849 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.849 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:23.849 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007795+0000", 2024-04-24T15:32:23.849 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.396e-06 2024-04-24T15:32:23.849 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.850 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.850 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:23.850 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007798+0000", 2024-04-24T15:32:23.850 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.2899999999999998e-06 2024-04-24T15:32:23.850 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.850 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.850 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:23.850 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007801+0000", 2024-04-24T15:32:23.850 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5080000000000001e-06 2024-04-24T15:32:23.850 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.850 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.850 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:23.850 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.009688+0000", 2024-04-24T15:32:23.850 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.001886956 2024-04-24T15:32:23.850 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.851 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.851 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:23.851 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.009765+0000", 2024-04-24T15:32:23.851 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.6483000000000006e-05 2024-04-24T15:32:23.851 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.851 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.851 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:23.851 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.009904+0000", 2024-04-24T15:32:23.851 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00013925100000000001 2024-04-24T15:32:23.851 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.851 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.851 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:23.851 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.011742+0000", 2024-04-24T15:32:23.851 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.001837878 2024-04-24T15:32:23.851 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.852 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.852 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:23.852 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.014260+0000", 2024-04-24T15:32:23.852 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0025179149999999999 2024-04-24T15:32:23.852 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.852 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.852 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:23.852 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.014282+0000", 2024-04-24T15:32:23.852 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.1923999999999999e-05 2024-04-24T15:32:23.852 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.852 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.852 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:23.852 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.014299+0000", 2024-04-24T15:32:23.852 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.6931000000000001e-05 2024-04-24T15:32:23.852 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:23.853 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:23.853 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:23.853 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.853 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.853 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4184.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-04-24T15:32:23.853 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:01.007879+0000", 2024-04-24T15:32:23.853 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 22.811407298999999, 2024-04-24T15:32:23.853 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0064617579999999997, 2024-04-24T15:32:23.853 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:23.853 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:23.853 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:23.853 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4184", 2024-04-24T15:32:23.853 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/1884602651", 2024-04-24T15:32:23.853 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 35 2024-04-24T15:32:23.853 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.853 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:23.854 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.854 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:23.854 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007879+0000", 2024-04-24T15:32:23.854 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:23.854 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.854 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.854 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:23.854 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007879+0000", 2024-04-24T15:32:23.854 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:23.854 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.854 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.854 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:23.854 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007878+0000", 2024-04-24T15:32:23.854 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:23.855 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.855 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.855 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:23.855 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007949+0000", 2024-04-24T15:32:23.855 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.0714000000000001e-05 2024-04-24T15:32:23.855 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.855 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.855 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:23.855 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007961+0000", 2024-04-24T15:32:23.855 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.1969e-05 2024-04-24T15:32:23.855 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.855 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.855 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:23.855 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007965+0000", 2024-04-24T15:32:23.855 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.4620000000000001e-06 2024-04-24T15:32:23.856 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.856 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.856 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:23.856 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.010182+0000", 2024-04-24T15:32:23.856 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0022175599999999999 2024-04-24T15:32:23.856 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.856 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.856 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:23.856 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.010238+0000", 2024-04-24T15:32:23.856 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.5583e-05 2024-04-24T15:32:23.856 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.856 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.856 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:23.856 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.010321+0000", 2024-04-24T15:32:23.856 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3515000000000003e-05 2024-04-24T15:32:23.856 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.857 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.857 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:23.857 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.012956+0000", 2024-04-24T15:32:23.857 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.002634426 2024-04-24T15:32:23.857 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.857 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.857 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:23.857 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.014303+0000", 2024-04-24T15:32:23.857 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.001347507 2024-04-24T15:32:23.857 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.857 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.857 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:23.857 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.014325+0000", 2024-04-24T15:32:23.857 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.1577999999999999e-05 2024-04-24T15:32:23.858 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.858 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.858 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:23.858 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.014341+0000", 2024-04-24T15:32:23.858 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.6121999999999999e-05 2024-04-24T15:32:23.858 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:23.858 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:23.858 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:23.858 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.858 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.858 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4184.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-04-24T15:32:23.858 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:01.007969+0000", 2024-04-24T15:32:23.858 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 22.811317281000001, 2024-04-24T15:32:23.858 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0090800020000000002, 2024-04-24T15:32:23.859 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:23.859 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:23.859 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:23.859 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4184", 2024-04-24T15:32:23.859 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/1884602651", 2024-04-24T15:32:23.859 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 36 2024-04-24T15:32:23.859 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.859 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:23.859 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.859 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:23.859 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007969+0000", 2024-04-24T15:32:23.859 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:23.859 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.859 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.859 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:23.860 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007969+0000", 2024-04-24T15:32:23.860 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:23.860 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.860 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.860 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:23.860 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007969+0000", 2024-04-24T15:32:23.860 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:23.860 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.860 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.860 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:23.860 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007971+0000", 2024-04-24T15:32:23.860 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7219999999999999e-06 2024-04-24T15:32:23.860 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.860 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.860 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:23.861 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007975+0000", 2024-04-24T15:32:23.861 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.7019999999999999e-06 2024-04-24T15:32:23.861 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.861 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.861 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:23.861 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007978+0000", 2024-04-24T15:32:23.861 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.4059999999999999e-06 2024-04-24T15:32:23.861 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.861 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.861 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:23.861 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.010779+0000", 2024-04-24T15:32:23.861 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0028010610000000001 2024-04-24T15:32:23.861 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.861 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.861 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:23.862 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.010887+0000", 2024-04-24T15:32:23.862 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00010871599999999999 2024-04-24T15:32:23.862 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.862 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.862 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:23.862 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.011026+0000", 2024-04-24T15:32:23.862 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00013850200000000001 2024-04-24T15:32:23.862 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.862 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.862 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:23.862 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.015873+0000", 2024-04-24T15:32:23.862 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0048467680000000004 2024-04-24T15:32:23.862 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.862 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.862 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:23.863 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.016993+0000", 2024-04-24T15:32:23.863 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0011203809999999999 2024-04-24T15:32:23.863 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.863 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.863 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:23.863 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.017027+0000", 2024-04-24T15:32:23.863 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.3637000000000003e-05 2024-04-24T15:32:23.863 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.863 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.863 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:23.863 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.017049+0000", 2024-04-24T15:32:23.863 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.2509000000000001e-05 2024-04-24T15:32:23.863 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:23.863 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:23.863 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:23.864 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.864 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.864 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4184.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-04-24T15:32:23.864 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:01.007981+0000", 2024-04-24T15:32:23.864 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 22.811305088000001, 2024-04-24T15:32:23.864 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0091181590000000007, 2024-04-24T15:32:23.864 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:23.864 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:23.864 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:23.864 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4184", 2024-04-24T15:32:23.864 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/1884602651", 2024-04-24T15:32:23.864 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 37 2024-04-24T15:32:23.864 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.864 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:23.864 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.865 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:23.865 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007981+0000", 2024-04-24T15:32:23.865 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:23.865 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.865 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.865 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:23.865 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007981+0000", 2024-04-24T15:32:23.865 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:23.865 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.865 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.865 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:23.865 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007981+0000", 2024-04-24T15:32:23.865 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:32:23.865 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.865 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.865 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:23.866 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007984+0000", 2024-04-24T15:32:23.866 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.3469999999999999e-06 2024-04-24T15:32:23.866 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.866 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.866 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:23.866 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007988+0000", 2024-04-24T15:32:23.866 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.4479999999999999e-06 2024-04-24T15:32:23.866 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.866 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.866 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:23.866 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007990+0000", 2024-04-24T15:32:23.866 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.26e-06 2024-04-24T15:32:23.866 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.866 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.866 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:23.867 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.011341+0000", 2024-04-24T15:32:23.867 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0033510459999999999 2024-04-24T15:32:23.867 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.867 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.867 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:23.867 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.011400+0000", 2024-04-24T15:32:23.867 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.9206000000000002e-05 2024-04-24T15:32:23.867 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.867 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.867 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:23.867 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.011488+0000", 2024-04-24T15:32:23.867 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.7301999999999999e-05 2024-04-24T15:32:23.867 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.867 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.867 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:23.867 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.015901+0000", 2024-04-24T15:32:23.868 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0044129549999999997 2024-04-24T15:32:23.868 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.868 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.868 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:23.868 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.017057+0000", 2024-04-24T15:32:23.868 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0011566790000000001 2024-04-24T15:32:23.868 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.868 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.868 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:23.868 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.017081+0000", 2024-04-24T15:32:23.868 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.3790000000000001e-05 2024-04-24T15:32:23.868 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.868 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.868 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:23.868 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.017099+0000", 2024-04-24T15:32:23.869 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.8346e-05 2024-04-24T15:32:23.869 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:23.869 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:23.869 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:23.869 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.869 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.869 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4184.0:69 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-04-24T15:32:23.869 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:01.057223+0000", 2024-04-24T15:32:23.869 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 22.762063063999999, 2024-04-24T15:32:23.869 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0080831549999999999, 2024-04-24T15:32:23.869 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:23.869 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:23.869 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:23.869 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4184", 2024-04-24T15:32:23.869 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/1884602651", 2024-04-24T15:32:23.869 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 69 2024-04-24T15:32:23.870 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.870 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:23.870 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.870 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:23.870 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.057223+0000", 2024-04-24T15:32:23.870 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:23.870 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.870 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.870 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:23.870 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.057223+0000", 2024-04-24T15:32:23.870 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:23.870 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.870 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.870 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:23.871 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.057223+0000", 2024-04-24T15:32:23.871 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:23.871 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.871 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.871 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:23.871 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.057261+0000", 2024-04-24T15:32:23.871 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.7905999999999999e-05 2024-04-24T15:32:23.871 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.871 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.871 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:23.871 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.057267+0000", 2024-04-24T15:32:23.871 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.4389999999999997e-06 2024-04-24T15:32:23.871 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.871 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.871 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:23.872 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.057270+0000", 2024-04-24T15:32:23.872 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.0489999999999999e-06 2024-04-24T15:32:23.872 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.872 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.872 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:23.872 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.058232+0000", 2024-04-24T15:32:23.872 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00096145599999999996 2024-04-24T15:32:23.872 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.872 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.872 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:23.872 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.058284+0000", 2024-04-24T15:32:23.872 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.2213999999999998e-05 2024-04-24T15:32:23.872 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.872 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.873 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:23.873 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.058364+0000", 2024-04-24T15:32:23.873 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0096e-05 2024-04-24T15:32:23.873 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.873 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.873 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:23.873 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.059686+0000", 2024-04-24T15:32:23.873 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0013213739999999999 2024-04-24T15:32:23.873 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.873 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.873 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:23.873 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.065225+0000", 2024-04-24T15:32:23.873 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.005539675 2024-04-24T15:32:23.873 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.873 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.873 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:23.874 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.065272+0000", 2024-04-24T15:32:23.874 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6369999999999998e-05 2024-04-24T15:32:23.874 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.874 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.874 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:23.874 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.065306+0000", 2024-04-24T15:32:23.874 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.4875000000000001e-05 2024-04-24T15:32:23.874 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:23.874 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:23.874 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:23.874 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.874 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.874 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4184.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-04-24T15:32:23.875 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:01.057275+0000", 2024-04-24T15:32:23.875 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 22.762011629, 2024-04-24T15:32:23.875 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0081317479999999994, 2024-04-24T15:32:23.875 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:23.875 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:23.875 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:23.875 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4184", 2024-04-24T15:32:23.875 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/1884602651", 2024-04-24T15:32:23.875 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 70 2024-04-24T15:32:23.875 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.875 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:23.875 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.875 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:23.875 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.057275+0000", 2024-04-24T15:32:23.876 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:23.876 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.876 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.876 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:23.876 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.057275+0000", 2024-04-24T15:32:23.876 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:23.876 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.876 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.876 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:23.876 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.057274+0000", 2024-04-24T15:32:23.876 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:23.876 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.876 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.876 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:23.876 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.057277+0000", 2024-04-24T15:32:23.877 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.9790000000000002e-06 2024-04-24T15:32:23.877 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.877 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.877 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:23.877 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.057281+0000", 2024-04-24T15:32:23.877 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5109999999999999e-06 2024-04-24T15:32:23.877 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.877 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.877 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:23.877 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.057289+0000", 2024-04-24T15:32:23.877 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.4300000000000006e-06 2024-04-24T15:32:23.877 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.877 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.877 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:23.877 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.058763+0000", 2024-04-24T15:32:23.878 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.001473238 2024-04-24T15:32:23.878 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.878 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.878 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:23.878 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.058850+0000", 2024-04-24T15:32:23.878 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.7600000000000002e-05 2024-04-24T15:32:23.878 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.878 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.878 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:23.878 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.058972+0000", 2024-04-24T15:32:23.878 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00012228099999999999 2024-04-24T15:32:23.878 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.878 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.878 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:23.878 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.060449+0000", 2024-04-24T15:32:23.879 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.001476705 2024-04-24T15:32:23.879 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.879 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.879 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:23.879 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.065340+0000", 2024-04-24T15:32:23.879 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0048906929999999998 2024-04-24T15:32:23.879 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.879 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.879 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:23.879 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.065370+0000", 2024-04-24T15:32:23.879 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.9887999999999999e-05 2024-04-24T15:32:23.879 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.879 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.879 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:23.879 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.065406+0000", 2024-04-24T15:32:23.880 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.6755999999999998e-05 2024-04-24T15:32:23.880 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:23.880 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:23.880 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:23.880 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.880 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.880 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4184.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-04-24T15:32:23.880 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:01.074168+0000", 2024-04-24T15:32:23.880 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 22.745118012999999, 2024-04-24T15:32:23.880 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0068309110000000003, 2024-04-24T15:32:23.880 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:23.880 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:23.880 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:23.880 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4184", 2024-04-24T15:32:23.880 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/1884602651", 2024-04-24T15:32:23.880 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 88 2024-04-24T15:32:23.881 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.881 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:23.881 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.881 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:23.881 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.074168+0000", 2024-04-24T15:32:23.881 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:23.881 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.881 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.881 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:23.881 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.074168+0000", 2024-04-24T15:32:23.881 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:23.881 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.881 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.881 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:23.881 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.074162+0000", 2024-04-24T15:32:23.882 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999943 2024-04-24T15:32:23.882 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.882 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.882 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:23.882 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.074171+0000", 2024-04-24T15:32:23.882 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.2230000000000007e-06 2024-04-24T15:32:23.882 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.882 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.882 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:23.882 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.074175+0000", 2024-04-24T15:32:23.882 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8800000000000001e-06 2024-04-24T15:32:23.882 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.882 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.882 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:23.882 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.074178+0000", 2024-04-24T15:32:23.883 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.1879999999999998e-06 2024-04-24T15:32:23.883 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.883 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.883 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:23.883 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.076404+0000", 2024-04-24T15:32:23.883 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0022259179999999999 2024-04-24T15:32:23.883 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.883 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.883 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:23.883 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.076447+0000", 2024-04-24T15:32:23.883 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2978000000000001e-05 2024-04-24T15:32:23.883 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.883 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.883 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:23.883 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.076529+0000", 2024-04-24T15:32:23.884 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.1940000000000005e-05 2024-04-24T15:32:23.884 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.884 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.884 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:23.884 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.077197+0000", 2024-04-24T15:32:23.884 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00066798300000000003 2024-04-24T15:32:23.884 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.884 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.884 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:23.884 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.080952+0000", 2024-04-24T15:32:23.884 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0037559410000000001 2024-04-24T15:32:23.884 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.884 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.884 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:23.884 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.080980+0000", 2024-04-24T15:32:23.885 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7134000000000001e-05 2024-04-24T15:32:23.885 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.885 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.885 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:23.885 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.080999+0000", 2024-04-24T15:32:23.885 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.9646000000000001e-05 2024-04-24T15:32:23.885 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:23.885 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:23.885 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:23.885 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.885 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.885 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4184.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-04-24T15:32:23.885 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:01.180426+0000", 2024-04-24T15:32:23.885 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 22.638860405999999, 2024-04-24T15:32:23.885 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0063214070000000002, 2024-04-24T15:32:23.886 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:23.886 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:23.886 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:23.886 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4184", 2024-04-24T15:32:23.886 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/1884602651", 2024-04-24T15:32:23.886 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 116 2024-04-24T15:32:23.886 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.886 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:23.886 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.886 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:23.886 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.180426+0000", 2024-04-24T15:32:23.886 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:23.886 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.886 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.887 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:23.887 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.180426+0000", 2024-04-24T15:32:23.887 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:23.887 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.887 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.887 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:23.887 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.180425+0000", 2024-04-24T15:32:23.887 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:23.887 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.887 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.887 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:23.887 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.180430+0000", 2024-04-24T15:32:23.887 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4569999999999998e-06 2024-04-24T15:32:23.887 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.887 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.887 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:23.888 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.180436+0000", 2024-04-24T15:32:23.888 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.4060000000000002e-06 2024-04-24T15:32:23.888 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.888 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.888 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:23.888 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.180442+0000", 2024-04-24T15:32:23.888 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.5809999999999996e-06 2024-04-24T15:32:23.888 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.888 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.888 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:23.888 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.185273+0000", 2024-04-24T15:32:23.888 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0048306069999999998 2024-04-24T15:32:23.888 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.888 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.888 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:23.889 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.185348+0000", 2024-04-24T15:32:23.889 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.538e-05 2024-04-24T15:32:23.889 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.889 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.889 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:23.889 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.185470+0000", 2024-04-24T15:32:23.889 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00012215300000000001 2024-04-24T15:32:23.889 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.889 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.889 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:23.889 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.186159+0000", 2024-04-24T15:32:23.889 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00068886399999999997 2024-04-24T15:32:23.889 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.889 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.889 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:23.890 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.186672+0000", 2024-04-24T15:32:23.890 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00051336100000000005 2024-04-24T15:32:23.890 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.890 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.890 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:23.890 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.186713+0000", 2024-04-24T15:32:23.890 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.104e-05 2024-04-24T15:32:23.890 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.890 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.890 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:23.890 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.186747+0000", 2024-04-24T15:32:23.890 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.4063000000000002e-05 2024-04-24T15:32:23.890 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:23.890 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:23.890 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:23.891 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.891 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.891 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4184.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-04-24T15:32:23.891 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:01.180448+0000", 2024-04-24T15:32:23.891 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 22.638838374999999, 2024-04-24T15:32:23.891 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0063607250000000002, 2024-04-24T15:32:23.891 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:23.891 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:23.891 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:23.891 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4184", 2024-04-24T15:32:23.891 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/1884602651", 2024-04-24T15:32:23.891 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 117 2024-04-24T15:32:23.891 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.891 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:23.891 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.892 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:23.892 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.180448+0000", 2024-04-24T15:32:23.892 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:23.892 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.892 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.892 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:23.892 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.180448+0000", 2024-04-24T15:32:23.892 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:23.892 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.892 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.892 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:23.892 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.180448+0000", 2024-04-24T15:32:23.892 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:23.892 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.892 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.892 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:23.893 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.180452+0000", 2024-04-24T15:32:23.893 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.997e-06 2024-04-24T15:32:23.893 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.893 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.893 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:23.893 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.180459+0000", 2024-04-24T15:32:23.893 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2389999999999996e-06 2024-04-24T15:32:23.893 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.893 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.893 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:23.893 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.180463+0000", 2024-04-24T15:32:23.893 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.7189999999999999e-06 2024-04-24T15:32:23.893 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.893 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.893 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:23.894 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.185758+0000", 2024-04-24T15:32:23.894 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0052953050000000002 2024-04-24T15:32:23.894 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.894 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.894 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:23.894 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.185805+0000", 2024-04-24T15:32:23.894 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.7490000000000001e-05 2024-04-24T15:32:23.894 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.894 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.894 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:23.894 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.185883+0000", 2024-04-24T15:32:23.894 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7824999999999994e-05 2024-04-24T15:32:23.894 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.894 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.894 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:23.895 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.186252+0000", 2024-04-24T15:32:23.895 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00036880799999999998 2024-04-24T15:32:23.895 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.895 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.895 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:23.895 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.186770+0000", 2024-04-24T15:32:23.895 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00051828499999999995 2024-04-24T15:32:23.895 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.895 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.895 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:23.895 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.186793+0000", 2024-04-24T15:32:23.895 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.2549999999999999e-05 2024-04-24T15:32:23.895 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.895 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.895 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:23.896 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.186809+0000", 2024-04-24T15:32:23.896 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.5883e-05 2024-04-24T15:32:23.896 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:23.896 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:23.896 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:23.896 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.896 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.896 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4184.0:154 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-04-24T15:32:23.896 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:01.928400+0000", 2024-04-24T15:32:23.896 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 21.890886618, 2024-04-24T15:32:23.896 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0077033919999999999, 2024-04-24T15:32:23.896 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:23.896 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:23.896 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:23.897 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4184", 2024-04-24T15:32:23.897 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/1884602651", 2024-04-24T15:32:23.897 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 154 2024-04-24T15:32:23.897 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.897 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:23.897 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.897 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:23.897 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928400+0000", 2024-04-24T15:32:23.897 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:23.897 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.897 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.897 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:23.897 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928400+0000", 2024-04-24T15:32:23.897 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:23.897 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.898 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.898 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:23.898 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928399+0000", 2024-04-24T15:32:23.898 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:23.898 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.898 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.898 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:23.898 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928402+0000", 2024-04-24T15:32:23.898 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.497e-06 2024-04-24T15:32:23.898 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.898 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.898 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:23.898 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928407+0000", 2024-04-24T15:32:23.898 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.1950000000000002e-06 2024-04-24T15:32:23.899 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.899 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.899 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:23.899 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928410+0000", 2024-04-24T15:32:23.899 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.5000000000000002e-06 2024-04-24T15:32:23.899 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.899 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.899 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:23.899 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928822+0000", 2024-04-24T15:32:23.899 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00041253900000000002 2024-04-24T15:32:23.899 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.899 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.899 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:23.899 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928865+0000", 2024-04-24T15:32:23.899 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2570000000000001e-05 2024-04-24T15:32:23.900 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.900 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.900 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:23.900 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928944+0000", 2024-04-24T15:32:23.900 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.9634000000000004e-05 2024-04-24T15:32:23.900 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.900 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.900 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:23.900 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.932122+0000", 2024-04-24T15:32:23.900 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.003177862 2024-04-24T15:32:23.900 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.900 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.900 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:23.900 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.936054+0000", 2024-04-24T15:32:23.900 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0039315569999999996 2024-04-24T15:32:23.901 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.901 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.901 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:23.901 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.936083+0000", 2024-04-24T15:32:23.901 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.9043e-05 2024-04-24T15:32:23.901 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.901 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.901 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:23.901 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.936103+0000", 2024-04-24T15:32:23.901 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.0292000000000002e-05 2024-04-24T15:32:23.901 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:23.901 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:23.901 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:23.901 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.901 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.901 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4184.0:155 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-04-24T15:32:23.902 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:01.928434+0000", 2024-04-24T15:32:23.902 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 21.890852075000002, 2024-04-24T15:32:23.902 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.011257120000000001, 2024-04-24T15:32:23.902 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:23.902 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:23.902 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:23.902 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4184", 2024-04-24T15:32:23.902 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/1884602651", 2024-04-24T15:32:23.902 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 155 2024-04-24T15:32:23.902 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.902 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:23.902 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.902 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:23.902 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928434+0000", 2024-04-24T15:32:23.903 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:23.903 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.903 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.903 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:23.903 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928434+0000", 2024-04-24T15:32:23.903 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:23.903 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.903 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.903 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:23.903 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928434+0000", 2024-04-24T15:32:23.903 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:23.903 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.903 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.903 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:23.903 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928562+0000", 2024-04-24T15:32:23.904 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00012848100000000001 2024-04-24T15:32:23.904 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.904 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.904 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:23.904 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928568+0000", 2024-04-24T15:32:23.904 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.4500000000000003e-06 2024-04-24T15:32:23.904 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.904 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.904 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:23.904 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928572+0000", 2024-04-24T15:32:23.904 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.1880000000000004e-06 2024-04-24T15:32:23.904 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.904 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.904 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:23.904 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.929192+0000", 2024-04-24T15:32:23.904 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00062016399999999998 2024-04-24T15:32:23.905 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.905 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.905 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:23.905 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.929240+0000", 2024-04-24T15:32:23.905 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8112999999999998e-05 2024-04-24T15:32:23.905 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.905 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.905 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:23.905 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.929316+0000", 2024-04-24T15:32:23.905 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.5486000000000002e-05 2024-04-24T15:32:23.905 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.905 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.905 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:23.905 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.935151+0000", 2024-04-24T15:32:23.906 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00583498 2024-04-24T15:32:23.906 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.906 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.906 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:23.906 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.939589+0000", 2024-04-24T15:32:23.906 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0044387840000000003 2024-04-24T15:32:23.906 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.906 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.906 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:23.906 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.939670+0000", 2024-04-24T15:32:23.906 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0389000000000006e-05 2024-04-24T15:32:23.906 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.906 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.906 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:23.906 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.939691+0000", 2024-04-24T15:32:23.907 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.1543999999999999e-05 2024-04-24T15:32:23.907 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:23.907 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:23.907 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:23.907 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.907 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.907 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4184.0:156 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-04-24T15:32:23.907 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:01.928576+0000", 2024-04-24T15:32:23.907 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 21.890710335000001, 2024-04-24T15:32:23.907 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.011200128, 2024-04-24T15:32:23.907 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:23.907 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:23.907 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:23.907 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4184", 2024-04-24T15:32:23.907 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/1884602651", 2024-04-24T15:32:23.908 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 156 2024-04-24T15:32:23.908 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.908 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:23.908 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.908 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:23.908 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928576+0000", 2024-04-24T15:32:23.908 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:23.908 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.908 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.908 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:23.908 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928576+0000", 2024-04-24T15:32:23.908 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:23.908 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.908 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.908 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:23.908 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928576+0000", 2024-04-24T15:32:23.909 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:23.909 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.909 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.909 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:23.909 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928578+0000", 2024-04-24T15:32:23.909 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.3070000000000001e-06 2024-04-24T15:32:23.909 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.909 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.909 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:23.909 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928581+0000", 2024-04-24T15:32:23.909 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.3419999999999999e-06 2024-04-24T15:32:23.909 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.909 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.909 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:23.909 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928584+0000", 2024-04-24T15:32:23.910 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.1780000000000002e-06 2024-04-24T15:32:23.910 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.910 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.910 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:23.910 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.929860+0000", 2024-04-24T15:32:23.910 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.001276578 2024-04-24T15:32:23.910 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.910 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.910 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:23.910 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.929907+0000", 2024-04-24T15:32:23.910 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.693e-05 2024-04-24T15:32:23.910 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.910 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.910 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:23.910 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.929985+0000", 2024-04-24T15:32:23.911 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7565999999999996e-05 2024-04-24T15:32:23.911 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.911 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.911 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:23.911 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.935155+0000", 2024-04-24T15:32:23.911 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0051699110000000001 2024-04-24T15:32:23.911 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.911 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.911 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:23.911 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.939724+0000", 2024-04-24T15:32:23.911 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0045697189999999999 2024-04-24T15:32:23.911 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.911 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.911 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:23.911 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.939753+0000", 2024-04-24T15:32:23.912 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8547e-05 2024-04-24T15:32:23.912 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.912 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.912 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:23.912 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.939776+0000", 2024-04-24T15:32:23.912 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.3348000000000001e-05 2024-04-24T15:32:23.912 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:23.912 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:23.912 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:23.912 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.912 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.912 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4184.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-04-24T15:32:23.912 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:01.928586+0000", 2024-04-24T15:32:23.912 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 21.890700026000001, 2024-04-24T15:32:23.913 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.011256990999999999, 2024-04-24T15:32:23.913 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:23.913 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:23.913 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:23.913 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4184", 2024-04-24T15:32:23.913 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/1884602651", 2024-04-24T15:32:23.913 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 157 2024-04-24T15:32:23.913 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.913 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:23.913 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.913 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:23.913 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928586+0000", 2024-04-24T15:32:23.913 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:23.913 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.913 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.914 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:23.914 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928586+0000", 2024-04-24T15:32:23.914 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:23.914 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.914 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.914 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:23.914 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928586+0000", 2024-04-24T15:32:23.914 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:32:23.914 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.914 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.914 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:23.914 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928592+0000", 2024-04-24T15:32:23.914 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6400000000000002e-06 2024-04-24T15:32:23.914 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.914 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.915 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:23.915 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928596+0000", 2024-04-24T15:32:23.915 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2169999999999996e-06 2024-04-24T15:32:23.915 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.915 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.915 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:23.915 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928605+0000", 2024-04-24T15:32:23.915 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.1260000000000004e-06 2024-04-24T15:32:23.915 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.915 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.915 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:23.915 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.930310+0000", 2024-04-24T15:32:23.915 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0017047200000000001 2024-04-24T15:32:23.915 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.915 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.916 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:23.916 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.930366+0000", 2024-04-24T15:32:23.916 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6357999999999998e-05 2024-04-24T15:32:23.916 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.916 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.916 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:23.916 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.930438+0000", 2024-04-24T15:32:23.916 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1469000000000003e-05 2024-04-24T15:32:23.916 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.916 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.916 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:23.916 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.935159+0000", 2024-04-24T15:32:23.916 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0047212809999999999 2024-04-24T15:32:23.916 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.916 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.917 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:23.917 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.939803+0000", 2024-04-24T15:32:23.917 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0046441920000000001 2024-04-24T15:32:23.917 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.917 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.917 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:23.917 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.939826+0000", 2024-04-24T15:32:23.917 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.2949e-05 2024-04-24T15:32:23.917 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.917 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.917 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:23.917 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.939843+0000", 2024-04-24T15:32:23.917 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.7266999999999999e-05 2024-04-24T15:32:23.917 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:23.917 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:23.917 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:23.918 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.918 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.918 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4184.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-04-24T15:32:23.918 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:01.928608+0000", 2024-04-24T15:32:23.918 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 21.890677880999998, 2024-04-24T15:32:23.918 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.011304719, 2024-04-24T15:32:23.918 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:23.918 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:23.918 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:23.918 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4184", 2024-04-24T15:32:23.918 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/1884602651", 2024-04-24T15:32:23.918 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 158 2024-04-24T15:32:23.918 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.918 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:23.918 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.919 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:23.919 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928608+0000", 2024-04-24T15:32:23.919 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:23.919 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.919 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.919 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:23.919 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928608+0000", 2024-04-24T15:32:23.919 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:23.919 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.919 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.919 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:23.919 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928608+0000", 2024-04-24T15:32:23.919 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:23.919 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.919 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.920 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:23.920 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928652+0000", 2024-04-24T15:32:23.920 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3856999999999997e-05 2024-04-24T15:32:23.920 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.920 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.920 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:23.920 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928660+0000", 2024-04-24T15:32:23.920 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.9619999999999993e-06 2024-04-24T15:32:23.920 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.920 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.920 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:23.920 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928667+0000", 2024-04-24T15:32:23.920 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.029e-06 2024-04-24T15:32:23.920 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.920 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.921 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:23.921 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.930668+0000", 2024-04-24T15:32:23.921 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0020010219999999999 2024-04-24T15:32:23.921 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.921 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.921 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:23.921 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.930712+0000", 2024-04-24T15:32:23.921 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3897999999999999e-05 2024-04-24T15:32:23.921 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.921 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.921 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:23.921 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.930784+0000", 2024-04-24T15:32:23.921 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1898999999999997e-05 2024-04-24T15:32:23.921 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.921 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.922 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:23.922 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.935162+0000", 2024-04-24T15:32:23.922 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.004378364 2024-04-24T15:32:23.922 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.922 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.922 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:23.922 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.939870+0000", 2024-04-24T15:32:23.922 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0047080439999999998 2024-04-24T15:32:23.922 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.922 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.922 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:23.922 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.939893+0000", 2024-04-24T15:32:23.922 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.2989e-05 2024-04-24T15:32:23.922 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.923 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.923 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:23.923 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.939913+0000", 2024-04-24T15:32:23.923 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.9925999999999999e-05 2024-04-24T15:32:23.923 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:23.923 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:23.923 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:23.923 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.923 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.923 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4184.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-04-24T15:32:23.923 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:01.928671+0000", 2024-04-24T15:32:23.923 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 21.890615354000001, 2024-04-24T15:32:23.923 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.011298967, 2024-04-24T15:32:23.923 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:23.923 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:23.923 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:23.924 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4184", 2024-04-24T15:32:23.924 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/1884602651", 2024-04-24T15:32:23.924 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 159 2024-04-24T15:32:23.924 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.924 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:23.924 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.924 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:23.924 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928671+0000", 2024-04-24T15:32:23.924 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:23.924 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.924 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.924 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:23.924 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928671+0000", 2024-04-24T15:32:23.924 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:23.924 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.925 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.925 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:23.925 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928671+0000", 2024-04-24T15:32:23.925 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:23.925 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.925 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.925 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:23.925 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928674+0000", 2024-04-24T15:32:23.925 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.715e-06 2024-04-24T15:32:23.925 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.925 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.925 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:23.925 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928680+0000", 2024-04-24T15:32:23.925 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.0939999999999998e-06 2024-04-24T15:32:23.926 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.926 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.926 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:23.926 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928683+0000", 2024-04-24T15:32:23.926 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.1360000000000001e-06 2024-04-24T15:32:23.926 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.926 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.926 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:23.926 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.931109+0000", 2024-04-24T15:32:23.926 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0024267379999999999 2024-04-24T15:32:23.926 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.926 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.926 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:23.926 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.931153+0000", 2024-04-24T15:32:23.926 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3970000000000001e-05 2024-04-24T15:32:23.927 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.927 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.927 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:23.927 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.931231+0000", 2024-04-24T15:32:23.927 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.8053999999999995e-05 2024-04-24T15:32:23.927 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.927 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.927 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:23.927 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.935166+0000", 2024-04-24T15:32:23.927 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0039344159999999996 2024-04-24T15:32:23.927 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.927 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.927 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:23.927 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.939932+0000", 2024-04-24T15:32:23.927 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0047658980000000002 2024-04-24T15:32:23.928 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.928 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.928 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:23.928 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.939953+0000", 2024-04-24T15:32:23.928 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.1107e-05 2024-04-24T15:32:23.928 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:23.928 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:23.928 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:23.928 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.939970+0000", 2024-04-24T15:32:23.928 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.7110999999999999e-05 2024-04-24T15:32:23.928 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:23.928 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:23.928 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:23.928 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:23.928 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:23.929 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:32:23.930 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:32:23.931+0000 7f23abcf6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 44021) UID: 0 2024-04-24T15:32:23.930 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-24T15:32:24.031 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:32:24.032+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:32:24.067 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:32:24.067 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:32:24.067 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-04-24T15:32:24.067 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:32:24.083 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-24T15:32:24.084 INFO:teuthology.orchestra.run.smithi096.stdout:149 write 0xc6ea15e thru 0xc6f913c (0xefdf bytes) 2024-04-24T15:32:24.099 INFO:teuthology.orchestra.run.smithi096.stdout:150 write 0x769e5d6 thru 0x76a4df4 (0x681f bytes) 2024-04-24T15:32:24.114 INFO:teuthology.orchestra.run.smithi096.stdout:152 write 0x7147875 thru 0x714f1f2 (0x797e bytes) 2024-04-24T15:32:24.130 INFO:teuthology.orchestra.run.smithi096.stdout:154 punch from 0x60170a2 to 0x601e09c, (0x6ffa bytes) 2024-04-24T15:32:24.132 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:32:24.133+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:32:24.133 INFO:teuthology.orchestra.run.smithi096.stdout:155 trunc from 0xc6f913d to 0xa140d51 2024-04-24T15:32:24.149 INFO:teuthology.orchestra.run.smithi096.stdout:156 write 0xa66fde thru 0xa73fb2 (0xcfd5 bytes) 2024-04-24T15:32:24.156 INFO:teuthology.orchestra.run.smithi096.stdout:157 read 0x6d9df66 thru 0x6da1ae5 (0x3b80 bytes) 2024-04-24T15:32:24.156 INFO:teuthology.orchestra.run.smithi096.stdout:160 punch from 0x976605a to 0x976ea72, (0x8a18 bytes) 2024-04-24T15:32:24.156 INFO:teuthology.orchestra.run.smithi096.stdout:161 punch from 0x138a051 to 0x1392a08, (0x89b7 bytes) 2024-04-24T15:32:24.156 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:24.161 INFO:teuthology.orchestra.run.smithi096.stdout:163 write 0xac4301b thru 0xac44ba9 (0x1b8f bytes) 2024-04-24T15:32:24.167 INFO:teuthology.orchestra.run.smithi096.stdout:165 punch from 0x2b772ac to 0x2b8115a, (0x9eae bytes) 2024-04-24T15:32:24.177 INFO:teuthology.orchestra.run.smithi096.stdout:166 writesame 0xe7ab8f2 thru 0xe7b7c11 data_size 0x20(0xc320 bytes) 2024-04-24T15:32:24.177 INFO:teuthology.orchestra.run.smithi096.stdout:167 write 0x140f5cd thru 0x141ace0 (0xb714 bytes) 2024-04-24T15:32:24.190 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:24.205 INFO:teuthology.orchestra.run.smithi096.stdout:170 trunc from 0xe7b7c12 to 0x3e8635c 2024-04-24T15:32:24.228 INFO:teuthology.orchestra.run.smithi096.stdout:171 write 0x64d9ec0 thru 0x64e1d5d (0x7e9e bytes) 2024-04-24T15:32:24.232 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:32:24.234+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:32:24.234 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:32:24.234 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:32:24.234 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-04-24T15:32:24.234 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-04-24T15:32:24.234 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:32:24.245 INFO:teuthology.orchestra.run.smithi096.stdout:172 trunc from 0x64e1d5e to 0x1e1f6c9 2024-04-24T15:32:24.251 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-24T15:32:24.287 INFO:teuthology.orchestra.run.smithi096.stdout:173 read 0x16c6b95 thru 0x16d063b (0x9aa7 bytes) 2024-04-24T15:32:24.287 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:24.287 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:24.288 INFO:teuthology.orchestra.run.smithi096.stdout:177 read 0x19d2b79 thru 0x19d77d0 (0x4c58 bytes) 2024-04-24T15:32:24.302 INFO:teuthology.orchestra.run.smithi096.stdout:178 write 0xa51eada thru 0xa522a5a (0x3f81 bytes) 2024-04-24T15:32:24.316 INFO:teuthology.orchestra.run.smithi096.stdout:179 write 0x81d3c4 thru 0x82ab2f (0xd76c bytes) 2024-04-24T15:32:24.333 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:32:24.333+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:32:24.388 INFO:teuthology.orchestra.run.smithi096.stdout:180 write 0x398c8a3 thru 0x39980c5 (0xb823 bytes) 2024-04-24T15:32:24.407 INFO:teuthology.orchestra.run.smithi096.stdout:181 write 0xb7d8298 thru 0xb7e7dcd (0xfb36 bytes) 2024-04-24T15:32:24.417 INFO:teuthology.orchestra.run.smithi096.stdout:183 read 0x71b2b0d thru 0x71c0783 (0xdc77 bytes) 2024-04-24T15:32:24.424 INFO:teuthology.orchestra.run.smithi096.stdout:184 writesame 0xcc755bd thru 0xcc7839f data_size 0x1(0x2de3 bytes) 2024-04-24T15:32:24.424 INFO:teuthology.orchestra.run.smithi096.stdout:185 writesame 0xbbae23f thru 0xbbb44da data_size 0x2(0x629c bytes) 2024-04-24T15:32:24.434 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:32:24.434+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:32:24.435 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:32:24.435 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-04-24T15:32:24.435 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-04-24T15:32:24.435 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-04-24T15:32:24.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.436 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:18 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'8, mlcod=15'5)", 2024-04-24T15:32:24.436 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:00.973932+0000", 2024-04-24T15:32:24.436 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.461320879999999, 2024-04-24T15:32:24.436 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0064286839999999996, 2024-04-24T15:32:24.436 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:24.436 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:24.436 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:24.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.436 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:24.436 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.973932+0000", 2024-04-24T15:32:24.436 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:24.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.437 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:24.437 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.973932+0000", 2024-04-24T15:32:24.437 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:24.437 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.437 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.437 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:24.437 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.973931+0000", 2024-04-24T15:32:24.437 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:24.437 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.437 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.437 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:24.437 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.973938+0000", 2024-04-24T15:32:24.437 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.029e-06 2024-04-24T15:32:24.437 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.438 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.438 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:24.438 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.973948+0000", 2024-04-24T15:32:24.438 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9129999999999998e-06 2024-04-24T15:32:24.438 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.438 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.438 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:24.438 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.973952+0000", 2024-04-24T15:32:24.438 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3069999999999996e-06 2024-04-24T15:32:24.438 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.438 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.438 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:24.439 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.973986+0000", 2024-04-24T15:32:24.439 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3643000000000003e-05 2024-04-24T15:32:24.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.439 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:24.439 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.974006+0000", 2024-04-24T15:32:24.439 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9658999999999999e-05 2024-04-24T15:32:24.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.439 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:24.439 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.980330+0000", 2024-04-24T15:32:24.439 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0063243479999999996 2024-04-24T15:32:24.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.439 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:24.440 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.980361+0000", 2024-04-24T15:32:24.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0527000000000001e-05 2024-04-24T15:32:24.440 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:24.440 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:24.440 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:24.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.440 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.440 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:21 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'11, mlcod=15'8)", 2024-04-24T15:32:24.440 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:00.986502+0000", 2024-04-24T15:32:24.440 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.448750947000001, 2024-04-24T15:32:24.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0062008819999999996, 2024-04-24T15:32:24.440 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:24.440 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:24.440 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:24.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.441 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:24.441 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.986502+0000", 2024-04-24T15:32:24.441 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:24.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.441 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:24.441 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.986502+0000", 2024-04-24T15:32:24.441 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:24.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.442 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:24.442 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.986501+0000", 2024-04-24T15:32:24.442 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:24.442 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.442 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.442 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:24.442 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.986639+0000", 2024-04-24T15:32:24.442 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013731400000000001 2024-04-24T15:32:24.442 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.442 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.442 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:24.442 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.986649+0000", 2024-04-24T15:32:24.442 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.041e-05 2024-04-24T15:32:24.443 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.443 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.443 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:24.443 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.986654+0000", 2024-04-24T15:32:24.443 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2000000000000002e-06 2024-04-24T15:32:24.443 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.443 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.443 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:24.443 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.986678+0000", 2024-04-24T15:32:24.443 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3731e-05 2024-04-24T15:32:24.443 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.443 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.443 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:24.443 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.986691+0000", 2024-04-24T15:32:24.444 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3078000000000001e-05 2024-04-24T15:32:24.444 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.444 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.444 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:24.444 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.992666+0000", 2024-04-24T15:32:24.444 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0059751650000000002 2024-04-24T15:32:24.444 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.444 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.444 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:24.444 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.992703+0000", 2024-04-24T15:32:24.444 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6622000000000003e-05 2024-04-24T15:32:24.444 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:24.444 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:24.444 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:24.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.445 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.445 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:22 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'12, mlcod=15'8)", 2024-04-24T15:32:24.445 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:00.987102+0000", 2024-04-24T15:32:24.445 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.448151101000001, 2024-04-24T15:32:24.445 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0056291650000000002, 2024-04-24T15:32:24.445 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:24.445 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:24.445 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:24.445 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.445 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:24.445 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.987102+0000", 2024-04-24T15:32:24.445 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:24.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.445 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.446 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:24.446 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.987102+0000", 2024-04-24T15:32:24.446 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:24.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.446 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.446 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:24.446 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.987101+0000", 2024-04-24T15:32:24.446 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:24.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.446 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.446 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:24.446 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.987106+0000", 2024-04-24T15:32:24.447 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0969999999999996e-06 2024-04-24T15:32:24.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.447 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:24.447 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.987113+0000", 2024-04-24T15:32:24.447 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6789999999999999e-06 2024-04-24T15:32:24.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.447 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:24.447 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.987117+0000", 2024-04-24T15:32:24.447 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6519999999999999e-06 2024-04-24T15:32:24.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.447 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:24.447 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.987143+0000", 2024-04-24T15:32:24.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6098999999999999e-05 2024-04-24T15:32:24.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.448 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.448 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:24.448 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.987157+0000", 2024-04-24T15:32:24.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4100999999999999e-05 2024-04-24T15:32:24.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.448 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.448 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:24.448 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.992706+0000", 2024-04-24T15:32:24.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0055495630000000004 2024-04-24T15:32:24.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.448 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.448 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:24.449 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.992731+0000", 2024-04-24T15:32:24.449 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4519e-05 2024-04-24T15:32:24.449 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:24.449 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:24.449 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:24.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.449 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:36 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'22, mlcod=15'15)", 2024-04-24T15:32:24.449 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.011092+0000", 2024-04-24T15:32:24.449 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.424160705999999, 2024-04-24T15:32:24.450 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004602887, 2024-04-24T15:32:24.450 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:24.450 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:24.450 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:24.450 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.450 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:24.450 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.011092+0000", 2024-04-24T15:32:24.450 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:24.450 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.450 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.451 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:24.451 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.011092+0000", 2024-04-24T15:32:24.451 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:24.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.451 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.451 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:24.451 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.011092+0000", 2024-04-24T15:32:24.451 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:24.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.451 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.451 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:24.451 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.011099+0000", 2024-04-24T15:32:24.452 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4900000000000003e-06 2024-04-24T15:32:24.452 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.452 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.452 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:24.452 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.011107+0000", 2024-04-24T15:32:24.452 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7370000000000006e-06 2024-04-24T15:32:24.452 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.452 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.453 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:24.453 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.011111+0000", 2024-04-24T15:32:24.453 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.822e-06 2024-04-24T15:32:24.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.453 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.453 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:24.453 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.011137+0000", 2024-04-24T15:32:24.453 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6092999999999999e-05 2024-04-24T15:32:24.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.453 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.453 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:24.453 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.011149+0000", 2024-04-24T15:32:24.453 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1727e-05 2024-04-24T15:32:24.454 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.454 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.454 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:24.454 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.015673+0000", 2024-04-24T15:32:24.454 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004524686 2024-04-24T15:32:24.454 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.454 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.454 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:24.454 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.015695+0000", 2024-04-24T15:32:24.454 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.179e-05 2024-04-24T15:32:24.454 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:24.454 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:24.454 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:24.455 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.455 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.455 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:67 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'41, mlcod=15'39)", 2024-04-24T15:32:24.455 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.057501+0000", 2024-04-24T15:32:24.455 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.377751612000001, 2024-04-24T15:32:24.455 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0042378980000000004, 2024-04-24T15:32:24.455 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:24.455 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:24.455 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:24.455 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.455 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:24.455 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.057501+0000", 2024-04-24T15:32:24.455 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:24.456 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.456 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.456 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:24.456 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.057501+0000", 2024-04-24T15:32:24.456 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:24.456 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.456 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.456 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:24.456 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.057500+0000", 2024-04-24T15:32:24.456 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:24.456 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.456 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.456 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:24.456 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.057509+0000", 2024-04-24T15:32:24.457 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9660000000000002e-06 2024-04-24T15:32:24.457 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.457 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:24.457 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.057523+0000", 2024-04-24T15:32:24.457 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3634e-05 2024-04-24T15:32:24.457 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.457 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:24.457 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.057529+0000", 2024-04-24T15:32:24.457 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6390000000000003e-06 2024-04-24T15:32:24.457 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.457 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:24.458 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.057565+0000", 2024-04-24T15:32:24.458 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.54e-05 2024-04-24T15:32:24.458 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.458 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.458 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:24.458 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.057581+0000", 2024-04-24T15:32:24.458 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6268000000000001e-05 2024-04-24T15:32:24.458 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.458 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.458 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:24.458 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.061711+0000", 2024-04-24T15:32:24.458 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0041296859999999996 2024-04-24T15:32:24.458 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.459 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.459 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:24.459 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.061739+0000", 2024-04-24T15:32:24.459 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8394e-05 2024-04-24T15:32:24.459 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:24.459 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:24.459 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:24.459 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.459 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.459 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:68 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'42, mlcod=15'39)", 2024-04-24T15:32:24.459 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.058047+0000", 2024-04-24T15:32:24.459 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.377205991, 2024-04-24T15:32:24.459 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004741706, 2024-04-24T15:32:24.460 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:24.460 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:24.460 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:24.460 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.460 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:24.460 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.058047+0000", 2024-04-24T15:32:24.460 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:24.460 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.460 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.460 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:24.460 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.058047+0000", 2024-04-24T15:32:24.460 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:24.460 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.460 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.460 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:24.461 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.058046+0000", 2024-04-24T15:32:24.461 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:24.461 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.461 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.461 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:24.461 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.058053+0000", 2024-04-24T15:32:24.461 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5660000000000003e-06 2024-04-24T15:32:24.461 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.461 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.461 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:24.461 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.058063+0000", 2024-04-24T15:32:24.461 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9350000000000006e-06 2024-04-24T15:32:24.461 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.461 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.462 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:24.462 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.058067+0000", 2024-04-24T15:32:24.462 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3200000000000001e-06 2024-04-24T15:32:24.462 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.462 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.462 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:24.462 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.060633+0000", 2024-04-24T15:32:24.462 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0025664189999999999 2024-04-24T15:32:24.462 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.462 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.463 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:24.463 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.060655+0000", 2024-04-24T15:32:24.463 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1492e-05 2024-04-24T15:32:24.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.463 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.463 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:24.463 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.062761+0000", 2024-04-24T15:32:24.463 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002105797 2024-04-24T15:32:24.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.463 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.463 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:24.463 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.062789+0000", 2024-04-24T15:32:24.464 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7847e-05 2024-04-24T15:32:24.464 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:24.464 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:24.464 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:24.464 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.464 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.464 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:69 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'43, mlcod=15'39)", 2024-04-24T15:32:24.464 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.058504+0000", 2024-04-24T15:32:24.464 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.376748627000001, 2024-04-24T15:32:24.464 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.006494606, 2024-04-24T15:32:24.464 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:24.464 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:24.464 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:24.464 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.464 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:24.465 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.058504+0000", 2024-04-24T15:32:24.465 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:24.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.465 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.465 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:24.465 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.058504+0000", 2024-04-24T15:32:24.465 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:24.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.465 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.465 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:24.465 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.058503+0000", 2024-04-24T15:32:24.465 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:24.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.465 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.465 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:24.466 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.058588+0000", 2024-04-24T15:32:24.466 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4087999999999999e-05 2024-04-24T15:32:24.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.466 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.466 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:24.466 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.058597+0000", 2024-04-24T15:32:24.466 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7459999999999995e-06 2024-04-24T15:32:24.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.466 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.466 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:24.466 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.058602+0000", 2024-04-24T15:32:24.466 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6929999999999998e-06 2024-04-24T15:32:24.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.466 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.467 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:24.467 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.060983+0000", 2024-04-24T15:32:24.467 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023812690000000001 2024-04-24T15:32:24.467 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.467 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.467 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:24.467 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.060996+0000", 2024-04-24T15:32:24.467 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2826999999999999e-05 2024-04-24T15:32:24.467 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.467 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.467 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:24.467 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.064972+0000", 2024-04-24T15:32:24.467 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0039757890000000004 2024-04-24T15:32:24.467 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.468 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.468 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:24.468 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.064999+0000", 2024-04-24T15:32:24.468 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6917999999999999e-05 2024-04-24T15:32:24.468 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:24.468 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:24.468 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:24.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.468 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.468 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:70 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'44, mlcod=15'39)", 2024-04-24T15:32:24.468 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.059180+0000", 2024-04-24T15:32:24.468 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.376072874999998, 2024-04-24T15:32:24.468 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0058429709999999998, 2024-04-24T15:32:24.468 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:24.469 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:24.469 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:24.469 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.469 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:24.469 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.059180+0000", 2024-04-24T15:32:24.469 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:24.469 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.469 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.469 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:24.469 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.059180+0000", 2024-04-24T15:32:24.469 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:24.469 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.469 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.469 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:24.469 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.059179+0000", 2024-04-24T15:32:24.470 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:24.470 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.470 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.470 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:24.470 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.059184+0000", 2024-04-24T15:32:24.470 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.074e-06 2024-04-24T15:32:24.470 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.470 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.470 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:24.470 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.059194+0000", 2024-04-24T15:32:24.470 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6050000000000004e-06 2024-04-24T15:32:24.470 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.470 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.470 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:24.471 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.059199+0000", 2024-04-24T15:32:24.471 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2639999999999999e-06 2024-04-24T15:32:24.471 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.471 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.471 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:24.471 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.061424+0000", 2024-04-24T15:32:24.471 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0022244880000000002 2024-04-24T15:32:24.471 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.471 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.471 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:24.471 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.061444+0000", 2024-04-24T15:32:24.471 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0633e-05 2024-04-24T15:32:24.471 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.471 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.472 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:24.472 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.065001+0000", 2024-04-24T15:32:24.472 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035567670000000002 2024-04-24T15:32:24.472 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.472 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.472 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:24.472 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.065023+0000", 2024-04-24T15:32:24.472 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1724000000000001e-05 2024-04-24T15:32:24.472 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:24.472 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:24.472 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:24.472 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.473 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.473 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:88 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'58, mlcod=15'51)", 2024-04-24T15:32:24.473 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.076587+0000", 2024-04-24T15:32:24.473 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.358665718000001, 2024-04-24T15:32:24.473 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0042421120000000001, 2024-04-24T15:32:24.473 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:24.473 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:24.473 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:24.474 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.474 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:24.474 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.076587+0000", 2024-04-24T15:32:24.474 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:24.474 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.474 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.474 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:24.474 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.076587+0000", 2024-04-24T15:32:24.474 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:24.474 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.474 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.475 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:24.475 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.076587+0000", 2024-04-24T15:32:24.475 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:24.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.475 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.475 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:24.475 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.076591+0000", 2024-04-24T15:32:24.475 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6990000000000002e-06 2024-04-24T15:32:24.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.475 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.475 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:24.475 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.076599+0000", 2024-04-24T15:32:24.475 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.464e-06 2024-04-24T15:32:24.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.475 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.476 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:24.476 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.076602+0000", 2024-04-24T15:32:24.476 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6689999999999999e-06 2024-04-24T15:32:24.476 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.476 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.476 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:24.476 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.077691+0000", 2024-04-24T15:32:24.476 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001088405 2024-04-24T15:32:24.476 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.476 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.476 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:24.476 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.077708+0000", 2024-04-24T15:32:24.476 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7142e-05 2024-04-24T15:32:24.477 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.477 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.477 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:24.477 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.080806+0000", 2024-04-24T15:32:24.477 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003097982 2024-04-24T15:32:24.477 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.477 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.477 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:24.477 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.080829+0000", 2024-04-24T15:32:24.477 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3280000000000001e-05 2024-04-24T15:32:24.477 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:24.477 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:24.477 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:24.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.478 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.478 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:154 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'108, mlcod=15'105)", 2024-04-24T15:32:24.478 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.929033+0000", 2024-04-24T15:32:24.478 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 22.506219494, 2024-04-24T15:32:24.478 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0068887990000000001, 2024-04-24T15:32:24.478 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:24.478 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:24.478 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:24.478 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.478 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:24.478 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929033+0000", 2024-04-24T15:32:24.478 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:24.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.479 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.479 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:24.479 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929033+0000", 2024-04-24T15:32:24.479 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:24.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.479 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.479 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:24.479 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929033+0000", 2024-04-24T15:32:24.479 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:24.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.480 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.480 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:24.480 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929042+0000", 2024-04-24T15:32:24.480 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5440000000000004e-06 2024-04-24T15:32:24.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.480 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.480 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:24.481 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929051+0000", 2024-04-24T15:32:24.481 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0580000000000004e-06 2024-04-24T15:32:24.481 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.481 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.481 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:24.481 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929055+0000", 2024-04-24T15:32:24.482 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.337e-06 2024-04-24T15:32:24.482 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.482 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.482 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:24.482 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929074+0000", 2024-04-24T15:32:24.482 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9724999999999998e-05 2024-04-24T15:32:24.483 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.483 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.483 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:24.483 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929084+0000", 2024-04-24T15:32:24.483 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7049999999999994e-06 2024-04-24T15:32:24.483 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.484 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.484 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:24.484 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.935895+0000", 2024-04-24T15:32:24.484 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0068104359999999996 2024-04-24T15:32:24.484 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.484 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.484 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:24.485 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.935922+0000", 2024-04-24T15:32:24.485 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7538999999999998e-05 2024-04-24T15:32:24.485 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:24.485 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:24.485 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:24.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.485 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.486 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:155 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'109, mlcod=15'105)", 2024-04-24T15:32:24.486 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.929398+0000", 2024-04-24T15:32:24.486 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 22.505854512999999, 2024-04-24T15:32:24.486 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0098754700000000008, 2024-04-24T15:32:24.486 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:24.486 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:24.486 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:24.487 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.487 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:24.487 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929398+0000", 2024-04-24T15:32:24.487 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:24.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.487 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.487 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:24.487 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929398+0000", 2024-04-24T15:32:24.487 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:24.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.488 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.488 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:24.488 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929398+0000", 2024-04-24T15:32:24.488 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:24.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.488 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.488 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:24.488 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929571+0000", 2024-04-24T15:32:24.488 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017281999999999999 2024-04-24T15:32:24.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.488 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.489 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:24.489 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929578+0000", 2024-04-24T15:32:24.489 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8929999999999999e-06 2024-04-24T15:32:24.489 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.489 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.489 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:24.489 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929581+0000", 2024-04-24T15:32:24.490 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1709999999999998e-06 2024-04-24T15:32:24.490 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.490 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.490 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:24.490 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929605+0000", 2024-04-24T15:32:24.490 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3938e-05 2024-04-24T15:32:24.491 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.491 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.491 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:24.491 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929618+0000", 2024-04-24T15:32:24.491 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3653e-05 2024-04-24T15:32:24.491 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.491 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.491 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:24.492 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939186+0000", 2024-04-24T15:32:24.492 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0095674260000000004 2024-04-24T15:32:24.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.492 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.492 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:24.492 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939274+0000", 2024-04-24T15:32:24.492 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7928000000000002e-05 2024-04-24T15:32:24.492 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:24.492 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:24.492 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:24.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.492 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.492 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:156 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'110, mlcod=15'105)", 2024-04-24T15:32:24.492 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.930050+0000", 2024-04-24T15:32:24.492 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 22.505203197, 2024-04-24T15:32:24.493 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0092911650000000005, 2024-04-24T15:32:24.493 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:24.493 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:24.493 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:24.493 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.493 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:24.493 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930050+0000", 2024-04-24T15:32:24.493 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:24.494 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.494 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.494 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:24.494 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930050+0000", 2024-04-24T15:32:24.494 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:24.494 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.494 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.494 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:24.494 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930049+0000", 2024-04-24T15:32:24.494 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:24.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.495 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.495 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:24.495 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930053+0000", 2024-04-24T15:32:24.495 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.98e-06 2024-04-24T15:32:24.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.495 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.495 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:24.495 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930058+0000", 2024-04-24T15:32:24.495 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3530000000000002e-06 2024-04-24T15:32:24.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.495 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.495 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:24.495 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930061+0000", 2024-04-24T15:32:24.496 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6970000000000002e-06 2024-04-24T15:32:24.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.496 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.496 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:24.496 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930082+0000", 2024-04-24T15:32:24.496 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0815e-05 2024-04-24T15:32:24.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.496 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.496 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:24.496 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930093+0000", 2024-04-24T15:32:24.496 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0699999999999999e-05 2024-04-24T15:32:24.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.497 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.497 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:24.497 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939280+0000", 2024-04-24T15:32:24.497 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0091874880000000006 2024-04-24T15:32:24.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.497 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.497 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:24.497 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939341+0000", 2024-04-24T15:32:24.497 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0608e-05 2024-04-24T15:32:24.497 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:24.497 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:24.497 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:24.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.497 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.497 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:157 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'111, mlcod=15'106)", 2024-04-24T15:32:24.498 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.930506+0000", 2024-04-24T15:32:24.498 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 22.504746339, 2024-04-24T15:32:24.498 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.008889246, 2024-04-24T15:32:24.498 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:24.498 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:24.498 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:24.498 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.498 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:24.498 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930506+0000", 2024-04-24T15:32:24.498 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:24.498 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.498 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.499 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:24.499 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930506+0000", 2024-04-24T15:32:24.499 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:24.499 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.499 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.499 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:24.499 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930506+0000", 2024-04-24T15:32:24.499 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:24.499 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.499 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.499 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:24.499 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930510+0000", 2024-04-24T15:32:24.499 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1810000000000003e-06 2024-04-24T15:32:24.499 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.500 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:24.500 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930516+0000", 2024-04-24T15:32:24.500 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1920000000000002e-06 2024-04-24T15:32:24.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.500 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:24.500 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930520+0000", 2024-04-24T15:32:24.500 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.501e-06 2024-04-24T15:32:24.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.500 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:24.501 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930539+0000", 2024-04-24T15:32:24.501 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8929000000000001e-05 2024-04-24T15:32:24.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.501 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.501 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:24.501 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930550+0000", 2024-04-24T15:32:24.501 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1168999999999999e-05 2024-04-24T15:32:24.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.501 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.501 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:24.502 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939343+0000", 2024-04-24T15:32:24.502 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0087932900000000005 2024-04-24T15:32:24.502 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.502 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.502 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:24.502 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939396+0000", 2024-04-24T15:32:24.502 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2454999999999998e-05 2024-04-24T15:32:24.502 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:24.502 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:24.502 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:24.502 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.502 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.502 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:158 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'112, mlcod=15'106)", 2024-04-24T15:32:24.502 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.930877+0000", 2024-04-24T15:32:24.503 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 22.504375648, 2024-04-24T15:32:24.503 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.008572741, 2024-04-24T15:32:24.503 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:24.503 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:24.503 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:24.503 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.503 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:24.503 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930877+0000", 2024-04-24T15:32:24.503 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:24.503 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.503 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.503 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:24.503 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930877+0000", 2024-04-24T15:32:24.503 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:24.504 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.504 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.504 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:24.504 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930877+0000", 2024-04-24T15:32:24.504 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:24.504 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.504 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.504 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:24.504 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930968+0000", 2024-04-24T15:32:24.504 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1638999999999995e-05 2024-04-24T15:32:24.504 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.504 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.505 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:24.505 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930975+0000", 2024-04-24T15:32:24.505 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2169999999999999e-06 2024-04-24T15:32:24.505 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.505 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.505 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:24.505 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930977+0000", 2024-04-24T15:32:24.505 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.791e-06 2024-04-24T15:32:24.505 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.505 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.505 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:24.505 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931002+0000", 2024-04-24T15:32:24.505 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4491e-05 2024-04-24T15:32:24.505 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.506 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.506 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:24.506 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931012+0000", 2024-04-24T15:32:24.506 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0217000000000001e-05 2024-04-24T15:32:24.506 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.506 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.506 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:24.506 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939398+0000", 2024-04-24T15:32:24.506 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.008385366 2024-04-24T15:32:24.506 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.506 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.506 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:24.506 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939450+0000", 2024-04-24T15:32:24.507 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2326e-05 2024-04-24T15:32:24.507 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:24.507 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:24.507 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:24.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.507 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:159 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'113, mlcod=15'106)", 2024-04-24T15:32:24.507 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.931313+0000", 2024-04-24T15:32:24.507 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 22.503940296, 2024-04-24T15:32:24.507 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0081994159999999993, 2024-04-24T15:32:24.507 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:24.507 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:24.508 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:24.508 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.508 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:24.508 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931313+0000", 2024-04-24T15:32:24.508 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:24.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.508 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.508 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:24.508 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931313+0000", 2024-04-24T15:32:24.508 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:24.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.508 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.508 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:24.508 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931312+0000", 2024-04-24T15:32:24.508 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:24.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.509 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:24.509 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931316+0000", 2024-04-24T15:32:24.509 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0589999999999996e-06 2024-04-24T15:32:24.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.509 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:24.509 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931322+0000", 2024-04-24T15:32:24.509 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2020000000000001e-06 2024-04-24T15:32:24.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.509 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:24.509 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931327+0000", 2024-04-24T15:32:24.509 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2440000000000004e-06 2024-04-24T15:32:24.510 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.510 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.510 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:24.510 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931348+0000", 2024-04-24T15:32:24.510 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1464e-05 2024-04-24T15:32:24.510 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.510 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.510 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:24.510 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931359+0000", 2024-04-24T15:32:24.510 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0699e-05 2024-04-24T15:32:24.510 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.510 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.510 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:24.511 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939452+0000", 2024-04-24T15:32:24.511 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0080931749999999993 2024-04-24T15:32:24.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.511 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:24.511 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939512+0000", 2024-04-24T15:32:24.511 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0050000000000003e-05 2024-04-24T15:32:24.511 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:24.511 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:24.511 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:24.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.511 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:2447 4.6 4:668081fa:::rbd_data.114a1f60cb66.00000000000001f0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-24T15:32:24.511 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:23.027189+0000", 2024-04-24T15:32:24.511 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.4080642720000001, 2024-04-24T15:32:24.512 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0056741200000000004, 2024-04-24T15:32:24.512 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:24.512 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:24.512 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:24.512 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:24.512 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:24.512 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2447 2024-04-24T15:32:24.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.512 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:24.512 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.512 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:24.512 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027189+0000", 2024-04-24T15:32:24.512 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:24.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.512 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.513 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:24.513 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027189+0000", 2024-04-24T15:32:24.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:24.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.513 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:24.513 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027188+0000", 2024-04-24T15:32:24.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:24.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.513 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:24.513 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027194+0000", 2024-04-24T15:32:24.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0499999999999997e-06 2024-04-24T15:32:24.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.514 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.514 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:24.514 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027205+0000", 2024-04-24T15:32:24.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1619e-05 2024-04-24T15:32:24.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.514 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.514 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:24.514 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027212+0000", 2024-04-24T15:32:24.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4289999999999998e-06 2024-04-24T15:32:24.515 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.515 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.515 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:24.515 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027243+0000", 2024-04-24T15:32:24.515 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1510000000000002e-05 2024-04-24T15:32:24.515 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.515 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.515 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:24.515 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027288+0000", 2024-04-24T15:32:24.515 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4484000000000003e-05 2024-04-24T15:32:24.515 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.515 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.516 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-24T15:32:24.516 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027351+0000", 2024-04-24T15:32:24.516 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3311000000000004e-05 2024-04-24T15:32:24.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.516 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.516 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:24.516 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.029874+0000", 2024-04-24T15:32:24.516 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00252315 2024-04-24T15:32:24.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.516 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.516 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:24.516 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.032786+0000", 2024-04-24T15:32:24.516 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0029115370000000001 2024-04-24T15:32:24.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.517 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.517 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:24.517 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.032828+0000", 2024-04-24T15:32:24.517 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2132000000000003e-05 2024-04-24T15:32:24.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.517 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.517 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:24.517 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.032863+0000", 2024-04-24T15:32:24.517 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4912e-05 2024-04-24T15:32:24.517 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:24.517 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:24.517 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:24.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.517 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.517 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:2449 4.6 4:61283b7c:::rbd_data.114a1f60cb66.00000000000001f2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-24T15:32:24.518 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:23.027620+0000", 2024-04-24T15:32:24.518 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.407632859, 2024-04-24T15:32:24.518 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0053273959999999999, 2024-04-24T15:32:24.518 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:24.518 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:24.518 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:24.518 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:24.518 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:24.518 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2449 2024-04-24T15:32:24.518 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.518 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:24.518 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.518 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:24.519 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027620+0000", 2024-04-24T15:32:24.519 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:24.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.519 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.519 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:24.519 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027620+0000", 2024-04-24T15:32:24.519 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:24.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.519 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.519 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:24.519 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027619+0000", 2024-04-24T15:32:24.519 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:24.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.519 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.520 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:24.520 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027623+0000", 2024-04-24T15:32:24.520 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7100000000000001e-06 2024-04-24T15:32:24.520 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.520 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.520 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:24.520 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027636+0000", 2024-04-24T15:32:24.520 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2609000000000001e-05 2024-04-24T15:32:24.520 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.520 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.521 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:24.521 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027643+0000", 2024-04-24T15:32:24.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.182e-06 2024-04-24T15:32:24.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.521 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.521 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:24.521 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027678+0000", 2024-04-24T15:32:24.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5641e-05 2024-04-24T15:32:24.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.521 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.521 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:24.521 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027765+0000", 2024-04-24T15:32:24.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6691000000000001e-05 2024-04-24T15:32:24.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.521 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.522 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-24T15:32:24.522 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027864+0000", 2024-04-24T15:32:24.522 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9257000000000002e-05 2024-04-24T15:32:24.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.522 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:24.522 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.032638+0000", 2024-04-24T15:32:24.522 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004773429 2024-04-24T15:32:24.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.522 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:24.522 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.032896+0000", 2024-04-24T15:32:24.522 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000258352 2024-04-24T15:32:24.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.523 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:24.523 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.032924+0000", 2024-04-24T15:32:24.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8337999999999999e-05 2024-04-24T15:32:24.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.523 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:24.523 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.032947+0000", 2024-04-24T15:32:24.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2849000000000001e-05 2024-04-24T15:32:24.523 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:24.523 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:24.523 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:24.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.524 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4426.0:2767 4.6 e34/33 4:6297db09:::rbd_data.114a1f60cb66.00000000000000e1:head v 34'25, mlcod=34'23)", 2024-04-24T15:32:24.524 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.256572+0000", 2024-04-24T15:32:24.524 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.17868088200000001, 2024-04-24T15:32:24.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0050987539999999996, 2024-04-24T15:32:24.524 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:24.524 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:24.524 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:24.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.524 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:24.524 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.256572+0000", 2024-04-24T15:32:24.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:24.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.525 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:24.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.256572+0000", 2024-04-24T15:32:24.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:24.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.525 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:24.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.256571+0000", 2024-04-24T15:32:24.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:24.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.525 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:24.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.256578+0000", 2024-04-24T15:32:24.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.161e-06 2024-04-24T15:32:24.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:24.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.256585+0000", 2024-04-24T15:32:24.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.391e-06 2024-04-24T15:32:24.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:24.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.256590+0000", 2024-04-24T15:32:24.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8189999999999999e-06 2024-04-24T15:32:24.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:24.527 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.256623+0000", 2024-04-24T15:32:24.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3618000000000002e-05 2024-04-24T15:32:24.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.527 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:24.527 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.256642+0000", 2024-04-24T15:32:24.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8479999999999999e-05 2024-04-24T15:32:24.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.527 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:24.527 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.261622+0000", 2024-04-24T15:32:24.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0049796959999999996 2024-04-24T15:32:24.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:24.528 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.261671+0000", 2024-04-24T15:32:24.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9147000000000002e-05 2024-04-24T15:32:24.528 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:24.528 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:24.528 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:24.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.528 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4426.0:2781 4.b e34/33 4:d743064f:::rbd_data.114a1f60cb66.0000000000000191:head v 34'24, mlcod=34'22)", 2024-04-24T15:32:24.528 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.262276+0000", 2024-04-24T15:32:24.528 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.172977041, 2024-04-24T15:32:24.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0049135359999999996, 2024-04-24T15:32:24.528 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:24.529 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:24.529 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:24.529 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.529 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:24.529 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.262276+0000", 2024-04-24T15:32:24.529 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:24.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.529 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.529 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:24.529 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.262276+0000", 2024-04-24T15:32:24.529 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:24.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.529 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.529 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:24.530 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.262275+0000", 2024-04-24T15:32:24.530 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:24.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.530 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.530 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:24.530 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.262281+0000", 2024-04-24T15:32:24.530 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5089999999999999e-06 2024-04-24T15:32:24.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.530 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.530 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:24.530 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.262291+0000", 2024-04-24T15:32:24.530 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4639999999999995e-06 2024-04-24T15:32:24.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:24.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.262297+0000", 2024-04-24T15:32:24.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5370000000000003e-06 2024-04-24T15:32:24.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:24.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.262325+0000", 2024-04-24T15:32:24.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7453e-05 2024-04-24T15:32:24.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:24.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.262337+0000", 2024-04-24T15:32:24.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1997999999999999e-05 2024-04-24T15:32:24.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:24.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.267161+0000", 2024-04-24T15:32:24.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0048238669999999999 2024-04-24T15:32:24.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:24.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.267189+0000", 2024-04-24T15:32:24.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8498999999999999e-05 2024-04-24T15:32:24.532 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:24.532 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:24.532 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:24.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.532 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4426.0:2783 4.1e e34/33 4:7b9df907:::rbd_data.114a1f60cb66.0000000000000193:head v 34'18, mlcod=34'15)", 2024-04-24T15:32:24.533 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.262583+0000", 2024-04-24T15:32:24.533 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.172670295, 2024-04-24T15:32:24.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0046302210000000003, 2024-04-24T15:32:24.533 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:24.533 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:24.533 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:24.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:24.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.262583+0000", 2024-04-24T15:32:24.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:24.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:24.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.262583+0000", 2024-04-24T15:32:24.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:24.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:24.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.262582+0000", 2024-04-24T15:32:24.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:24.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:24.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.262587+0000", 2024-04-24T15:32:24.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0050000000000004e-06 2024-04-24T15:32:24.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:24.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.262595+0000", 2024-04-24T15:32:24.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0700000000000007e-06 2024-04-24T15:32:24.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:24.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.262601+0000", 2024-04-24T15:32:24.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1890000000000004e-06 2024-04-24T15:32:24.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:24.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.262629+0000", 2024-04-24T15:32:24.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8147999999999999e-05 2024-04-24T15:32:24.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:24.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.262640+0000", 2024-04-24T15:32:24.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0829000000000001e-05 2024-04-24T15:32:24.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:24.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.267175+0000", 2024-04-24T15:32:24.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0045347629999999998 2024-04-24T15:32:24.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:24.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:24.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:24.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.267213+0000", 2024-04-24T15:32:24.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7799999999999997e-05 2024-04-24T15:32:24.536 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:24.537 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:24.537 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:24.537 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:24.537 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:24.537 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:32:24.537 INFO:teuthology.orchestra.run.smithi096.stdout:186 writesame 0x902e7fe thru 0x903e16b data_size 0x7(0xf96e bytes) 2024-04-24T15:32:24.538 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-24T15:32:24.540 INFO:teuthology.orchestra.run.smithi096.stdout:187 read 0x931ef44 thru 0x9322c4f (0x3d0c bytes) 2024-04-24T15:32:24.540 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:24.540 INFO:teuthology.orchestra.run.smithi096.stdout:189 writesame 0x5cedc69 thru 0x5cf7987 data_size 0x1d(0x9d1f bytes) 2024-04-24T15:32:24.540 INFO:teuthology.orchestra.run.smithi096.stdout:190 writesame 0x3819a60 thru 0x3822cd8 data_size 0x1d(0x9279 bytes) 2024-04-24T15:32:24.541 INFO:teuthology.orchestra.run.smithi096.stdout:191 write 0x57b2150 thru 0x57b447d (0x232e bytes) 2024-04-24T15:32:24.541 INFO:teuthology.orchestra.run.smithi096.stdout:193 read 0x3396362 thru 0x33a4e4e (0xeaed bytes) 2024-04-24T15:32:24.541 INFO:teuthology.orchestra.run.smithi096.stdout:194 write 0xf9d962 thru 0xfa337b (0x5a1a bytes) 2024-04-24T15:32:24.541 INFO:teuthology.orchestra.run.smithi096.stdout:195 read 0x6d479ce thru 0x6d52acd (0xb100 bytes) 2024-04-24T15:32:24.541 INFO:teuthology.orchestra.run.smithi096.stdout:196 read 0x9b746eb thru 0x9b7d6a2 (0x8fb8 bytes) 2024-04-24T15:32:24.541 INFO:teuthology.orchestra.run.smithi096.stdout:198 read 0x1bf1e9c thru 0x1bfab42 (0x8ca7 bytes) 2024-04-24T15:32:24.541 INFO:teuthology.orchestra.run.smithi096.stdout:199 punch from 0xb53aefe to 0xb548bdc, (0xdcde bytes) 2024-04-24T15:32:24.541 INFO:teuthology.orchestra.run.smithi096.stdout:200 punch from 0x61483d9 to 0x614f3da, (0x7001 bytes) 2024-04-24T15:32:24.541 INFO:teuthology.orchestra.run.smithi096.stdout:202 read 0x1695519 thru 0x16a306b (0xdb53 bytes) 2024-04-24T15:32:24.541 INFO:teuthology.orchestra.run.smithi096.stdout:203 punch from 0x56bd8e3 to 0x56cad1c, (0xd439 bytes) 2024-04-24T15:32:24.541 INFO:teuthology.orchestra.run.smithi096.stdout:205 write 0x2602d4d thru 0x2610b08 (0xddbc bytes) 2024-04-24T15:32:24.541 INFO:teuthology.orchestra.run.smithi096.stdout:206 write 0xdf145c7 thru 0xdf22762 (0xe19c bytes) 2024-04-24T15:32:24.542 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:32:24.541+0000 7f23abcf6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 44021) UID: 0 2024-04-24T15:32:24.552 INFO:teuthology.orchestra.run.smithi096.stdout:208 trunc from 0xdf22763 to 0xacad652 2024-04-24T15:32:24.574 INFO:teuthology.orchestra.run.smithi096.stdout:209 writesame 0x58d7b2c thru 0x58e6084 data_size 0x1(0xe559 bytes) 2024-04-24T15:32:24.589 INFO:teuthology.orchestra.run.smithi096.stdout:210 read 0x7989631 thru 0x799652e (0xcefe bytes) 2024-04-24T15:32:24.589 INFO:teuthology.orchestra.run.smithi096.stdout:211 writesame 0x165694c thru 0x165800d data_size 0x3(0x16c2 bytes) 2024-04-24T15:32:24.592 INFO:teuthology.orchestra.run.smithi096.stdout:213 read 0x6d8e4aa thru 0x6d99dd9 (0xb930 bytes) 2024-04-24T15:32:24.599 INFO:teuthology.orchestra.run.smithi096.stdout:214 writesame 0xd0239a1 thru 0xd0314e6 data_size 0x7f(0xdb46 bytes) 2024-04-24T15:32:24.600 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:24.606 INFO:teuthology.orchestra.run.smithi096.stdout:216 write 0xea53c75 thru 0xea600dd (0xc469 bytes) 2024-04-24T15:32:24.617 INFO:teuthology.orchestra.run.smithi096.stdout:217 write 0x5c8be06 thru 0x5c98077 (0xc272 bytes) 2024-04-24T15:32:24.620 INFO:teuthology.orchestra.run.smithi096.stdout:218 read 0x189934b thru 0x18a45c7 (0xb27d bytes) 2024-04-24T15:32:24.620 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:24.620 INFO:teuthology.orchestra.run.smithi096.stdout:222 write 0x1b66967 thru 0x1b74f14 (0xe5ae bytes) 2024-04-24T15:32:24.633 INFO:teuthology.orchestra.run.smithi096.stdout:223 write 0x3b4aa27 thru 0x3b5842d (0xda07 bytes) 2024-04-24T15:32:24.641 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:32:24.641+0000 7f23abcf6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 44021) UID: 0 2024-04-24T15:32:24.648 INFO:teuthology.orchestra.run.smithi096.stdout:224 read 0xcaf0e49 thru 0xcaf1b9d (0xd55 bytes) 2024-04-24T15:32:24.651 INFO:teuthology.orchestra.run.smithi096.stdout:225 trunc from 0xea600de to 0xbba09c5 2024-04-24T15:32:24.667 INFO:teuthology.orchestra.run.smithi096.stdout:226 punch from 0x45d7804 to 0x45dd4e0, (0x5cdc bytes) 2024-04-24T15:32:24.667 INFO:teuthology.orchestra.run.smithi096.stdout:227 write 0x6efb9f8 thru 0x6f05d23 (0xa32c bytes) 2024-04-24T15:32:24.684 INFO:teuthology.orchestra.run.smithi096.stdout:228 write 0xbec7dce thru 0xbec86ad (0x8e0 bytes) 2024-04-24T15:32:24.690 INFO:teuthology.orchestra.run.smithi096.stdout:229 write 0x99c7cb3 thru 0x99c8d6a (0x10b8 bytes) 2024-04-24T15:32:24.694 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:32:24.695 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:32:24.695 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-04-24T15:32:24.695 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:32:24.697 INFO:teuthology.orchestra.run.smithi096.stdout:230 writesame 0x6daf59 thru 0x6e86c7 data_size 0x1(0xd76f bytes) 2024-04-24T15:32:24.706 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-24T15:32:24.706 INFO:teuthology.orchestra.run.smithi096.stdout:231 write 0x4c0dde7 thru 0x4c1621a (0x8434 bytes) 2024-04-24T15:32:24.721 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:24.721 INFO:teuthology.orchestra.run.smithi096.stdout:235 read 0x6bb789d thru 0x6bc4723 (0xce87 bytes) 2024-04-24T15:32:24.721 INFO:teuthology.orchestra.run.smithi096.stdout:236 read 0x68a6efb thru 0x68b38b8 (0xc9be bytes) 2024-04-24T15:32:24.722 INFO:teuthology.orchestra.run.smithi096.stdout:237 read 0x2b94e59 thru 0x2ba02af (0xb457 bytes) 2024-04-24T15:32:24.722 INFO:teuthology.orchestra.run.smithi096.stdout:238 writesame 0x91fea7 thru 0x92e8d6 data_size 0x1(0xea30 bytes) 2024-04-24T15:32:24.735 INFO:teuthology.orchestra.run.smithi096.stdout:240 write 0x6d1107d thru 0x6d18a5b (0x79df bytes) 2024-04-24T15:32:24.742 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:32:24.743+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:32:24.753 INFO:teuthology.orchestra.run.smithi096.stdout:241 trunc from 0xbec86ae to 0x911b668 2024-04-24T15:32:24.766 INFO:teuthology.orchestra.run.smithi096.stdout:242 read 0xe3e8b7 thru 0xe43922 (0x506c bytes) 2024-04-24T15:32:24.767 INFO:teuthology.orchestra.run.smithi096.stdout:243 writesame 0x5182a8f thru 0x51899d8 data_size 0x6e(0x6f4a bytes) 2024-04-24T15:32:24.779 INFO:teuthology.orchestra.run.smithi096.stdout:245 writesame 0x1b713e4 thru 0x1b7729c data_size 0x3b(0x5eb9 bytes) 2024-04-24T15:32:24.793 INFO:teuthology.orchestra.run.smithi096.stdout:246 trunc from 0x911b668 to 0x1241104 2024-04-24T15:32:24.843 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:32:24.844+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:32:24.852 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:32:24.852 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:32:24.852 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-04-24T15:32:24.852 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-04-24T15:32:24.852 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:32:24.864 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-24T15:32:24.871 INFO:teuthology.orchestra.run.smithi096.stdout:249 writesame 0x1eddf81 thru 0x1eecd37 data_size 0x5(0xedb7 bytes) 2024-04-24T15:32:24.871 INFO:teuthology.orchestra.run.smithi096.stdout:251 read 0x1a542d5 thru 0x1a5e546 (0xa272 bytes) 2024-04-24T15:32:24.879 INFO:teuthology.orchestra.run.smithi096.stdout:253 trunc from 0x1eecd38 to 0x8473978 2024-04-24T15:32:24.884 INFO:teuthology.orchestra.run.smithi096.stdout:255 read 0x7dd7075 thru 0x7de493d (0xd8c9 bytes) 2024-04-24T15:32:24.885 INFO:teuthology.orchestra.run.smithi096.stdout:256 read 0x66e4f2c thru 0x66e8386 (0x345b bytes) 2024-04-24T15:32:24.885 INFO:teuthology.orchestra.run.smithi096.stdout:258 read 0x80df671 thru 0x80e8a4b (0x93db bytes) 2024-04-24T15:32:24.885 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:24.893 INFO:teuthology.orchestra.run.smithi096.stdout:262 writesame 0xabca5a8 thru 0xabd69ac data_size 0x81(0xc405 bytes) 2024-04-24T15:32:24.893 INFO:teuthology.orchestra.run.smithi096.stdout:263 read 0x3a1b6ba thru 0x3a22aa6 (0x73ed bytes) 2024-04-24T15:32:24.893 INFO:teuthology.orchestra.run.smithi096.stdout:265 read 0x2d77211 thru 0x2d7edeb (0x7bdb bytes) 2024-04-24T15:32:24.893 INFO:teuthology.orchestra.run.smithi096.stdout:267 writesame 0x5c35fb9 thru 0x5c3d314 data_size 0x2e(0x735c bytes) 2024-04-24T15:32:24.911 INFO:teuthology.orchestra.run.smithi096.stdout:269 writesame 0x4ee9889 thru 0x4ef8c08 data_size 0x40(0xf380 bytes) 2024-04-24T15:32:24.926 INFO:teuthology.orchestra.run.smithi096.stdout:270 write 0x5d8f270 thru 0x5d973b4 (0x8145 bytes) 2024-04-24T15:32:24.944 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:32:24.945+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:32:24.948 INFO:teuthology.orchestra.run.smithi096.stdout:271 trunc from 0xabd69ad to 0x4c79353 2024-04-24T15:32:24.965 INFO:teuthology.orchestra.run.smithi096.stdout:273 write 0x3d4970c thru 0x3d505ed (0x6ee2 bytes) 2024-04-24T15:32:24.984 INFO:teuthology.orchestra.run.smithi096.stdout:274 trunc from 0x4c79353 to 0xb9e570d 2024-04-24T15:32:24.989 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:24.997 INFO:teuthology.orchestra.run.smithi096.stdout:276 write 0xdf1b7f0 thru 0xdf21db7 (0x65c8 bytes) 2024-04-24T15:32:25.013 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:32:25.013 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-04-24T15:32:25.013 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-04-24T15:32:25.013 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [] 2024-04-24T15:32:25.013 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:32:25.018 INFO:teuthology.orchestra.run.smithi096.stdout:277 writesame 0xe1c6d73 thru 0xe1d06aa data_size 0x1(0x9938 bytes) 2024-04-24T15:32:25.018 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:25.018 INFO:teuthology.orchestra.run.smithi096.stdout:280 write 0xc540689 thru 0xc54be54 (0xb7cc bytes) 2024-04-24T15:32:25.023 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-24T15:32:25.047 INFO:teuthology.orchestra.run.smithi096.stdout:281 trunc from 0xe1d06ab to 0x18b6851 2024-04-24T15:32:25.047 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:32:25.048+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:32:25.078 INFO:teuthology.orchestra.run.smithi096.stdout:282 writesame 0x3e515b9 thru 0x3e5e714 data_size 0x2(0xd15c bytes) 2024-04-24T15:32:25.078 INFO:teuthology.orchestra.run.smithi096.stdout:283 punch from 0x253e7cd to 0x25468f9, (0x812c bytes) 2024-04-24T15:32:25.089 INFO:teuthology.orchestra.run.smithi096.stdout:284 writesame 0x914a49d thru 0x9158422 data_size 0x33(0xdf86 bytes) 2024-04-24T15:32:25.090 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-24T15:32:25.095 INFO:teuthology.orchestra.run.smithi096.stdout:285 trunc from 0x9158423 to 0x3ce9786 2024-04-24T15:32:25.103 INFO:teuthology.orchestra.run.smithi096.stdout:286 trunc from 0x3ce9786 to 0x5d7ac71 2024-04-24T15:32:25.109 INFO:teuthology.orchestra.run.smithi096.stdout:287 punch from 0x364d719 to 0x365720c, (0x9af3 bytes) 2024-04-24T15:32:25.122 INFO:teuthology.orchestra.run.smithi096.stdout:288 write 0xb1ac7cf thru 0xb1bb39c (0xebce bytes) 2024-04-24T15:32:25.128 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-24T15:32:25.138 INFO:teuthology.orchestra.run.smithi096.stdout:289 writesame 0x5dada1b thru 0x5dba388 data_size 0x2(0xc96e bytes) 2024-04-24T15:32:25.146 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:32:25.146+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:32:25.167 INFO:teuthology.orchestra.run.smithi096.stdout:290 writesame 0xc23e3b1 thru 0xc24cc49 data_size 0x1(0xe899 bytes) 2024-04-24T15:32:25.172 INFO:teuthology.orchestra.run.smithi096.stdout:292 write 0xc3824e5 thru 0xc38bb7a (0x9696 bytes) 2024-04-24T15:32:25.194 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:32:25.195 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:32:25.195 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-04-24T15:32:25.195 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:32:25.196 INFO:teuthology.orchestra.run.smithi096.stdout:294 write 0xedd3dfa thru 0xedd6acd (0x2cd4 bytes) 2024-04-24T15:32:25.206 INFO:teuthology.orchestra.run.smithi096.stdout:297 read 0xd951807 thru 0xd95ba2d (0xa227 bytes) 2024-04-24T15:32:25.206 INFO:teuthology.orchestra.run.smithi096.stdout:298 read 0xb2d63b9 thru 0xb2df221 (0x8e69 bytes) 2024-04-24T15:32:25.206 INFO:teuthology.orchestra.run.smithi096.stdout:299 punch from 0xe369719 to 0xe37764e, (0xdf35 bytes) 2024-04-24T15:32:25.206 INFO:teuthology.orchestra.run.smithi096.stdout:301 write 0xce21eca thru 0xce2eec3 (0xcffa bytes) 2024-04-24T15:32:25.207 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-24T15:32:25.219 INFO:teuthology.orchestra.run.smithi096.stdout:302 write 0xeb305e9 thru 0xeb30bc9 (0x5e1 bytes) 2024-04-24T15:32:25.227 INFO:teuthology.orchestra.run.smithi096.stdout:303 write 0x445e48e thru 0x445e66d (0x1e0 bytes) 2024-04-24T15:32:25.230 INFO:teuthology.orchestra.run.smithi096.stdout:304 punch from 0x1b444ce to 0x1b4f689, (0xb1bb bytes) 2024-04-24T15:32:25.245 INFO:teuthology.orchestra.run.smithi096.stdout:305 trunc from 0xedd6ace to 0x320f040 2024-04-24T15:32:25.248 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:32:25.248+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:32:25.290 INFO:teuthology.orchestra.run.smithi096.stdout:306 write 0xb738a97 thru 0xb7468ab (0xde15 bytes) 2024-04-24T15:32:25.299 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-04-24T15:32:25.305 INFO:teuthology.orchestra.run.smithi096.stdout:307 write 0x6714234 thru 0x6720230 (0xbffd bytes) 2024-04-24T15:32:25.321 INFO:teuthology.orchestra.run.smithi096.stdout:308 read 0x2ae8675 thru 0x2af4cc3 (0xc64f bytes) 2024-04-24T15:32:25.322 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-04-24T15:32:25.331 INFO:teuthology.orchestra.run.smithi096.stdout:309 trunc from 0xb7468ac to 0x31da344 2024-04-24T15:32:25.344 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-04-24T15:32:25.348 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:32:25.349+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:32:25.366 INFO:teuthology.orchestra.run.smithi096.stdout:310 writesame 0xb38163a thru 0xb38a0a1 data_size 0x8(0x8a68 bytes) 2024-04-24T15:32:25.371 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-04-24T15:32:25.374 INFO:teuthology.orchestra.run.smithi096.stdout:311 writesame 0xd08d399 thru 0xd09d24b data_size 0x1(0xfeb3 bytes) 2024-04-24T15:32:25.374 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:25.375 INFO:teuthology.orchestra.run.smithi096.stdout:314 trunc from 0xd09d24c to 0xcd0f7bd 2024-04-24T15:32:25.377 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:25.377 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:25.379 INFO:teuthology.orchestra.run.smithi096.stdout:315 writesame 0xaacc34c thru 0xaad772e data_size 0x1(0xb3e3 bytes) 2024-04-24T15:32:25.389 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-24T15:32:25.391 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-04-24T15:32:25.397 INFO:teuthology.orchestra.run.smithi096.stdout:316 read 0x5c8b50e thru 0x5c967d1 (0xb2c4 bytes) 2024-04-24T15:32:25.397 INFO:teuthology.orchestra.run.smithi096.stdout:317 read 0x6bdea28 thru 0x6be9d42 (0xb31b bytes) 2024-04-24T15:32:25.397 INFO:teuthology.orchestra.run.smithi096.stdout:318 write 0x77e6225 thru 0x77e815b (0x1f37 bytes) 2024-04-24T15:32:25.408 INFO:teuthology.orchestra.run.smithi096.stdout:320 read 0x9d394bb thru 0x9d3ba8a (0x25d0 bytes) 2024-04-24T15:32:25.408 INFO:teuthology.orchestra.run.smithi096.stdout:322 read 0x47cdf4b thru 0x47dd458 (0xf50e bytes) 2024-04-24T15:32:25.408 INFO:teuthology.orchestra.run.smithi096.stdout:327 write 0xaa98623 thru 0xaa9a579 (0x1f57 bytes) 2024-04-24T15:32:25.412 INFO:teuthology.orchestra.run.smithi096.stdout:328 read 0xad2a894 thru 0xad3978c (0xeef9 bytes) 2024-04-24T15:32:25.412 INFO:teuthology.orchestra.run.smithi096.stdout:329 write 0xc00da66 thru 0xc0131a3 (0x573e bytes) 2024-04-24T15:32:25.413 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-04-24T15:32:25.425 INFO:teuthology.orchestra.run.smithi096.stdout:330 writesame 0xc075681 thru 0xc07e211 data_size 0x1(0x8b91 bytes) 2024-04-24T15:32:25.430 INFO:teuthology.orchestra.run.smithi096.stdout:331 punch from 0xb03417e to 0xb03f516, (0xb398 bytes) 2024-04-24T15:32:25.430 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:25.430 INFO:teuthology.orchestra.run.smithi096.stdout:334 punch from 0xa48a94f to 0xa497375, (0xca26 bytes) 2024-04-24T15:32:25.430 INFO:teuthology.orchestra.run.smithi096.stdout:335 write 0x731d479 thru 0x7321d69 (0x48f1 bytes) 2024-04-24T15:32:25.437 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-04-24T15:32:25.444 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:25.444 INFO:teuthology.orchestra.run.smithi096.stdout:338 write 0xbcd5742 thru 0xbce0481 (0xad40 bytes) 2024-04-24T15:32:25.449 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:32:25.449+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:32:25.459 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-04-24T15:32:25.459 INFO:teuthology.orchestra.run.smithi096.stdout:339 read 0x74f0c96 thru 0x74f85a5 (0x7910 bytes) 2024-04-24T15:32:25.461 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T15:32:25.461 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T15:32:25.461 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T15:32:25.462 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T15:32:25.462 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T15:32:25.462 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T15:32:25.462 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T15:32:25.462 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T15:32:25.467 INFO:teuthology.orchestra.run.smithi096.stdout:340 writesame 0xe09eeb7 thru 0xe0a275c data_size 0x3(0x38a6 bytes) 2024-04-24T15:32:25.467 INFO:teuthology.orchestra.run.smithi096.stdout:341 read 0x8edbbc2 thru 0x8ee8f84 (0xd3c3 bytes) 2024-04-24T15:32:25.467 INFO:teuthology.orchestra.run.smithi096.stdout:342 read 0x3d2a959 thru 0x3d35f70 (0xb618 bytes) 2024-04-24T15:32:25.467 INFO:teuthology.orchestra.run.smithi096.stdout:343 read 0x9ff7c93 thru 0x9ff82ec (0x65a bytes) 2024-04-24T15:32:25.467 INFO:teuthology.orchestra.run.smithi096.stdout:344 writesame 0x33f657d thru 0x3401bd4 data_size 0x3c(0xb658 bytes) 2024-04-24T15:32:25.480 INFO:teuthology.orchestra.run.smithi096.stdout:345 read 0xb1e6c0c thru 0xb1f2e17 (0xc20c bytes) 2024-04-24T15:32:25.480 INFO:teuthology.orchestra.run.smithi096.stdout:346 write 0x267bc72 thru 0x2688d40 (0xd0cf bytes) 2024-04-24T15:32:25.495 INFO:teuthology.orchestra.run.smithi096.stdout:347 write 0x836e586 thru 0x837cbbc (0xe637 bytes) 2024-04-24T15:32:25.512 INFO:teuthology.orchestra.run.smithi096.stdout:348 read 0x417df68 thru 0x4188532 (0xa5cb bytes) 2024-04-24T15:32:25.512 INFO:teuthology.orchestra.run.smithi096.stdout:350 read 0x20f4d2b thru 0x2103ac1 (0xed97 bytes) 2024-04-24T15:32:25.512 INFO:teuthology.orchestra.run.smithi096.stdout:351 write 0x7e3f672 thru 0x7e44eba (0x5849 bytes) 2024-04-24T15:32:25.531 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:25.531 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:25.540 INFO:teuthology.orchestra.run.smithi096.stdout:352 trunc from 0xe0a275d to 0x3356192 2024-04-24T15:32:25.548 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-24T15:32:25.549 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:32:25.549+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:32:25.592 INFO:teuthology.orchestra.run.smithi096.stdout:353 write 0xe2f2e59 thru 0xe2f8048 (0x51f0 bytes) 2024-04-24T15:32:25.604 INFO:teuthology.orchestra.run.smithi096.stdout:354 write 0xa6d695c thru 0xa6d7e1e (0x14c3 bytes) 2024-04-24T15:32:25.608 INFO:teuthology.orchestra.run.smithi096.stdout:355 punch from 0xbd4031d to 0xbd484d7, (0x81ba bytes) 2024-04-24T15:32:25.609 INFO:teuthology.orchestra.run.smithi096.stdout:356 writesame 0x57527d7 thru 0x575ee31 data_size 0x1d(0xc65b bytes) 2024-04-24T15:32:25.620 INFO:teuthology.orchestra.run.smithi096.stdout:357 writesame 0x469b136 thru 0x469c318 data_size 0x1(0x11e3 bytes) 2024-04-24T15:32:25.628 INFO:teuthology.orchestra.run.smithi096.stdout:361 read 0x5bfb99f thru 0x5c045a9 (0x8c0b bytes) 2024-04-24T15:32:25.641 INFO:teuthology.orchestra.run.smithi096.stdout:364 trunc from 0xe2f8049 to 0x36c7162 2024-04-24T15:32:25.649 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:32:25.650+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:32:25.661 INFO:teuthology.orchestra.run.smithi096.stdout:366 write 0x35e1053 thru 0x35ea8d3 (0x9881 bytes) 2024-04-24T15:32:25.683 INFO:teuthology.orchestra.run.smithi096.stdout:367 trunc from 0x36c7162 to 0x97aed96 2024-04-24T15:32:25.693 INFO:teuthology.orchestra.run.smithi096.stdout:368 trunc from 0x97aed96 to 0x4bea5b1 2024-04-24T15:32:25.698 INFO:teuthology.orchestra.run.smithi096.stdout:369 read 0xf28942 thru 0xf28b7e (0x23d bytes) 2024-04-24T15:32:25.699 INFO:teuthology.orchestra.run.smithi096.stdout:370 write 0x2dd35f2 thru 0x2de2db9 (0xf7c8 bytes) 2024-04-24T15:32:25.715 INFO:teuthology.orchestra.run.smithi096.stdout:371 read 0x3ca2c48 thru 0x3ca9d96 (0x714f bytes) 2024-04-24T15:32:25.715 INFO:teuthology.orchestra.run.smithi096.stdout:373 punch from 0x256a23a to 0x2576d38, (0xcafe bytes) 2024-04-24T15:32:25.715 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:25.723 INFO:teuthology.orchestra.run.smithi096.stdout:375 trunc from 0x4bea5b1 to 0xb3b1882 2024-04-24T15:32:25.728 INFO:teuthology.orchestra.run.smithi096.stdout:378 read 0x9ed6f9e thru 0x9ed8979 (0x19dc bytes) 2024-04-24T15:32:25.729 INFO:teuthology.orchestra.run.smithi096.stdout:379 read 0x7e9ee13 thru 0x7eab922 (0xcb10 bytes) 2024-04-24T15:32:25.729 INFO:teuthology.orchestra.run.smithi096.stdout:380 punch from 0x6833d8c to 0x6840e0a, (0xd07e bytes) 2024-04-24T15:32:25.729 INFO:teuthology.orchestra.run.smithi096.stdout:381 write 0x432b09 thru 0x43d05d (0xa555 bytes) 2024-04-24T15:32:25.729 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:25.729 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:25.741 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-24T15:32:25.747 INFO:teuthology.orchestra.run.smithi096.stdout:385 read 0x46f5520 thru 0x46f5545 (0x26 bytes) 2024-04-24T15:32:25.750 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:32:25.751+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:32:25.754 INFO:teuthology.orchestra.run.smithi096.stdout:386 writesame 0xca7db90 thru 0xca843bb data_size 0x3b(0x682c bytes) 2024-04-24T15:32:25.754 INFO:teuthology.orchestra.run.smithi096.stdout:387 read 0x941259c thru 0x9420a5b (0xe4c0 bytes) 2024-04-24T15:32:25.754 INFO:teuthology.orchestra.run.smithi096.stdout:388 write 0x6f071dd thru 0x6f15bfd (0xea21 bytes) 2024-04-24T15:32:25.768 INFO:teuthology.orchestra.run.smithi096.stdout:390 write 0x49d3bb7 thru 0x49ddb53 (0x9f9d bytes) 2024-04-24T15:32:25.778 INFO:teuthology.orchestra.run.smithi096.stdout:393 read 0xa7fc2d7 thru 0xa802a97 (0x67c1 bytes) 2024-04-24T15:32:25.778 INFO:teuthology.orchestra.run.smithi096.stdout:395 punch from 0x84b2172 to 0x84b9e3f, (0x7ccd bytes) 2024-04-24T15:32:25.778 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:25.779 INFO:teuthology.orchestra.run.smithi096.stdout:398 punch from 0xb31b458 to 0xb326536, (0xb0de bytes) 2024-04-24T15:32:25.779 INFO:teuthology.orchestra.run.smithi096.stdout:399 write 0x725dd98 thru 0x726ce67 (0xf0d0 bytes) 2024-04-24T15:32:25.791 INFO:teuthology.orchestra.run.smithi096.stdout:401 write 0x66f07e7 thru 0x66f9a67 (0x9281 bytes) 2024-04-24T15:32:25.804 INFO:teuthology.orchestra.run.smithi096.stdout:402 punch from 0x611fa03 to 0x6123617, (0x3c14 bytes) 2024-04-24T15:32:25.804 INFO:teuthology.orchestra.run.smithi096.stdout:403 write 0x604790d thru 0x6052e8e (0xb582 bytes) 2024-04-24T15:32:25.819 INFO:teuthology.orchestra.run.smithi096.stdout:404 write 0x656d591 thru 0x657a54e (0xcfbe bytes) 2024-04-24T15:32:25.825 INFO:teuthology.orchestra.run.smithi096.stdout:407 read 0x663cdba thru 0x6644676 (0x78bd bytes) 2024-04-24T15:32:25.834 INFO:teuthology.orchestra.run.smithi096.stdout:408 write 0xeac4ffd thru 0xead1ac1 (0xcac5 bytes) 2024-04-24T15:32:25.851 INFO:teuthology.orchestra.run.smithi096.stdout:410 read 0xc4d3dc3 thru 0xc4d9863 (0x5aa1 bytes) 2024-04-24T15:32:25.851 INFO:teuthology.orchestra.run.smithi096.stdout:412 read 0x2ffdb3c thru 0x3009df0 (0xc2b5 bytes) 2024-04-24T15:32:25.851 INFO:teuthology.orchestra.run.smithi096.stdout:413 write 0x93e00e5 thru 0x93e76b4 (0x75d0 bytes) 2024-04-24T15:32:25.852 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:32:25.852+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:32:25.867 INFO:teuthology.orchestra.run.smithi096.stdout:414 write 0xccd7e23 thru 0xcce2253 (0xa431 bytes) 2024-04-24T15:32:25.882 INFO:teuthology.orchestra.run.smithi096.stdout:415 write 0x18f87f3 thru 0x1907d09 (0xf517 bytes) 2024-04-24T15:32:25.884 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:25.885 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:25.896 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-24T15:32:25.896 INFO:teuthology.orchestra.run.smithi096.stdout:418 write 0x3458f84 thru 0x345dd4e (0x4dcb bytes) 2024-04-24T15:32:25.907 INFO:teuthology.orchestra.run.smithi096.stdout:419 trunc from 0xead1ac2 to 0x8aaa7b5 2024-04-24T15:32:25.937 INFO:teuthology.orchestra.run.smithi096.stdout:420 trunc from 0x8aaa7b5 to 0xface6a 2024-04-24T15:32:25.952 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:32:25.953+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:32:26.003 INFO:teuthology.orchestra.run.smithi096.stdout:422 write 0xbbdb153 thru 0xbbdbfdd (0xe8b bytes) 2024-04-24T15:32:26.009 INFO:teuthology.orchestra.run.smithi096.stdout:423 trunc from 0xbbdbfde to 0xc5b862e 2024-04-24T15:32:26.020 INFO:teuthology.orchestra.run.smithi096.stdout:424 trunc from 0xc5b862e to 0x7b026d7 2024-04-24T15:32:26.038 INFO:teuthology.orchestra.run.smithi096.stdout:425 write 0x919eba8 thru 0x91a3429 (0x4882 bytes) 2024-04-24T15:32:26.051 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:26.051 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:26.052 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:32:26.054+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:32:26.062 INFO:teuthology.orchestra.run.smithi096.stdout:426 write 0xc20b4db thru 0xc21a64c (0xf172 bytes) 2024-04-24T15:32:26.065 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-24T15:32:26.076 INFO:teuthology.orchestra.run.smithi096.stdout:427 trunc from 0xc21a64d to 0xbf03e4f 2024-04-24T15:32:26.097 INFO:teuthology.orchestra.run.smithi096.stderr:nodeep-scrub is unset 2024-04-24T15:32:26.100 INFO:teuthology.orchestra.run.smithi096.stdout:429 read 0x8d8bec4 thru 0x8d91408 (0x5545 bytes) 2024-04-24T15:32:26.100 INFO:teuthology.orchestra.run.smithi096.stdout:430 write 0x94dab29 thru 0x94e7fd0 (0xd4a8 bytes) 2024-04-24T15:32:26.112 INFO:teuthology.orchestra.run.smithi096.stdout:431 writesame 0xadbed7c thru 0xadc9669 data_size 0x7(0xa8ee bytes) 2024-04-24T15:32:26.124 INFO:teuthology.orchestra.run.smithi096.stdout:434 write 0x928606 thru 0x929a63 (0x145e bytes) 2024-04-24T15:32:26.126 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:26.126 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:26.126 INFO:teuthology.orchestra.run.smithi096.stdout:437 trunc from 0xbf03e4f to 0xc5192bb 2024-04-24T15:32:26.131 INFO:teuthology.orchestra.run.smithi096.stdout:438 read 0x80846da thru 0x8084c52 (0x579 bytes) 2024-04-24T15:32:26.131 INFO:teuthology.orchestra.run.smithi096.stdout:441 write 0xbe2f1e9 thru 0xbe3461e (0x5436 bytes) 2024-04-24T15:32:26.144 INFO:teuthology.orchestra.run.smithi096.stdout:442 trunc from 0xc5192bb to 0xa5a9a7d 2024-04-24T15:32:26.153 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:32:26.154+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:32:26.159 INFO:teuthology.orchestra.run.smithi096.stdout:443 punch from 0x11c2b78 to 0x11cea39, (0xbec1 bytes) 2024-04-24T15:32:26.166 INFO:teuthology.orchestra.run.smithi096.stdout:445 trunc from 0xa5a9a7d to 0x476391c 2024-04-24T15:32:26.184 INFO:teuthology.orchestra.run.smithi096.stdout:446 read 0x191b96c thru 0x191ba2c (0xc1 bytes) 2024-04-24T15:32:26.185 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:26.185 INFO:teuthology.orchestra.run.smithi096.stdout:448 punch from 0xc4e480 to 0xc50567, (0x20e7 bytes) 2024-04-24T15:32:26.192 INFO:teuthology.orchestra.run.smithi096.stdout:449 write 0x6aff7cc thru 0x6b044d9 (0x4d0e bytes) 2024-04-24T15:32:26.209 INFO:teuthology.orchestra.run.smithi096.stdout:450 flatten 2024-04-24T15:32:26.214 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:26.214 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:26.224 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-24T15:32:26.259 INFO:teuthology.orchestra.run.smithi096.stdout:451 write 0x7c20da thru 0x7c75d0 (0x54f7 bytes) 2024-04-24T15:32:26.264 INFO:teuthology.orchestra.run.smithi096.stdout:452 punch from 0x523e6a to 0x533c2f, (0xfdc5 bytes) 2024-04-24T15:32:26.264 INFO:teuthology.orchestra.run.smithi096.stdout:454 writesame 0x1aaa69b thru 0x1aac722 data_size 0xc(0x2088 bytes) 2024-04-24T15:32:26.268 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:32:26.270+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:32:26.276 INFO:teuthology.orchestra.run.smithi096.stdout:456 read 0x32d557d thru 0x32d6603 (0x1087 bytes) 2024-04-24T15:32:26.276 INFO:teuthology.orchestra.run.smithi096.stdout:457 read 0x3b15f08 thru 0x3b22b15 (0xcc0e bytes) 2024-04-24T15:32:26.276 INFO:teuthology.orchestra.run.smithi096.stdout:458 read 0x470d67b thru 0x471608e (0x8a14 bytes) 2024-04-24T15:32:26.276 INFO:teuthology.orchestra.run.smithi096.stdout:459 read 0x9ae9a0 thru 0x9b4702 (0x5d63 bytes) 2024-04-24T15:32:26.277 INFO:teuthology.orchestra.run.smithi096.stdout:460 punch from 0x383762 to 0x383b5f, (0x3fd bytes) 2024-04-24T15:32:26.277 INFO:teuthology.orchestra.run.smithi096.stdout:462 read 0x1c63cab thru 0x1c6add3 (0x7129 bytes) 2024-04-24T15:32:26.277 INFO:teuthology.orchestra.run.smithi096.stdout:464 write 0x3b63860 thru 0x3b6e52d (0xacce bytes) 2024-04-24T15:32:26.304 INFO:teuthology.orchestra.run.smithi096.stdout:465 write 0x8d7ec67 thru 0x8d8a45d (0xb7f7 bytes) 2024-04-24T15:32:26.317 INFO:teuthology.orchestra.run.smithi096.stdout:466 writesame 0x26b7665 thru 0x26bd9c7 data_size 0x21(0x6363 bytes) 2024-04-24T15:32:26.329 INFO:teuthology.orchestra.run.smithi096.stdout:467 trunc from 0x8d8a45e to 0x9a1f07c 2024-04-24T15:32:26.334 INFO:teuthology.orchestra.run.smithi096.stdout:468 write 0x2f8f1b4 thru 0x2f908ab (0x16f8 bytes) 2024-04-24T15:32:26.339 INFO:teuthology.orchestra.run.smithi096.stdout:470 read 0x35cc6fc thru 0x35cf3be (0x2cc3 bytes) 2024-04-24T15:32:26.339 INFO:teuthology.orchestra.run.smithi096.stdout:471 read 0x7e5bd2f thru 0x7e658ff (0x9bd1 bytes) 2024-04-24T15:32:26.343 INFO:teuthology.orchestra.run.smithi096.stdout:472 trunc from 0x9a1f07c to 0x67b71f3 2024-04-24T15:32:26.354 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:32:26.355+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:32:26.366 INFO:teuthology.orchestra.run.smithi096.stdout:473 write 0xaf575bd thru 0xaf61dd9 (0xa81d bytes) 2024-04-24T15:32:26.366 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:26.367 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:26.379 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-24T15:32:26.381 INFO:teuthology.orchestra.run.smithi096.stdout:474 writesame 0x5c51b4c thru 0x5c5bf88 data_size 0x5(0xa43d bytes) 2024-04-24T15:32:26.396 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:26.396 INFO:teuthology.orchestra.run.smithi096.stdout:477 write 0xa1b4d41 thru 0xa1bd6aa (0x896a bytes) 2024-04-24T15:32:26.412 INFO:teuthology.orchestra.run.smithi096.stdout:480 trunc from 0xaf61dda to 0xc62a118 2024-04-24T15:32:26.417 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:26.417 INFO:teuthology.orchestra.run.smithi096.stdout:482 read 0x2b31ba4 thru 0x2b3a0f8 (0x8555 bytes) 2024-04-24T15:32:26.418 INFO:teuthology.orchestra.run.smithi096.stdout:483 write 0x14affa9 thru 0x14bad50 (0xada8 bytes) 2024-04-24T15:32:26.430 INFO:teuthology.orchestra.run.smithi096.stdout:484 read 0x6d75651 thru 0x6d7d87e (0x822e bytes) 2024-04-24T15:32:26.431 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:26.431 INFO:teuthology.orchestra.run.smithi096.stdout:486 read 0xa6420ed thru 0xa64f10a (0xd01e bytes) 2024-04-24T15:32:26.431 INFO:teuthology.orchestra.run.smithi096.stdout:487 punch from 0x46bdb33 to 0x46cc2a7, (0xe774 bytes) 2024-04-24T15:32:26.431 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:26.431 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:26.431 INFO:teuthology.orchestra.run.smithi096.stdout:490 write 0x7bbbc2c thru 0x7bbcd28 (0x10fd bytes) 2024-04-24T15:32:26.440 INFO:teuthology.orchestra.run.smithi096.stdout:491 punch from 0x1543763 to 0x1552930, (0xf1cd bytes) 2024-04-24T15:32:26.440 INFO:teuthology.orchestra.run.smithi096.stdout:492 write 0x944a030 thru 0x944cd12 (0x2ce3 bytes) 2024-04-24T15:32:26.453 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:26.454 INFO:teuthology.orchestra.run.smithi096.stdout:495 read 0x4e7d5bb thru 0x4e840e8 (0x6b2e bytes) 2024-04-24T15:32:26.454 INFO:teuthology.orchestra.run.smithi096.stdout:496 writesame 0x907f571 thru 0x908f344 data_size 0x72(0xfdd4 bytes) 2024-04-24T15:32:26.454 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:32:26.456+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:32:26.465 INFO:teuthology.orchestra.run.smithi096.stdout:497 read 0x3925da8 thru 0x3928028 (0x2281 bytes) 2024-04-24T15:32:26.465 INFO:teuthology.orchestra.run.smithi096.stdout:499 read 0x56f14e5 thru 0x56f3c15 (0x2731 bytes) 2024-04-24T15:32:26.465 INFO:teuthology.orchestra.run.smithi096.stdout:500 read 0x838e8f2 thru 0x839e05a (0xf769 bytes) 2024-04-24T15:32:26.465 INFO:teuthology.orchestra.run.smithi096.stdout:501 writesame 0x41d2da2 thru 0x41d8e6d data_size 0x15(0x60cc bytes) 2024-04-24T15:32:26.481 INFO:teuthology.orchestra.run.smithi096.stdout:502 write 0x1cb18db thru 0x1cb3481 (0x1ba7 bytes) 2024-04-24T15:32:26.487 INFO:teuthology.orchestra.run.smithi096.stdout:504 trunc from 0xc62a118 to 0xcc71f2a 2024-04-24T15:32:26.492 INFO:teuthology.orchestra.run.smithi096.stdout:505 write 0x3741b7f thru 0x374fbe4 (0xe066 bytes) 2024-04-24T15:32:26.515 INFO:teuthology.orchestra.run.smithi096.stdout:506 trunc from 0xcc71f2a to 0x23a8019 2024-04-24T15:32:26.555 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:32:26.556+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:32:26.566 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:26.566 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:26.577 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-24T15:32:26.583 INFO:teuthology.orchestra.run.smithi096.stdout:507 write 0xe94529e thru 0xe953532 (0xe295 bytes) 2024-04-24T15:32:26.599 INFO:teuthology.orchestra.run.smithi096.stdout:508 trunc from 0xe953533 to 0xed6ba95 2024-04-24T15:32:26.604 INFO:teuthology.orchestra.run.smithi096.stdout:509 punch from 0xc9e0bd9 to 0xc9ea64b, (0x9a72 bytes) 2024-04-24T15:32:26.604 INFO:teuthology.orchestra.run.smithi096.stdout:510 writesame 0xa87cda1 thru 0xa88635d data_size 0x1(0x95bd bytes) 2024-04-24T15:32:26.641 INFO:teuthology.orchestra.run.smithi096.stdout:511 trunc from 0xed6ba95 to 0x2e13299 2024-04-24T15:32:26.656 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:32:26.658+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:32:26.660 INFO:teuthology.orchestra.run.smithi096.stdout:512 read 0x14b5b54 thru 0x14c4437 (0xe8e4 bytes) 2024-04-24T15:32:26.660 INFO:teuthology.orchestra.run.smithi096.stdout:513 read 0x2bb08a thru 0x2c5133 (0xa0aa bytes) 2024-04-24T15:32:26.661 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:26.661 INFO:teuthology.orchestra.run.smithi096.stdout:516 write 0x1a1df91 thru 0x1a1f686 (0x16f6 bytes) 2024-04-24T15:32:26.678 INFO:teuthology.orchestra.run.smithi096.stdout:518 write 0x90513af thru 0x906043e (0xf090 bytes) 2024-04-24T15:32:26.689 INFO:teuthology.orchestra.run.smithi096.stdout:519 read 0x25c9fe9 thru 0x25cd92d (0x3945 bytes) 2024-04-24T15:32:26.689 INFO:teuthology.orchestra.run.smithi096.stdout:520 read 0x2658c8 thru 0x270bff (0xb338 bytes) 2024-04-24T15:32:26.689 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:26.698 INFO:teuthology.orchestra.run.smithi096.stdout:523 writesame 0xbecfcdb thru 0xbed7064 data_size 0x17(0x738a bytes) 2024-04-24T15:32:26.700 INFO:teuthology.orchestra.run.smithi096.stdout:525 trunc from 0xbed7065 to 0xd4802d0 2024-04-24T15:32:26.705 INFO:teuthology.orchestra.run.smithi096.stdout:526 write 0xab24865 thru 0xab2e461 (0x9bfd bytes) 2024-04-24T15:32:26.717 INFO:teuthology.orchestra.run.smithi096.stdout:529 punch from 0xab63abf to 0xab6934a, (0x588b bytes) 2024-04-24T15:32:26.717 INFO:teuthology.orchestra.run.smithi096.stdout:532 read 0x5c52964 thru 0x5c61f50 (0xf5ed bytes) 2024-04-24T15:32:26.718 INFO:teuthology.orchestra.run.smithi096.stdout:533 read 0x18aa7ab thru 0x18b5910 (0xb166 bytes) 2024-04-24T15:32:26.718 INFO:teuthology.orchestra.run.smithi096.stdout:534 clone 3 order 25 su 2097152 sc 2 2024-04-24T15:32:26.721 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:26.721 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:26.731 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-24T15:32:26.774 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:32:26.776+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:32:26.854 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:26.854 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:26.859 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:32:26.860+0000 7f23abcf6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 44021) UID: 0 2024-04-24T15:32:26.864 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-24T15:32:26.960 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:32:26.961+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:32:27.026 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:27.026 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:27.036 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-24T15:32:27.078 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:32:27.079+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:32:27.161 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:32:27.163+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:32:27.204 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:27.204 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:27.213 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-24T15:32:27.279 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #1, image image_client.0-clone1 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent2 2024-04-24T15:32:27.297 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:32:27.299+0000 7f23abcf6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 44021) UID: 0 2024-04-24T15:32:27.337 INFO:teuthology.orchestra.run.smithi096.stdout:536 write 0x845dba4 thru 0x845e1eb (0x648 bytes) 2024-04-24T15:32:27.350 INFO:teuthology.orchestra.run.smithi096.stdout:538 write 0x209e31e thru 0x209efbd (0xca0 bytes) 2024-04-24T15:32:27.363 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:32:27.364+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:32:27.364 INFO:teuthology.orchestra.run.smithi096.stdout:539 read 0xd2111fd thru 0xd216ae8 (0x58ec bytes) 2024-04-24T15:32:27.364 INFO:teuthology.orchestra.run.smithi096.stdout:540 write 0x18b1464 thru 0x18ba8d7 (0x9474 bytes) 2024-04-24T15:32:27.366 INFO:teuthology.orchestra.run.smithi096.stdout:541 punch from 0xcd6dc67 to 0xcd76991, (0x8d2a bytes) 2024-04-24T15:32:27.366 INFO:teuthology.orchestra.run.smithi096.stdout:542 write 0x9604166 thru 0x960d11c (0x8fb7 bytes) 2024-04-24T15:32:27.373 INFO:teuthology.orchestra.run.smithi096.stdout:544 write 0x77339eb thru 0x773d33b (0x9951 bytes) 2024-04-24T15:32:27.382 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:27.382 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:27.389 INFO:teuthology.orchestra.run.smithi096.stdout:545 trunc from 0xd4802d0 to 0x56a9617 2024-04-24T15:32:27.392 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-24T15:32:27.408 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:27.410 INFO:teuthology.orchestra.run.smithi096.stdout:548 trunc from 0x56a9617 to 0x7633743 2024-04-24T15:32:27.419 INFO:teuthology.orchestra.run.smithi096.stdout:549 trunc from 0x7633743 to 0x8b9f7d3 2024-04-24T15:32:27.425 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:27.432 INFO:teuthology.orchestra.run.smithi096.stdout:552 write 0x91630c6 thru 0x9166504 (0x343f bytes) 2024-04-24T15:32:27.439 INFO:teuthology.orchestra.run.smithi096.stdout:553 write 0x91d3b14 thru 0x91d6a23 (0x2f10 bytes) 2024-04-24T15:32:27.441 INFO:teuthology.orchestra.run.smithi096.stdout:554 read 0x8b817b7 thru 0x8b8eb6b (0xd3b5 bytes) 2024-04-24T15:32:27.450 INFO:teuthology.orchestra.run.smithi096.stdout:555 trunc from 0x91d6a24 to 0xdc3c19 2024-04-24T15:32:27.463 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:32:27.464+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:32:27.469 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:27.469 INFO:teuthology.orchestra.run.smithi096.stdout:558 read 0xccfa53 thru 0xcd6cfd (0x72ab bytes) 2024-04-24T15:32:27.474 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-24T15:32:27.477 INFO:teuthology.orchestra.run.smithi096.stdout:560 trunc from 0xdc3c19 to 0x6373e64 2024-04-24T15:32:27.483 INFO:teuthology.orchestra.run.smithi096.stdout:561 write 0x5ac1008 thru 0x5ac169b (0x694 bytes) 2024-04-24T15:32:27.485 INFO:teuthology.orchestra.run.smithi096.stdout:562 read 0x580283d thru 0x580698b (0x414f bytes) 2024-04-24T15:32:27.488 INFO:teuthology.orchestra.run.smithi096.stdout:564 trunc from 0x6373e64 to 0x819776e 2024-04-24T15:32:27.494 INFO:teuthology.orchestra.run.smithi096.stdout:566 writesame 0x283e47 thru 0x28cf74 data_size 0x1(0x912e bytes) 2024-04-24T15:32:27.521 INFO:teuthology.orchestra.run.smithi096.stdout:567 write 0x95ba0ca thru 0x95ba85d (0x794 bytes) 2024-04-24T15:32:27.532 INFO:teuthology.orchestra.run.smithi096.stdout:568 trunc from 0x95ba85e to 0x396e3fe 2024-04-24T15:32:27.543 INFO:teuthology.orchestra.run.smithi096.stdout:569 read 0x360cc4b thru 0x360db88 (0xf3e bytes) 2024-04-24T15:32:27.543 INFO:teuthology.orchestra.run.smithi096.stdout:572 punch from 0x7df4bf to 0x7e5a43, (0x6584 bytes) 2024-04-24T15:32:27.546 INFO:teuthology.orchestra.run.smithi096.stdout:574 trunc from 0x396e3fe to 0x1716e99 2024-04-24T15:32:27.552 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:27.561 INFO:teuthology.orchestra.run.smithi096.stdout:576 trunc from 0x1716e99 to 0x90cf3f2 2024-04-24T15:32:27.564 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:27.564 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:27.565 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:32:27.565+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:32:27.566 INFO:teuthology.orchestra.run.smithi096.stdout:577 read 0x267cc4f thru 0x267e52c (0x18de bytes) 2024-04-24T15:32:27.574 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-24T15:32:27.575 INFO:teuthology.orchestra.run.smithi096.stdout:578 trunc from 0x90cf3f2 to 0x2a3a742 2024-04-24T15:32:27.583 INFO:teuthology.orchestra.run.smithi096.stdout:580 punch from 0x12714bb to 0x127c1fc, (0xad41 bytes) 2024-04-24T15:32:27.583 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:27.596 INFO:teuthology.orchestra.run.smithi096.stdout:583 trunc from 0x2a3a742 to 0xd64c8fd 2024-04-24T15:32:27.602 INFO:teuthology.orchestra.run.smithi096.stdout:584 write 0x4247420 thru 0x4254133 (0xcd14 bytes) 2024-04-24T15:32:27.606 INFO:teuthology.orchestra.run.smithi096.stdout:586 writesame 0xc33f5de thru 0xc3477d4 data_size 0x31(0x81f7 bytes) 2024-04-24T15:32:27.609 INFO:teuthology.orchestra.run.smithi096.stdout:587 write 0x381f3f5 thru 0x38258c6 (0x64d2 bytes) 2024-04-24T15:32:27.611 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:27.611 INFO:teuthology.orchestra.run.smithi096.stdout:589 read 0x5bdc17c thru 0x5be028b (0x4110 bytes) 2024-04-24T15:32:27.612 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:27.612 INFO:teuthology.orchestra.run.smithi096.stdout:592 write 0xae4ef80 thru 0xae523bb (0x343c bytes) 2024-04-24T15:32:27.615 INFO:teuthology.orchestra.run.smithi096.stdout:593 punch from 0x8ad3ac to 0x8b950d, (0xc161 bytes) 2024-04-24T15:32:27.615 INFO:teuthology.orchestra.run.smithi096.stdout:594 punch from 0x8463c71 to 0x8470706, (0xca95 bytes) 2024-04-24T15:32:27.615 INFO:teuthology.orchestra.run.smithi096.stdout:596 writesame 0x2085192 thru 0x2092003 data_size 0x46(0xce72 bytes) 2024-04-24T15:32:27.616 INFO:teuthology.orchestra.run.smithi096.stdout:597 writesame 0xc12c7b2 thru 0xc12e290 data_size 0x1(0x1adf bytes) 2024-04-24T15:32:27.621 INFO:teuthology.orchestra.run.smithi096.stdout:598 punch from 0xd5a3a5c to 0xd5a4687, (0xc2b bytes) 2024-04-24T15:32:27.621 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:27.622 INFO:teuthology.orchestra.run.smithi096.stdout:600 writesame 0x3a1f0a8 thru 0x3a1fa94 data_size 0xb(0x9ed bytes) 2024-04-24T15:32:27.631 INFO:teuthology.orchestra.run.smithi096.stdout:602 trunc from 0xd64c8fd to 0x6fe92df 2024-04-24T15:32:27.633 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-04-24T15:32:27.654 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-04-24T15:32:27.660 INFO:teuthology.orchestra.run.smithi096.stdout:603 write 0xad52f22 thru 0xad53463 (0x542 bytes) 2024-04-24T15:32:27.663 INFO:teuthology.orchestra.run.smithi096.stdout:605 read 0x59501cb thru 0x5952eb2 (0x2ce8 bytes) 2024-04-24T15:32:27.663 INFO:teuthology.orchestra.run.smithi096.stdout:607 read 0x56069a2 thru 0x560a82a (0x3e89 bytes) 2024-04-24T15:32:27.666 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:32:27.667+0000 7f23abcf6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 44021) UID: 0 2024-04-24T15:32:27.672 INFO:teuthology.orchestra.run.smithi096.stdout:608 write 0xd9a7135 thru 0xd9ac55e (0x542a bytes) 2024-04-24T15:32:27.673 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-04-24T15:32:27.677 INFO:teuthology.orchestra.run.smithi096.stdout:609 read 0x685c0f1 thru 0x686a5fd (0xe50d bytes) 2024-04-24T15:32:27.678 INFO:teuthology.orchestra.run.smithi096.stdout:610 write 0x401bd84 thru 0x402b39e (0xf61b bytes) 2024-04-24T15:32:27.679 INFO:teuthology.orchestra.run.smithi096.stdout:612 write 0x26bb21c thru 0x26cad61 (0xfb46 bytes) 2024-04-24T15:32:27.693 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-04-24T15:32:27.700 INFO:teuthology.orchestra.run.smithi096.stdout:614 trunc from 0xd9ac55f to 0xdfcb8b 2024-04-24T15:32:27.707 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:27.707 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:27.715 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-04-24T15:32:27.719 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-24T15:32:27.731 INFO:teuthology.orchestra.run.smithi096.stdout:615 writesame 0xbe5b44a thru 0xbe6760f data_size 0x2(0xc1c6 bytes) 2024-04-24T15:32:27.731 INFO:teuthology.orchestra.run.smithi096.stdout:616 write 0x7f71bdb thru 0x7f7e440 (0xc866 bytes) 2024-04-24T15:32:27.734 INFO:teuthology.orchestra.run.smithi096.stdout:617 punch from 0x667dd4f to 0x668285c, (0x4b0d bytes) 2024-04-24T15:32:27.734 INFO:teuthology.orchestra.run.smithi096.stdout:619 write 0xa8a3c53 thru 0xa8a985e (0x5c0c bytes) 2024-04-24T15:32:27.735 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-04-24T15:32:27.736 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:27.736 INFO:teuthology.orchestra.run.smithi096.stdout:622 write 0x129f2ad thru 0x12ac5dd (0xd331 bytes) 2024-04-24T15:32:27.738 INFO:teuthology.orchestra.run.smithi096.stdout:623 punch from 0x4023f5a to 0x40280d6, (0x417c bytes) 2024-04-24T15:32:27.738 INFO:teuthology.orchestra.run.smithi096.stdout:624 read 0x46b67db thru 0x46ba5ca (0x3df0 bytes) 2024-04-24T15:32:27.738 INFO:teuthology.orchestra.run.smithi096.stdout:625 punch from 0x1086917 to 0x1087db1, (0x149a bytes) 2024-04-24T15:32:27.738 INFO:teuthology.orchestra.run.smithi096.stdout:626 write 0x1d62c0a thru 0x1d6df16 (0xb30d bytes) 2024-04-24T15:32:27.740 INFO:teuthology.orchestra.run.smithi096.stdout:627 read 0x6ad44cf thru 0x6ae066d (0xc19f bytes) 2024-04-24T15:32:27.740 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:27.740 INFO:teuthology.orchestra.run.smithi096.stdout:630 read 0xb918049 thru 0xb9246a7 (0xc65f bytes) 2024-04-24T15:32:27.741 INFO:teuthology.orchestra.run.smithi096.stdout:631 writesame 0x14a58b2 thru 0x14a6801 data_size 0x7(0xf50 bytes) 2024-04-24T15:32:27.743 INFO:teuthology.orchestra.run.smithi096.stdout:632 trunc from 0xbe67610 to 0xb0a7f8a 2024-04-24T15:32:27.749 INFO:teuthology.orchestra.run.smithi096.stdout:634 read 0x28d47d0 thru 0x28dab55 (0x6386 bytes) 2024-04-24T15:32:27.749 INFO:teuthology.orchestra.run.smithi096.stdout:636 punch from 0x6529877 to 0x652d144, (0x38cd bytes) 2024-04-24T15:32:27.749 INFO:teuthology.orchestra.run.smithi096.stdout:637 write 0x9d850e2 thru 0x9d8e850 (0x976f bytes) 2024-04-24T15:32:27.751 INFO:teuthology.orchestra.run.smithi096.stdout:639 writesame 0x5dab58 thru 0x5de715 data_size 0x3(0x3bbe bytes) 2024-04-24T15:32:27.754 INFO:teuthology.orchestra.run.smithi096.stdout:640 read 0x4a8980a thru 0x4a8a38c (0xb83 bytes) 2024-04-24T15:32:27.755 INFO:teuthology.orchestra.run.smithi096.stdout:642 read 0x97c424b thru 0x97d19f8 (0xd7ae bytes) 2024-04-24T15:32:27.755 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:27.755 INFO:teuthology.orchestra.run.smithi096.stdout:648 writesame 0x6ad0b26 thru 0x6adf4a9 data_size 0xa(0xe984 bytes) 2024-04-24T15:32:27.755 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-04-24T15:32:27.758 INFO:teuthology.orchestra.run.smithi096.stdout:652 writesame 0x2a86359 thru 0x2a88637 data_size 0x1(0x22df bytes) 2024-04-24T15:32:27.762 INFO:teuthology.orchestra.run.smithi096.stdout:653 writesame 0x43ea62 thru 0x44007a data_size 0x1(0x1619 bytes) 2024-04-24T15:32:27.766 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:27.766 INFO:teuthology.orchestra.run.smithi096.stdout:655 read 0x8893bbd thru 0x88945a5 (0x9e9 bytes) 2024-04-24T15:32:27.766 INFO:teuthology.orchestra.run.smithi096.stdout:656 write 0x12d5b21 thru 0x12dd744 (0x7c24 bytes) 2024-04-24T15:32:27.768 INFO:teuthology.orchestra.run.smithi096.stdout:658 writesame 0x6c212a3 thru 0x6c28a9b data_size 0x1(0x77f9 bytes) 2024-04-24T15:32:27.775 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-04-24T15:32:27.776 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T15:32:27.776 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T15:32:27.776 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T15:32:27.776 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T15:32:27.776 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T15:32:27.776 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T15:32:27.776 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T15:32:27.776 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T15:32:27.786 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:27.786 INFO:teuthology.orchestra.run.smithi096.stdout:660 read 0x925d6c6 thru 0x925d965 (0x2a0 bytes) 2024-04-24T15:32:27.786 INFO:teuthology.orchestra.run.smithi096.stdout:661 write 0xadbcae thru 0xae777b (0xbace bytes) 2024-04-24T15:32:27.788 INFO:teuthology.orchestra.run.smithi096.stdout:663 read 0x4a208a thru 0x4a4f8b (0x2f02 bytes) 2024-04-24T15:32:27.788 INFO:teuthology.orchestra.run.smithi096.stdout:664 punch from 0x4d92b4c to 0x4d97641, (0x4af5 bytes) 2024-04-24T15:32:27.788 INFO:teuthology.orchestra.run.smithi096.stdout:665 write 0xa3892 thru 0xb333c (0xfaab bytes) 2024-04-24T15:32:27.797 INFO:teuthology.orchestra.run.smithi096.stdout:666 writesame 0xc7c810f thru 0xc7c8e28 data_size 0x6(0xd1a bytes) 2024-04-24T15:32:27.797 INFO:teuthology.orchestra.run.smithi096.stdout:667 read 0x6a51334 thru 0x6a61131 (0xfdfe bytes) 2024-04-24T15:32:27.797 INFO:teuthology.orchestra.run.smithi096.stdout:668 writesame 0x1256bd9 thru 0x125f017 data_size 0x3d(0x843f bytes) 2024-04-24T15:32:27.799 INFO:teuthology.orchestra.run.smithi096.stdout:669 writesame 0xbb93331 thru 0xbba1e14 data_size 0x6(0xeae4 bytes) 2024-04-24T15:32:27.801 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:32:27.803+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:32:27.806 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:32:27.807+0000 7faf004a6700 -1 reset not still connected to 0x5565c61ecf70 2024-04-24T15:32:27.806 INFO:teuthology.orchestra.run.smithi096.stdout:670 read 0x28cccb8 thru 0x28d0b91 (0x3eda bytes) 2024-04-24T15:32:27.806 INFO:teuthology.orchestra.run.smithi096.stdout:671 write 0x5b8da2c thru 0x5b9d51b (0xfaf0 bytes) 2024-04-24T15:32:27.813 INFO:teuthology.orchestra.run.smithi096.stdout:673 write 0xd527338 thru 0xd53208c (0xad55 bytes) 2024-04-24T15:32:27.816 INFO:teuthology.orchestra.run.smithi096.stdout:674 write 0x3931cca thru 0x3940b40 (0xee77 bytes) 2024-04-24T15:32:27.818 INFO:teuthology.orchestra.run.smithi096.stdout:676 writesame 0x323b702 thru 0x324a975 data_size 0x76(0xf274 bytes) 2024-04-24T15:32:27.820 INFO:teuthology.orchestra.run.smithi096.stdout:678 punch from 0x74a4a04 to 0x74a9048, (0x4644 bytes) 2024-04-24T15:32:27.820 INFO:teuthology.orchestra.run.smithi096.stdout:679 read 0xbafb0b7 thru 0xbb051bc (0xa106 bytes) 2024-04-24T15:32:27.820 INFO:teuthology.orchestra.run.smithi096.stdout:680 writesame 0x8f1c870 thru 0x8f2589e data_size 0x1(0x902f bytes) 2024-04-24T15:32:27.837 INFO:teuthology.orchestra.run.smithi096.stdout:681 write 0x921a875 thru 0x921c00b (0x1797 bytes) 2024-04-24T15:32:27.838 INFO:teuthology.orchestra.run.smithi096.stdout:682 writesame 0x3fece2 thru 0x401e62 data_size 0x1(0x3181 bytes) 2024-04-24T15:32:27.844 INFO:teuthology.orchestra.run.smithi096.stdout:684 write 0x44be8f4 thru 0x44c59a7 (0x70b4 bytes) 2024-04-24T15:32:27.845 INFO:teuthology.orchestra.run.smithi096.stdout:685 write 0x16688f1 thru 0x16769a5 (0xe0b5 bytes) 2024-04-24T15:32:27.847 INFO:teuthology.orchestra.run.smithi096.stdout:689 write 0x37f906 thru 0x38508d (0x5788 bytes) 2024-04-24T15:32:27.857 INFO:teuthology.orchestra.run.smithi096.stdout:690 trunc from 0xd53208d to 0x5f6e8fb 2024-04-24T15:32:27.867 INFO:teuthology.orchestra.run.smithi096.stdout:692 punch from 0x37141c6 to 0x37219d7, (0xd811 bytes) 2024-04-24T15:32:27.867 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:27.867 INFO:teuthology.orchestra.run.smithi096.stdout:695 write 0x31a0e thru 0x36a47 (0x503a bytes) 2024-04-24T15:32:27.868 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:32:27.868+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:32:27.873 INFO:teuthology.orchestra.run.smithi096.stdout:696 write 0x6220243 thru 0x622a555 (0xa313 bytes) 2024-04-24T15:32:27.874 INFO:teuthology.orchestra.run.smithi096.stdout:697 punch from 0x1bde250 to 0x1bea06c, (0xbe1c bytes) 2024-04-24T15:32:27.874 INFO:teuthology.orchestra.run.smithi096.stdout:698 write 0x2fcb453 thru 0x2fcb998 (0x546 bytes) 2024-04-24T15:32:27.887 INFO:teuthology.orchestra.run.smithi096.stdout:699 write 0xaa756d7 thru 0xaa7bcf6 (0x6620 bytes) 2024-04-24T15:32:27.894 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:32:27.894 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:32:27.894 INFO:teuthology.orchestra.run.smithi151.stdout: "complaint_time": 30, 2024-04-24T15:32:27.894 INFO:teuthology.orchestra.run.smithi151.stdout: "num_blocked_ops": 0 2024-04-24T15:32:27.894 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:32:27.895 INFO:teuthology.orchestra.run.smithi096.stdout:700 write 0xc0eab02 thru 0xc0f6d7e (0xc27d bytes) 2024-04-24T15:32:27.898 INFO:teuthology.orchestra.run.smithi096.stdout:701 write 0xb5ee8d4 thru 0xb5f9c72 (0xb39f bytes) 2024-04-24T15:32:27.901 INFO:teuthology.orchestra.run.smithi096.stdout:704 punch from 0xb92283f to 0xb92a910, (0x80d1 bytes) 2024-04-24T15:32:27.901 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:27.901 INFO:teuthology.orchestra.run.smithi096.stdout:706 write 0x41e4eaf thru 0x41e65e7 (0x1739 bytes) 2024-04-24T15:32:27.902 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:27.904 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-24T15:32:27.914 INFO:teuthology.orchestra.run.smithi096.stdout:708 trunc from 0xc0f6d7f to 0x1a75863 2024-04-24T15:32:27.926 INFO:teuthology.orchestra.run.smithi096.stdout:709 writesame 0xa3c74c thru 0xa3f6fa data_size 0xd(0x2faf bytes) 2024-04-24T15:32:27.933 INFO:teuthology.orchestra.run.smithi096.stdout:711 write 0x29cb992 thru 0x29da47a (0xeae9 bytes) 2024-04-24T15:32:27.934 INFO:teuthology.orchestra.run.smithi096.stdout:712 read 0x191bbd1 thru 0x1929f41 (0xe371 bytes) 2024-04-24T15:32:27.936 INFO:teuthology.orchestra.run.smithi096.stdout:716 trunc from 0x29da47b to 0x367d733 2024-04-24T15:32:27.940 INFO:teuthology.orchestra.run.smithi096.stdout:717 punch from 0x14efd1d to 0x14fcfdc, (0xd2bf bytes) 2024-04-24T15:32:27.941 INFO:teuthology.orchestra.run.smithi096.stdout:719 write 0x16eb8ee thru 0x16fa133 (0xe846 bytes) 2024-04-24T15:32:27.954 INFO:teuthology.orchestra.run.smithi096.stdout:720 trunc from 0x367d733 to 0xd6cc17b 2024-04-24T15:32:27.959 INFO:teuthology.orchestra.run.smithi096.stdout:723 write 0x49a3414 thru 0x49b186d (0xe45a bytes) 2024-04-24T15:32:27.962 INFO:teuthology.orchestra.run.smithi096.stdout:724 write 0x7c73c31 thru 0x7c81ab0 (0xde80 bytes) 2024-04-24T15:32:27.974 INFO:teuthology.orchestra.run.smithi096.stdout:726 trunc from 0xd6cc17b to 0x463c009 2024-04-24T15:32:27.980 INFO:teuthology.orchestra.run.smithi096.stdout:727 read 0x12c573f thru 0x12cb6a5 (0x5f67 bytes) 2024-04-24T15:32:27.989 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:32:27.991+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:32:27.990 INFO:teuthology.orchestra.run.smithi096.stdout:729 write 0x6f447ce thru 0x6f4d4fe (0x8d31 bytes) 2024-04-24T15:32:27.993 INFO:teuthology.orchestra.run.smithi096.stdout:731 read 0x1b742f0 thru 0x1b7b3f4 (0x7105 bytes) 2024-04-24T15:32:27.993 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:28.001 INFO:teuthology.orchestra.run.smithi096.stdout:734 write 0x80bcb0b thru 0x80c4ffe (0x84f4 bytes) 2024-04-24T15:32:28.004 INFO:teuthology.orchestra.run.smithi096.stdout:736 writesame 0x9b4df9 thru 0x9b7dd8 data_size 0x10(0x2fe0 bytes) 2024-04-24T15:32:28.006 INFO:teuthology.orchestra.run.smithi096.stdout:737 punch from 0x7d7ca0e to 0x7d7ddf6, (0x13e8 bytes) 2024-04-24T15:32:28.007 INFO:teuthology.orchestra.run.smithi096.stdout:738 read 0x7a5b754 thru 0x7a6825a (0xcb07 bytes) 2024-04-24T15:32:28.007 INFO:teuthology.orchestra.run.smithi096.stdout:739 punch from 0x5765443 to 0x5767fbb, (0x2b78 bytes) 2024-04-24T15:32:28.013 INFO:teuthology.orchestra.run.smithi096.stdout:741 writesame 0x99b3daf thru 0x99b764f data_size 0x13(0x38a1 bytes) 2024-04-24T15:32:28.025 INFO:teuthology.orchestra.run.smithi096.stdout:744 trunc from 0x99b7650 to 0x6932c7 2024-04-24T15:32:28.056 INFO:teuthology.orchestra.run.smithi096.stdout:746 write 0xa4c1cec thru 0xa4c83da (0x66ef bytes) 2024-04-24T15:32:28.059 INFO:teuthology.orchestra.run.smithi096.stdout:747 read 0x52efa42 thru 0x52ff6f4 (0xfcb3 bytes) 2024-04-24T15:32:28.059 INFO:teuthology.orchestra.run.smithi096.stdout:748 write 0x7a28bc1 thru 0x7a2da66 (0x4ea6 bytes) 2024-04-24T15:32:28.061 INFO:teuthology.orchestra.run.smithi096.stdout:750 write 0x1ca3247 thru 0x1ca8b1a (0x58d4 bytes) 2024-04-24T15:32:28.069 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:32:28.070+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:32:28.070 INFO:teuthology.orchestra.run.smithi096.stdout:751 write 0xc30ae89 thru 0xc30d1be (0x2336 bytes) 2024-04-24T15:32:28.075 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:32:28.075 INFO:teuthology.orchestra.run.smithi151.stdout: "size": 20, 2024-04-24T15:32:28.076 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 600, 2024-04-24T15:32:28.076 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [ 2024-04-24T15:32:28.076 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.076 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4184.0:18 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-04-24T15:32:28.076 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:00.973237+0000", 2024-04-24T15:32:28.076 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.102099321000001, 2024-04-24T15:32:28.076 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0075407699999999996, 2024-04-24T15:32:28.076 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:28.076 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:28.076 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:28.076 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4184", 2024-04-24T15:32:28.076 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/1884602651", 2024-04-24T15:32:28.076 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18 2024-04-24T15:32:28.076 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.077 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:28.077 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.077 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:28.077 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.973237+0000", 2024-04-24T15:32:28.077 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:28.077 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.077 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.077 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:28.077 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.973237+0000", 2024-04-24T15:32:28.077 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:28.077 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.077 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.077 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:28.077 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.973237+0000", 2024-04-24T15:32:28.077 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:28.078 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.078 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.078 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:28.078 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.973241+0000", 2024-04-24T15:32:28.078 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9750000000000001e-06 2024-04-24T15:32:28.078 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.078 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.079 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:28.079 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.973246+0000", 2024-04-24T15:32:28.079 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.5670000000000002e-06 2024-04-24T15:32:28.079 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.079 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.079 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:28.079 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.973258+0000", 2024-04-24T15:32:28.080 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.1707e-05 2024-04-24T15:32:28.080 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.080 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.080 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:28.080 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.973684+0000", 2024-04-24T15:32:28.080 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00042587099999999998 2024-04-24T15:32:28.080 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.080 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.080 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:28.080 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.973745+0000", 2024-04-24T15:32:28.080 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.0976e-05 2024-04-24T15:32:28.080 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.080 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.080 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:28.081 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.973849+0000", 2024-04-24T15:32:28.081 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000103999 2024-04-24T15:32:28.081 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.081 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.081 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:28.081 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.976929+0000", 2024-04-24T15:32:28.081 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0030801689999999998 2024-04-24T15:32:28.081 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.081 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.081 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:28.081 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.980645+0000", 2024-04-24T15:32:28.081 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0037165060000000001 2024-04-24T15:32:28.081 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.081 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.081 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:28.082 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.980743+0000", 2024-04-24T15:32:28.082 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.7728999999999999e-05 2024-04-24T15:32:28.082 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.082 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.082 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:28.082 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.980778+0000", 2024-04-24T15:32:28.082 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.485e-05 2024-04-24T15:32:28.082 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:28.082 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:28.082 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:28.082 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.082 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.082 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4184.0:21 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 65536~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-04-24T15:32:28.082 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:00.985588+0000", 2024-04-24T15:32:28.083 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.089748991, 2024-04-24T15:32:28.083 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0073858090000000001, 2024-04-24T15:32:28.083 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:28.083 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:28.083 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:28.083 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4184", 2024-04-24T15:32:28.083 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/1884602651", 2024-04-24T15:32:28.083 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 21 2024-04-24T15:32:28.083 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.083 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:28.083 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.083 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:28.083 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.985588+0000", 2024-04-24T15:32:28.083 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:28.083 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.084 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.084 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:28.084 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.985588+0000", 2024-04-24T15:32:28.084 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:28.084 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.084 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.084 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:28.084 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.985587+0000", 2024-04-24T15:32:28.084 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:28.084 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.084 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.084 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:28.084 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.985653+0000", 2024-04-24T15:32:28.084 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.6086000000000004e-05 2024-04-24T15:32:28.085 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.085 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.085 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:28.085 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.985660+0000", 2024-04-24T15:32:28.085 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.3790000000000003e-06 2024-04-24T15:32:28.085 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.085 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.085 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:28.085 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.985664+0000", 2024-04-24T15:32:28.085 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.053e-06 2024-04-24T15:32:28.085 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.085 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.085 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:28.085 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.986266+0000", 2024-04-24T15:32:28.086 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00060216800000000004 2024-04-24T15:32:28.086 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.086 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.086 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:28.086 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.986321+0000", 2024-04-24T15:32:28.086 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.5275999999999997e-05 2024-04-24T15:32:28.086 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.086 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.086 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:28.086 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.986410+0000", 2024-04-24T15:32:28.086 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.9137999999999999e-05 2024-04-24T15:32:28.086 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.086 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.086 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:28.087 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.989564+0000", 2024-04-24T15:32:28.087 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0031535209999999998 2024-04-24T15:32:28.087 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.087 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.087 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:28.087 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.992882+0000", 2024-04-24T15:32:28.087 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0033180190000000002 2024-04-24T15:32:28.087 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.087 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.087 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:28.087 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.992934+0000", 2024-04-24T15:32:28.087 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.24e-05 2024-04-24T15:32:28.087 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.087 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.088 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:28.088 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.992973+0000", 2024-04-24T15:32:28.088 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9274999999999999e-05 2024-04-24T15:32:28.088 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:28.088 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:28.088 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:28.088 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.088 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.088 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4184.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-04-24T15:32:28.088 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:00.985670+0000", 2024-04-24T15:32:28.088 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.089666149999999, 2024-04-24T15:32:28.088 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0074023860000000004, 2024-04-24T15:32:28.088 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:28.088 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:28.088 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:28.089 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4184", 2024-04-24T15:32:28.089 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/1884602651", 2024-04-24T15:32:28.089 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 22 2024-04-24T15:32:28.089 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.089 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:28.089 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.089 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:28.089 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.985670+0000", 2024-04-24T15:32:28.089 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:28.089 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.089 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.089 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:28.089 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.985670+0000", 2024-04-24T15:32:28.089 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:28.090 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.090 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.090 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:28.090 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.985670+0000", 2024-04-24T15:32:28.090 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:28.090 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.090 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.090 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:28.090 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.985673+0000", 2024-04-24T15:32:28.090 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.1939999999999998e-06 2024-04-24T15:32:28.090 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.090 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.090 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:28.091 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.985677+0000", 2024-04-24T15:32:28.091 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.7079999999999999e-06 2024-04-24T15:32:28.091 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.091 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.091 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:28.091 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.985679+0000", 2024-04-24T15:32:28.091 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.3470000000000001e-06 2024-04-24T15:32:28.091 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.091 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.091 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:28.091 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.986801+0000", 2024-04-24T15:32:28.091 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0011212609999999999 2024-04-24T15:32:28.091 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.091 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.091 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:28.092 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.986887+0000", 2024-04-24T15:32:28.092 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.6877000000000002e-05 2024-04-24T15:32:28.092 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.092 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.092 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:28.092 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.987022+0000", 2024-04-24T15:32:28.092 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000134991 2024-04-24T15:32:28.092 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.092 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.092 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:28.092 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.992467+0000", 2024-04-24T15:32:28.092 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0054450219999999999 2024-04-24T15:32:28.092 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.092 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.093 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:28.093 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.993004+0000", 2024-04-24T15:32:28.093 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00053681400000000004 2024-04-24T15:32:28.093 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.093 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.093 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:28.093 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.993049+0000", 2024-04-24T15:32:28.093 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4709999999999997e-05 2024-04-24T15:32:28.093 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.093 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.093 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:28.093 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:00.993073+0000", 2024-04-24T15:32:28.093 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.3886000000000001e-05 2024-04-24T15:32:28.093 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:28.093 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:28.094 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:28.094 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.094 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.094 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4184.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-04-24T15:32:28.094 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:01.007969+0000", 2024-04-24T15:32:28.094 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.067367521000001, 2024-04-24T15:32:28.094 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0090800020000000002, 2024-04-24T15:32:28.094 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:28.094 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:28.094 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:28.094 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4184", 2024-04-24T15:32:28.094 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/1884602651", 2024-04-24T15:32:28.094 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 36 2024-04-24T15:32:28.094 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.095 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:28.095 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.095 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:28.095 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007969+0000", 2024-04-24T15:32:28.095 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:28.095 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.095 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.095 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:28.095 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007969+0000", 2024-04-24T15:32:28.095 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:28.095 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.095 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.095 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:28.095 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007969+0000", 2024-04-24T15:32:28.095 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:28.096 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.096 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.096 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:28.096 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007971+0000", 2024-04-24T15:32:28.096 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7219999999999999e-06 2024-04-24T15:32:28.096 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.096 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.096 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:28.096 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007975+0000", 2024-04-24T15:32:28.096 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.7019999999999999e-06 2024-04-24T15:32:28.096 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.096 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.096 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:28.096 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007978+0000", 2024-04-24T15:32:28.096 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.4059999999999999e-06 2024-04-24T15:32:28.097 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.097 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.097 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:28.097 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.010779+0000", 2024-04-24T15:32:28.097 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0028010610000000001 2024-04-24T15:32:28.097 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.097 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.097 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:28.097 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.010887+0000", 2024-04-24T15:32:28.097 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00010871599999999999 2024-04-24T15:32:28.097 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.097 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.097 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:28.097 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.011026+0000", 2024-04-24T15:32:28.097 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00013850200000000001 2024-04-24T15:32:28.097 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.098 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.098 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:28.098 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.015873+0000", 2024-04-24T15:32:28.098 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0048467680000000004 2024-04-24T15:32:28.098 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.098 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.098 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:28.098 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.016993+0000", 2024-04-24T15:32:28.098 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0011203809999999999 2024-04-24T15:32:28.098 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.098 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.098 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:28.098 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.017027+0000", 2024-04-24T15:32:28.098 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.3637000000000003e-05 2024-04-24T15:32:28.099 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.099 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.099 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:28.099 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.017049+0000", 2024-04-24T15:32:28.099 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.2509000000000001e-05 2024-04-24T15:32:28.099 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:28.099 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:28.099 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:28.099 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.099 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.099 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4184.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-04-24T15:32:28.099 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:01.007981+0000", 2024-04-24T15:32:28.099 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.067355328000001, 2024-04-24T15:32:28.099 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0091181590000000007, 2024-04-24T15:32:28.099 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:28.100 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:28.100 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:28.100 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4184", 2024-04-24T15:32:28.100 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/1884602651", 2024-04-24T15:32:28.100 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 37 2024-04-24T15:32:28.100 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.100 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:28.100 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.100 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:28.100 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007981+0000", 2024-04-24T15:32:28.100 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:28.100 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.100 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.100 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:28.101 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007981+0000", 2024-04-24T15:32:28.101 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:28.101 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.101 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.101 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:28.101 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007981+0000", 2024-04-24T15:32:28.101 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:32:28.101 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.101 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.101 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:28.101 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007984+0000", 2024-04-24T15:32:28.101 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.3469999999999999e-06 2024-04-24T15:32:28.101 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.101 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.101 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:28.102 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007988+0000", 2024-04-24T15:32:28.102 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.4479999999999999e-06 2024-04-24T15:32:28.102 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.102 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.102 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:28.102 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.007990+0000", 2024-04-24T15:32:28.102 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.26e-06 2024-04-24T15:32:28.102 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.102 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.102 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:28.102 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.011341+0000", 2024-04-24T15:32:28.102 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0033510459999999999 2024-04-24T15:32:28.102 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.102 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.103 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:28.103 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.011400+0000", 2024-04-24T15:32:28.103 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.9206000000000002e-05 2024-04-24T15:32:28.103 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.103 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.103 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:28.103 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.011488+0000", 2024-04-24T15:32:28.103 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.7301999999999999e-05 2024-04-24T15:32:28.103 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.103 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.103 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:28.103 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.015901+0000", 2024-04-24T15:32:28.103 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0044129549999999997 2024-04-24T15:32:28.103 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.104 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.104 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:28.104 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.017057+0000", 2024-04-24T15:32:28.104 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0011566790000000001 2024-04-24T15:32:28.104 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.104 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.104 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:28.104 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.017081+0000", 2024-04-24T15:32:28.104 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.3790000000000001e-05 2024-04-24T15:32:28.104 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.104 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.104 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:28.104 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.017099+0000", 2024-04-24T15:32:28.104 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.8346e-05 2024-04-24T15:32:28.105 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:28.105 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:28.105 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:28.105 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.105 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.105 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4184.0:69 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-04-24T15:32:28.105 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:01.057223+0000", 2024-04-24T15:32:28.105 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.018113304, 2024-04-24T15:32:28.105 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0080831549999999999, 2024-04-24T15:32:28.105 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:28.105 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:28.105 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:28.105 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4184", 2024-04-24T15:32:28.105 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/1884602651", 2024-04-24T15:32:28.106 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 69 2024-04-24T15:32:28.106 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.106 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:28.106 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.106 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:28.106 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.057223+0000", 2024-04-24T15:32:28.106 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:28.106 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.106 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.106 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:28.106 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.057223+0000", 2024-04-24T15:32:28.106 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:28.106 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.106 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.107 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:28.107 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.057223+0000", 2024-04-24T15:32:28.107 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:28.107 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.107 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.107 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:28.107 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.057261+0000", 2024-04-24T15:32:28.107 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.7905999999999999e-05 2024-04-24T15:32:28.107 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.107 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.107 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:28.107 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.057267+0000", 2024-04-24T15:32:28.107 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.4389999999999997e-06 2024-04-24T15:32:28.107 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.108 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.108 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:28.108 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.057270+0000", 2024-04-24T15:32:28.108 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.0489999999999999e-06 2024-04-24T15:32:28.108 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.108 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.108 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:28.108 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.058232+0000", 2024-04-24T15:32:28.108 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00096145599999999996 2024-04-24T15:32:28.108 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.108 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.108 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:28.108 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.058284+0000", 2024-04-24T15:32:28.108 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.2213999999999998e-05 2024-04-24T15:32:28.108 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.109 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.109 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:28.109 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.058364+0000", 2024-04-24T15:32:28.109 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0096e-05 2024-04-24T15:32:28.109 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.109 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.109 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:28.109 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.059686+0000", 2024-04-24T15:32:28.109 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0013213739999999999 2024-04-24T15:32:28.109 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.109 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.109 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:28.109 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.065225+0000", 2024-04-24T15:32:28.109 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.005539675 2024-04-24T15:32:28.109 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.110 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.110 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:28.110 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.065272+0000", 2024-04-24T15:32:28.110 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6369999999999998e-05 2024-04-24T15:32:28.110 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.110 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.110 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:28.110 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.065306+0000", 2024-04-24T15:32:28.110 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.4875000000000001e-05 2024-04-24T15:32:28.110 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:28.110 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:28.110 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:28.110 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.110 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.110 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4184.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-04-24T15:32:28.111 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:01.057275+0000", 2024-04-24T15:32:28.111 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.018061869, 2024-04-24T15:32:28.111 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0081317479999999994, 2024-04-24T15:32:28.111 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:28.111 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:28.111 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:28.111 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4184", 2024-04-24T15:32:28.111 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/1884602651", 2024-04-24T15:32:28.111 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 70 2024-04-24T15:32:28.111 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.111 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:28.111 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.111 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:28.111 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.057275+0000", 2024-04-24T15:32:28.111 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:28.112 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.112 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.112 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:28.112 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.057275+0000", 2024-04-24T15:32:28.112 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:28.112 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.112 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.112 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:28.112 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.057274+0000", 2024-04-24T15:32:28.112 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:28.112 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.112 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.112 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:28.112 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.057277+0000", 2024-04-24T15:32:28.112 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.9790000000000002e-06 2024-04-24T15:32:28.113 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.113 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.113 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:28.113 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.057281+0000", 2024-04-24T15:32:28.113 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5109999999999999e-06 2024-04-24T15:32:28.113 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.113 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.113 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:28.113 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.057289+0000", 2024-04-24T15:32:28.113 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.4300000000000006e-06 2024-04-24T15:32:28.113 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.113 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.113 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:28.113 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.058763+0000", 2024-04-24T15:32:28.113 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.001473238 2024-04-24T15:32:28.114 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.114 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.114 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:28.114 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.058850+0000", 2024-04-24T15:32:28.114 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.7600000000000002e-05 2024-04-24T15:32:28.114 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.114 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.114 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:28.114 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.058972+0000", 2024-04-24T15:32:28.114 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00012228099999999999 2024-04-24T15:32:28.114 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.114 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.114 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:28.114 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.060449+0000", 2024-04-24T15:32:28.114 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.001476705 2024-04-24T15:32:28.115 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.115 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.115 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:28.115 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.065340+0000", 2024-04-24T15:32:28.115 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0048906929999999998 2024-04-24T15:32:28.115 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.115 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.115 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:28.115 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.065370+0000", 2024-04-24T15:32:28.115 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.9887999999999999e-05 2024-04-24T15:32:28.115 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.115 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.115 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:28.115 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.065406+0000", 2024-04-24T15:32:28.115 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.6755999999999998e-05 2024-04-24T15:32:28.116 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:28.116 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:28.116 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:28.116 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.116 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.116 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4184.0:154 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-04-24T15:32:28.116 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:01.928400+0000", 2024-04-24T15:32:28.116 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 26.146936858, 2024-04-24T15:32:28.116 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0077033919999999999, 2024-04-24T15:32:28.116 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:28.116 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:28.116 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:28.116 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4184", 2024-04-24T15:32:28.116 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/1884602651", 2024-04-24T15:32:28.117 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 154 2024-04-24T15:32:28.117 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.117 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:28.117 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.117 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:28.117 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928400+0000", 2024-04-24T15:32:28.117 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:28.117 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.117 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.117 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:28.117 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928400+0000", 2024-04-24T15:32:28.117 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:28.117 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.117 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.117 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:28.118 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928399+0000", 2024-04-24T15:32:28.118 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:28.118 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.118 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.118 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:28.118 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928402+0000", 2024-04-24T15:32:28.118 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.497e-06 2024-04-24T15:32:28.118 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.118 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.118 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:28.118 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928407+0000", 2024-04-24T15:32:28.118 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.1950000000000002e-06 2024-04-24T15:32:28.118 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.119 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.119 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:28.119 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928410+0000", 2024-04-24T15:32:28.119 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.5000000000000002e-06 2024-04-24T15:32:28.119 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.119 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.119 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:28.119 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928822+0000", 2024-04-24T15:32:28.119 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00041253900000000002 2024-04-24T15:32:28.119 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.119 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.119 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:28.119 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928865+0000", 2024-04-24T15:32:28.119 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2570000000000001e-05 2024-04-24T15:32:28.119 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.120 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.120 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:28.120 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928944+0000", 2024-04-24T15:32:28.120 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.9634000000000004e-05 2024-04-24T15:32:28.120 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.120 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.120 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:28.120 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.932122+0000", 2024-04-24T15:32:28.120 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.003177862 2024-04-24T15:32:28.120 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.120 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.120 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:28.120 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.936054+0000", 2024-04-24T15:32:28.120 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0039315569999999996 2024-04-24T15:32:28.121 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.121 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.121 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:28.121 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.936083+0000", 2024-04-24T15:32:28.121 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.9043e-05 2024-04-24T15:32:28.121 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.121 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.121 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:28.121 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.936103+0000", 2024-04-24T15:32:28.121 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.0292000000000002e-05 2024-04-24T15:32:28.121 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:28.121 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:28.121 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:28.121 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.121 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.122 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4184.0:155 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-04-24T15:32:28.122 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:01.928434+0000", 2024-04-24T15:32:28.122 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 26.146902314999998, 2024-04-24T15:32:28.122 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.011257120000000001, 2024-04-24T15:32:28.122 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:28.122 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:28.122 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:28.122 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4184", 2024-04-24T15:32:28.122 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/1884602651", 2024-04-24T15:32:28.122 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 155 2024-04-24T15:32:28.122 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.122 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:28.122 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.123 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:28.123 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928434+0000", 2024-04-24T15:32:28.123 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:28.123 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.123 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.123 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:28.123 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928434+0000", 2024-04-24T15:32:28.123 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:28.123 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.123 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.123 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:28.123 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928434+0000", 2024-04-24T15:32:28.123 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:28.123 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.123 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.124 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:28.124 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928562+0000", 2024-04-24T15:32:28.124 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00012848100000000001 2024-04-24T15:32:28.124 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.124 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.124 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:28.124 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928568+0000", 2024-04-24T15:32:28.124 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.4500000000000003e-06 2024-04-24T15:32:28.124 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.124 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.124 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:28.124 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928572+0000", 2024-04-24T15:32:28.124 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.1880000000000004e-06 2024-04-24T15:32:28.125 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.125 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.125 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:28.125 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.929192+0000", 2024-04-24T15:32:28.125 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00062016399999999998 2024-04-24T15:32:28.125 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.125 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.125 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:28.125 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.929240+0000", 2024-04-24T15:32:28.125 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8112999999999998e-05 2024-04-24T15:32:28.125 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.125 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.125 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:28.125 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.929316+0000", 2024-04-24T15:32:28.126 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.5486000000000002e-05 2024-04-24T15:32:28.126 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.126 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.126 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:28.126 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.935151+0000", 2024-04-24T15:32:28.126 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00583498 2024-04-24T15:32:28.126 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.126 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.126 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:28.126 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.939589+0000", 2024-04-24T15:32:28.126 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0044387840000000003 2024-04-24T15:32:28.126 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.126 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.126 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:28.127 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.939670+0000", 2024-04-24T15:32:28.127 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0389000000000006e-05 2024-04-24T15:32:28.127 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.127 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.127 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:28.127 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.939691+0000", 2024-04-24T15:32:28.127 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.1543999999999999e-05 2024-04-24T15:32:28.127 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:28.127 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:28.127 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:28.127 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.127 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.127 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4184.0:156 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-04-24T15:32:28.127 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:01.928576+0000", 2024-04-24T15:32:28.128 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 26.146760574999998, 2024-04-24T15:32:28.128 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.011200128, 2024-04-24T15:32:28.128 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:28.128 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:28.128 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:28.128 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4184", 2024-04-24T15:32:28.128 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/1884602651", 2024-04-24T15:32:28.128 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 156 2024-04-24T15:32:28.128 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.128 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:28.128 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.128 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:28.128 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928576+0000", 2024-04-24T15:32:28.128 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:28.128 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.129 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.129 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:28.129 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928576+0000", 2024-04-24T15:32:28.129 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:28.129 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.129 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.129 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:28.129 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928576+0000", 2024-04-24T15:32:28.129 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:28.129 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.129 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.129 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:28.129 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928578+0000", 2024-04-24T15:32:28.129 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.3070000000000001e-06 2024-04-24T15:32:28.129 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.130 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.130 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:28.130 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928581+0000", 2024-04-24T15:32:28.130 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.3419999999999999e-06 2024-04-24T15:32:28.130 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.130 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.130 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:28.130 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928584+0000", 2024-04-24T15:32:28.130 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.1780000000000002e-06 2024-04-24T15:32:28.130 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.130 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.130 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:28.130 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.929860+0000", 2024-04-24T15:32:28.131 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.001276578 2024-04-24T15:32:28.131 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.131 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.131 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:28.131 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.929907+0000", 2024-04-24T15:32:28.131 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.693e-05 2024-04-24T15:32:28.131 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.131 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.131 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:28.131 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.929985+0000", 2024-04-24T15:32:28.131 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7565999999999996e-05 2024-04-24T15:32:28.131 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.131 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.131 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:28.132 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.935155+0000", 2024-04-24T15:32:28.132 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0051699110000000001 2024-04-24T15:32:28.132 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.132 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.132 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:28.132 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.939724+0000", 2024-04-24T15:32:28.132 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0045697189999999999 2024-04-24T15:32:28.132 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.132 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.132 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:28.132 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.939753+0000", 2024-04-24T15:32:28.132 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8547e-05 2024-04-24T15:32:28.132 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.132 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.133 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:28.133 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.939776+0000", 2024-04-24T15:32:28.133 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.3348000000000001e-05 2024-04-24T15:32:28.133 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:28.133 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:28.133 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:28.133 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.133 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.133 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4184.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-04-24T15:32:28.133 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:01.928586+0000", 2024-04-24T15:32:28.133 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 26.146750266000002, 2024-04-24T15:32:28.133 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.011256990999999999, 2024-04-24T15:32:28.133 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:28.133 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:28.133 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:28.134 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4184", 2024-04-24T15:32:28.134 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/1884602651", 2024-04-24T15:32:28.134 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 157 2024-04-24T15:32:28.134 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.134 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:28.134 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.134 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:28.134 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928586+0000", 2024-04-24T15:32:28.134 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:28.134 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.134 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.134 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:28.134 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928586+0000", 2024-04-24T15:32:28.135 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:28.135 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.135 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.135 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:28.135 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928586+0000", 2024-04-24T15:32:28.135 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:32:28.135 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.135 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.135 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:28.135 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928592+0000", 2024-04-24T15:32:28.135 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6400000000000002e-06 2024-04-24T15:32:28.135 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.135 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.135 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:28.136 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928596+0000", 2024-04-24T15:32:28.136 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2169999999999996e-06 2024-04-24T15:32:28.136 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.136 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.136 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:28.136 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928605+0000", 2024-04-24T15:32:28.136 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.1260000000000004e-06 2024-04-24T15:32:28.136 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.136 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.136 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:28.136 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.930310+0000", 2024-04-24T15:32:28.136 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0017047200000000001 2024-04-24T15:32:28.136 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.136 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.136 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:28.137 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.930366+0000", 2024-04-24T15:32:28.137 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6357999999999998e-05 2024-04-24T15:32:28.137 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.137 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.137 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:28.137 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.930438+0000", 2024-04-24T15:32:28.137 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1469000000000003e-05 2024-04-24T15:32:28.137 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.137 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.137 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:28.137 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.935159+0000", 2024-04-24T15:32:28.137 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0047212809999999999 2024-04-24T15:32:28.137 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.137 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.137 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:28.138 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.939803+0000", 2024-04-24T15:32:28.138 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0046441920000000001 2024-04-24T15:32:28.138 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.138 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.138 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:28.138 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.939826+0000", 2024-04-24T15:32:28.138 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.2949e-05 2024-04-24T15:32:28.138 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.138 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.138 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:28.138 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.939843+0000", 2024-04-24T15:32:28.138 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.7266999999999999e-05 2024-04-24T15:32:28.138 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:28.138 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:28.139 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:28.139 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.139 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.139 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4184.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-04-24T15:32:28.139 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:01.928608+0000", 2024-04-24T15:32:28.139 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 26.146728120999999, 2024-04-24T15:32:28.139 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.011304719, 2024-04-24T15:32:28.139 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:28.139 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:28.139 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:28.139 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4184", 2024-04-24T15:32:28.139 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/1884602651", 2024-04-24T15:32:28.139 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 158 2024-04-24T15:32:28.140 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.140 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:28.140 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.140 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:28.140 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928608+0000", 2024-04-24T15:32:28.140 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:28.140 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.140 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.140 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:28.140 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928608+0000", 2024-04-24T15:32:28.140 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:28.140 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.140 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.140 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:28.140 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928608+0000", 2024-04-24T15:32:28.141 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:28.141 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.141 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.141 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:28.141 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928652+0000", 2024-04-24T15:32:28.141 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3856999999999997e-05 2024-04-24T15:32:28.141 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.141 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.141 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:28.141 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928660+0000", 2024-04-24T15:32:28.141 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.9619999999999993e-06 2024-04-24T15:32:28.141 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.141 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.141 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:28.142 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928667+0000", 2024-04-24T15:32:28.142 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.029e-06 2024-04-24T15:32:28.142 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.142 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.142 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:28.142 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.930668+0000", 2024-04-24T15:32:28.142 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0020010219999999999 2024-04-24T15:32:28.142 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.142 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.142 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:28.142 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.930712+0000", 2024-04-24T15:32:28.142 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3897999999999999e-05 2024-04-24T15:32:28.142 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.142 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.143 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:28.143 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.930784+0000", 2024-04-24T15:32:28.143 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1898999999999997e-05 2024-04-24T15:32:28.143 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.143 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.143 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:28.143 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.935162+0000", 2024-04-24T15:32:28.143 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.004378364 2024-04-24T15:32:28.143 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.143 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.143 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:28.143 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.939870+0000", 2024-04-24T15:32:28.143 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0047080439999999998 2024-04-24T15:32:28.143 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.143 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.144 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:28.144 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.939893+0000", 2024-04-24T15:32:28.144 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.2989e-05 2024-04-24T15:32:28.144 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.144 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.144 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:28.144 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.939913+0000", 2024-04-24T15:32:28.144 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.9925999999999999e-05 2024-04-24T15:32:28.144 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:28.144 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:28.144 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:28.144 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.144 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.144 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4184.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-04-24T15:32:28.145 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:01.928671+0000", 2024-04-24T15:32:28.145 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 26.146665594000002, 2024-04-24T15:32:28.145 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.011298967, 2024-04-24T15:32:28.145 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:28.145 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:28.145 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:28.145 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4184", 2024-04-24T15:32:28.145 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/1884602651", 2024-04-24T15:32:28.145 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 159 2024-04-24T15:32:28.145 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.145 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:28.145 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.145 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:28.145 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928671+0000", 2024-04-24T15:32:28.146 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:28.146 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.146 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.146 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:28.146 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928671+0000", 2024-04-24T15:32:28.146 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:28.146 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.146 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.146 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:28.146 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928671+0000", 2024-04-24T15:32:28.146 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:28.146 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.146 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.146 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:28.147 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928674+0000", 2024-04-24T15:32:28.147 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.715e-06 2024-04-24T15:32:28.147 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.147 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.147 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:28.147 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928680+0000", 2024-04-24T15:32:28.147 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.0939999999999998e-06 2024-04-24T15:32:28.147 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.147 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.147 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:28.147 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.928683+0000", 2024-04-24T15:32:28.147 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.1360000000000001e-06 2024-04-24T15:32:28.147 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.147 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.147 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:28.148 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.931109+0000", 2024-04-24T15:32:28.148 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0024267379999999999 2024-04-24T15:32:28.148 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.148 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.148 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:28.148 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.931153+0000", 2024-04-24T15:32:28.148 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3970000000000001e-05 2024-04-24T15:32:28.148 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.148 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.148 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:28.148 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.931231+0000", 2024-04-24T15:32:28.148 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.8053999999999995e-05 2024-04-24T15:32:28.148 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.148 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.149 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:28.149 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.935166+0000", 2024-04-24T15:32:28.149 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0039344159999999996 2024-04-24T15:32:28.149 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.149 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.149 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:28.149 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.939932+0000", 2024-04-24T15:32:28.149 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0047658980000000002 2024-04-24T15:32:28.149 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.149 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.149 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:28.149 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.939953+0000", 2024-04-24T15:32:28.149 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.1107e-05 2024-04-24T15:32:28.149 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.149 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.150 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:28.150 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:01.939970+0000", 2024-04-24T15:32:28.150 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.7110999999999999e-05 2024-04-24T15:32:28.150 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:28.150 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:28.150 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:28.150 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.150 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.150 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_repop(client.4426.0:3993 4.a e34/27 4:577a4030:::rbd_data.114a1f60cb66.00000000000001b0:head v 34'29, mlcod=34'27)", 2024-04-24T15:32:28.150 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.837439+0000", 2024-04-24T15:32:28.150 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 3.2378979010000002, 2024-04-24T15:32:28.150 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0072208100000000002, 2024-04-24T15:32:28.150 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:28.150 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:28.150 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:28.151 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.151 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:28.151 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.837439+0000", 2024-04-24T15:32:28.151 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:28.151 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.151 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.151 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:28.151 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.837439+0000", 2024-04-24T15:32:28.151 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:28.151 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.151 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.151 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:28.151 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.837438+0000", 2024-04-24T15:32:28.151 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:28.151 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.152 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.152 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:28.152 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.837445+0000", 2024-04-24T15:32:28.152 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.6499999999999999e-06 2024-04-24T15:32:28.152 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.152 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.152 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:28.152 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.837460+0000", 2024-04-24T15:32:28.152 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.539e-05 2024-04-24T15:32:28.152 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.152 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.152 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:28.152 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.837466+0000", 2024-04-24T15:32:28.152 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.2469999999999997e-06 2024-04-24T15:32:28.152 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.153 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.153 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:28.153 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.837503+0000", 2024-04-24T15:32:28.153 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.6919999999999999e-05 2024-04-24T15:32:28.153 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.153 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.153 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:28.153 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.837523+0000", 2024-04-24T15:32:28.153 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.9652000000000001e-05 2024-04-24T15:32:28.153 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.153 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.153 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:28.153 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.844631+0000", 2024-04-24T15:32:28.153 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0071081060000000003 2024-04-24T15:32:28.153 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.154 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.154 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:28.154 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.844660+0000", 2024-04-24T15:32:28.154 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8504999999999999e-05 2024-04-24T15:32:28.154 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:28.154 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:28.154 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:28.154 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.154 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.154 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_repop(client.4426.0:4011 4.17 e34/26 4:ea569883:::rbd_data.114a1f60cb66.0000000000000240:head v 34'46, mlcod=34'44)", 2024-04-24T15:32:28.154 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.845809+0000", 2024-04-24T15:32:28.154 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 3.2295272019999999, 2024-04-24T15:32:28.154 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0070089840000000002, 2024-04-24T15:32:28.154 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:28.154 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:28.155 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:28.155 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.155 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:28.155 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.845809+0000", 2024-04-24T15:32:28.155 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:28.155 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.155 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.155 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:28.155 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.845809+0000", 2024-04-24T15:32:28.155 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:28.155 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.155 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.155 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:28.155 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.845809+0000", 2024-04-24T15:32:28.155 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:28.156 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.156 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.156 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:28.156 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.845816+0000", 2024-04-24T15:32:28.156 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.7889999999999997e-06 2024-04-24T15:32:28.156 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.156 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.156 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:28.156 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.845827+0000", 2024-04-24T15:32:28.156 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.1681e-05 2024-04-24T15:32:28.156 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.156 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.156 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:28.156 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.845833+0000", 2024-04-24T15:32:28.156 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.7259999999999999e-06 2024-04-24T15:32:28.157 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.157 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.157 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:28.157 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.845865+0000", 2024-04-24T15:32:28.157 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.1755999999999999e-05 2024-04-24T15:32:28.157 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.157 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.157 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:28.157 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.845884+0000", 2024-04-24T15:32:28.157 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.9568999999999998e-05 2024-04-24T15:32:28.157 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.157 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.157 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:28.157 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.852788+0000", 2024-04-24T15:32:28.157 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0069035169999999996 2024-04-24T15:32:28.158 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.158 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.158 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:28.158 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.852818+0000", 2024-04-24T15:32:28.158 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.0565999999999998e-05 2024-04-24T15:32:28.158 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:28.158 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:28.158 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:28.158 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.158 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.158 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_repop(client.4426.0:4018 4.0 e34/26 4:01f056f4:::rbd_data.114a1f60cb66.0000000000000247:head v 34'47, mlcod=34'44)", 2024-04-24T15:32:28.158 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.848484+0000", 2024-04-24T15:32:28.158 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 3.2268525829999999, 2024-04-24T15:32:28.159 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0078882899999999992, 2024-04-24T15:32:28.159 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:28.159 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:28.159 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:28.159 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.159 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:28.159 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.848484+0000", 2024-04-24T15:32:28.159 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:28.159 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.159 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.159 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:28.159 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.848484+0000", 2024-04-24T15:32:28.159 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:28.159 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.159 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.160 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:28.160 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.848483+0000", 2024-04-24T15:32:28.160 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:28.160 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.160 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.160 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:28.160 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.848489+0000", 2024-04-24T15:32:28.160 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.1410000000000004e-06 2024-04-24T15:32:28.160 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.160 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.160 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:28.160 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.848498+0000", 2024-04-24T15:32:28.160 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.2380000000000003e-06 2024-04-24T15:32:28.160 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.160 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.161 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:28.161 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.848502+0000", 2024-04-24T15:32:28.161 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2529999999999998e-06 2024-04-24T15:32:28.161 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.161 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.161 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:28.161 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.848536+0000", 2024-04-24T15:32:28.161 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.4156999999999998e-05 2024-04-24T15:32:28.161 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.161 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.161 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:28.161 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.848554+0000", 2024-04-24T15:32:28.161 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.7646999999999999e-05 2024-04-24T15:32:28.161 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.161 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.162 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:28.162 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.856339+0000", 2024-04-24T15:32:28.162 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0077853799999999997 2024-04-24T15:32:28.162 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.162 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.162 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:28.162 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.856372+0000", 2024-04-24T15:32:28.162 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.3042e-05 2024-04-24T15:32:28.162 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:28.162 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:28.162 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:28.162 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.162 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.162 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_repop(client.4426.0:4081 4.15 e34/26 4:a8696d39:::rbd_data.114a1f60cb66.000000000000016e:head v 34'33, mlcod=34'31)", 2024-04-24T15:32:28.163 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.905704+0000", 2024-04-24T15:32:28.163 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 3.1696324059999998, 2024-04-24T15:32:28.163 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0069589170000000002, 2024-04-24T15:32:28.163 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:28.163 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:28.163 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:28.163 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.163 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:28.163 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.905704+0000", 2024-04-24T15:32:28.163 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:28.163 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.163 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.163 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:28.163 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.905704+0000", 2024-04-24T15:32:28.163 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:28.164 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.164 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.164 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:28.164 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.905704+0000", 2024-04-24T15:32:28.164 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:28.164 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.164 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.164 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:28.164 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.905712+0000", 2024-04-24T15:32:28.164 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.6260000000000001e-06 2024-04-24T15:32:28.164 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.164 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.164 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:28.165 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.905725+0000", 2024-04-24T15:32:28.165 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.2935999999999999e-05 2024-04-24T15:32:28.165 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.165 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.165 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:28.165 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.905731+0000", 2024-04-24T15:32:28.165 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.1299999999999998e-06 2024-04-24T15:32:28.165 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.165 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.165 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:28.165 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.905766+0000", 2024-04-24T15:32:28.165 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.4337e-05 2024-04-24T15:32:28.165 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.166 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.166 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:28.166 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.905786+0000", 2024-04-24T15:32:28.166 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.0496000000000002e-05 2024-04-24T15:32:28.166 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.166 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.166 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:28.166 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.912632+0000", 2024-04-24T15:32:28.166 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0068458740000000001 2024-04-24T15:32:28.166 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.167 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.167 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:28.167 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.912663+0000", 2024-04-24T15:32:28.167 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.1148000000000001e-05 2024-04-24T15:32:28.167 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:28.167 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:28.167 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:28.167 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.167 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.167 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_repop(client.4426.0:4253 4.a e34/27 4:53a9cf93:::rbd_data.114a1f60cb66.0000000000000313:head v 34'37, mlcod=34'35)", 2024-04-24T15:32:28.167 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:25.054583+0000", 2024-04-24T15:32:28.167 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 3.020753633, 2024-04-24T15:32:28.167 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00788172, 2024-04-24T15:32:28.167 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:28.167 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:28.168 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:28.168 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.168 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:28.168 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.054583+0000", 2024-04-24T15:32:28.168 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:28.168 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.168 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.168 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:28.168 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.054583+0000", 2024-04-24T15:32:28.168 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:28.168 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.168 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.168 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:28.168 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.054582+0000", 2024-04-24T15:32:28.168 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999999 2024-04-24T15:32:28.169 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.169 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.169 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:28.169 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.054589+0000", 2024-04-24T15:32:28.169 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.0720000000000002e-06 2024-04-24T15:32:28.169 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.169 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.169 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:28.169 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.054602+0000", 2024-04-24T15:32:28.169 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.2303999999999999e-05 2024-04-24T15:32:28.169 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.169 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.169 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:28.169 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.054611+0000", 2024-04-24T15:32:28.169 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.0469999999999999e-06 2024-04-24T15:32:28.170 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.170 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.170 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:28.170 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.054652+0000", 2024-04-24T15:32:28.170 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0887999999999999e-05 2024-04-24T15:32:28.170 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.170 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.170 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:28.170 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.054670+0000", 2024-04-24T15:32:28.170 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.8890999999999999e-05 2024-04-24T15:32:28.170 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.170 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.170 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:28.171 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.062430+0000", 2024-04-24T15:32:28.171 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.007759841 2024-04-24T15:32:28.171 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.171 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.171 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:28.171 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.062465+0000", 2024-04-24T15:32:28.171 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.4434000000000003e-05 2024-04-24T15:32:28.171 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:28.171 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:28.171 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:28.171 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.171 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.171 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_repop(client.4426.0:4455 4.0 e34/26 4:05029c7a:::rbd_data.114a1f60cb66.00000000000002c1:head v 34'50, mlcod=34'48)", 2024-04-24T15:32:28.171 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:25.251382+0000", 2024-04-24T15:32:28.171 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 2.8239549899999998, 2024-04-24T15:32:28.172 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0077590339999999997, 2024-04-24T15:32:28.172 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:28.172 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:28.172 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:28.172 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.172 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:28.172 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.251382+0000", 2024-04-24T15:32:28.172 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:28.172 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.172 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.172 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:28.172 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.251382+0000", 2024-04-24T15:32:28.172 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:28.172 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.172 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.173 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:28.173 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.251381+0000", 2024-04-24T15:32:28.173 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:28.173 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.173 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.173 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:28.173 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.251386+0000", 2024-04-24T15:32:28.173 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.3859999999999998e-06 2024-04-24T15:32:28.173 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.173 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.173 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:28.173 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.251395+0000", 2024-04-24T15:32:28.173 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.0389999999999993e-06 2024-04-24T15:32:28.173 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.173 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.174 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:28.174 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.251400+0000", 2024-04-24T15:32:28.174 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.1300000000000003e-06 2024-04-24T15:32:28.174 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.174 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.174 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:28.174 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.251429+0000", 2024-04-24T15:32:28.174 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.9584e-05 2024-04-24T15:32:28.174 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.174 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.174 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:28.174 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.251442+0000", 2024-04-24T15:32:28.174 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3050999999999999e-05 2024-04-24T15:32:28.175 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.175 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.175 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:28.175 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.259104+0000", 2024-04-24T15:32:28.175 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0076613510000000003 2024-04-24T15:32:28.175 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.175 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.175 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:28.175 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.259141+0000", 2024-04-24T15:32:28.175 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.7131000000000001e-05 2024-04-24T15:32:28.175 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:28.175 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:28.175 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:28.175 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.175 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.176 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_repop(client.4426.0:4454 4.11 e34/26 4:8d4f9ae5:::rbd_data.114a1f60cb66.00000000000002c0:head v 34'44, mlcod=34'41)", 2024-04-24T15:32:28.176 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:25.251407+0000", 2024-04-24T15:32:28.176 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 2.823929422, 2024-04-24T15:32:28.176 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0077369680000000003, 2024-04-24T15:32:28.176 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:28.176 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:28.176 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:28.176 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.176 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:28.176 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.251407+0000", 2024-04-24T15:32:28.176 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:28.176 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.176 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.176 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:28.176 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.251407+0000", 2024-04-24T15:32:28.176 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:28.177 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.177 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.177 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:28.177 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.251407+0000", 2024-04-24T15:32:28.177 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:32:28.177 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.177 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.177 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:28.177 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.251410+0000", 2024-04-24T15:32:28.177 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.5979999999999999e-06 2024-04-24T15:32:28.177 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.177 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.177 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:28.177 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.251413+0000", 2024-04-24T15:32:28.178 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5429999999999998e-06 2024-04-24T15:32:28.178 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.178 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.178 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:28.178 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.251415+0000", 2024-04-24T15:32:28.178 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.3269999999999999e-06 2024-04-24T15:32:28.178 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.178 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.178 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:28.178 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.251440+0000", 2024-04-24T15:32:28.178 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.4380999999999999e-05 2024-04-24T15:32:28.178 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.178 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.178 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:28.179 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.251455+0000", 2024-04-24T15:32:28.179 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.4972e-05 2024-04-24T15:32:28.179 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.179 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.179 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:28.179 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.259113+0000", 2024-04-24T15:32:28.179 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0076581230000000002 2024-04-24T15:32:28.179 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:28.179 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:28.179 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:28.179 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.259144+0000", 2024-04-24T15:32:28.179 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.1180999999999998e-05 2024-04-24T15:32:28.179 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:28.179 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:28.180 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:28.180 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:28.180 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:28.180 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:32:28.180 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7] out_osds: [1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-04-24T15:32:28.180 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-24T15:32:28.180 INFO:tasks.thrashosds.thrasher:Removing osd 4, in_osds are: [0, 2, 3, 4, 5, 6, 7] 2024-04-24T15:32:28.180 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 4 2024-04-24T15:32:28.181 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-24T15:32:28.183 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:32:28.184+0000 7f23abcf6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 44021) UID: 0 2024-04-24T15:32:28.183 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-24T15:32:28.185 INFO:teuthology.orchestra.run.smithi096.stdout:753 trunc from 0xc30d1bf to 0x42f0cc9 2024-04-24T15:32:28.185 INFO:teuthology.orchestra.run.smithi096.stdout:754 writesame 0x6e0b4ae thru 0x6e0f66b data_size 0x66(0x41be bytes) 2024-04-24T15:32:28.186 INFO:teuthology.orchestra.run.smithi096.stdout:755 writesame 0x215487 thru 0x21fc52 data_size 0x2(0xa7cc bytes) 2024-04-24T15:32:28.186 INFO:teuthology.orchestra.run.smithi096.stdout:757 trunc from 0x6e0f66c to 0x63ffa2a 2024-04-24T15:32:28.186 INFO:teuthology.orchestra.run.smithi096.stdout:758 read 0x566cc1 thru 0x573c87 (0xcfc7 bytes) 2024-04-24T15:32:28.186 INFO:teuthology.orchestra.run.smithi096.stdout:759 read 0x218b498 thru 0x21995fa (0xe163 bytes) 2024-04-24T15:32:28.186 INFO:teuthology.orchestra.run.smithi096.stdout:760 write 0xa472484 thru 0xa47b900 (0x947d bytes) 2024-04-24T15:32:28.186 INFO:teuthology.orchestra.run.smithi096.stdout:761 write 0x9eba6af thru 0x9eca480 (0xfdd2 bytes) 2024-04-24T15:32:28.186 INFO:teuthology.orchestra.run.smithi096.stdout:763 read 0x37c759f thru 0x37cfad4 (0x8536 bytes) 2024-04-24T15:32:28.186 INFO:teuthology.orchestra.run.smithi096.stdout:764 read 0x7ca08a6 thru 0x7caa1aa (0x9905 bytes) 2024-04-24T15:32:28.186 INFO:teuthology.orchestra.run.smithi096.stdout:765 read 0x561a48a thru 0x5629ffc (0xfb73 bytes) 2024-04-24T15:32:28.186 INFO:teuthology.orchestra.run.smithi096.stdout:766 write 0x78a5228 thru 0x78a64a6 (0x127f bytes) 2024-04-24T15:32:28.186 INFO:teuthology.orchestra.run.smithi096.stdout:769 writesame 0x98d0c5a thru 0x98de1b9 data_size 0x10(0xd560 bytes) 2024-04-24T15:32:28.186 INFO:teuthology.orchestra.run.smithi096.stdout:771 read 0x6a99835 thru 0x6aa3ac6 (0xa292 bytes) 2024-04-24T15:32:28.186 INFO:teuthology.orchestra.run.smithi096.stdout:773 write 0xdedcfb5 thru 0xdee112c (0x4178 bytes) 2024-04-24T15:32:28.187 INFO:teuthology.orchestra.run.smithi096.stdout:774 write 0x275771a thru 0x275e6ae (0x6f95 bytes) 2024-04-24T15:32:28.187 INFO:teuthology.orchestra.run.smithi096.stdout:775 write 0x969b236 thru 0x969e064 (0x2e2f bytes) 2024-04-24T15:32:28.187 INFO:teuthology.orchestra.run.smithi096.stdout:776 read 0x55b7428 thru 0x55b9d29 (0x2902 bytes) 2024-04-24T15:32:28.187 INFO:teuthology.orchestra.run.smithi096.stdout:777 writesame 0x46d2f0e thru 0x46df525 data_size 0xc(0xc618 bytes) 2024-04-24T15:32:28.187 INFO:teuthology.orchestra.run.smithi096.stdout:778 writesame 0xcbf53ec thru 0xcbfa19f data_size 0x2(0x4db4 bytes) 2024-04-24T15:32:28.187 INFO:teuthology.orchestra.run.smithi096.stdout:780 trunc from 0xdee112d to 0x612ea08 2024-04-24T15:32:28.187 INFO:teuthology.orchestra.run.smithi096.stdout:781 write 0x40de8a6 thru 0x40deae8 (0x243 bytes) 2024-04-24T15:32:28.189 INFO:teuthology.orchestra.run.smithi096.stdout:782 write 0x8a63874 thru 0x8a7060c (0xcd99 bytes) 2024-04-24T15:32:28.194 INFO:teuthology.orchestra.run.smithi096.stdout:783 trunc from 0x8a7060d to 0xb1858b1 2024-04-24T15:32:28.200 INFO:teuthology.orchestra.run.smithi096.stdout:784 punch from 0xb1443 to 0xb9a65, (0x8622 bytes) 2024-04-24T15:32:28.200 INFO:teuthology.orchestra.run.smithi096.stdout:786 writesame 0x1401829 thru 0x14076ab data_size 0x5(0x5e83 bytes) 2024-04-24T15:32:28.212 INFO:teuthology.orchestra.run.smithi096.stdout:787 writesame 0xd269976 thru 0xd2787ec data_size 0x3f(0xee77 bytes) 2024-04-24T15:32:28.212 INFO:teuthology.orchestra.run.smithi096.stdout:789 read 0x71d2170 thru 0x71d7125 (0x4fb6 bytes) 2024-04-24T15:32:28.212 INFO:teuthology.orchestra.run.smithi096.stdout:790 write 0x4d6b039 thru 0x4d6c7f8 (0x17c0 bytes) 2024-04-24T15:32:28.214 INFO:teuthology.orchestra.run.smithi096.stdout:791 read 0xb57528c thru 0xb57e00f (0x8d84 bytes) 2024-04-24T15:32:28.214 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:28.214 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:28.214 INFO:teuthology.orchestra.run.smithi096.stdout:796 writesame 0x8e36ce4 thru 0x8e3d64a data_size 0xb(0x6967 bytes) 2024-04-24T15:32:28.216 INFO:teuthology.orchestra.run.smithi096.stdout:797 read 0x133fcda thru 0x13483bb (0x86e2 bytes) 2024-04-24T15:32:28.217 INFO:teuthology.orchestra.run.smithi096.stdout:800 write 0xcdd959f thru 0xcde2ede (0x9940 bytes) 2024-04-24T15:32:28.220 INFO:teuthology.orchestra.run.smithi096.stdout:801 punch from 0x5a545af to 0x5a55fb2, (0x1a03 bytes) 2024-04-24T15:32:28.220 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:28.220 INFO:teuthology.orchestra.run.smithi096.stdout:804 read 0xa224f7f thru 0xa231dac (0xce2e bytes) 2024-04-24T15:32:28.220 INFO:teuthology.orchestra.run.smithi096.stdout:805 read 0xa911bdb thru 0xa913b20 (0x1f46 bytes) 2024-04-24T15:32:28.220 INFO:teuthology.orchestra.run.smithi096.stdout:806 write 0x922e0ee thru 0x922e64d (0x560 bytes) 2024-04-24T15:32:28.234 INFO:teuthology.orchestra.run.smithi096.stdout:807 trunc from 0xd2787ed to 0x3c61537 2024-04-24T15:32:28.245 INFO:teuthology.orchestra.run.smithi096.stdout:811 read 0x27ccc70 thru 0x27d574e (0x8adf bytes) 2024-04-24T15:32:28.258 INFO:teuthology.orchestra.run.smithi096.stdout:813 trunc from 0x3c61537 to 0xe0186a9 2024-04-24T15:32:28.262 INFO:teuthology.orchestra.run.smithi096.stdout:814 read 0xa219aa5 thru 0xa21a7f2 (0xd4e bytes) 2024-04-24T15:32:28.269 INFO:teuthology.orchestra.run.smithi096.stdout:815 trunc from 0xe0186a9 to 0x7d09b72 2024-04-24T15:32:28.273 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:28.273 INFO:teuthology.orchestra.run.smithi096.stdout:818 read 0x357f067 thru 0x3582e08 (0x3da2 bytes) 2024-04-24T15:32:28.274 INFO:teuthology.orchestra.run.smithi096.stdout:819 write 0x28074eb thru 0x280b823 (0x4339 bytes) 2024-04-24T15:32:28.275 INFO:teuthology.orchestra.run.smithi096.stdout:822 write 0x2a98506 thru 0x2a9d01b (0x4b16 bytes) 2024-04-24T15:32:28.277 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:28.277 INFO:teuthology.orchestra.run.smithi096.stdout:826 write 0x7018f45 thru 0x7021a2c (0x8ae8 bytes) 2024-04-24T15:32:28.280 INFO:teuthology.orchestra.run.smithi096.stdout:828 read 0x63cba60 thru 0x63d378d (0x7d2e bytes) 2024-04-24T15:32:28.283 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:32:28.284+0000 7f23abcf6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 44021) UID: 0 2024-04-24T15:32:28.285 INFO:teuthology.orchestra.run.smithi096.stdout:830 trunc from 0x7d09b72 to 0xbd057ad 2024-04-24T15:32:28.289 INFO:teuthology.orchestra.run.smithi096.stdout:832 writesame 0x66de962 thru 0x66e9737 data_size 0x3(0xadd6 bytes) 2024-04-24T15:32:28.301 INFO:teuthology.orchestra.run.smithi096.stdout:834 trunc from 0xbd057ad to 0xe9ee329 2024-04-24T15:32:28.306 INFO:teuthology.orchestra.run.smithi096.stdout:835 write 0x28b19f1 thru 0x28be19f (0xc7af bytes) 2024-04-24T15:32:28.308 INFO:teuthology.orchestra.run.smithi096.stdout:837 read 0xd32a04e thru 0xd32a936 (0x8e9 bytes) 2024-04-24T15:32:28.308 INFO:teuthology.orchestra.run.smithi096.stdout:838 read 0x9d46570 thru 0x9d472cc (0xd5d bytes) 2024-04-24T15:32:28.308 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:28.308 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:28.308 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:28.308 INFO:teuthology.orchestra.run.smithi096.stdout:842 write 0x601677 thru 0x60ca63 (0xb3ed bytes) 2024-04-24T15:32:28.309 INFO:teuthology.orchestra.run.smithi096.stdout:843 writesame 0x7083abc thru 0x708ee50 data_size 0x1(0xb395 bytes) 2024-04-24T15:32:28.329 INFO:teuthology.orchestra.run.smithi096.stdout:844 read 0x4689213 thru 0x468aa2c (0x181a bytes) 2024-04-24T15:32:28.329 INFO:teuthology.orchestra.run.smithi096.stdout:845 write 0x7ecf99 thru 0x7f8edb (0xbf43 bytes) 2024-04-24T15:32:28.331 INFO:teuthology.orchestra.run.smithi096.stdout:846 punch from 0xc03158 to 0xc088a5, (0x574d bytes) 2024-04-24T15:32:28.335 INFO:teuthology.orchestra.run.smithi096.stdout:849 trunc from 0xe9ee329 to 0xbc4d7bd 2024-04-24T15:32:28.347 INFO:teuthology.orchestra.run.smithi096.stdout:850 trunc from 0xbc4d7bd to 0x79ba235 2024-04-24T15:32:28.353 INFO:teuthology.orchestra.run.smithi096.stdout:851 read 0x566f10c thru 0x56796dc (0xa5d1 bytes) 2024-04-24T15:32:28.353 INFO:teuthology.orchestra.run.smithi096.stdout:852 punch from 0x581a567 to 0x581bece, (0x1967 bytes) 2024-04-24T15:32:28.353 INFO:teuthology.orchestra.run.smithi096.stdout:854 punch from 0x6dff8be to 0x6dffbab, (0x2ed bytes) 2024-04-24T15:32:28.362 INFO:teuthology.orchestra.run.smithi096.stdout:855 write 0xa0495c3 thru 0xa04c2a7 (0x2ce5 bytes) 2024-04-24T15:32:28.365 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:28.365 INFO:teuthology.orchestra.run.smithi096.stdout:857 read 0x868dad5 thru 0x868f73e (0x1c6a bytes) 2024-04-24T15:32:28.365 INFO:teuthology.orchestra.run.smithi096.stdout:858 writesame 0xceaff0 thru 0xcf09b8 data_size 0x1(0x59c9 bytes) 2024-04-24T15:32:28.376 INFO:teuthology.orchestra.run.smithi096.stdout:862 write 0x527e1d thru 0x52cedf (0x50c3 bytes) 2024-04-24T15:32:28.378 INFO:teuthology.orchestra.run.smithi096.stdout:863 punch from 0x62fde04 to 0x63005ac, (0x27a8 bytes) 2024-04-24T15:32:28.381 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:32:28.382 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:32:28.382 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-04-24T15:32:28.382 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:32:28.382 INFO:teuthology.orchestra.run.smithi096.stdout:864 trunc from 0xa04c2a8 to 0xcfdf20a 2024-04-24T15:32:28.384 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:32:28.385+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:32:28.386 INFO:teuthology.orchestra.run.smithi096.stdout:865 write 0xcde45c5 thru 0xcde7fdf (0x3a1b bytes) 2024-04-24T15:32:28.394 INFO:teuthology.orchestra.run.smithi096.stdout:866 trunc from 0xcfdf20a to 0x8c950f0 2024-04-24T15:32:28.395 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-24T15:32:28.405 INFO:teuthology.orchestra.run.smithi096.stdout:870 trunc from 0x8c950f0 to 0x75e999f 2024-04-24T15:32:28.427 INFO:teuthology.orchestra.run.smithi096.stdout:871 write 0xb9a97b6 thru 0xb9b15e7 (0x7e32 bytes) 2024-04-24T15:32:28.430 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:28.430 INFO:teuthology.orchestra.run.smithi096.stdout:873 write 0x3ada7a9 thru 0x3ae8cb5 (0xe50d bytes) 2024-04-24T15:32:28.433 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:28.433 INFO:teuthology.orchestra.run.smithi096.stdout:875 read 0x932cb24 thru 0x933b02a (0xe507 bytes) 2024-04-24T15:32:28.433 INFO:teuthology.orchestra.run.smithi096.stdout:876 write 0x82af5d4 thru 0x82b3dc6 (0x47f3 bytes) 2024-04-24T15:32:28.435 INFO:teuthology.orchestra.run.smithi096.stdout:877 read 0x8153d2c thru 0x81613ad (0xd682 bytes) 2024-04-24T15:32:28.442 INFO:teuthology.orchestra.run.smithi096.stdout:878 write 0xc67594f thru 0xc67aef4 (0x55a6 bytes) 2024-04-24T15:32:28.444 INFO:teuthology.orchestra.run.smithi096.stdout:879 read 0x5d9a66a thru 0x5da5402 (0xad99 bytes) 2024-04-24T15:32:28.445 INFO:teuthology.orchestra.run.smithi096.stdout:880 read 0x8a3e49c thru 0x8a44fc8 (0x6b2d bytes) 2024-04-24T15:32:28.445 INFO:teuthology.orchestra.run.smithi096.stdout:881 read 0x14f0a20 thru 0x14fc539 (0xbb1a bytes) 2024-04-24T15:32:28.446 INFO:teuthology.orchestra.run.smithi096.stdout:882 read 0x646876a thru 0x6469097 (0x92e bytes) 2024-04-24T15:32:28.446 INFO:teuthology.orchestra.run.smithi096.stdout:883 read 0x7dbe7a6 thru 0x7dcacdd (0xc538 bytes) 2024-04-24T15:32:28.460 INFO:teuthology.orchestra.run.smithi096.stdout:884 trunc from 0xc67aef5 to 0xc80cf6 2024-04-24T15:32:28.484 INFO:teuthology.orchestra.run.smithi096.stdout:885 write 0x44562b7 thru 0x445b07e (0x4dc8 bytes) 2024-04-24T15:32:28.485 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:32:28.486+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:32:28.487 INFO:teuthology.orchestra.run.smithi096.stdout:886 punch from 0x2355efd to 0x235c3c7, (0x64ca bytes) 2024-04-24T15:32:28.487 INFO:teuthology.orchestra.run.smithi096.stdout:887 read 0xf5757a thru 0xf63407 (0xbe8e bytes) 2024-04-24T15:32:28.487 INFO:teuthology.orchestra.run.smithi096.stdout:889 punch from 0x308fb09 to 0x309812e, (0x8625 bytes) 2024-04-24T15:32:28.498 INFO:teuthology.orchestra.run.smithi096.stdout:890 trunc from 0x445b07f to 0xd85c65b 2024-04-24T15:32:28.517 INFO:teuthology.orchestra.run.smithi096.stdout:891 trunc from 0xd85c65b to 0x3d13bb7 2024-04-24T15:32:28.545 INFO:teuthology.orchestra.run.smithi096.stdout:893 write 0xc042d84 thru 0xc0521f2 (0xf46f bytes) 2024-04-24T15:32:28.548 INFO:teuthology.orchestra.run.smithi096.stdout:894 punch from 0x2397173 to 0x239e009, (0x6e96 bytes) 2024-04-24T15:32:28.548 INFO:teuthology.orchestra.run.smithi096.stdout:895 punch from 0x4e939e4 to 0x4e9832c, (0x4948 bytes) 2024-04-24T15:32:28.548 INFO:teuthology.orchestra.run.smithi096.stdout:896 write 0xb0cd495 thru 0xb0d0a81 (0x35ed bytes) 2024-04-24T15:32:28.551 INFO:teuthology.orchestra.run.smithi096.stdout:897 write 0x21b9cf9 thru 0x21c63b5 (0xc6bd bytes) 2024-04-24T15:32:28.553 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:32:28.553 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:32:28.553 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-04-24T15:32:28.553 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-04-24T15:32:28.553 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:32:28.553 INFO:teuthology.orchestra.run.smithi096.stdout:899 write 0x1e22259 thru 0x1e23c53 (0x19fb bytes) 2024-04-24T15:32:28.554 INFO:teuthology.orchestra.run.smithi096.stdout:900 read 0x40a23df thru 0x40b063d (0xe25f bytes) 2024-04-24T15:32:28.554 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:28.555 INFO:teuthology.orchestra.run.smithi096.stdout:902 write 0x93e85d7 thru 0x93ec37f (0x3da9 bytes) 2024-04-24T15:32:28.558 INFO:teuthology.orchestra.run.smithi096.stdout:905 write 0x6c9d881 thru 0x6ca2bef (0x536f bytes) 2024-04-24T15:32:28.560 INFO:teuthology.orchestra.run.smithi096.stdout:908 punch from 0x77deeaf to 0x77eb473, (0xc5c4 bytes) 2024-04-24T15:32:28.560 INFO:teuthology.orchestra.run.smithi096.stdout:909 read 0x2840dc7 thru 0x284f42d (0xe667 bytes) 2024-04-24T15:32:28.561 INFO:teuthology.orchestra.run.smithi096.stdout:910 trunc from 0xc0521f3 to 0xbe6a595 2024-04-24T15:32:28.564 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-24T15:32:28.571 INFO:teuthology.orchestra.run.smithi096.stdout:911 write 0x4fd38f4 thru 0x4fe07fc (0xcf09 bytes) 2024-04-24T15:32:28.574 INFO:teuthology.orchestra.run.smithi096.stdout:912 punch from 0x6b9bb54 to 0x6ba9f13, (0xe3bf bytes) 2024-04-24T15:32:28.574 INFO:teuthology.orchestra.run.smithi096.stdout:913 write 0x3d29304 thru 0x3d37cf9 (0xe9f6 bytes) 2024-04-24T15:32:28.576 INFO:teuthology.orchestra.run.smithi096.stdout:914 writesame 0x9f44fc8 thru 0x9f45193 data_size 0xa(0x1cc bytes) 2024-04-24T15:32:28.577 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:28.578 INFO:teuthology.orchestra.run.smithi096.stdout:916 punch from 0x365b559 to 0x3665074, (0x9b1b bytes) 2024-04-24T15:32:28.580 INFO:teuthology.orchestra.run.smithi096.stdout:917 trunc from 0xbe6a595 to 0xe1d00b6 2024-04-24T15:32:28.585 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:32:28.587+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:32:28.586 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:28.586 INFO:teuthology.orchestra.run.smithi096.stdout:919 punch from 0x7b5b40e to 0x7b60dd8, (0x59ca bytes) 2024-04-24T15:32:28.586 INFO:teuthology.orchestra.run.smithi096.stdout:920 write 0x6b096bf thru 0x6b15da7 (0xc6e9 bytes) 2024-04-24T15:32:28.588 INFO:teuthology.orchestra.run.smithi096.stdout:921 read 0x598c677 thru 0x5998b7e (0xc508 bytes) 2024-04-24T15:32:28.594 INFO:tasks.ceph.mon.a.smithi096.stderr:2024-04-24T15:32:28.595+0000 7fa88ddc8700 -1 mon.a@0(leader).osd e37 definitely_dead 0 2024-04-24T15:32:28.597 INFO:teuthology.orchestra.run.smithi096.stdout:922 trunc from 0xe1d00b6 to 0x80e75cf 2024-04-24T15:32:28.608 INFO:teuthology.orchestra.run.smithi096.stdout:923 read 0x286d09e thru 0x286d61f (0x582 bytes) 2024-04-24T15:32:28.609 INFO:teuthology.orchestra.run.smithi096.stdout:924 punch from 0x7f457a2 to 0x7f4d5c1, (0x7e1f bytes) 2024-04-24T15:32:28.609 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:28.609 INFO:teuthology.orchestra.run.smithi096.stdout:926 read 0x63e6206 thru 0x63eb931 (0x572c bytes) 2024-04-24T15:32:28.609 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:28.623 INFO:teuthology.orchestra.run.smithi096.stdout:929 write 0xd70a330 thru 0xd70aa92 (0x763 bytes) 2024-04-24T15:32:28.637 INFO:teuthology.orchestra.run.smithi096.stdout:930 trunc from 0xd70aa93 to 0x5da4d62 2024-04-24T15:32:28.653 INFO:teuthology.orchestra.run.smithi096.stdout:931 write 0x6c7126c thru 0x6c7a8ef (0x9684 bytes) 2024-04-24T15:32:28.654 INFO:teuthology.orchestra.run.smithi096.stdout:934 read 0x2f4a0fc thru 0x2f50dc8 (0x6ccd bytes) 2024-04-24T15:32:28.655 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:28.667 INFO:teuthology.orchestra.run.smithi096.stdout:936 writesame 0xdcd846d thru 0xdce6f4f data_size 0x1(0xeae3 bytes) 2024-04-24T15:32:28.674 INFO:teuthology.orchestra.run.smithi096.stdout:939 trunc from 0xdce6f50 to 0x81d7881 2024-04-24T15:32:28.680 INFO:teuthology.orchestra.run.smithi096.stdout:940 read 0x3741046 thru 0x37502e6 (0xf2a1 bytes) 2024-04-24T15:32:28.680 INFO:teuthology.orchestra.run.smithi096.stdout:941 write 0x3a6f8f1 thru 0x3a79ae4 (0xa1f4 bytes) 2024-04-24T15:32:28.686 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:32:28.688+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:32:28.687 INFO:teuthology.orchestra.run.smithi096.stdout:943 write 0x8a20a5b thru 0x8a3018b (0xf731 bytes) 2024-04-24T15:32:28.690 INFO:teuthology.orchestra.run.smithi096.stdout:944 read 0x33da06a thru 0x33dc5c9 (0x2560 bytes) 2024-04-24T15:32:28.700 INFO:teuthology.orchestra.run.smithi096.stdout:945 trunc from 0x8a3018c to 0xed01cc 2024-04-24T15:32:28.720 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:32:28.720 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-04-24T15:32:28.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-04-24T15:32:28.720 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-04-24T15:32:28.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.721 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:18 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'8, mlcod=15'5)", 2024-04-24T15:32:28.721 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:00.973932+0000", 2024-04-24T15:32:28.721 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.746194649, 2024-04-24T15:32:28.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0064286839999999996, 2024-04-24T15:32:28.721 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:28.721 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:28.721 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:28.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.721 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:28.721 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.973932+0000", 2024-04-24T15:32:28.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:28.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.722 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:28.722 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.973932+0000", 2024-04-24T15:32:28.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:28.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.722 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.722 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:28.722 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.973931+0000", 2024-04-24T15:32:28.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:28.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.722 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.722 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:28.722 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.973938+0000", 2024-04-24T15:32:28.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.029e-06 2024-04-24T15:32:28.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:28.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.973948+0000", 2024-04-24T15:32:28.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9129999999999998e-06 2024-04-24T15:32:28.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:28.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.973952+0000", 2024-04-24T15:32:28.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3069999999999996e-06 2024-04-24T15:32:28.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:28.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.973986+0000", 2024-04-24T15:32:28.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3643000000000003e-05 2024-04-24T15:32:28.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.724 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:28.724 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.974006+0000", 2024-04-24T15:32:28.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9658999999999999e-05 2024-04-24T15:32:28.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.724 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:28.724 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.980330+0000", 2024-04-24T15:32:28.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0063243479999999996 2024-04-24T15:32:28.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.724 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:28.724 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.980361+0000", 2024-04-24T15:32:28.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0527000000000001e-05 2024-04-24T15:32:28.725 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:28.725 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:28.725 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:28.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.725 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:21 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'11, mlcod=15'8)", 2024-04-24T15:32:28.725 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:00.986502+0000", 2024-04-24T15:32:28.725 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.733624716000001, 2024-04-24T15:32:28.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0062008819999999996, 2024-04-24T15:32:28.725 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:28.725 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:28.725 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:28.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.726 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:28.726 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.986502+0000", 2024-04-24T15:32:28.726 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:28.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.726 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.726 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:28.726 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.986502+0000", 2024-04-24T15:32:28.726 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:28.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.726 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.726 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:28.726 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.986501+0000", 2024-04-24T15:32:28.726 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:28.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:28.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.986639+0000", 2024-04-24T15:32:28.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013731400000000001 2024-04-24T15:32:28.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:28.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.986649+0000", 2024-04-24T15:32:28.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.041e-05 2024-04-24T15:32:28.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:28.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.986654+0000", 2024-04-24T15:32:28.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2000000000000002e-06 2024-04-24T15:32:28.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:28.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.986678+0000", 2024-04-24T15:32:28.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3731e-05 2024-04-24T15:32:28.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:28.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.986691+0000", 2024-04-24T15:32:28.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3078000000000001e-05 2024-04-24T15:32:28.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:28.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.992666+0000", 2024-04-24T15:32:28.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0059751650000000002 2024-04-24T15:32:28.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:28.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.992703+0000", 2024-04-24T15:32:28.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6622000000000003e-05 2024-04-24T15:32:28.729 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:28.729 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:28.729 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:28.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.729 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:22 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'12, mlcod=15'8)", 2024-04-24T15:32:28.729 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:00.987102+0000", 2024-04-24T15:32:28.729 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.733024870000001, 2024-04-24T15:32:28.730 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0056291650000000002, 2024-04-24T15:32:28.730 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:28.730 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:28.730 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:28.730 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.730 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:28.730 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.987102+0000", 2024-04-24T15:32:28.730 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:28.730 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.730 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.730 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:28.730 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.987102+0000", 2024-04-24T15:32:28.730 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:28.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.731 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.731 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:28.731 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.987101+0000", 2024-04-24T15:32:28.731 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:28.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.731 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.731 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:28.731 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.987106+0000", 2024-04-24T15:32:28.731 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0969999999999996e-06 2024-04-24T15:32:28.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.731 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.731 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:28.731 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.987113+0000", 2024-04-24T15:32:28.731 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6789999999999999e-06 2024-04-24T15:32:28.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:28.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.987117+0000", 2024-04-24T15:32:28.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6519999999999999e-06 2024-04-24T15:32:28.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:28.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.987143+0000", 2024-04-24T15:32:28.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6098999999999999e-05 2024-04-24T15:32:28.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:28.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.987157+0000", 2024-04-24T15:32:28.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4100999999999999e-05 2024-04-24T15:32:28.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:28.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.992706+0000", 2024-04-24T15:32:28.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0055495630000000004 2024-04-24T15:32:28.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:28.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.992731+0000", 2024-04-24T15:32:28.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4519e-05 2024-04-24T15:32:28.733 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:28.733 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:28.734 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:28.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.734 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.734 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:69 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'43, mlcod=15'39)", 2024-04-24T15:32:28.734 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.058504+0000", 2024-04-24T15:32:28.734 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.661622395999999, 2024-04-24T15:32:28.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.006494606, 2024-04-24T15:32:28.734 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:28.734 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:28.734 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:28.734 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.734 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:28.734 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.058504+0000", 2024-04-24T15:32:28.735 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:28.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.735 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:28.735 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.058504+0000", 2024-04-24T15:32:28.735 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:28.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.735 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:28.735 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.058503+0000", 2024-04-24T15:32:28.735 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:28.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.735 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:28.736 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.058588+0000", 2024-04-24T15:32:28.736 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4087999999999999e-05 2024-04-24T15:32:28.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.736 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.736 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:28.736 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.058597+0000", 2024-04-24T15:32:28.736 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7459999999999995e-06 2024-04-24T15:32:28.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.736 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.736 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:28.736 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.058602+0000", 2024-04-24T15:32:28.736 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6929999999999998e-06 2024-04-24T15:32:28.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.736 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:28.737 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.060983+0000", 2024-04-24T15:32:28.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023812690000000001 2024-04-24T15:32:28.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:28.737 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.060996+0000", 2024-04-24T15:32:28.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2826999999999999e-05 2024-04-24T15:32:28.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:28.737 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.064972+0000", 2024-04-24T15:32:28.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0039757890000000004 2024-04-24T15:32:28.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.738 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.738 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:28.738 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.064999+0000", 2024-04-24T15:32:28.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6917999999999999e-05 2024-04-24T15:32:28.738 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:28.738 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:28.738 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:28.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.738 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.738 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:70 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'44, mlcod=15'39)", 2024-04-24T15:32:28.738 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.059180+0000", 2024-04-24T15:32:28.738 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.660946643999999, 2024-04-24T15:32:28.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0058429709999999998, 2024-04-24T15:32:28.739 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:28.739 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:28.739 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:28.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:28.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.059180+0000", 2024-04-24T15:32:28.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:28.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:28.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.059180+0000", 2024-04-24T15:32:28.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:28.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:28.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.059179+0000", 2024-04-24T15:32:28.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:28.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:28.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.059184+0000", 2024-04-24T15:32:28.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.074e-06 2024-04-24T15:32:28.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:28.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.059194+0000", 2024-04-24T15:32:28.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6050000000000004e-06 2024-04-24T15:32:28.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:28.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.059199+0000", 2024-04-24T15:32:28.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2639999999999999e-06 2024-04-24T15:32:28.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:28.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.061424+0000", 2024-04-24T15:32:28.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0022244880000000002 2024-04-24T15:32:28.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:28.742 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.061444+0000", 2024-04-24T15:32:28.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0633e-05 2024-04-24T15:32:28.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.742 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:28.742 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.065001+0000", 2024-04-24T15:32:28.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035567670000000002 2024-04-24T15:32:28.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.742 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:28.742 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.065023+0000", 2024-04-24T15:32:28.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1724000000000001e-05 2024-04-24T15:32:28.743 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:28.743 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:28.743 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:28.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.743 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:154 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'108, mlcod=15'105)", 2024-04-24T15:32:28.743 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.929033+0000", 2024-04-24T15:32:28.743 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.791093263, 2024-04-24T15:32:28.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0068887990000000001, 2024-04-24T15:32:28.743 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:28.743 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:28.743 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:28.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:28.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929033+0000", 2024-04-24T15:32:28.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:28.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:28.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929033+0000", 2024-04-24T15:32:28.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:28.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:28.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929033+0000", 2024-04-24T15:32:28.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:28.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:28.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929042+0000", 2024-04-24T15:32:28.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5440000000000004e-06 2024-04-24T15:32:28.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:28.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929051+0000", 2024-04-24T15:32:28.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0580000000000004e-06 2024-04-24T15:32:28.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:28.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929055+0000", 2024-04-24T15:32:28.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.337e-06 2024-04-24T15:32:28.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.746 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.746 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:28.746 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929074+0000", 2024-04-24T15:32:28.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9724999999999998e-05 2024-04-24T15:32:28.746 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.746 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.746 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:28.746 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929084+0000", 2024-04-24T15:32:28.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7049999999999994e-06 2024-04-24T15:32:28.746 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.746 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.746 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:28.746 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.935895+0000", 2024-04-24T15:32:28.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0068104359999999996 2024-04-24T15:32:28.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:28.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.935922+0000", 2024-04-24T15:32:28.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7538999999999998e-05 2024-04-24T15:32:28.747 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:28.747 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:28.747 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:28.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.747 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:155 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'109, mlcod=15'105)", 2024-04-24T15:32:28.747 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.929398+0000", 2024-04-24T15:32:28.747 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.790728282, 2024-04-24T15:32:28.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0098754700000000008, 2024-04-24T15:32:28.748 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:28.748 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:28.748 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:28.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:28.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929398+0000", 2024-04-24T15:32:28.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:28.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:28.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929398+0000", 2024-04-24T15:32:28.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:28.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:28.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929398+0000", 2024-04-24T15:32:28.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:28.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:28.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929571+0000", 2024-04-24T15:32:28.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017281999999999999 2024-04-24T15:32:28.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:28.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929578+0000", 2024-04-24T15:32:28.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8929999999999999e-06 2024-04-24T15:32:28.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:28.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929581+0000", 2024-04-24T15:32:28.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1709999999999998e-06 2024-04-24T15:32:28.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:28.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929605+0000", 2024-04-24T15:32:28.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3938e-05 2024-04-24T15:32:28.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:28.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929618+0000", 2024-04-24T15:32:28.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3653e-05 2024-04-24T15:32:28.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:28.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939186+0000", 2024-04-24T15:32:28.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0095674260000000004 2024-04-24T15:32:28.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:28.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939274+0000", 2024-04-24T15:32:28.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7928000000000002e-05 2024-04-24T15:32:28.751 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:28.751 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:28.751 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:28.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.752 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:156 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'110, mlcod=15'105)", 2024-04-24T15:32:28.752 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.930050+0000", 2024-04-24T15:32:28.752 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.790076966000001, 2024-04-24T15:32:28.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0092911650000000005, 2024-04-24T15:32:28.752 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:28.752 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:28.752 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:28.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:28.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930050+0000", 2024-04-24T15:32:28.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:28.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:28.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930050+0000", 2024-04-24T15:32:28.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:28.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:28.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930049+0000", 2024-04-24T15:32:28.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:28.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:28.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930053+0000", 2024-04-24T15:32:28.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.98e-06 2024-04-24T15:32:28.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:28.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930058+0000", 2024-04-24T15:32:28.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3530000000000002e-06 2024-04-24T15:32:28.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:28.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930061+0000", 2024-04-24T15:32:28.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6970000000000002e-06 2024-04-24T15:32:28.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:28.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930082+0000", 2024-04-24T15:32:28.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0815e-05 2024-04-24T15:32:28.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:28.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930093+0000", 2024-04-24T15:32:28.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0699999999999999e-05 2024-04-24T15:32:28.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:28.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939280+0000", 2024-04-24T15:32:28.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0091874880000000006 2024-04-24T15:32:28.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:28.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939341+0000", 2024-04-24T15:32:28.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0608e-05 2024-04-24T15:32:28.756 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:28.756 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:28.756 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:28.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.756 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:157 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'111, mlcod=15'106)", 2024-04-24T15:32:28.756 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.930506+0000", 2024-04-24T15:32:28.756 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.789620108000001, 2024-04-24T15:32:28.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.008889246, 2024-04-24T15:32:28.756 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:28.756 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:28.756 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:28.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:28.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930506+0000", 2024-04-24T15:32:28.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:28.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:28.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930506+0000", 2024-04-24T15:32:28.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:28.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:28.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930506+0000", 2024-04-24T15:32:28.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:28.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:28.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930510+0000", 2024-04-24T15:32:28.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1810000000000003e-06 2024-04-24T15:32:28.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:28.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930516+0000", 2024-04-24T15:32:28.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1920000000000002e-06 2024-04-24T15:32:28.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:28.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930520+0000", 2024-04-24T15:32:28.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.501e-06 2024-04-24T15:32:28.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:28.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930539+0000", 2024-04-24T15:32:28.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8929000000000001e-05 2024-04-24T15:32:28.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:28.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930550+0000", 2024-04-24T15:32:28.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1168999999999999e-05 2024-04-24T15:32:28.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:28.760 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939343+0000", 2024-04-24T15:32:28.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0087932900000000005 2024-04-24T15:32:28.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:28.760 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939396+0000", 2024-04-24T15:32:28.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2454999999999998e-05 2024-04-24T15:32:28.760 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:28.760 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:28.760 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:28.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.760 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:158 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'112, mlcod=15'106)", 2024-04-24T15:32:28.760 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.930877+0000", 2024-04-24T15:32:28.761 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.789249417000001, 2024-04-24T15:32:28.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.008572741, 2024-04-24T15:32:28.761 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:28.761 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:28.761 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:28.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:28.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930877+0000", 2024-04-24T15:32:28.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:28.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:28.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930877+0000", 2024-04-24T15:32:28.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:28.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:28.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930877+0000", 2024-04-24T15:32:28.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:28.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:28.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930968+0000", 2024-04-24T15:32:28.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1638999999999995e-05 2024-04-24T15:32:28.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:28.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930975+0000", 2024-04-24T15:32:28.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2169999999999999e-06 2024-04-24T15:32:28.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:28.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930977+0000", 2024-04-24T15:32:28.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.791e-06 2024-04-24T15:32:28.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:28.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931002+0000", 2024-04-24T15:32:28.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4491e-05 2024-04-24T15:32:28.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:28.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931012+0000", 2024-04-24T15:32:28.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0217000000000001e-05 2024-04-24T15:32:28.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:28.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939398+0000", 2024-04-24T15:32:28.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.008385366 2024-04-24T15:32:28.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:28.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939450+0000", 2024-04-24T15:32:28.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2326e-05 2024-04-24T15:32:28.765 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:28.765 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:28.765 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:28.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.765 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:159 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'113, mlcod=15'106)", 2024-04-24T15:32:28.765 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.931313+0000", 2024-04-24T15:32:28.765 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.788814065, 2024-04-24T15:32:28.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0081994159999999993, 2024-04-24T15:32:28.765 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:28.765 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:28.765 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:28.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:28.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931313+0000", 2024-04-24T15:32:28.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:28.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:28.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931313+0000", 2024-04-24T15:32:28.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:28.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:28.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931312+0000", 2024-04-24T15:32:28.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:28.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:28.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931316+0000", 2024-04-24T15:32:28.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0589999999999996e-06 2024-04-24T15:32:28.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:28.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931322+0000", 2024-04-24T15:32:28.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2020000000000001e-06 2024-04-24T15:32:28.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:28.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931327+0000", 2024-04-24T15:32:28.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2440000000000004e-06 2024-04-24T15:32:28.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:28.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931348+0000", 2024-04-24T15:32:28.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1464e-05 2024-04-24T15:32:28.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:28.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931359+0000", 2024-04-24T15:32:28.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0699e-05 2024-04-24T15:32:28.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:28.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939452+0000", 2024-04-24T15:32:28.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0080931749999999993 2024-04-24T15:32:28.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:28.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939512+0000", 2024-04-24T15:32:28.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0050000000000003e-05 2024-04-24T15:32:28.769 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:28.769 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:28.769 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:28.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.769 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:2447 4.6 4:668081fa:::rbd_data.114a1f60cb66.00000000000001f0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-24T15:32:28.770 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:23.027189+0000", 2024-04-24T15:32:28.770 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 5.6929380409999997, 2024-04-24T15:32:28.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0056741200000000004, 2024-04-24T15:32:28.770 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:28.770 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:28.770 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:28.770 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:28.770 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:28.770 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2447 2024-04-24T15:32:28.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.770 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:28.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:28.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027189+0000", 2024-04-24T15:32:28.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:28.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:28.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027189+0000", 2024-04-24T15:32:28.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:28.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:28.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027188+0000", 2024-04-24T15:32:28.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:28.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:28.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027194+0000", 2024-04-24T15:32:28.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0499999999999997e-06 2024-04-24T15:32:28.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:28.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027205+0000", 2024-04-24T15:32:28.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1619e-05 2024-04-24T15:32:28.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:28.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027212+0000", 2024-04-24T15:32:28.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4289999999999998e-06 2024-04-24T15:32:28.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:28.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027243+0000", 2024-04-24T15:32:28.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1510000000000002e-05 2024-04-24T15:32:28.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:28.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027288+0000", 2024-04-24T15:32:28.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4484000000000003e-05 2024-04-24T15:32:28.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-24T15:32:28.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027351+0000", 2024-04-24T15:32:28.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3311000000000004e-05 2024-04-24T15:32:28.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:28.774 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.029874+0000", 2024-04-24T15:32:28.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00252315 2024-04-24T15:32:28.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:28.774 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.032786+0000", 2024-04-24T15:32:28.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0029115370000000001 2024-04-24T15:32:28.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:28.774 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.032828+0000", 2024-04-24T15:32:28.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2132000000000003e-05 2024-04-24T15:32:28.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:28.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.032863+0000", 2024-04-24T15:32:28.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4912e-05 2024-04-24T15:32:28.775 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:28.775 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:28.775 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:28.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.775 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:2449 4.6 4:61283b7c:::rbd_data.114a1f60cb66.00000000000001f2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-24T15:32:28.775 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:23.027620+0000", 2024-04-24T15:32:28.775 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 5.6925066280000003, 2024-04-24T15:32:28.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0053273959999999999, 2024-04-24T15:32:28.776 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:28.776 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:28.776 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:28.776 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:28.776 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:28.776 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2449 2024-04-24T15:32:28.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.776 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:28.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:28.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027620+0000", 2024-04-24T15:32:28.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:28.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:28.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027620+0000", 2024-04-24T15:32:28.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:28.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:28.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027619+0000", 2024-04-24T15:32:28.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:28.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:28.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027623+0000", 2024-04-24T15:32:28.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7100000000000001e-06 2024-04-24T15:32:28.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:28.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027636+0000", 2024-04-24T15:32:28.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2609000000000001e-05 2024-04-24T15:32:28.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:28.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027643+0000", 2024-04-24T15:32:28.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.182e-06 2024-04-24T15:32:28.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:28.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027678+0000", 2024-04-24T15:32:28.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5641e-05 2024-04-24T15:32:28.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:28.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027765+0000", 2024-04-24T15:32:28.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6691000000000001e-05 2024-04-24T15:32:28.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-24T15:32:28.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027864+0000", 2024-04-24T15:32:28.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9257000000000002e-05 2024-04-24T15:32:28.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:28.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.032638+0000", 2024-04-24T15:32:28.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004773429 2024-04-24T15:32:28.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:28.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.032896+0000", 2024-04-24T15:32:28.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000258352 2024-04-24T15:32:28.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:28.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.032924+0000", 2024-04-24T15:32:28.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8337999999999999e-05 2024-04-24T15:32:28.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:28.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.032947+0000", 2024-04-24T15:32:28.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2849000000000001e-05 2024-04-24T15:32:28.781 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:28.781 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:28.781 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:28.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.781 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4426.0:2767 4.6 e34/33 4:6297db09:::rbd_data.114a1f60cb66.00000000000000e1:head v 34'25, mlcod=34'23)", 2024-04-24T15:32:28.781 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.256572+0000", 2024-04-24T15:32:28.781 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.4635546509999999, 2024-04-24T15:32:28.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0050987539999999996, 2024-04-24T15:32:28.781 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:28.781 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:28.782 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:28.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:28.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.256572+0000", 2024-04-24T15:32:28.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:28.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:28.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.256572+0000", 2024-04-24T15:32:28.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:28.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:28.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.256571+0000", 2024-04-24T15:32:28.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:28.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:28.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.256578+0000", 2024-04-24T15:32:28.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.161e-06 2024-04-24T15:32:28.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:28.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.256585+0000", 2024-04-24T15:32:28.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.391e-06 2024-04-24T15:32:28.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:28.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.256590+0000", 2024-04-24T15:32:28.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8189999999999999e-06 2024-04-24T15:32:28.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:28.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.256623+0000", 2024-04-24T15:32:28.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3618000000000002e-05 2024-04-24T15:32:28.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:28.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.256642+0000", 2024-04-24T15:32:28.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8479999999999999e-05 2024-04-24T15:32:28.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:28.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.261622+0000", 2024-04-24T15:32:28.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0049796959999999996 2024-04-24T15:32:28.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:28.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.261671+0000", 2024-04-24T15:32:28.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9147000000000002e-05 2024-04-24T15:32:28.785 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:28.785 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:28.785 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:28.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.786 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4426.0:3917 4.3 e34/33 4:c23f5710:::rbd_data.114a1f60cb66.0000000000000091:head v 34'28, mlcod=34'26)", 2024-04-24T15:32:28.786 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.804899+0000", 2024-04-24T15:32:28.786 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.9152280020000001, 2024-04-24T15:32:28.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0050025249999999999, 2024-04-24T15:32:28.786 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:28.786 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:28.786 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:28.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:28.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.804899+0000", 2024-04-24T15:32:28.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:28.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:28.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.804899+0000", 2024-04-24T15:32:28.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:28.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:28.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.804898+0000", 2024-04-24T15:32:28.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:28.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:28.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.804903+0000", 2024-04-24T15:32:28.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9679999999999997e-06 2024-04-24T15:32:28.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:28.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.804911+0000", 2024-04-24T15:32:28.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8490000000000006e-06 2024-04-24T15:32:28.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:28.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.804916+0000", 2024-04-24T15:32:28.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5220000000000001e-06 2024-04-24T15:32:28.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:28.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.804958+0000", 2024-04-24T15:32:28.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2607000000000001e-05 2024-04-24T15:32:28.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:28.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.804975+0000", 2024-04-24T15:32:28.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6433999999999999e-05 2024-04-24T15:32:28.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:28.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.809868+0000", 2024-04-24T15:32:28.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0048933839999999998 2024-04-24T15:32:28.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:28.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.809901+0000", 2024-04-24T15:32:28.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3158999999999999e-05 2024-04-24T15:32:28.789 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:28.790 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:28.790 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:28.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.790 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4426.0:3934 4.6 e34/33 4:6297db09:::rbd_data.114a1f60cb66.00000000000000e1:head v 34'40, mlcod=34'37)", 2024-04-24T15:32:28.790 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.812857+0000", 2024-04-24T15:32:28.790 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.9072699440000003, 2024-04-24T15:32:28.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0052230590000000004, 2024-04-24T15:32:28.790 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:28.790 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:28.790 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:28.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:28.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.812857+0000", 2024-04-24T15:32:28.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:28.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:28.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.812857+0000", 2024-04-24T15:32:28.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:28.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:28.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.812856+0000", 2024-04-24T15:32:28.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:28.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:28.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.812861+0000", 2024-04-24T15:32:28.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4e-06 2024-04-24T15:32:28.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:28.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.812871+0000", 2024-04-24T15:32:28.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8679999999999994e-06 2024-04-24T15:32:28.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:28.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.812877+0000", 2024-04-24T15:32:28.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5310000000000001e-06 2024-04-24T15:32:28.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:28.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.812913+0000", 2024-04-24T15:32:28.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6548999999999999e-05 2024-04-24T15:32:28.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:28.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.812934+0000", 2024-04-24T15:32:28.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0590000000000001e-05 2024-04-24T15:32:28.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:28.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.818047+0000", 2024-04-24T15:32:28.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0051134090000000002 2024-04-24T15:32:28.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:28.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.818080+0000", 2024-04-24T15:32:28.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2292000000000002e-05 2024-04-24T15:32:28.794 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:28.794 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:28.794 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:28.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.794 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4426.0:3943 4.6 e34/33 4:639bc13c:::rbd_data.114a1f60cb66.00000000000000ea:head v 34'41, mlcod=34'38)", 2024-04-24T15:32:28.794 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.815615+0000", 2024-04-24T15:32:28.794 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.9045116520000001, 2024-04-24T15:32:28.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0050425490000000003, 2024-04-24T15:32:28.795 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:28.795 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:28.795 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:28.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:28.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.815615+0000", 2024-04-24T15:32:28.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:28.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:28.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.815615+0000", 2024-04-24T15:32:28.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:28.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:28.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.815614+0000", 2024-04-24T15:32:28.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:28.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:28.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.815620+0000", 2024-04-24T15:32:28.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1460000000000001e-06 2024-04-24T15:32:28.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:28.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.815632+0000", 2024-04-24T15:32:28.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2026e-05 2024-04-24T15:32:28.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:28.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.815638+0000", 2024-04-24T15:32:28.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8479999999999997e-06 2024-04-24T15:32:28.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:28.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.815678+0000", 2024-04-24T15:32:28.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9981000000000002e-05 2024-04-24T15:32:28.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:28.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.815702+0000", 2024-04-24T15:32:28.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3917000000000002e-05 2024-04-24T15:32:28.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:28.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.820633+0000", 2024-04-24T15:32:28.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0049306380000000002 2024-04-24T15:32:28.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:28.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.820657+0000", 2024-04-24T15:32:28.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.476e-05 2024-04-24T15:32:28.798 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:28.798 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:28.799 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:28.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.799 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4426.0:4005 4.6 e34/33 4:64968d7a:::rbd_data.114a1f60cb66.00000000000001bc:head v 34'45, mlcod=34'43)", 2024-04-24T15:32:28.799 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.842218+0000", 2024-04-24T15:32:28.799 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.8779083970000001, 2024-04-24T15:32:28.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0052518060000000004, 2024-04-24T15:32:28.799 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:28.799 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:28.799 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:28.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:28.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.842218+0000", 2024-04-24T15:32:28.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:28.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:28.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.842218+0000", 2024-04-24T15:32:28.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:28.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:28.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.842218+0000", 2024-04-24T15:32:28.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:28.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:28.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.842223+0000", 2024-04-24T15:32:28.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0579999999999997e-06 2024-04-24T15:32:28.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:28.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.842231+0000", 2024-04-24T15:32:28.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5259999999999993e-06 2024-04-24T15:32:28.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:28.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.842235+0000", 2024-04-24T15:32:28.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0990000000000001e-06 2024-04-24T15:32:28.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:28.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.842271+0000", 2024-04-24T15:32:28.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5716000000000002e-05 2024-04-24T15:32:28.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.802 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:28.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.842291+0000", 2024-04-24T15:32:28.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9581000000000001e-05 2024-04-24T15:32:28.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.802 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:28.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.847434+0000", 2024-04-24T15:32:28.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0051435480000000004 2024-04-24T15:32:28.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:28.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.847470+0000", 2024-04-24T15:32:28.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5929000000000001e-05 2024-04-24T15:32:28.803 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:28.803 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:28.803 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:28.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.803 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4426.0:4119 4.6 e34/33 4:6588fd86:::rbd_data.114a1f60cb66.0000000000000179:head v 34'48, mlcod=34'45)", 2024-04-24T15:32:28.803 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.936723+0000", 2024-04-24T15:32:28.803 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.7834035999999998, 2024-04-24T15:32:28.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0058208640000000002, 2024-04-24T15:32:28.803 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:28.803 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:28.804 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:28.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:28.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.936723+0000", 2024-04-24T15:32:28.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:28.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:28.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.936723+0000", 2024-04-24T15:32:28.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:28.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:28.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.936722+0000", 2024-04-24T15:32:28.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:28.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.805 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:28.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.936735+0000", 2024-04-24T15:32:28.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2153999999999999e-05 2024-04-24T15:32:28.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.805 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:28.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.936743+0000", 2024-04-24T15:32:28.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9360000000000005e-06 2024-04-24T15:32:28.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.805 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:28.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.936749+0000", 2024-04-24T15:32:28.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5040000000000002e-06 2024-04-24T15:32:28.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.806 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:28.806 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.936783+0000", 2024-04-24T15:32:28.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3689999999999998e-05 2024-04-24T15:32:28.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.806 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:28.806 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.936803+0000", 2024-04-24T15:32:28.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0047999999999998e-05 2024-04-24T15:32:28.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.806 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:28.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.942517+0000", 2024-04-24T15:32:28.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057144220000000003 2024-04-24T15:32:28.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:28.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.942544+0000", 2024-04-24T15:32:28.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6735999999999999e-05 2024-04-24T15:32:28.807 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:28.807 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:28.807 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:28.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.807 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4426.0:4353 4.3 e34/33 4:c38b6006:::rbd_data.114a1f60cb66.0000000000000175:head v 34'35, mlcod=34'33)", 2024-04-24T15:32:28.807 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:25.154297+0000", 2024-04-24T15:32:28.807 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.5658295280000001, 2024-04-24T15:32:28.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0051580410000000004, 2024-04-24T15:32:28.808 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:28.808 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:28.808 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:28.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:28.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:25.154297+0000", 2024-04-24T15:32:28.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:28.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:28.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:25.154297+0000", 2024-04-24T15:32:28.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:28.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:28.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:25.154296+0000", 2024-04-24T15:32:28.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:28.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:28.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:25.154312+0000", 2024-04-24T15:32:28.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6271000000000001e-05 2024-04-24T15:32:28.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:28.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:25.154330+0000", 2024-04-24T15:32:28.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8076000000000001e-05 2024-04-24T15:32:28.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.810 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.810 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:28.810 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:25.154340+0000", 2024-04-24T15:32:28.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7210000000000006e-06 2024-04-24T15:32:28.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.810 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.810 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:28.810 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:25.154399+0000", 2024-04-24T15:32:28.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9082000000000001e-05 2024-04-24T15:32:28.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.810 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.810 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:28.810 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:25.154422+0000", 2024-04-24T15:32:28.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2569e-05 2024-04-24T15:32:28.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:28.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:25.159411+0000", 2024-04-24T15:32:28.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0049891609999999998 2024-04-24T15:32:28.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:28.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:28.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:28.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:25.159455+0000", 2024-04-24T15:32:28.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4288e-05 2024-04-24T15:32:28.811 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:28.811 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:28.811 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:28.811 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:28.812 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:28.812 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:32:28.813 INFO:teuthology.orchestra.run.smithi096.stdout:947 write 0x8420cad thru 0x842376b (0x2abf bytes) 2024-04-24T15:32:28.813 INFO:teuthology.orchestra.run.smithi096.stdout:948 write 0x285f879 thru 0x2865c7f (0x6407 bytes) 2024-04-24T15:32:28.813 INFO:teuthology.orchestra.run.smithi096.stdout:949 read 0x1c79621 thru 0x1c88896 (0xf276 bytes) 2024-04-24T15:32:28.813 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:28.813 INFO:teuthology.orchestra.run.smithi096.stdout:953 write 0x3c95c90 thru 0x3c97124 (0x1495 bytes) 2024-04-24T15:32:28.814 INFO:teuthology.orchestra.run.smithi096.stdout:954 read 0x37772a3 thru 0x3783c8d (0xc9eb bytes) 2024-04-24T15:32:28.814 INFO:teuthology.orchestra.run.smithi096.stdout:955 read 0x1eadbca thru 0x1ebae93 (0xd2ca bytes) 2024-04-24T15:32:28.814 INFO:teuthology.orchestra.run.smithi096.stdout:956 read 0x5e9b0f4 thru 0x5e9cc87 (0x1b94 bytes) 2024-04-24T15:32:28.814 INFO:teuthology.orchestra.run.smithi096.stdout:957 read 0x13fbbf6 thru 0x140a09f (0xe4aa bytes) 2024-04-24T15:32:28.815 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-24T15:32:28.817 INFO:teuthology.orchestra.run.smithi096.stdout:958 write 0xc1dd957 thru 0xc1e4228 (0x68d2 bytes) 2024-04-24T15:32:28.817 INFO:teuthology.orchestra.run.smithi096.stdout:959 write 0xcf8b4c thru 0xd009cc (0x7e81 bytes) 2024-04-24T15:32:28.817 INFO:teuthology.orchestra.run.smithi096.stdout:960 write 0x3154234 thru 0x315687c (0x2649 bytes) 2024-04-24T15:32:28.817 INFO:teuthology.orchestra.run.smithi096.stdout:961 writesame 0x38207a4 thru 0x3823283 data_size 0x38(0x2ae0 bytes) 2024-04-24T15:32:28.817 INFO:teuthology.orchestra.run.smithi096.stdout:963 punch from 0x1acd8c0 to 0x1ad33b3, (0x5af3 bytes) 2024-04-24T15:32:28.817 INFO:teuthology.orchestra.run.smithi096.stdout:964 read 0x9257961 thru 0x92641dc (0xc87c bytes) 2024-04-24T15:32:28.817 INFO:teuthology.orchestra.run.smithi096.stdout:965 write 0x8480704 thru 0x8485d06 (0x5603 bytes) 2024-04-24T15:32:28.817 INFO:teuthology.orchestra.run.smithi096.stdout:966 writesame 0x3590ead thru 0x3591a24 data_size 0x4(0xb78 bytes) 2024-04-24T15:32:28.817 INFO:teuthology.orchestra.run.smithi096.stdout:967 trunc from 0xc1e4229 to 0x269876 2024-04-24T15:32:28.817 INFO:teuthology.orchestra.run.smithi096.stdout:968 punch from 0x83d2d to 0x86a29, (0x2cfc bytes) 2024-04-24T15:32:28.817 INFO:teuthology.orchestra.run.smithi096.stdout:969 write 0x7bdb68d thru 0x7be246f (0x6de3 bytes) 2024-04-24T15:32:28.817 INFO:teuthology.orchestra.run.smithi096.stdout:970 trunc from 0x7be2470 to 0xebeab1c 2024-04-24T15:32:28.817 INFO:teuthology.orchestra.run.smithi096.stdout:973 write 0x8d28fc5 thru 0x8d36d64 (0xdda0 bytes) 2024-04-24T15:32:28.818 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:28.818 INFO:teuthology.orchestra.run.smithi096.stdout:979 read 0x702a8ad thru 0x70373d4 (0xcb28 bytes) 2024-04-24T15:32:28.818 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:28.818 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:32:28.814+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:32:28.818 INFO:teuthology.orchestra.run.smithi096.stdout:982 trunc from 0xebeab1c to 0x39edbba 2024-04-24T15:32:28.829 INFO:teuthology.orchestra.run.smithi096.stdout:983 read 0x26c5cb1 thru 0x26ccca7 (0x6ff7 bytes) 2024-04-24T15:32:28.836 INFO:teuthology.orchestra.run.smithi096.stdout:984 trunc from 0x39edbba to 0x95986c6 2024-04-24T15:32:28.841 INFO:teuthology.orchestra.run.smithi096.stdout:985 write 0x71b2061 thru 0x71b35a3 (0x1543 bytes) 2024-04-24T15:32:28.843 INFO:teuthology.orchestra.run.smithi096.stdout:987 read 0x67aca0 thru 0x6800c4 (0x5425 bytes) 2024-04-24T15:32:28.849 INFO:teuthology.orchestra.run.smithi096.stdout:988 writesame 0x9f4d692 thru 0x9f5761c data_size 0x2f(0x9f8b bytes) 2024-04-24T15:32:28.859 INFO:teuthology.orchestra.run.smithi096.stdout:989 write 0xdf4ec8a thru 0xdf52087 (0x33fe bytes) 2024-04-24T15:32:28.862 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:28.862 INFO:teuthology.orchestra.run.smithi096.stdout:991 read 0xbfd4411 thru 0xbfd63ec (0x1fdc bytes) 2024-04-24T15:32:28.862 INFO:teuthology.orchestra.run.smithi096.stdout:992 write 0x5571a0e thru 0x5577ac5 (0x60b8 bytes) 2024-04-24T15:32:28.863 INFO:teuthology.orchestra.run.smithi096.stdout:993 read 0x6066019 thru 0x606a552 (0x453a bytes) 2024-04-24T15:32:28.864 INFO:teuthology.orchestra.run.smithi096.stdout:994 read 0x15c306e thru 0x15c9a9f (0x6a32 bytes) 2024-04-24T15:32:28.864 INFO:teuthology.orchestra.run.smithi096.stdout:995 write 0x660d3dd thru 0x661aa01 (0xd625 bytes) 2024-04-24T15:32:28.867 INFO:teuthology.orchestra.run.smithi096.stdout:997 read 0x8615ba thru 0x865c12 (0x4659 bytes) 2024-04-24T15:32:28.868 INFO:teuthology.orchestra.run.smithi096.stdout:998 read 0x856f8a8 thru 0x857d760 (0xdeb9 bytes) 2024-04-24T15:32:28.868 INFO:teuthology.orchestra.run.smithi096.stdout:999 read 0xc24fa94 thru 0xc2520ee (0x265b bytes) 2024-04-24T15:32:28.868 INFO:teuthology.orchestra.run.smithi096.stdout:1000 read 0x9ec6b0e thru 0x9ed3b75 (0xd068 bytes) 2024-04-24T15:32:28.868 INFO:teuthology.orchestra.run.smithi096.stdout:1001 punch from 0x234c3e0 to 0x2355527, (0x9147 bytes) 2024-04-24T15:32:28.868 INFO:teuthology.orchestra.run.smithi096.stdout:1003 read 0x2403616 thru 0x240a188 (0x6b73 bytes) 2024-04-24T15:32:28.868 INFO:teuthology.orchestra.run.smithi096.stdout:1004 writesame 0xc1edc2 thru 0xc26079 data_size 0x4(0x72b8 bytes) 2024-04-24T15:32:28.871 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:28.877 INFO:teuthology.orchestra.run.smithi096.stdout:1007 write 0xe8dde36 thru 0xe8e296d (0x4b38 bytes) 2024-04-24T15:32:28.880 INFO:teuthology.orchestra.run.smithi096.stdout:1008 read 0x7e8de89 thru 0x7e91964 (0x3adc bytes) 2024-04-24T15:32:28.880 INFO:teuthology.orchestra.run.smithi096.stdout:1010 write 0x75aa thru 0xe6f0 (0x7147 bytes) 2024-04-24T15:32:28.882 INFO:teuthology.orchestra.run.smithi096.stdout:1011 read 0x4e41412 thru 0x4e4487e (0x346d bytes) 2024-04-24T15:32:28.882 INFO:teuthology.orchestra.run.smithi096.stdout:1012 read 0x6c74c90 thru 0x6c7a964 (0x5cd5 bytes) 2024-04-24T15:32:28.883 INFO:teuthology.orchestra.run.smithi096.stdout:1013 writesame 0x197fd70 thru 0x198a058 data_size 0x13(0xa2e9 bytes) 2024-04-24T15:32:28.884 INFO:teuthology.orchestra.run.smithi096.stdout:1014 write 0xfccbaf thru 0xfcfdf0 (0x3242 bytes) 2024-04-24T15:32:28.886 INFO:teuthology.orchestra.run.smithi096.stdout:1015 read 0xe18cb6e thru 0xe196899 (0x9d2c bytes) 2024-04-24T15:32:28.886 INFO:teuthology.orchestra.run.smithi096.stdout:1016 writesame 0x4fdd916 thru 0x4fe838a data_size 0x1(0xaa75 bytes) 2024-04-24T15:32:28.906 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:28.906 INFO:teuthology.orchestra.run.smithi096.stdout:1018 read 0x2d59652 thru 0x2d59c31 (0x5e0 bytes) 2024-04-24T15:32:28.906 INFO:teuthology.orchestra.run.smithi096.stdout:1021 write 0xc8b2f8e thru 0xc8bfce3 (0xcd56 bytes) 2024-04-24T15:32:28.908 INFO:teuthology.orchestra.run.smithi096.stdout:1022 write 0x935d717 thru 0x9369904 (0xc1ee bytes) 2024-04-24T15:32:28.911 INFO:teuthology.orchestra.run.smithi096.stdout:1023 writesame 0x51cbf52 thru 0x51cf901 data_size 0x47(0x39b0 bytes) 2024-04-24T15:32:28.913 INFO:teuthology.orchestra.run.smithi096.stdout:1024 read 0x1a666f6 thru 0x1a6ebcd (0x84d8 bytes) 2024-04-24T15:32:28.913 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:28.913 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:32:28.914+0000 7f23abcf6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 44021) UID: 0 2024-04-24T15:32:28.917 INFO:teuthology.orchestra.run.smithi096.stdout:1026 trunc from 0xe8e296e to 0xb41adc2 2024-04-24T15:32:28.928 INFO:teuthology.orchestra.run.smithi096.stdout:1028 write 0x7e1169a thru 0x7e15d4a (0x46b1 bytes) 2024-04-24T15:32:28.930 INFO:teuthology.orchestra.run.smithi096.stdout:1029 punch from 0x37e1d36 to 0x37e3e07, (0x20d1 bytes) 2024-04-24T15:32:28.930 INFO:teuthology.orchestra.run.smithi096.stdout:1030 read 0x56b4d54 thru 0x56b635b (0x1608 bytes) 2024-04-24T15:32:28.930 INFO:teuthology.orchestra.run.smithi096.stdout:1031 punch from 0x7fc5071 to 0x7fcce0f, (0x7d9e bytes) 2024-04-24T15:32:28.940 INFO:teuthology.orchestra.run.smithi096.stdout:1032 write 0xedd8479 thru 0xeddda26 (0x55ae bytes) 2024-04-24T15:32:28.943 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:28.943 INFO:teuthology.orchestra.run.smithi096.stdout:1034 read 0x98e9f15 thru 0x98f91f9 (0xf2e5 bytes) 2024-04-24T15:32:28.943 INFO:teuthology.orchestra.run.smithi096.stdout:1035 writesame 0x661b24b thru 0x661ba3c data_size 0x9(0x7f2 bytes) 2024-04-24T15:32:28.950 INFO:teuthology.orchestra.run.smithi096.stdout:1036 trunc from 0xeddda27 to 0xa86f614 2024-04-24T15:32:28.958 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:32:28.958 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:32:28.958 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-04-24T15:32:28.959 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:32:28.966 INFO:teuthology.orchestra.run.smithi096.stdout:1037 write 0xbfe8131 thru 0xbfecadd (0x49ad bytes) 2024-04-24T15:32:28.968 INFO:teuthology.orchestra.run.smithi096.stdout:1038 write 0x4cc6d1d thru 0x4cd1f02 (0xb1e6 bytes) 2024-04-24T15:32:28.970 INFO:teuthology.orchestra.run.smithi096.stdout:1040 read 0xd75dba thru 0xd83641 (0xd888 bytes) 2024-04-24T15:32:28.971 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-24T15:32:28.971 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:28.971 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:28.976 INFO:teuthology.orchestra.run.smithi096.stdout:1044 trunc from 0xbfecade to 0x7386376 2024-04-24T15:32:28.986 INFO:teuthology.orchestra.run.smithi096.stdout:1045 read 0x1d068b6 thru 0x1d07887 (0xfd2 bytes) 2024-04-24T15:32:28.986 INFO:teuthology.orchestra.run.smithi096.stdout:1047 read 0x617684d thru 0x617c6ec (0x5ea0 bytes) 2024-04-24T15:32:29.000 INFO:teuthology.orchestra.run.smithi096.stdout:1048 write 0xe33c3c7 thru 0xe3451c7 (0x8e01 bytes) 2024-04-24T15:32:29.003 INFO:teuthology.orchestra.run.smithi096.stdout:1049 write 0x4a061b9 thru 0x4a0e66b (0x84b3 bytes) 2024-04-24T15:32:29.005 INFO:teuthology.orchestra.run.smithi096.stdout:1050 read 0x9635a8a thru 0x96393b1 (0x3928 bytes) 2024-04-24T15:32:29.005 INFO:teuthology.orchestra.run.smithi096.stdout:1052 write 0x5ede841 thru 0x5eebfdd (0xd79d bytes) 2024-04-24T15:32:29.009 INFO:teuthology.orchestra.run.smithi096.stdout:1055 trunc from 0xe3451c8 to 0xbffcd80 2024-04-24T15:32:29.014 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:32:29.015+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:32:29.017 INFO:teuthology.orchestra.run.smithi096.stdout:1057 read 0x37cd82 thru 0x38c878 (0xfaf7 bytes) 2024-04-24T15:32:29.017 INFO:teuthology.orchestra.run.smithi096.stdout:1058 read 0x4c8978e thru 0x4c93be4 (0xa457 bytes) 2024-04-24T15:32:29.029 INFO:teuthology.orchestra.run.smithi096.stdout:1059 trunc from 0xbffcd80 to 0x23feefc 2024-04-24T15:32:29.042 INFO:teuthology.orchestra.run.smithi096.stdout:1060 punch from 0xbc4be4 to 0xbd2b51, (0xdf6d bytes) 2024-04-24T15:32:29.042 INFO:teuthology.orchestra.run.smithi096.stdout:1061 punch from 0x1622e80 to 0x162f240, (0xc3c0 bytes) 2024-04-24T15:32:29.042 INFO:teuthology.orchestra.run.smithi096.stdout:1063 read 0x23db5df thru 0x23eb227 (0xfc49 bytes) 2024-04-24T15:32:29.062 INFO:teuthology.orchestra.run.smithi096.stdout:1065 write 0xd2190d4 thru 0xd21feaf (0x6ddc bytes) 2024-04-24T15:32:29.065 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:29.065 INFO:teuthology.orchestra.run.smithi096.stdout:1068 read 0x5392842 thru 0x5393a6c (0x122b bytes) 2024-04-24T15:32:29.065 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:29.070 INFO:teuthology.orchestra.run.smithi096.stdout:1070 writesame 0xd94f63c thru 0xd95c203 data_size 0x4(0xcbc8 bytes) 2024-04-24T15:32:29.070 INFO:teuthology.orchestra.run.smithi096.stdout:1071 read 0x29aa9a7 thru 0x29ae16b (0x37c5 bytes) 2024-04-24T15:32:29.070 INFO:teuthology.orchestra.run.smithi096.stdout:1072 punch from 0x504577 to 0x5048a4, (0x32d bytes) 2024-04-24T15:32:29.082 INFO:teuthology.orchestra.run.smithi096.stdout:1073 trunc from 0xd95c204 to 0x36f1cea 2024-04-24T15:32:29.112 INFO:teuthology.orchestra.run.smithi096.stderr:noscrub is set 2024-04-24T15:32:29.114 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:32:29.116+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:32:29.118 INFO:teuthology.orchestra.run.smithi096.stderr:marked out osd.4. 2024-04-24T15:32:29.127 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:32:29.128 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:32:29.128 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-04-24T15:32:29.128 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-04-24T15:32:29.128 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:32:29.138 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-24T15:32:29.224 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:32:29.224+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:32:29.302 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:32:29.303 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-04-24T15:32:29.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-04-24T15:32:29.303 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [] 2024-04-24T15:32:29.303 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:32:29.312 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-24T15:32:29.353 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:32:29.354+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:32:29.417 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:32:29.418+0000 7f23abcf6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 44021) UID: 0 2024-04-24T15:32:29.475 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:32:29.475 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:32:29.476 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-04-24T15:32:29.476 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:32:29.485 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-24T15:32:29.530 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:32:29.530+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:32:29.608 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:32:29.608 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:32:29.609 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-04-24T15:32:29.609 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-04-24T15:32:29.609 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:32:29.618 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-24T15:32:29.619 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:32:29.619+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:32:29.719 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:32:29.721+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:32:29.784 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:32:29.785 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-04-24T15:32:29.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-04-24T15:32:29.785 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-04-24T15:32:29.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.785 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4426.0:2434 4.d e32/26 4:b7840674:::rbd_data.114a1f60cb66.00000000000001a7:head v 32'12, mlcod=32'10)", 2024-04-24T15:32:29.785 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:23.024172+0000", 2024-04-24T15:32:29.785 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 6.7595563920000004, 2024-04-24T15:32:29.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057919520000000004, 2024-04-24T15:32:29.785 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:29.785 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:29.785 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:29.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:29.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.024172+0000", 2024-04-24T15:32:29.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:29.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:29.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.024172+0000", 2024-04-24T15:32:29.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:29.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:29.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.024171+0000", 2024-04-24T15:32:29.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:29.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:29.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.024178+0000", 2024-04-24T15:32:29.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2420000000000003e-06 2024-04-24T15:32:29.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:29.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.024191+0000", 2024-04-24T15:32:29.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2444e-05 2024-04-24T15:32:29.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:29.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.024197+0000", 2024-04-24T15:32:29.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2249999999999997e-06 2024-04-24T15:32:29.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:29.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.024251+0000", 2024-04-24T15:32:29.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4221000000000001e-05 2024-04-24T15:32:29.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:29.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.024270+0000", 2024-04-24T15:32:29.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9151999999999999e-05 2024-04-24T15:32:29.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:29.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.029925+0000", 2024-04-24T15:32:29.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0056546160000000003 2024-04-24T15:32:29.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:29.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.029964+0000", 2024-04-24T15:32:29.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8855e-05 2024-04-24T15:32:29.789 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:29.789 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:29.789 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:29.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.789 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:2450 4.1c 4:3f9dbb33:::rbd_data.114a1f60cb66.00000000000001f3:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-24T15:32:29.789 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:23.027406+0000", 2024-04-24T15:32:29.789 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 6.7563220189999997, 2024-04-24T15:32:29.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.005672379, 2024-04-24T15:32:29.789 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:29.789 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:29.790 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:29.790 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:29.790 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:29.790 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2450 2024-04-24T15:32:29.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.790 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:29.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:29.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027406+0000", 2024-04-24T15:32:29.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:29.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:29.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027406+0000", 2024-04-24T15:32:29.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:29.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:29.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027405+0000", 2024-04-24T15:32:29.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:29.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:29.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027411+0000", 2024-04-24T15:32:29.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7610000000000004e-06 2024-04-24T15:32:29.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:29.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027422+0000", 2024-04-24T15:32:29.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1462e-05 2024-04-24T15:32:29.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:29.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027429+0000", 2024-04-24T15:32:29.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3339999999999998e-06 2024-04-24T15:32:29.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:29.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027478+0000", 2024-04-24T15:32:29.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9804e-05 2024-04-24T15:32:29.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:29.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027559+0000", 2024-04-24T15:32:29.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1068000000000004e-05 2024-04-24T15:32:29.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:29.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027672+0000", 2024-04-24T15:32:29.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000112632 2024-04-24T15:32:29.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:29.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.032737+0000", 2024-04-24T15:32:29.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0050646429999999997 2024-04-24T15:32:29.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:29.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.033029+0000", 2024-04-24T15:32:29.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00029226300000000002 2024-04-24T15:32:29.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:29.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.033057+0000", 2024-04-24T15:32:29.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7872e-05 2024-04-24T15:32:29.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:29.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.033078+0000", 2024-04-24T15:32:29.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.158e-05 2024-04-24T15:32:29.795 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:29.795 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:29.795 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:29.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.795 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4426.0:2447 4.6 e32/27 4:668081fa:::rbd_data.114a1f60cb66.00000000000001f0:head v 32'16, mlcod=32'13)", 2024-04-24T15:32:29.795 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:23.027434+0000", 2024-04-24T15:32:29.795 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 6.7562941370000003, 2024-04-24T15:32:29.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0052559449999999997, 2024-04-24T15:32:29.795 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:29.795 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:29.795 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:29.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:29.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027434+0000", 2024-04-24T15:32:29.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:29.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:29.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027434+0000", 2024-04-24T15:32:29.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:29.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:29.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027433+0000", 2024-04-24T15:32:29.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:29.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:29.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027440+0000", 2024-04-24T15:32:29.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2099999999999998e-06 2024-04-24T15:32:29.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:29.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027450+0000", 2024-04-24T15:32:29.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0278000000000001e-05 2024-04-24T15:32:29.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:29.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027456+0000", 2024-04-24T15:32:29.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3720000000000001e-06 2024-04-24T15:32:29.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:29.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027488+0000", 2024-04-24T15:32:29.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1427000000000003e-05 2024-04-24T15:32:29.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:29.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027510+0000", 2024-04-24T15:32:29.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.243e-05 2024-04-24T15:32:29.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:29.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.032654+0000", 2024-04-24T15:32:29.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0051436210000000001 2024-04-24T15:32:29.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:29.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.032690+0000", 2024-04-24T15:32:29.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6188e-05 2024-04-24T15:32:29.799 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:29.799 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:29.799 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:29.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.799 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:2454 4.4 4:21c2614c:::rbd_data.114a1f60cb66.00000000000001f7:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-24T15:32:29.799 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:23.030259+0000", 2024-04-24T15:32:29.799 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 6.753469451, 2024-04-24T15:32:29.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0056217740000000004, 2024-04-24T15:32:29.799 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:29.799 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:29.799 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:29.799 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:29.799 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:29.800 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2454 2024-04-24T15:32:29.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.800 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:29.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:29.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.030259+0000", 2024-04-24T15:32:29.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:29.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:29.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.030259+0000", 2024-04-24T15:32:29.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:29.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:29.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.030258+0000", 2024-04-24T15:32:29.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:29.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:29.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.030263+0000", 2024-04-24T15:32:29.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1869999999999996e-06 2024-04-24T15:32:29.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:29.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.030275+0000", 2024-04-24T15:32:29.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.199e-05 2024-04-24T15:32:29.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:29.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.030281+0000", 2024-04-24T15:32:29.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1680000000000001e-06 2024-04-24T15:32:29.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.802 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:29.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.030311+0000", 2024-04-24T15:32:29.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0008000000000001e-05 2024-04-24T15:32:29.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.802 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:29.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.030355+0000", 2024-04-24T15:32:29.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3489999999999999e-05 2024-04-24T15:32:29.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.802 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:29.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.030408+0000", 2024-04-24T15:32:29.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3737999999999999e-05 2024-04-24T15:32:29.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:29.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.035580+0000", 2024-04-24T15:32:29.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0051716560000000002 2024-04-24T15:32:29.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:29.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.035820+0000", 2024-04-24T15:32:29.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00023958900000000001 2024-04-24T15:32:29.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:29.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.035847+0000", 2024-04-24T15:32:29.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7495999999999999e-05 2024-04-24T15:32:29.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:29.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.035880+0000", 2024-04-24T15:32:29.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3367000000000001e-05 2024-04-24T15:32:29.804 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:29.804 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:29.804 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:29.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.804 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4426.0:2510 4.12 e33/27 4:4fee0517:::rbd_data.114a1f60cb66.000000000000029d:head v 33'19, mlcod=32'17)", 2024-04-24T15:32:29.804 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:23.078201+0000", 2024-04-24T15:32:29.804 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 6.705526893, 2024-04-24T15:32:29.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0060097689999999999, 2024-04-24T15:32:29.804 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:29.804 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:29.804 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:29.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.805 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:29.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.078201+0000", 2024-04-24T15:32:29.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:29.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.805 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:29.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.078201+0000", 2024-04-24T15:32:29.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:29.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.805 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:29.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.078201+0000", 2024-04-24T15:32:29.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:29.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.806 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:29.806 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.078213+0000", 2024-04-24T15:32:29.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2261e-05 2024-04-24T15:32:29.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.806 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:29.806 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.078223+0000", 2024-04-24T15:32:29.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4709999999999996e-06 2024-04-24T15:32:29.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.806 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:29.806 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.078230+0000", 2024-04-24T15:32:29.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8979999999999999e-06 2024-04-24T15:32:29.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:29.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.079842+0000", 2024-04-24T15:32:29.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001612048 2024-04-24T15:32:29.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:29.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.079870+0000", 2024-04-24T15:32:29.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8487e-05 2024-04-24T15:32:29.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:29.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.084169+0000", 2024-04-24T15:32:29.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0042991490000000004 2024-04-24T15:32:29.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:29.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.084211+0000", 2024-04-24T15:32:29.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1792000000000002e-05 2024-04-24T15:32:29.808 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:29.808 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:29.808 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:29.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.808 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:2522 4.6 4:60ceedf0:::rbd_data.114a1f60cb66.000000000000011d:head [set-alloc-hint object_size 262144 write_size 262144,write 110592~2048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-24T15:32:29.808 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:23.088240+0000", 2024-04-24T15:32:29.808 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 6.6954879790000001, 2024-04-24T15:32:29.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.988232797, 2024-04-24T15:32:29.808 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:29.808 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:29.809 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:29.809 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:29.809 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:29.809 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2522 2024-04-24T15:32:29.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.809 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:29.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:29.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088240+0000", 2024-04-24T15:32:29.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:29.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:29.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088240+0000", 2024-04-24T15:32:29.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:29.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.810 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.810 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:29.810 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088238+0000", 2024-04-24T15:32:29.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999981 2024-04-24T15:32:29.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.810 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.810 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:29.810 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088254+0000", 2024-04-24T15:32:29.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5562000000000001e-05 2024-04-24T15:32:29.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.810 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.810 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:29.810 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088275+0000", 2024-04-24T15:32:29.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1256000000000001e-05 2024-04-24T15:32:29.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:29.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088283+0000", 2024-04-24T15:32:29.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3219999999999993e-06 2024-04-24T15:32:29.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:29.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088351+0000", 2024-04-24T15:32:29.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7595000000000006e-05 2024-04-24T15:32:29.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-04-24T15:32:29.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088362+0000", 2024-04-24T15:32:29.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1764e-05 2024-04-24T15:32:29.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:29.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.073748+0000", 2024-04-24T15:32:29.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98538541899999998 2024-04-24T15:32:29.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:29.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.073925+0000", 2024-04-24T15:32:29.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017747600000000001 2024-04-24T15:32:29.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:29.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.074092+0000", 2024-04-24T15:32:29.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000166622 2024-04-24T15:32:29.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:29.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.074774+0000", 2024-04-24T15:32:29.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00068225000000000002 2024-04-24T15:32:29.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:29.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.076412+0000", 2024-04-24T15:32:29.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001638189 2024-04-24T15:32:29.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:29.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.076446+0000", 2024-04-24T15:32:29.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3587000000000002e-05 2024-04-24T15:32:29.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.814 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:29.814 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.076473+0000", 2024-04-24T15:32:29.814 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6886999999999999e-05 2024-04-24T15:32:29.814 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:29.814 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:29.814 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:29.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.814 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:2755 4.1c 4:3c8baac7:::rbd_data.114a1f60cb66.00000000000000a8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:29.814 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.252831+0000", 2024-04-24T15:32:29.814 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 5.5308971649999998, 2024-04-24T15:32:29.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057169500000000002, 2024-04-24T15:32:29.815 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:29.815 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:29.815 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:29.815 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:29.815 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:29.815 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2755 2024-04-24T15:32:29.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.815 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:29.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:29.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.252831+0000", 2024-04-24T15:32:29.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:29.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:29.816 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.252831+0000", 2024-04-24T15:32:29.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:29.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:29.816 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.252830+0000", 2024-04-24T15:32:29.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:29.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:29.816 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.252835+0000", 2024-04-24T15:32:29.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3739999999999997e-06 2024-04-24T15:32:29.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:29.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.252846+0000", 2024-04-24T15:32:29.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.039e-05 2024-04-24T15:32:29.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:29.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.252850+0000", 2024-04-24T15:32:29.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4100000000000001e-06 2024-04-24T15:32:29.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:29.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.252885+0000", 2024-04-24T15:32:29.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5213999999999999e-05 2024-04-24T15:32:29.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.818 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:29.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.252957+0000", 2024-04-24T15:32:29.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2285000000000003e-05 2024-04-24T15:32:29.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.818 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:29.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.253053+0000", 2024-04-24T15:32:29.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5841999999999995e-05 2024-04-24T15:32:29.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.818 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:29.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.255905+0000", 2024-04-24T15:32:29.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028514539999999998 2024-04-24T15:32:29.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.819 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:29.819 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.258474+0000", 2024-04-24T15:32:29.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0025695679999999999 2024-04-24T15:32:29.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.819 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:29.819 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.258518+0000", 2024-04-24T15:32:29.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3242999999999999e-05 2024-04-24T15:32:29.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.819 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:29.819 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.258548+0000", 2024-04-24T15:32:29.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0318999999999999e-05 2024-04-24T15:32:29.819 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:29.820 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:29.820 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:29.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.820 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4426.0:2761 4.19 e34/26 4:9e865ec5:::rbd_data.114a1f60cb66.00000000000000c0:head v 34'14, mlcod=34'12)", 2024-04-24T15:32:29.820 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.255783+0000", 2024-04-24T15:32:29.820 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 5.5279455679999998, 2024-04-24T15:32:29.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057401149999999996, 2024-04-24T15:32:29.820 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:29.820 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:29.820 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:29.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.820 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:29.820 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.255783+0000", 2024-04-24T15:32:29.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:29.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:29.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.255783+0000", 2024-04-24T15:32:29.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:29.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:29.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.255782+0000", 2024-04-24T15:32:29.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:29.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:29.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.255794+0000", 2024-04-24T15:32:29.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2429000000000001e-05 2024-04-24T15:32:29.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.822 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:29.822 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.255802+0000", 2024-04-24T15:32:29.822 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2139999999999999e-06 2024-04-24T15:32:29.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.822 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:29.822 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.255807+0000", 2024-04-24T15:32:29.822 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6500000000000001e-06 2024-04-24T15:32:29.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.822 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:29.822 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.255846+0000", 2024-04-24T15:32:29.822 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8497000000000001e-05 2024-04-24T15:32:29.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.823 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:29.823 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.255867+0000", 2024-04-24T15:32:29.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0733999999999998e-05 2024-04-24T15:32:29.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.823 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:29.823 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.261440+0000", 2024-04-24T15:32:29.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0055733099999999997 2024-04-24T15:32:29.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.823 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:29.823 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.261523+0000", 2024-04-24T15:32:29.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2764999999999998e-05 2024-04-24T15:32:29.824 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:29.824 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:29.824 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:29.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.824 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.824 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:2767 4.6 4:6297db09:::rbd_data.114a1f60cb66.00000000000000e1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:29.824 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.256201+0000", 2024-04-24T15:32:29.824 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 5.5275270289999998, 2024-04-24T15:32:29.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0056859040000000003, 2024-04-24T15:32:29.824 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:29.824 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:29.824 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:29.824 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:29.824 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:29.824 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2767 2024-04-24T15:32:29.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.825 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:29.825 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.825 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:29.825 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.256201+0000", 2024-04-24T15:32:29.825 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:29.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.825 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.825 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:29.825 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.256201+0000", 2024-04-24T15:32:29.825 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:29.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.825 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.826 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:29.826 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.256200+0000", 2024-04-24T15:32:29.826 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:29.826 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.826 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:29.826 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.256205+0000", 2024-04-24T15:32:29.826 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.587e-06 2024-04-24T15:32:29.826 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.826 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:29.826 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.256218+0000", 2024-04-24T15:32:29.826 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2362e-05 2024-04-24T15:32:29.826 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.827 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:29.827 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.256224+0000", 2024-04-24T15:32:29.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6120000000000004e-06 2024-04-24T15:32:29.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.827 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.827 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:29.827 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.256274+0000", 2024-04-24T15:32:29.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9084e-05 2024-04-24T15:32:29.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.827 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.827 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:29.827 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.256392+0000", 2024-04-24T15:32:29.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000118127 2024-04-24T15:32:29.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.827 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.828 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:29.828 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.256508+0000", 2024-04-24T15:32:29.828 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000116777 2024-04-24T15:32:29.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.828 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.828 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:29.828 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.261537+0000", 2024-04-24T15:32:29.828 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0050290380000000004 2024-04-24T15:32:29.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.828 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.828 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:29.828 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.261801+0000", 2024-04-24T15:32:29.828 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00026362400000000001 2024-04-24T15:32:29.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.828 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.828 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:29.829 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.261848+0000", 2024-04-24T15:32:29.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6700999999999999e-05 2024-04-24T15:32:29.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.829 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:29.829 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.261887+0000", 2024-04-24T15:32:29.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9159999999999998e-05 2024-04-24T15:32:29.829 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:29.829 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:29.829 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:29.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.829 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:3477 4.6 4:6588fd86:::rbd_data.114a1f60cb66.0000000000000179:head [set-alloc-hint object_size 262144 write_size 262144,write 77824~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:29.829 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.466375+0000", 2024-04-24T15:32:29.829 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 5.3173527490000003, 2024-04-24T15:32:29.830 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0051811119999999999, 2024-04-24T15:32:29.830 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:29.830 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:29.830 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:29.830 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:29.830 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:29.830 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3477 2024-04-24T15:32:29.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.830 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:29.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.830 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:29.830 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.466375+0000", 2024-04-24T15:32:29.830 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:29.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.831 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:29.831 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.466375+0000", 2024-04-24T15:32:29.831 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:29.831 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.831 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.831 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:29.831 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.466375+0000", 2024-04-24T15:32:29.831 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:29.831 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.831 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.831 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:29.831 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.466381+0000", 2024-04-24T15:32:29.831 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6499999999999999e-06 2024-04-24T15:32:29.831 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.831 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.832 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:29.832 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.466391+0000", 2024-04-24T15:32:29.832 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4129999999999995e-06 2024-04-24T15:32:29.832 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.832 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.832 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:29.832 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.466395+0000", 2024-04-24T15:32:29.832 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4209999999999997e-06 2024-04-24T15:32:29.832 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.832 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.832 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:29.832 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.466436+0000", 2024-04-24T15:32:29.832 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1179e-05 2024-04-24T15:32:29.832 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.833 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:29.833 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.466567+0000", 2024-04-24T15:32:29.833 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013121500000000001 2024-04-24T15:32:29.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.833 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:29.833 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.466722+0000", 2024-04-24T15:32:29.833 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015405599999999999 2024-04-24T15:32:29.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.833 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:29.833 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.468854+0000", 2024-04-24T15:32:29.833 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0021328440000000001 2024-04-24T15:32:29.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.834 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:29.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.471509+0000", 2024-04-24T15:32:29.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0026541659999999999 2024-04-24T15:32:29.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.834 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:29.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.471535+0000", 2024-04-24T15:32:29.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6699e-05 2024-04-24T15:32:29.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.834 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:29.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.471556+0000", 2024-04-24T15:32:29.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1222000000000001e-05 2024-04-24T15:32:29.834 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:29.835 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:29.835 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:29.835 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.835 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.835 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:3772 4.4 4:24a7cfbc:::rbd_data.114a1f60cb66.0000000000000020:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:29.835 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.699611+0000", 2024-04-24T15:32:29.835 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 5.0841171489999999, 2024-04-24T15:32:29.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0060871340000000001, 2024-04-24T15:32:29.835 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:29.835 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:29.835 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:29.835 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:29.835 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:29.835 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3772 2024-04-24T15:32:29.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.836 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:29.836 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.836 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:29.836 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.699611+0000", 2024-04-24T15:32:29.836 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:29.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.836 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.836 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:29.836 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.699611+0000", 2024-04-24T15:32:29.836 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:29.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.836 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.836 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:29.836 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.699609+0000", 2024-04-24T15:32:29.836 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999981 2024-04-24T15:32:29.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.837 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:29.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.699618+0000", 2024-04-24T15:32:29.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3859999999999996e-06 2024-04-24T15:32:29.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.837 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:29.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.699637+0000", 2024-04-24T15:32:29.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8688000000000001e-05 2024-04-24T15:32:29.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.837 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:29.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.699647+0000", 2024-04-24T15:32:29.838 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.029e-05 2024-04-24T15:32:29.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.838 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:29.838 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.699692+0000", 2024-04-24T15:32:29.838 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4400000000000002e-05 2024-04-24T15:32:29.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.838 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:29.838 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.699751+0000", 2024-04-24T15:32:29.838 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8953999999999998e-05 2024-04-24T15:32:29.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.838 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:29.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.700249+0000", 2024-04-24T15:32:29.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00049869900000000002 2024-04-24T15:32:29.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.839 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:29.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.705560+0000", 2024-04-24T15:32:29.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0053110550000000003 2024-04-24T15:32:29.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.839 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:29.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.705602+0000", 2024-04-24T15:32:29.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1689e-05 2024-04-24T15:32:29.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.839 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:29.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.705652+0000", 2024-04-24T15:32:29.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9740000000000001e-05 2024-04-24T15:32:29.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.840 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:29.840 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.705698+0000", 2024-04-24T15:32:29.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6329000000000003e-05 2024-04-24T15:32:29.840 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:29.840 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:29.840 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:29.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.840 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:3774 4.4 4:25b094fe:::rbd_data.114a1f60cb66.0000000000000022:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:29.840 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.699669+0000", 2024-04-24T15:32:29.840 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 5.0840590179999996, 2024-04-24T15:32:29.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.006296678, 2024-04-24T15:32:29.840 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:29.841 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:29.841 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:29.841 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:29.841 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:29.841 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3774 2024-04-24T15:32:29.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.841 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:29.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.841 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:29.841 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.699669+0000", 2024-04-24T15:32:29.841 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:29.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.841 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:29.841 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.699669+0000", 2024-04-24T15:32:29.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:29.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.842 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.842 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:29.842 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.699669+0000", 2024-04-24T15:32:29.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:29.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.842 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.842 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:29.842 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.699672+0000", 2024-04-24T15:32:29.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2830000000000002e-06 2024-04-24T15:32:29.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.842 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.842 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:29.842 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.699677+0000", 2024-04-24T15:32:29.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7530000000000001e-06 2024-04-24T15:32:29.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.843 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:29.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.699680+0000", 2024-04-24T15:32:29.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0130000000000002e-06 2024-04-24T15:32:29.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.843 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:29.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.702736+0000", 2024-04-24T15:32:29.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0030567379999999998 2024-04-24T15:32:29.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.843 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:29.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.702797+0000", 2024-04-24T15:32:29.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0563000000000003e-05 2024-04-24T15:32:29.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.844 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:29.844 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.703297+0000", 2024-04-24T15:32:29.844 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00049955300000000002 2024-04-24T15:32:29.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.844 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:29.844 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.705595+0000", 2024-04-24T15:32:29.844 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002298076 2024-04-24T15:32:29.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.844 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:29.844 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.705870+0000", 2024-04-24T15:32:29.844 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000275285 2024-04-24T15:32:29.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.844 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:29.845 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.705923+0000", 2024-04-24T15:32:29.845 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3229999999999997e-05 2024-04-24T15:32:29.845 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.845 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.845 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:29.845 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.705966+0000", 2024-04-24T15:32:29.845 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2636999999999999e-05 2024-04-24T15:32:29.845 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:29.845 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:29.845 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:29.845 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.845 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.845 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:3795 4.1c 4:3a2027c5:::rbd_data.114a1f60cb66.0000000000000025:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:29.845 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.723461+0000", 2024-04-24T15:32:29.846 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 5.0602671609999996, 2024-04-24T15:32:29.846 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0066452719999999998, 2024-04-24T15:32:29.846 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:29.846 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:29.846 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:29.846 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:29.846 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:29.846 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3795 2024-04-24T15:32:29.846 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.846 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:29.846 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.846 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:29.846 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.723461+0000", 2024-04-24T15:32:29.846 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:29.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.847 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.847 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:29.847 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.723461+0000", 2024-04-24T15:32:29.847 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:29.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.847 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.847 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:29.847 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.723459+0000", 2024-04-24T15:32:29.847 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999986 2024-04-24T15:32:29.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.847 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.847 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:29.847 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.723470+0000", 2024-04-24T15:32:29.848 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0097e-05 2024-04-24T15:32:29.848 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.848 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.848 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:29.848 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.723484+0000", 2024-04-24T15:32:29.848 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4338000000000001e-05 2024-04-24T15:32:29.848 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.848 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.848 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:29.848 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.723496+0000", 2024-04-24T15:32:29.848 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2125e-05 2024-04-24T15:32:29.848 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.848 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.848 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:29.848 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.723547+0000", 2024-04-24T15:32:29.849 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0909000000000003e-05 2024-04-24T15:32:29.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.849 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.849 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:29.849 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.723639+0000", 2024-04-24T15:32:29.849 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1844999999999999e-05 2024-04-24T15:32:29.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.849 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.849 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:29.849 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.724463+0000", 2024-04-24T15:32:29.849 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00082389099999999999 2024-04-24T15:32:29.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.849 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.849 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:29.849 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.728863+0000", 2024-04-24T15:32:29.849 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0044007610000000004 2024-04-24T15:32:29.850 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.850 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.850 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:29.850 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.730017+0000", 2024-04-24T15:32:29.850 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0011534410000000001 2024-04-24T15:32:29.850 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.850 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.850 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:29.850 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.730067+0000", 2024-04-24T15:32:29.850 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0430000000000003e-05 2024-04-24T15:32:29.850 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.850 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.850 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:29.850 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.730106+0000", 2024-04-24T15:32:29.850 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8902000000000001e-05 2024-04-24T15:32:29.850 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:29.851 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:29.851 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:29.851 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.851 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.851 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:3929 4.6 4:640a4fea:::rbd_data.114a1f60cb66.00000000000000dc:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:29.851 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.809952+0000", 2024-04-24T15:32:29.851 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.9737759769999998, 2024-04-24T15:32:29.851 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0053076180000000001, 2024-04-24T15:32:29.851 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:29.851 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:29.851 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:29.851 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:29.851 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:29.852 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3929 2024-04-24T15:32:29.852 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.852 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:29.852 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.852 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:29.852 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.809952+0000", 2024-04-24T15:32:29.852 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:29.852 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.852 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.852 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:29.852 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.809952+0000", 2024-04-24T15:32:29.852 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:29.852 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.852 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.852 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:29.853 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.809951+0000", 2024-04-24T15:32:29.853 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999986 2024-04-24T15:32:29.853 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.853 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.853 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:29.853 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.809957+0000", 2024-04-24T15:32:29.853 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2879999999999998e-06 2024-04-24T15:32:29.853 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.853 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.853 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:29.853 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.809971+0000", 2024-04-24T15:32:29.853 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4258e-05 2024-04-24T15:32:29.853 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.854 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.854 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:29.854 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.809978+0000", 2024-04-24T15:32:29.854 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5400000000000001e-06 2024-04-24T15:32:29.854 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.854 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.854 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:29.854 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.810029+0000", 2024-04-24T15:32:29.854 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1289000000000003e-05 2024-04-24T15:32:29.854 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.854 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.854 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:29.854 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.810130+0000", 2024-04-24T15:32:29.855 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000100301 2024-04-24T15:32:29.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.855 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.855 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:29.855 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.810260+0000", 2024-04-24T15:32:29.855 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013075700000000001 2024-04-24T15:32:29.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.855 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.855 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:29.855 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.810856+0000", 2024-04-24T15:32:29.855 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00059576900000000001 2024-04-24T15:32:29.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.856 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.856 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:29.856 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.815165+0000", 2024-04-24T15:32:29.856 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0043092169999999997 2024-04-24T15:32:29.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.856 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.856 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:29.856 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.815217+0000", 2024-04-24T15:32:29.856 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1539999999999998e-05 2024-04-24T15:32:29.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.857 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.857 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:29.857 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.815260+0000", 2024-04-24T15:32:29.857 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2883999999999998e-05 2024-04-24T15:32:29.857 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:29.857 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:29.857 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:29.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.857 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.857 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:3934 4.6 4:6297db09:::rbd_data.114a1f60cb66.00000000000000e1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:29.857 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.812623+0000", 2024-04-24T15:32:29.857 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.971104822, 2024-04-24T15:32:29.858 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.005653712, 2024-04-24T15:32:29.858 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:29.858 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:29.858 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:29.858 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:29.858 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:29.858 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3934 2024-04-24T15:32:29.858 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.858 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:29.858 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.858 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:29.858 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.812623+0000", 2024-04-24T15:32:29.858 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:29.859 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.859 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.859 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:29.859 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.812623+0000", 2024-04-24T15:32:29.859 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:29.859 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.859 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.859 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:29.859 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.812622+0000", 2024-04-24T15:32:29.859 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:29.859 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.859 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.859 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:29.859 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.812627+0000", 2024-04-24T15:32:29.859 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1150000000000002e-06 2024-04-24T15:32:29.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.860 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.860 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:29.860 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.812640+0000", 2024-04-24T15:32:29.860 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3095000000000001e-05 2024-04-24T15:32:29.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.860 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.860 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:29.860 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.812647+0000", 2024-04-24T15:32:29.860 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5069999999999997e-06 2024-04-24T15:32:29.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.860 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.860 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:29.860 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.812685+0000", 2024-04-24T15:32:29.861 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8359000000000003e-05 2024-04-24T15:32:29.861 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.861 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.861 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:29.861 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.812748+0000", 2024-04-24T15:32:29.861 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2389000000000002e-05 2024-04-24T15:32:29.861 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.861 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.861 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:29.861 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.812808+0000", 2024-04-24T15:32:29.861 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0817999999999999e-05 2024-04-24T15:32:29.861 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.861 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.861 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:29.861 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.813294+0000", 2024-04-24T15:32:29.862 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000485763 2024-04-24T15:32:29.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.862 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.862 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:29.862 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.818204+0000", 2024-04-24T15:32:29.862 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0049093119999999999 2024-04-24T15:32:29.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.862 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.862 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:29.862 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.818240+0000", 2024-04-24T15:32:29.862 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6550000000000001e-05 2024-04-24T15:32:29.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.862 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.862 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:29.863 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.818277+0000", 2024-04-24T15:32:29.863 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6918999999999997e-05 2024-04-24T15:32:29.863 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:29.863 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:29.863 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:29.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.863 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.863 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:3943 4.6 4:639bc13c:::rbd_data.114a1f60cb66.00000000000000ea:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:29.863 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.815303+0000", 2024-04-24T15:32:29.863 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.9684246280000002, 2024-04-24T15:32:29.863 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0055918529999999999, 2024-04-24T15:32:29.863 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:29.863 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:29.863 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:29.863 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:29.864 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:29.864 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3943 2024-04-24T15:32:29.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.864 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:29.864 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.864 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:29.864 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.815303+0000", 2024-04-24T15:32:29.864 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:29.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.864 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.864 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:29.864 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.815303+0000", 2024-04-24T15:32:29.864 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:29.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.865 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.865 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:29.865 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.815303+0000", 2024-04-24T15:32:29.865 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:29.865 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.865 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.865 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:29.865 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.815308+0000", 2024-04-24T15:32:29.865 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0180000000000001e-06 2024-04-24T15:32:29.865 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.865 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.865 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:29.865 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.815320+0000", 2024-04-24T15:32:29.865 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.272e-05 2024-04-24T15:32:29.865 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.866 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.866 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:29.866 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.815327+0000", 2024-04-24T15:32:29.866 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7889999999999997e-06 2024-04-24T15:32:29.866 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.866 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.866 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:29.866 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.815370+0000", 2024-04-24T15:32:29.866 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2608000000000003e-05 2024-04-24T15:32:29.866 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.866 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.866 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:29.866 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.815450+0000", 2024-04-24T15:32:29.867 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9944999999999994e-05 2024-04-24T15:32:29.867 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.867 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.867 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:29.867 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.815560+0000", 2024-04-24T15:32:29.867 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000110153 2024-04-24T15:32:29.867 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.867 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.867 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:29.867 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.817736+0000", 2024-04-24T15:32:29.867 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0021763889999999999 2024-04-24T15:32:29.867 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.867 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.867 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:29.867 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.820810+0000", 2024-04-24T15:32:29.867 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0030733449999999999 2024-04-24T15:32:29.868 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.868 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.868 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:29.868 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.820857+0000", 2024-04-24T15:32:29.868 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6928000000000002e-05 2024-04-24T15:32:29.868 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.868 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.868 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:29.868 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.820895+0000", 2024-04-24T15:32:29.868 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8705999999999998e-05 2024-04-24T15:32:29.868 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:29.868 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:29.868 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:29.868 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.868 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.869 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:3976 4.1c 4:396ab59e:::rbd_data.114a1f60cb66.0000000000000160:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:29.869 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.827920+0000", 2024-04-24T15:32:29.869 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.9558082800000003, 2024-04-24T15:32:29.869 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057086940000000003, 2024-04-24T15:32:29.869 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:29.869 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:29.869 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:29.869 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:29.869 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:29.869 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3976 2024-04-24T15:32:29.869 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.869 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:29.869 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.869 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:29.869 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.827920+0000", 2024-04-24T15:32:29.870 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:29.870 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.870 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.870 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:29.870 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.827920+0000", 2024-04-24T15:32:29.870 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:29.870 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.870 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.870 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:29.870 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.827919+0000", 2024-04-24T15:32:29.870 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:29.870 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.870 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.870 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:29.871 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.827924+0000", 2024-04-24T15:32:29.871 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4270000000000001e-06 2024-04-24T15:32:29.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.871 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.871 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:29.871 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.827934+0000", 2024-04-24T15:32:29.871 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0606e-05 2024-04-24T15:32:29.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.871 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.871 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:29.871 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.827939+0000", 2024-04-24T15:32:29.871 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.792e-06 2024-04-24T15:32:29.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.871 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.871 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:29.871 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.827967+0000", 2024-04-24T15:32:29.872 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7919000000000002e-05 2024-04-24T15:32:29.872 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.872 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.872 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:29.872 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.828011+0000", 2024-04-24T15:32:29.872 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3896000000000001e-05 2024-04-24T15:32:29.872 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.872 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.872 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:29.872 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.828070+0000", 2024-04-24T15:32:29.872 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9348999999999997e-05 2024-04-24T15:32:29.872 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.872 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.872 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:29.872 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.830513+0000", 2024-04-24T15:32:29.873 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024429669999999999 2024-04-24T15:32:29.873 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.873 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.873 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:29.873 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.833567+0000", 2024-04-24T15:32:29.873 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0030542590000000001 2024-04-24T15:32:29.873 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.873 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.873 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:29.873 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.833608+0000", 2024-04-24T15:32:29.873 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1148e-05 2024-04-24T15:32:29.873 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.873 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.873 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:29.873 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.833629+0000", 2024-04-24T15:32:29.874 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0066999999999999e-05 2024-04-24T15:32:29.874 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:29.874 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:29.874 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:29.874 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.874 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.874 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:3987 4.1c 4:3a30a8f2:::rbd_data.114a1f60cb66.0000000000000174:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:29.874 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.833638+0000", 2024-04-24T15:32:29.874 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.9500900000000003, 2024-04-24T15:32:29.874 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057825430000000002, 2024-04-24T15:32:29.874 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:29.874 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:29.874 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:29.874 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:29.875 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:29.875 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3987 2024-04-24T15:32:29.875 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.875 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:29.875 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.875 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:29.875 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.833638+0000", 2024-04-24T15:32:29.875 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:29.875 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.875 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.875 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:29.875 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.833638+0000", 2024-04-24T15:32:29.875 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:29.875 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.875 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.875 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:29.876 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.833637+0000", 2024-04-24T15:32:29.876 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:29.876 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.876 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.876 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:29.876 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.833646+0000", 2024-04-24T15:32:29.876 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0180000000000007e-06 2024-04-24T15:32:29.876 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.876 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.876 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:29.876 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.833659+0000", 2024-04-24T15:32:29.876 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.313e-05 2024-04-24T15:32:29.876 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.876 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.877 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:29.877 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.833664+0000", 2024-04-24T15:32:29.877 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.006e-06 2024-04-24T15:32:29.877 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.877 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.877 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:29.877 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.833697+0000", 2024-04-24T15:32:29.877 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2978000000000002e-05 2024-04-24T15:32:29.877 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.877 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.877 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:29.877 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.833756+0000", 2024-04-24T15:32:29.877 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8811000000000003e-05 2024-04-24T15:32:29.877 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.877 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.878 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:29.878 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.833823+0000", 2024-04-24T15:32:29.878 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7225999999999996e-05 2024-04-24T15:32:29.878 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.878 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.878 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:29.878 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.838767+0000", 2024-04-24T15:32:29.878 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0049435809999999998 2024-04-24T15:32:29.878 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.878 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.878 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:29.878 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.839341+0000", 2024-04-24T15:32:29.878 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00057366800000000005 2024-04-24T15:32:29.878 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.878 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.879 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:29.879 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.839385+0000", 2024-04-24T15:32:29.879 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4597e-05 2024-04-24T15:32:29.879 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.879 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.879 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:29.879 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.839421+0000", 2024-04-24T15:32:29.879 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5516999999999999e-05 2024-04-24T15:32:29.879 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:29.879 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:29.879 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:29.879 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.879 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.879 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:4005 4.6 4:64968d7a:::rbd_data.114a1f60cb66.00000000000001bc:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:29.879 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.841940+0000", 2024-04-24T15:32:29.879 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.9417877409999997, 2024-04-24T15:32:29.880 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.005717481, 2024-04-24T15:32:29.880 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:29.880 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:29.880 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:29.880 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:29.880 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:29.880 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4005 2024-04-24T15:32:29.880 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.880 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:29.880 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.880 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:29.880 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.841940+0000", 2024-04-24T15:32:29.880 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:29.880 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.880 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.881 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:29.881 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.841940+0000", 2024-04-24T15:32:29.881 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:29.881 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.881 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.881 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:29.881 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.841940+0000", 2024-04-24T15:32:29.881 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:29.881 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.881 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.881 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:29.881 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.841944+0000", 2024-04-24T15:32:29.881 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.904e-06 2024-04-24T15:32:29.881 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.881 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.881 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:29.882 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.841954+0000", 2024-04-24T15:32:29.882 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7499999999999998e-06 2024-04-24T15:32:29.882 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.882 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.882 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:29.882 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.841960+0000", 2024-04-24T15:32:29.882 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4399999999999996e-06 2024-04-24T15:32:29.882 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.882 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.882 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:29.882 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.841991+0000", 2024-04-24T15:32:29.882 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1615000000000001e-05 2024-04-24T15:32:29.882 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.882 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.882 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:29.882 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.842067+0000", 2024-04-24T15:32:29.883 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6002999999999997e-05 2024-04-24T15:32:29.883 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.883 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.883 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:29.883 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.842168+0000", 2024-04-24T15:32:29.883 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000100474 2024-04-24T15:32:29.883 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.883 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.883 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:29.883 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.847470+0000", 2024-04-24T15:32:29.883 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0053021830000000002 2024-04-24T15:32:29.883 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.883 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.883 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:29.883 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.847577+0000", 2024-04-24T15:32:29.883 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000107425 2024-04-24T15:32:29.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.884 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.884 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:29.884 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.847622+0000", 2024-04-24T15:32:29.884 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5015999999999997e-05 2024-04-24T15:32:29.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.884 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.884 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:29.884 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.847658+0000", 2024-04-24T15:32:29.884 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5509999999999997e-05 2024-04-24T15:32:29.884 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:29.884 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:29.884 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:29.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.884 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.884 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:4119 4.6 4:6588fd86:::rbd_data.114a1f60cb66.0000000000000179:head [set-alloc-hint object_size 262144 write_size 262144,write 151552~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:29.885 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.935980+0000", 2024-04-24T15:32:29.885 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8477478720000002, 2024-04-24T15:32:29.885 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.006739391, 2024-04-24T15:32:29.885 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:29.885 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:29.885 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:29.885 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:29.885 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:29.885 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4119 2024-04-24T15:32:29.885 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.885 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:29.885 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.885 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:29.885 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.935980+0000", 2024-04-24T15:32:29.885 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:29.885 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.886 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.886 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:29.886 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.935980+0000", 2024-04-24T15:32:29.886 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:29.886 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.886 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.886 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:29.886 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.935980+0000", 2024-04-24T15:32:29.886 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:29.886 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.886 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.886 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:29.886 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.935987+0000", 2024-04-24T15:32:29.886 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1590000000000004e-06 2024-04-24T15:32:29.886 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.886 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.887 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:29.887 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.935996+0000", 2024-04-24T15:32:29.887 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8389999999999996e-06 2024-04-24T15:32:29.887 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.887 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.887 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:29.887 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.936001+0000", 2024-04-24T15:32:29.887 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6890000000000001e-06 2024-04-24T15:32:29.887 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.887 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.887 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:29.887 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.936456+0000", 2024-04-24T15:32:29.887 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00045432600000000001 2024-04-24T15:32:29.887 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.887 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.887 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:29.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.936530+0000", 2024-04-24T15:32:29.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4550999999999998e-05 2024-04-24T15:32:29.888 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.888 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.888 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:29.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.936663+0000", 2024-04-24T15:32:29.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000133225 2024-04-24T15:32:29.888 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.888 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.888 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:29.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.938066+0000", 2024-04-24T15:32:29.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0014031969999999999 2024-04-24T15:32:29.888 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.888 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.888 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:29.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.942636+0000", 2024-04-24T15:32:29.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0045691270000000001 2024-04-24T15:32:29.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:29.889 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.942684+0000", 2024-04-24T15:32:29.889 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8123e-05 2024-04-24T15:32:29.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:29.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:29.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:29.889 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.942720+0000", 2024-04-24T15:32:29.889 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5880000000000002e-05 2024-04-24T15:32:29.889 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:29.889 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:29.889 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:29.889 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:29.889 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:29.889 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:32:29.890 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-24T15:32:29.891 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-24T15:32:29.892 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:32:29.892+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:32:29.992 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:32:29.993+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:32:30.061 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:32:30.061 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-04-24T15:32:30.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:30.061 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:8432 4.f 4.cd033def (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-04-24T15:32:30.061 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:29.102885+0000", 2024-04-24T15:32:30.061 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.95942120099999995, 2024-04-24T15:32:30.061 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.95944719300000003, 2024-04-24T15:32:30.062 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:30.062 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for peered", 2024-04-24T15:32:30.062 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:30.062 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:30.062 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:30.062 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 8432 2024-04-24T15:32:30.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:30.062 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:30.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:30.062 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:30.062 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:29.102885+0000", 2024-04-24T15:32:30.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:30.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:30.063 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:30.063 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:30.063 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:29.102885+0000", 2024-04-24T15:32:30.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:30.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:30.063 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:30.063 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:30.063 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:29.102882+0000", 2024-04-24T15:32:30.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999976 2024-04-24T15:32:30.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:30.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:30.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:30.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:29.102896+0000", 2024-04-24T15:32:30.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3291e-05 2024-04-24T15:32:30.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:30.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:30.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:30.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:29.102920+0000", 2024-04-24T15:32:30.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4695999999999999e-05 2024-04-24T15:32:30.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:30.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:30.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:30.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:29.102930+0000", 2024-04-24T15:32:30.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9720000000000004e-06 2024-04-24T15:32:30.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:30.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:30.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:30.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:29.102981+0000", 2024-04-24T15:32:30.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0748000000000003e-05 2024-04-24T15:32:30.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:30.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:30.066 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op must wait for map", 2024-04-24T15:32:30.066 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:29.102985+0000", 2024-04-24T15:32:30.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3780000000000003e-06 2024-04-24T15:32:30.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:30.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:30.066 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:30.066 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:29.110631+0000", 2024-04-24T15:32:30.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0076457030000000002 2024-04-24T15:32:30.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:30.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:30.066 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-04-24T15:32:30.067 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:29.110641+0000", 2024-04-24T15:32:30.067 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.01e-05 2024-04-24T15:32:30.067 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:30.067 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:30.067 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:30.067 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:30.067 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:30.067 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:8433 4.f 4.45fc8f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-04-24T15:32:30.067 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:29.109363+0000", 2024-04-24T15:32:30.067 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.95294335399999996, 2024-04-24T15:32:30.067 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.95302215599999995, 2024-04-24T15:32:30.067 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:30.067 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for peered", 2024-04-24T15:32:30.068 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:30.068 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:30.068 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:30.068 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 8433 2024-04-24T15:32:30.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:30.068 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:30.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:30.068 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:30.068 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:29.109363+0000", 2024-04-24T15:32:30.068 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:30.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:30.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:30.068 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:30.069 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:29.109363+0000", 2024-04-24T15:32:30.069 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:30.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:30.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:30.069 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:30.069 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:29.109361+0000", 2024-04-24T15:32:30.069 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999981 2024-04-24T15:32:30.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:30.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:30.069 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:30.069 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:29.109370+0000", 2024-04-24T15:32:30.069 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1970000000000002e-06 2024-04-24T15:32:30.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:30.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:30.070 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:30.070 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:29.109385+0000", 2024-04-24T15:32:30.070 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5480000000000001e-05 2024-04-24T15:32:30.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:30.070 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:30.070 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:30.070 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:29.109404+0000", 2024-04-24T15:32:30.070 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8970999999999998e-05 2024-04-24T15:32:30.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:30.070 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:30.070 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:30.071 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:29.109461+0000", 2024-04-24T15:32:30.071 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6736e-05 2024-04-24T15:32:30.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:30.071 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:30.071 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2024-04-24T15:32:30.071 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:29.109467+0000", 2024-04-24T15:32:30.071 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2920000000000001e-06 2024-04-24T15:32:30.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:30.071 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:30.071 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:30.071 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:29.110662+0000", 2024-04-24T15:32:30.071 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001194458 2024-04-24T15:32:30.072 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:30.072 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:30.072 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-04-24T15:32:30.072 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:29.110668+0000", 2024-04-24T15:32:30.072 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4649999999999999e-06 2024-04-24T15:32:30.072 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:30.072 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:30.072 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:30.072 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:30.072 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-24T15:32:30.072 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 2 2024-04-24T15:32:30.072 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:32:30.074 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-24T15:32:30.074 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-04-24T15:32:30.091 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-04-24T15:32:30.093 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:32:30.094+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:32:30.114 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-04-24T15:32:30.142 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-04-24T15:32:30.162 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-04-24T15:32:30.193 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:32:30.194+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:32:30.198 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-04-24T15:32:30.232 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-04-24T15:32:30.254 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-04-24T15:32:30.255 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T15:32:30.255 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T15:32:30.255 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T15:32:30.255 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T15:32:30.255 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T15:32:30.256 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T15:32:30.256 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T15:32:30.256 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T15:32:30.261 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:30.261 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:30.272 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-24T15:32:30.295 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:32:30.296+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:32:30.396 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:32:30.397+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:32:30.437 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:30.437 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:30.447 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-24T15:32:30.496 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:32:30.497+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:32:30.570 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:30.570 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:30.580 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-24T15:32:30.623 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:32:30.625+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:32:30.698 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:32:30.699+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:32:30.702 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:30.702 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:30.712 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-24T15:32:30.799 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:32:30.800+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:32:30.875 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:30.875 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:30.884 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-24T15:32:30.928 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:32:30.930+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:32:31.001 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:32:31.002+0000 7f23abcf6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 44021) UID: 0 2024-04-24T15:32:31.007 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:31.007 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:31.017 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-24T15:32:31.102 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:32:31.104+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:32:31.127 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-24T15:32:31.170 INFO:teuthology.orchestra.run.smithi096.stdout:1074 writesame 0xcc5c1f4 thru 0xcc5cf8a data_size 0x7(0xd97 bytes) 2024-04-24T15:32:31.170 INFO:teuthology.orchestra.run.smithi096.stdout:1075 write 0x206376d thru 0x206b47d (0x7d11 bytes) 2024-04-24T15:32:31.173 INFO:teuthology.orchestra.run.smithi096.stdout:1076 write 0x1db0b20 thru 0x1dbcdff (0xc2e0 bytes) 2024-04-24T15:32:31.175 INFO:teuthology.orchestra.run.smithi096.stdout:1077 read 0x43f87da thru 0x440254a (0x9d71 bytes) 2024-04-24T15:32:31.176 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:31.176 INFO:teuthology.orchestra.run.smithi096.stdout:1081 read 0x94d0311 thru 0x94d1f90 (0x1c80 bytes) 2024-04-24T15:32:31.184 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:31.184 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:31.188 INFO:teuthology.orchestra.run.smithi096.stdout:1082 trunc from 0xcc5cf8b to 0x319511d 2024-04-24T15:32:31.194 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-24T15:32:31.199 INFO:teuthology.orchestra.run.smithi096.stdout:1083 read 0x23a3b96 thru 0x23aeb16 (0xaf81 bytes) 2024-04-24T15:32:31.200 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:31.200 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:31.200 INFO:teuthology.orchestra.run.smithi096.stdout:1087 punch from 0x1c46e86 to 0x1c4e4ce, (0x7648 bytes) 2024-04-24T15:32:31.208 INFO:teuthology.orchestra.run.smithi096.stdout:1089 trunc from 0x319511d to 0x9c49b4a 2024-04-24T15:32:31.215 INFO:teuthology.orchestra.run.smithi096.stdout:1090 write 0x1eda393 thru 0x1ee6f39 (0xcba7 bytes) 2024-04-24T15:32:31.218 INFO:teuthology.orchestra.run.smithi096.stdout:1091 writesame 0x74f7b16 thru 0x75070c8 data_size 0x1(0xf5b3 bytes) 2024-04-24T15:32:31.235 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:32:31.237+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:32:31.262 INFO:teuthology.orchestra.run.smithi096.stdout:1092 write 0xd917602 thru 0xd91e247 (0x6c46 bytes) 2024-04-24T15:32:31.265 INFO:teuthology.orchestra.run.smithi096.stdout:1095 write 0x4b22cc8 thru 0x4b26587 (0x38c0 bytes) 2024-04-24T15:32:31.269 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:31.277 INFO:teuthology.orchestra.run.smithi096.stdout:1099 trunc from 0xd91e248 to 0x73f5d3f 2024-04-24T15:32:31.295 INFO:teuthology.orchestra.run.smithi096.stdout:1100 trunc from 0x73f5d3f to 0x12c01ed 2024-04-24T15:32:31.304 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:32:31.305+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:32:31.305 INFO:teuthology.orchestra.run.smithi096.stdout:1101 punch from 0x97979a to 0x97e168, (0x49ce bytes) 2024-04-24T15:32:31.305 INFO:teuthology.orchestra.run.smithi096.stdout:1102 punch from 0x1110b94 to 0x111300b, (0x2477 bytes) 2024-04-24T15:32:31.313 INFO:teuthology.orchestra.run.smithi096.stdout:1103 writesame 0x294dbf2 thru 0x29559f2 data_size 0x1(0x7e01 bytes) 2024-04-24T15:32:31.315 INFO:teuthology.orchestra.run.smithi096.stdout:1104 write 0x27f2470 thru 0x27f3cf8 (0x1889 bytes) 2024-04-24T15:32:31.316 INFO:teuthology.orchestra.run.smithi096.stdout:1105 punch from 0xaf45c4 to 0xafe2b0, (0x9cec bytes) 2024-04-24T15:32:31.319 INFO:teuthology.orchestra.run.smithi096.stdout:1107 trunc from 0x29559f3 to 0x361722 2024-04-24T15:32:31.328 INFO:teuthology.orchestra.run.smithi096.stdout:1109 read 0x33404b thru 0x33bb0a (0x7ac0 bytes) 2024-04-24T15:32:31.329 INFO:teuthology.orchestra.run.smithi096.stdout:1110 read 0x8f0cf thru 0x99619 (0xa54b bytes) 2024-04-24T15:32:31.353 INFO:teuthology.orchestra.run.smithi096.stdout:1111 writesame 0xe4b5138 thru 0xe4bcab8 data_size 0x1(0x7981 bytes) 2024-04-24T15:32:31.353 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:31.353 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:31.353 INFO:teuthology.orchestra.run.smithi096.stdout:1114 write 0x5cbd7e3 thru 0x5cc452f (0x6d4d bytes) 2024-04-24T15:32:31.358 INFO:teuthology.orchestra.run.smithi096.stdout:1115 read 0x5ccc64c thru 0x5cd2dd4 (0x6789 bytes) 2024-04-24T15:32:31.359 INFO:teuthology.orchestra.run.smithi096.stdout:1116 write 0x6ef8efd thru 0x6f08440 (0xf544 bytes) 2024-04-24T15:32:31.362 INFO:teuthology.orchestra.run.smithi096.stdout:1118 read 0x6e34b77 thru 0x6e3e377 (0x9801 bytes) 2024-04-24T15:32:31.362 INFO:teuthology.orchestra.run.smithi096.stdout:1120 punch from 0xb9d5f45 to 0xb9d9a3e, (0x3af9 bytes) 2024-04-24T15:32:31.362 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:31.365 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:31.366 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:31.370 INFO:teuthology.orchestra.run.smithi096.stdout:1123 trunc from 0xe4bcab9 to 0x7ede8de 2024-04-24T15:32:31.373 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-24T15:32:31.376 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:31.376 INFO:teuthology.orchestra.run.smithi096.stdout:1125 read 0xcca8ec thru 0xcd2cb5 (0x83ca bytes) 2024-04-24T15:32:31.377 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:31.377 INFO:teuthology.orchestra.run.smithi096.stdout:1127 punch from 0x450783e to 0x450da2b, (0x61ed bytes) 2024-04-24T15:32:31.377 INFO:teuthology.orchestra.run.smithi096.stdout:1129 punch from 0x76414d1 to 0x76485fa, (0x7129 bytes) 2024-04-24T15:32:31.377 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:31.377 INFO:teuthology.orchestra.run.smithi096.stdout:1131 punch from 0x4963629 to 0x496b983, (0x835a bytes) 2024-04-24T15:32:31.377 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:31.383 INFO:teuthology.orchestra.run.smithi096.stdout:1133 write 0x9a6e5f1 thru 0x9a71b7a (0x358a bytes) 2024-04-24T15:32:31.391 INFO:teuthology.orchestra.run.smithi096.stdout:1135 write 0xa0a0ba9 thru 0xa0a56c6 (0x4b1e bytes) 2024-04-24T15:32:31.394 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:31.394 INFO:teuthology.orchestra.run.smithi096.stdout:1137 read 0x1b8dcb1 thru 0x1b8f8f9 (0x1c49 bytes) 2024-04-24T15:32:31.394 INFO:teuthology.orchestra.run.smithi096.stdout:1138 read 0x5651ab thru 0x56eba5 (0x99fb bytes) 2024-04-24T15:32:31.395 INFO:teuthology.orchestra.run.smithi096.stdout:1139 write 0x3532c06 thru 0x3542938 (0xfd33 bytes) 2024-04-24T15:32:31.396 INFO:teuthology.orchestra.run.smithi096.stdout:1141 write 0x75718e7 thru 0x757824e (0x6968 bytes) 2024-04-24T15:32:31.398 INFO:teuthology.orchestra.run.smithi096.stdout:1142 read 0x78e4263 thru 0x78ecfa8 (0x8d46 bytes) 2024-04-24T15:32:31.398 INFO:teuthology.orchestra.run.smithi096.stdout:1143 writesame 0x1ab66f0 thru 0x1ab9480 data_size 0x1(0x2d91 bytes) 2024-04-24T15:32:31.404 INFO:teuthology.orchestra.run.smithi096.stdout:1144 punch from 0x87d4508 to 0x87d4556, (0x4e bytes) 2024-04-24T15:32:31.411 INFO:teuthology.orchestra.run.smithi096.stdout:1145 write 0xbb272a7 thru 0xbb2c174 (0x4ece bytes) 2024-04-24T15:32:31.415 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:32:31.416+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:32:31.419 INFO:teuthology.orchestra.run.smithi096.stdout:1146 write 0xd1b1bcb thru 0xd1b49e5 (0x2e1b bytes) 2024-04-24T15:32:31.422 INFO:teuthology.orchestra.run.smithi096.stdout:1148 writesame 0x87f5d5f thru 0x87f69ea data_size 0x2(0xc8c bytes) 2024-04-24T15:32:31.423 INFO:teuthology.orchestra.run.smithi096.stdout:1149 write 0x2e29364 thru 0x2e37e54 (0xeaf1 bytes) 2024-04-24T15:32:31.425 INFO:teuthology.orchestra.run.smithi096.stdout:1150 write 0xaace10b thru 0xaad8943 (0xa839 bytes) 2024-04-24T15:32:31.434 INFO:teuthology.orchestra.run.smithi096.stdout:1151 trunc from 0xd1b49e6 to 0x67a84b4 2024-04-24T15:32:31.450 INFO:teuthology.orchestra.run.smithi096.stdout:1153 trunc from 0x67a84b4 to 0x46b73b0 2024-04-24T15:32:31.458 INFO:teuthology.orchestra.run.smithi096.stdout:1154 read 0x15dc3f5 thru 0x15e8bf3 (0xc7ff bytes) 2024-04-24T15:32:31.459 INFO:teuthology.orchestra.run.smithi096.stdout:1155 punch from 0x20915d5 to 0x20938bc, (0x22e7 bytes) 2024-04-24T15:32:31.470 INFO:teuthology.orchestra.run.smithi096.stdout:1156 trunc from 0x46b73b0 to 0xd67e7f4 2024-04-24T15:32:31.477 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:31.477 INFO:teuthology.orchestra.run.smithi096.stdout:1159 read 0xae5cc66 thru 0xae66cb6 (0xa051 bytes) 2024-04-24T15:32:31.477 INFO:teuthology.orchestra.run.smithi096.stdout:1161 read 0x9f0dfb thru 0x9fecd4 (0xdeda bytes) 2024-04-24T15:32:31.478 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:31.478 INFO:teuthology.orchestra.run.smithi096.stdout:1164 clone 4 order 21 su 8192 sc 5 2024-04-24T15:32:31.505 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:32:31.506+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:32:31.538 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:31.539 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:31.548 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-24T15:32:31.606 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:32:31.607+0000 7f23abcf6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 44021) UID: 0 2024-04-24T15:32:31.671 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:31.672 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:31.682 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-24T15:32:31.706 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:32:31.707+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:32:31.807 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:32:31.808+0000 7f23abcf6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 44021) UID: 0 2024-04-24T15:32:31.845 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:31.845 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:31.855 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-24T15:32:31.938 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:32:31.940+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:32:32.009 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:32:32.010+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:32:32.018 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:32.018 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:32.027 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-24T15:32:32.111 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:32:32.113+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:32:32.144 INFO:teuthology.orchestra.run.smithi096.stderr:nodeep-scrub is set 2024-04-24T15:32:32.192 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:32.192 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:32.202 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-24T15:32:32.244 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:32:32.245+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:32:32.268 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-24T15:32:32.312 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:32:32.313+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:32:32.366 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:32.367 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:32.376 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-24T15:32:32.411 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-04-24T15:32:32.417 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:32:32.419+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:32:32.429 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-04-24T15:32:32.447 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-04-24T15:32:32.466 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-04-24T15:32:32.484 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-04-24T15:32:32.502 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:32:32.503 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:32:32.503 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-04-24T15:32:32.503 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:32:32.503 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-04-24T15:32:32.513 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-24T15:32:32.514 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:32:32.514+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:32:32.522 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-04-24T15:32:32.543 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-04-24T15:32:32.545 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T15:32:32.545 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T15:32:32.545 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T15:32:32.545 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T15:32:32.545 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T15:32:32.545 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T15:32:32.545 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T15:32:32.545 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T15:32:32.614 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:32:32.616+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:32:32.645 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:32:32.645 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:32:32.645 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-04-24T15:32:32.645 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-04-24T15:32:32.645 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:32:32.655 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-24T15:32:32.716 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:32:32.717+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:32:32.779 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:32:32.779 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-04-24T15:32:32.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-04-24T15:32:32.779 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-04-24T15:32:32.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.780 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:18 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'8, mlcod=15'5)", 2024-04-24T15:32:32.780 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:00.973932+0000", 2024-04-24T15:32:32.780 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.805386146, 2024-04-24T15:32:32.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0064286839999999996, 2024-04-24T15:32:32.780 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:32.780 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:32.780 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:32.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:32.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.973932+0000", 2024-04-24T15:32:32.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:32.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:32.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.973932+0000", 2024-04-24T15:32:32.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:32.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:32.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.973931+0000", 2024-04-24T15:32:32.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:32.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:32.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.973938+0000", 2024-04-24T15:32:32.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.029e-06 2024-04-24T15:32:32.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:32.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.973948+0000", 2024-04-24T15:32:32.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9129999999999998e-06 2024-04-24T15:32:32.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:32.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.973952+0000", 2024-04-24T15:32:32.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3069999999999996e-06 2024-04-24T15:32:32.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:32.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.973986+0000", 2024-04-24T15:32:32.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3643000000000003e-05 2024-04-24T15:32:32.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:32.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.974006+0000", 2024-04-24T15:32:32.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9658999999999999e-05 2024-04-24T15:32:32.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:32.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.980330+0000", 2024-04-24T15:32:32.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0063243479999999996 2024-04-24T15:32:32.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:32.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.980361+0000", 2024-04-24T15:32:32.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0527000000000001e-05 2024-04-24T15:32:32.784 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:32.784 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:32.784 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:32.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.784 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:21 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'11, mlcod=15'8)", 2024-04-24T15:32:32.784 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:00.986502+0000", 2024-04-24T15:32:32.784 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.792816213000002, 2024-04-24T15:32:32.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0062008819999999996, 2024-04-24T15:32:32.784 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:32.784 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:32.784 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:32.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:32.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.986502+0000", 2024-04-24T15:32:32.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:32.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:32.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.986502+0000", 2024-04-24T15:32:32.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:32.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:32.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.986501+0000", 2024-04-24T15:32:32.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:32.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:32.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.986639+0000", 2024-04-24T15:32:32.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013731400000000001 2024-04-24T15:32:32.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:32.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.986649+0000", 2024-04-24T15:32:32.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.041e-05 2024-04-24T15:32:32.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:32.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.986654+0000", 2024-04-24T15:32:32.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2000000000000002e-06 2024-04-24T15:32:32.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:32.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.986678+0000", 2024-04-24T15:32:32.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3731e-05 2024-04-24T15:32:32.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:32.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.986691+0000", 2024-04-24T15:32:32.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3078000000000001e-05 2024-04-24T15:32:32.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:32.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.992666+0000", 2024-04-24T15:32:32.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0059751650000000002 2024-04-24T15:32:32.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:32.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.992703+0000", 2024-04-24T15:32:32.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6622000000000003e-05 2024-04-24T15:32:32.788 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:32.788 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:32.788 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:32.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.788 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:22 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'12, mlcod=15'8)", 2024-04-24T15:32:32.789 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:00.987102+0000", 2024-04-24T15:32:32.789 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.792216367000002, 2024-04-24T15:32:32.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0056291650000000002, 2024-04-24T15:32:32.789 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:32.789 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:32.789 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:32.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:32.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.987102+0000", 2024-04-24T15:32:32.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:32.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:32.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.987102+0000", 2024-04-24T15:32:32.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:32.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:32.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.987101+0000", 2024-04-24T15:32:32.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:32.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:32.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.987106+0000", 2024-04-24T15:32:32.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0969999999999996e-06 2024-04-24T15:32:32.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:32.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.987113+0000", 2024-04-24T15:32:32.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6789999999999999e-06 2024-04-24T15:32:32.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:32.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.987117+0000", 2024-04-24T15:32:32.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6519999999999999e-06 2024-04-24T15:32:32.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:32.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.987143+0000", 2024-04-24T15:32:32.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6098999999999999e-05 2024-04-24T15:32:32.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:32.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.987157+0000", 2024-04-24T15:32:32.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4100999999999999e-05 2024-04-24T15:32:32.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:32.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.992706+0000", 2024-04-24T15:32:32.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0055495630000000004 2024-04-24T15:32:32.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:32.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.992731+0000", 2024-04-24T15:32:32.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4519e-05 2024-04-24T15:32:32.792 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:32.792 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:32.792 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:32.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.793 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:69 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'43, mlcod=15'39)", 2024-04-24T15:32:32.793 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.058504+0000", 2024-04-24T15:32:32.793 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.720813892999999, 2024-04-24T15:32:32.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.006494606, 2024-04-24T15:32:32.793 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:32.793 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:32.793 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:32.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:32.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.058504+0000", 2024-04-24T15:32:32.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:32.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:32.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.058504+0000", 2024-04-24T15:32:32.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:32.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:32.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.058503+0000", 2024-04-24T15:32:32.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:32.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:32.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.058588+0000", 2024-04-24T15:32:32.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4087999999999999e-05 2024-04-24T15:32:32.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:32.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.058597+0000", 2024-04-24T15:32:32.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7459999999999995e-06 2024-04-24T15:32:32.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:32.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.058602+0000", 2024-04-24T15:32:32.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6929999999999998e-06 2024-04-24T15:32:32.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:32.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.060983+0000", 2024-04-24T15:32:32.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023812690000000001 2024-04-24T15:32:32.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:32.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.060996+0000", 2024-04-24T15:32:32.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2826999999999999e-05 2024-04-24T15:32:32.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:32.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.064972+0000", 2024-04-24T15:32:32.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0039757890000000004 2024-04-24T15:32:32.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:32.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.064999+0000", 2024-04-24T15:32:32.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6917999999999999e-05 2024-04-24T15:32:32.797 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:32.797 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:32.797 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:32.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.797 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:70 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'44, mlcod=15'39)", 2024-04-24T15:32:32.797 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.059180+0000", 2024-04-24T15:32:32.797 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.720138141, 2024-04-24T15:32:32.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0058429709999999998, 2024-04-24T15:32:32.797 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:32.797 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:32.797 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:32.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:32.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.059180+0000", 2024-04-24T15:32:32.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:32.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:32.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.059180+0000", 2024-04-24T15:32:32.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:32.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:32.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.059179+0000", 2024-04-24T15:32:32.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:32.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:32.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.059184+0000", 2024-04-24T15:32:32.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.074e-06 2024-04-24T15:32:32.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:32.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.059194+0000", 2024-04-24T15:32:32.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6050000000000004e-06 2024-04-24T15:32:32.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:32.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.059199+0000", 2024-04-24T15:32:32.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2639999999999999e-06 2024-04-24T15:32:32.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:32.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.061424+0000", 2024-04-24T15:32:32.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0022244880000000002 2024-04-24T15:32:32.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:32.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.061444+0000", 2024-04-24T15:32:32.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0633e-05 2024-04-24T15:32:32.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:32.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.065001+0000", 2024-04-24T15:32:32.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035567670000000002 2024-04-24T15:32:32.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:32.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.065023+0000", 2024-04-24T15:32:32.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1724000000000001e-05 2024-04-24T15:32:32.801 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:32.801 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:32.801 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:32.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.801 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:154 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'108, mlcod=15'105)", 2024-04-24T15:32:32.801 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.929033+0000", 2024-04-24T15:32:32.801 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 30.850284760000001, 2024-04-24T15:32:32.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0068887990000000001, 2024-04-24T15:32:32.801 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:32.801 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:32.802 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:32.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.802 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:32.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929033+0000", 2024-04-24T15:32:32.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:32.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.802 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:32.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929033+0000", 2024-04-24T15:32:32.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:32.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.802 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:32.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929033+0000", 2024-04-24T15:32:32.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:32.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:32.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929042+0000", 2024-04-24T15:32:32.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5440000000000004e-06 2024-04-24T15:32:32.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:32.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929051+0000", 2024-04-24T15:32:32.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0580000000000004e-06 2024-04-24T15:32:32.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:32.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929055+0000", 2024-04-24T15:32:32.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.337e-06 2024-04-24T15:32:32.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:32.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929074+0000", 2024-04-24T15:32:32.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9724999999999998e-05 2024-04-24T15:32:32.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:32.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929084+0000", 2024-04-24T15:32:32.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7049999999999994e-06 2024-04-24T15:32:32.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:32.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.935895+0000", 2024-04-24T15:32:32.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0068104359999999996 2024-04-24T15:32:32.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.805 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:32.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.935922+0000", 2024-04-24T15:32:32.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7538999999999998e-05 2024-04-24T15:32:32.805 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:32.805 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:32.805 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:32.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.805 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:155 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'109, mlcod=15'105)", 2024-04-24T15:32:32.805 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.929398+0000", 2024-04-24T15:32:32.805 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 30.849919779, 2024-04-24T15:32:32.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0098754700000000008, 2024-04-24T15:32:32.805 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:32.806 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:32.806 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:32.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.806 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:32.806 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929398+0000", 2024-04-24T15:32:32.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:32.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.806 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:32.806 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929398+0000", 2024-04-24T15:32:32.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:32.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.806 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:32.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929398+0000", 2024-04-24T15:32:32.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:32.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:32.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929571+0000", 2024-04-24T15:32:32.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017281999999999999 2024-04-24T15:32:32.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:32.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929578+0000", 2024-04-24T15:32:32.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8929999999999999e-06 2024-04-24T15:32:32.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:32.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929581+0000", 2024-04-24T15:32:32.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1709999999999998e-06 2024-04-24T15:32:32.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:32.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929605+0000", 2024-04-24T15:32:32.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3938e-05 2024-04-24T15:32:32.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:32.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929618+0000", 2024-04-24T15:32:32.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3653e-05 2024-04-24T15:32:32.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:32.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939186+0000", 2024-04-24T15:32:32.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0095674260000000004 2024-04-24T15:32:32.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:32.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939274+0000", 2024-04-24T15:32:32.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7928000000000002e-05 2024-04-24T15:32:32.809 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:32.809 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:32.809 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:32.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.809 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:156 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'110, mlcod=15'105)", 2024-04-24T15:32:32.809 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.930050+0000", 2024-04-24T15:32:32.809 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 30.849268463000001, 2024-04-24T15:32:32.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0092911650000000005, 2024-04-24T15:32:32.809 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:32.810 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:32.810 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:32.810 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.810 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:32.810 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930050+0000", 2024-04-24T15:32:32.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:32.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.810 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.810 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:32.810 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930050+0000", 2024-04-24T15:32:32.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:32.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.810 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.810 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:32.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930049+0000", 2024-04-24T15:32:32.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:32.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:32.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930053+0000", 2024-04-24T15:32:32.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.98e-06 2024-04-24T15:32:32.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:32.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930058+0000", 2024-04-24T15:32:32.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3530000000000002e-06 2024-04-24T15:32:32.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:32.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930061+0000", 2024-04-24T15:32:32.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6970000000000002e-06 2024-04-24T15:32:32.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:32.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930082+0000", 2024-04-24T15:32:32.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0815e-05 2024-04-24T15:32:32.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:32.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930093+0000", 2024-04-24T15:32:32.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0699999999999999e-05 2024-04-24T15:32:32.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:32.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939280+0000", 2024-04-24T15:32:32.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0091874880000000006 2024-04-24T15:32:32.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:32.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939341+0000", 2024-04-24T15:32:32.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0608e-05 2024-04-24T15:32:32.813 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:32.813 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:32.814 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:32.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.814 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:157 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'111, mlcod=15'106)", 2024-04-24T15:32:32.814 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.930506+0000", 2024-04-24T15:32:32.814 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 30.848811605000002, 2024-04-24T15:32:32.814 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.008889246, 2024-04-24T15:32:32.814 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:32.814 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:32.814 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:32.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:32.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930506+0000", 2024-04-24T15:32:32.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:32.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:32.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930506+0000", 2024-04-24T15:32:32.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:32.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:32.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930506+0000", 2024-04-24T15:32:32.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:32.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:32.816 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930510+0000", 2024-04-24T15:32:32.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1810000000000003e-06 2024-04-24T15:32:32.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:32.816 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930516+0000", 2024-04-24T15:32:32.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1920000000000002e-06 2024-04-24T15:32:32.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:32.816 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930520+0000", 2024-04-24T15:32:32.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.501e-06 2024-04-24T15:32:32.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:32.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930539+0000", 2024-04-24T15:32:32.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8929000000000001e-05 2024-04-24T15:32:32.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:32.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930550+0000", 2024-04-24T15:32:32.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1168999999999999e-05 2024-04-24T15:32:32.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:32.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939343+0000", 2024-04-24T15:32:32.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0087932900000000005 2024-04-24T15:32:32.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.818 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:32.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939396+0000", 2024-04-24T15:32:32.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2454999999999998e-05 2024-04-24T15:32:32.818 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:32.818 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:32.818 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:32.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.818 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:158 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'112, mlcod=15'106)", 2024-04-24T15:32:32.818 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.930877+0000", 2024-04-24T15:32:32.818 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 30.848440914000001, 2024-04-24T15:32:32.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.008572741, 2024-04-24T15:32:32.818 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:32.819 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:32.819 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:32.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.819 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:32.819 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930877+0000", 2024-04-24T15:32:32.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:32.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.819 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:32.819 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930877+0000", 2024-04-24T15:32:32.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:32.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.819 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:32.820 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930877+0000", 2024-04-24T15:32:32.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:32.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.820 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:32.820 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930968+0000", 2024-04-24T15:32:32.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1638999999999995e-05 2024-04-24T15:32:32.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.820 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:32.820 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930975+0000", 2024-04-24T15:32:32.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2169999999999999e-06 2024-04-24T15:32:32.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.820 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:32.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930977+0000", 2024-04-24T15:32:32.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.791e-06 2024-04-24T15:32:32.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:32.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931002+0000", 2024-04-24T15:32:32.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4491e-05 2024-04-24T15:32:32.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:32.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931012+0000", 2024-04-24T15:32:32.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0217000000000001e-05 2024-04-24T15:32:32.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:32.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939398+0000", 2024-04-24T15:32:32.822 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.008385366 2024-04-24T15:32:32.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.822 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:32.822 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939450+0000", 2024-04-24T15:32:32.822 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2326e-05 2024-04-24T15:32:32.822 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:32.822 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:32.822 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:32.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.822 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:159 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'113, mlcod=15'106)", 2024-04-24T15:32:32.822 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.931313+0000", 2024-04-24T15:32:32.822 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 30.848005562000001, 2024-04-24T15:32:32.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0081994159999999993, 2024-04-24T15:32:32.823 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:32.823 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:32.823 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:32.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.823 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:32.823 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931313+0000", 2024-04-24T15:32:32.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:32.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.823 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:32.823 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931313+0000", 2024-04-24T15:32:32.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:32.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.823 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:32.824 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931312+0000", 2024-04-24T15:32:32.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:32.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.824 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.824 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:32.824 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931316+0000", 2024-04-24T15:32:32.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0589999999999996e-06 2024-04-24T15:32:32.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.824 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.824 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:32.824 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931322+0000", 2024-04-24T15:32:32.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2020000000000001e-06 2024-04-24T15:32:32.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.824 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.825 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:32.825 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931327+0000", 2024-04-24T15:32:32.825 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2440000000000004e-06 2024-04-24T15:32:32.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.825 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.825 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:32.825 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931348+0000", 2024-04-24T15:32:32.825 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1464e-05 2024-04-24T15:32:32.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.825 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.825 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:32.825 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931359+0000", 2024-04-24T15:32:32.825 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0699e-05 2024-04-24T15:32:32.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.826 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:32.826 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939452+0000", 2024-04-24T15:32:32.826 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0080931749999999993 2024-04-24T15:32:32.826 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.826 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:32.826 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939512+0000", 2024-04-24T15:32:32.826 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0050000000000003e-05 2024-04-24T15:32:32.826 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:32.826 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:32.826 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:32.826 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.827 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:2447 4.6 4:668081fa:::rbd_data.114a1f60cb66.00000000000001f0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-24T15:32:32.827 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:23.027189+0000", 2024-04-24T15:32:32.827 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.7521295380000002, 2024-04-24T15:32:32.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0056741200000000004, 2024-04-24T15:32:32.827 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:32.827 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:32.827 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:32.827 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:32.827 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:32.827 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2447 2024-04-24T15:32:32.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.827 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:32.827 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.827 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:32.827 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027189+0000", 2024-04-24T15:32:32.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:32.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.828 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.828 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:32.828 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027189+0000", 2024-04-24T15:32:32.828 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:32.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.828 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.828 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:32.828 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027188+0000", 2024-04-24T15:32:32.828 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:32.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.828 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.828 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:32.828 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027194+0000", 2024-04-24T15:32:32.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0499999999999997e-06 2024-04-24T15:32:32.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.829 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:32.829 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027205+0000", 2024-04-24T15:32:32.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1619e-05 2024-04-24T15:32:32.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.829 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:32.829 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027212+0000", 2024-04-24T15:32:32.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4289999999999998e-06 2024-04-24T15:32:32.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.829 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:32.830 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027243+0000", 2024-04-24T15:32:32.830 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1510000000000002e-05 2024-04-24T15:32:32.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.830 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:32.830 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027288+0000", 2024-04-24T15:32:32.830 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4484000000000003e-05 2024-04-24T15:32:32.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.830 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-24T15:32:32.830 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027351+0000", 2024-04-24T15:32:32.830 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3311000000000004e-05 2024-04-24T15:32:32.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.831 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.831 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:32.831 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.029874+0000", 2024-04-24T15:32:32.831 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00252315 2024-04-24T15:32:32.831 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.831 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.831 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:32.831 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.032786+0000", 2024-04-24T15:32:32.831 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0029115370000000001 2024-04-24T15:32:32.831 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.831 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.831 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:32.831 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.032828+0000", 2024-04-24T15:32:32.831 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2132000000000003e-05 2024-04-24T15:32:32.832 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.832 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.832 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:32.832 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.032863+0000", 2024-04-24T15:32:32.832 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4912e-05 2024-04-24T15:32:32.832 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:32.832 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:32.832 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:32.832 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.832 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.832 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:2449 4.6 4:61283b7c:::rbd_data.114a1f60cb66.00000000000001f2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-24T15:32:32.832 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:23.027620+0000", 2024-04-24T15:32:32.832 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.7516981250000008, 2024-04-24T15:32:32.832 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0053273959999999999, 2024-04-24T15:32:32.833 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:32.833 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:32.833 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:32.833 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:32.833 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:32.833 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2449 2024-04-24T15:32:32.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.833 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:32.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.833 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:32.833 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027620+0000", 2024-04-24T15:32:32.833 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:32.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.833 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:32.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027620+0000", 2024-04-24T15:32:32.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:32.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.834 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:32.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027619+0000", 2024-04-24T15:32:32.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:32.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.834 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:32.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027623+0000", 2024-04-24T15:32:32.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7100000000000001e-06 2024-04-24T15:32:32.835 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.835 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.835 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:32.835 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027636+0000", 2024-04-24T15:32:32.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2609000000000001e-05 2024-04-24T15:32:32.835 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.835 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.835 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:32.835 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027643+0000", 2024-04-24T15:32:32.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.182e-06 2024-04-24T15:32:32.835 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.835 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.835 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:32.836 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027678+0000", 2024-04-24T15:32:32.836 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5641e-05 2024-04-24T15:32:32.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.836 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.836 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:32.836 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027765+0000", 2024-04-24T15:32:32.836 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6691000000000001e-05 2024-04-24T15:32:32.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.836 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.836 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-24T15:32:32.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027864+0000", 2024-04-24T15:32:32.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9257000000000002e-05 2024-04-24T15:32:32.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.837 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:32.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.032638+0000", 2024-04-24T15:32:32.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004773429 2024-04-24T15:32:32.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.837 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:32.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.032896+0000", 2024-04-24T15:32:32.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000258352 2024-04-24T15:32:32.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.838 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:32.838 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.032924+0000", 2024-04-24T15:32:32.838 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8337999999999999e-05 2024-04-24T15:32:32.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.838 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:32.838 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.032947+0000", 2024-04-24T15:32:32.838 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2849000000000001e-05 2024-04-24T15:32:32.838 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:32.838 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:32.839 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:32.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.839 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4426.0:2767 4.6 e34/33 4:6297db09:::rbd_data.114a1f60cb66.00000000000000e1:head v 34'25, mlcod=34'23)", 2024-04-24T15:32:32.839 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.256572+0000", 2024-04-24T15:32:32.839 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 8.5227461479999995, 2024-04-24T15:32:32.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0050987539999999996, 2024-04-24T15:32:32.839 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:32.839 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:32.839 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:32.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.839 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:32.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.256572+0000", 2024-04-24T15:32:32.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:32.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.840 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:32.840 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.256572+0000", 2024-04-24T15:32:32.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:32.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.840 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:32.840 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.256571+0000", 2024-04-24T15:32:32.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:32.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.840 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:32.840 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.256578+0000", 2024-04-24T15:32:32.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.161e-06 2024-04-24T15:32:32.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.841 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:32.841 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.256585+0000", 2024-04-24T15:32:32.841 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.391e-06 2024-04-24T15:32:32.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.841 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:32.841 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.256590+0000", 2024-04-24T15:32:32.841 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8189999999999999e-06 2024-04-24T15:32:32.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.841 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:32.841 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.256623+0000", 2024-04-24T15:32:32.841 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3618000000000002e-05 2024-04-24T15:32:32.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.841 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:32.842 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.256642+0000", 2024-04-24T15:32:32.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8479999999999999e-05 2024-04-24T15:32:32.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.842 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.842 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:32.842 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.261622+0000", 2024-04-24T15:32:32.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0049796959999999996 2024-04-24T15:32:32.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.842 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.842 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:32.842 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.261671+0000", 2024-04-24T15:32:32.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9147000000000002e-05 2024-04-24T15:32:32.842 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:32.842 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:32.843 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:32.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.843 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4426.0:3917 4.3 e34/33 4:c23f5710:::rbd_data.114a1f60cb66.0000000000000091:head v 34'28, mlcod=34'26)", 2024-04-24T15:32:32.843 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.804899+0000", 2024-04-24T15:32:32.843 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 7.9744194989999997, 2024-04-24T15:32:32.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0050025249999999999, 2024-04-24T15:32:32.843 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:32.843 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:32.843 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:32.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.843 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:32.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.804899+0000", 2024-04-24T15:32:32.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:32.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.844 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:32.844 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.804899+0000", 2024-04-24T15:32:32.844 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:32.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.844 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:32.844 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.804898+0000", 2024-04-24T15:32:32.844 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:32.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.844 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:32.844 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.804903+0000", 2024-04-24T15:32:32.844 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9679999999999997e-06 2024-04-24T15:32:32.845 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.845 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.845 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:32.845 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.804911+0000", 2024-04-24T15:32:32.845 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8490000000000006e-06 2024-04-24T15:32:32.845 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.845 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.845 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:32.845 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.804916+0000", 2024-04-24T15:32:32.845 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5220000000000001e-06 2024-04-24T15:32:32.845 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.845 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.845 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:32.845 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.804958+0000", 2024-04-24T15:32:32.845 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2607000000000001e-05 2024-04-24T15:32:32.846 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.846 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.846 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:32.846 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.804975+0000", 2024-04-24T15:32:32.846 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6433999999999999e-05 2024-04-24T15:32:32.846 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.846 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.846 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:32.846 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.809868+0000", 2024-04-24T15:32:32.846 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0048933839999999998 2024-04-24T15:32:32.846 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.846 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.846 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:32.846 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.809901+0000", 2024-04-24T15:32:32.846 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3158999999999999e-05 2024-04-24T15:32:32.847 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:32.847 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:32.847 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:32.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.847 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.847 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4426.0:3934 4.6 e34/33 4:6297db09:::rbd_data.114a1f60cb66.00000000000000e1:head v 34'40, mlcod=34'37)", 2024-04-24T15:32:32.847 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.812857+0000", 2024-04-24T15:32:32.847 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 7.9664614409999999, 2024-04-24T15:32:32.847 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0052230590000000004, 2024-04-24T15:32:32.847 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:32.847 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:32.847 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:32.847 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.847 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:32.847 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.812857+0000", 2024-04-24T15:32:32.848 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:32.848 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.848 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.848 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:32.848 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.812857+0000", 2024-04-24T15:32:32.848 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:32.848 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.848 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.848 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:32.848 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.812856+0000", 2024-04-24T15:32:32.848 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:32.848 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.848 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.848 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:32.849 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.812861+0000", 2024-04-24T15:32:32.849 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4e-06 2024-04-24T15:32:32.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.849 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.849 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:32.849 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.812871+0000", 2024-04-24T15:32:32.849 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8679999999999994e-06 2024-04-24T15:32:32.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.849 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.849 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:32.849 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.812877+0000", 2024-04-24T15:32:32.849 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5310000000000001e-06 2024-04-24T15:32:32.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.849 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.849 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:32.850 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.812913+0000", 2024-04-24T15:32:32.850 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6548999999999999e-05 2024-04-24T15:32:32.850 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.850 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.850 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:32.850 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.812934+0000", 2024-04-24T15:32:32.850 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0590000000000001e-05 2024-04-24T15:32:32.850 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.850 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.850 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:32.850 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.818047+0000", 2024-04-24T15:32:32.850 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0051134090000000002 2024-04-24T15:32:32.850 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.850 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.851 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:32.851 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.818080+0000", 2024-04-24T15:32:32.851 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2292000000000002e-05 2024-04-24T15:32:32.851 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:32.851 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:32.851 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:32.851 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.851 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.851 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4426.0:3943 4.6 e34/33 4:639bc13c:::rbd_data.114a1f60cb66.00000000000000ea:head v 34'41, mlcod=34'38)", 2024-04-24T15:32:32.851 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.815615+0000", 2024-04-24T15:32:32.851 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 7.9637031489999996, 2024-04-24T15:32:32.851 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0050425490000000003, 2024-04-24T15:32:32.852 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:32.852 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:32.852 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:32.852 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.852 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:32.852 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.815615+0000", 2024-04-24T15:32:32.852 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:32.852 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.852 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.852 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:32.852 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.815615+0000", 2024-04-24T15:32:32.852 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:32.853 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.853 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.853 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:32.853 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.815614+0000", 2024-04-24T15:32:32.853 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:32.853 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.853 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.853 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:32.853 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.815620+0000", 2024-04-24T15:32:32.853 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1460000000000001e-06 2024-04-24T15:32:32.853 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.853 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.854 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:32.854 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.815632+0000", 2024-04-24T15:32:32.854 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2026e-05 2024-04-24T15:32:32.854 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.854 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.854 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:32.854 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.815638+0000", 2024-04-24T15:32:32.854 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8479999999999997e-06 2024-04-24T15:32:32.854 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.854 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.854 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:32.854 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.815678+0000", 2024-04-24T15:32:32.855 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9981000000000002e-05 2024-04-24T15:32:32.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.855 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.855 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:32.855 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.815702+0000", 2024-04-24T15:32:32.855 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3917000000000002e-05 2024-04-24T15:32:32.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.855 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.855 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:32.855 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.820633+0000", 2024-04-24T15:32:32.855 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0049306380000000002 2024-04-24T15:32:32.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.855 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.856 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:32.856 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.820657+0000", 2024-04-24T15:32:32.856 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.476e-05 2024-04-24T15:32:32.856 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:32.856 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:32.856 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:32.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.856 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.856 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4426.0:4005 4.6 e34/33 4:64968d7a:::rbd_data.114a1f60cb66.00000000000001bc:head v 34'45, mlcod=34'43)", 2024-04-24T15:32:32.856 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.842218+0000", 2024-04-24T15:32:32.856 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 7.9370998940000002, 2024-04-24T15:32:32.856 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0052518060000000004, 2024-04-24T15:32:32.856 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:32.856 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:32.857 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:32.857 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.857 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:32.857 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.842218+0000", 2024-04-24T15:32:32.857 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:32.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.857 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.857 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:32.857 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.842218+0000", 2024-04-24T15:32:32.857 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:32.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.857 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.857 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:32.857 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.842218+0000", 2024-04-24T15:32:32.858 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:32.858 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.858 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.858 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:32.858 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.842223+0000", 2024-04-24T15:32:32.858 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0579999999999997e-06 2024-04-24T15:32:32.858 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.858 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.858 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:32.858 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.842231+0000", 2024-04-24T15:32:32.858 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5259999999999993e-06 2024-04-24T15:32:32.858 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.858 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.859 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:32.859 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.842235+0000", 2024-04-24T15:32:32.859 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0990000000000001e-06 2024-04-24T15:32:32.859 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.859 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.859 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:32.859 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.842271+0000", 2024-04-24T15:32:32.859 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5716000000000002e-05 2024-04-24T15:32:32.859 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.859 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.859 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:32.859 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.842291+0000", 2024-04-24T15:32:32.859 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9581000000000001e-05 2024-04-24T15:32:32.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.860 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.860 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:32.860 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.847434+0000", 2024-04-24T15:32:32.860 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0051435480000000004 2024-04-24T15:32:32.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.860 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.860 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:32.860 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.847470+0000", 2024-04-24T15:32:32.860 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5929000000000001e-05 2024-04-24T15:32:32.860 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:32.860 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:32.860 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:32.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.860 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.861 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4426.0:4119 4.6 e34/33 4:6588fd86:::rbd_data.114a1f60cb66.0000000000000179:head v 34'48, mlcod=34'45)", 2024-04-24T15:32:32.861 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.936723+0000", 2024-04-24T15:32:32.861 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 7.8425950970000002, 2024-04-24T15:32:32.861 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0058208640000000002, 2024-04-24T15:32:32.861 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:32.861 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:32.861 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:32.861 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.861 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:32.861 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.936723+0000", 2024-04-24T15:32:32.861 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:32.861 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.861 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.861 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:32.861 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.936723+0000", 2024-04-24T15:32:32.861 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:32.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.862 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.862 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:32.862 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.936722+0000", 2024-04-24T15:32:32.862 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:32.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.862 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.862 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:32.862 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.936735+0000", 2024-04-24T15:32:32.862 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2153999999999999e-05 2024-04-24T15:32:32.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.862 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.862 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:32.862 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.936743+0000", 2024-04-24T15:32:32.863 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9360000000000005e-06 2024-04-24T15:32:32.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.863 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.863 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:32.863 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.936749+0000", 2024-04-24T15:32:32.863 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5040000000000002e-06 2024-04-24T15:32:32.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.863 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.863 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:32.863 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.936783+0000", 2024-04-24T15:32:32.863 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3689999999999998e-05 2024-04-24T15:32:32.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.863 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.863 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:32.863 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.936803+0000", 2024-04-24T15:32:32.863 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0047999999999998e-05 2024-04-24T15:32:32.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.864 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.864 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:32.864 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.942517+0000", 2024-04-24T15:32:32.864 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057144220000000003 2024-04-24T15:32:32.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.864 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.864 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:32.864 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.942544+0000", 2024-04-24T15:32:32.864 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6735999999999999e-05 2024-04-24T15:32:32.864 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:32.864 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:32.864 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:32.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.864 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.865 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4426.0:4353 4.3 e34/33 4:c38b6006:::rbd_data.114a1f60cb66.0000000000000175:head v 34'35, mlcod=34'33)", 2024-04-24T15:32:32.865 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:25.154297+0000", 2024-04-24T15:32:32.865 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 7.6250210249999997, 2024-04-24T15:32:32.865 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0051580410000000004, 2024-04-24T15:32:32.865 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:32.865 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:32.865 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:32.865 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.865 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:32.865 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:25.154297+0000", 2024-04-24T15:32:32.865 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:32.865 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.865 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.865 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:32.865 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:25.154297+0000", 2024-04-24T15:32:32.866 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:32.866 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.866 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.866 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:32.866 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:25.154296+0000", 2024-04-24T15:32:32.866 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:32.866 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.866 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.866 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:32.866 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:25.154312+0000", 2024-04-24T15:32:32.866 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6271000000000001e-05 2024-04-24T15:32:32.866 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.866 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.866 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:32.866 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:25.154330+0000", 2024-04-24T15:32:32.867 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8076000000000001e-05 2024-04-24T15:32:32.867 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.867 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.867 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:32.867 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:25.154340+0000", 2024-04-24T15:32:32.867 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7210000000000006e-06 2024-04-24T15:32:32.867 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.867 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.867 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:32.867 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:25.154399+0000", 2024-04-24T15:32:32.867 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9082000000000001e-05 2024-04-24T15:32:32.867 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.867 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.867 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:32.867 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:25.154422+0000", 2024-04-24T15:32:32.867 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2569e-05 2024-04-24T15:32:32.868 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.868 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.868 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:32.868 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:25.159411+0000", 2024-04-24T15:32:32.868 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0049891609999999998 2024-04-24T15:32:32.868 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:32.868 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:32.868 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:32.868 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:25.159455+0000", 2024-04-24T15:32:32.868 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4288e-05 2024-04-24T15:32:32.868 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:32.868 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:32.868 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:32.868 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:32.868 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:32.868 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:32:32.870 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-24T15:32:32.871 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:32:32.871+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:32:32.971 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:32:32.972+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:32:32.994 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:32:32.994 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:32:32.994 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-04-24T15:32:32.994 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:32:33.004 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-24T15:32:33.088 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:32:33.089+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:32:33.172 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:32:33.173+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:32:33.173 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:32:33.173 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:32:33.173 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-04-24T15:32:33.173 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-04-24T15:32:33.173 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:32:33.184 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-24T15:32:33.202 INFO:teuthology.orchestra.run.smithi096.stdout:truncating image image_client.0-clone3 from 0xd67e7f4 (overlap 0x269876) to 0x0 2024-04-24T15:32:33.246 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #2, image image_client.0-clone2 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent3 2024-04-24T15:32:33.273 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:32:33.274+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:32:33.345 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:32:33.345 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-04-24T15:32:33.345 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-04-24T15:32:33.345 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [] 2024-04-24T15:32:33.345 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:32:33.360 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-24T15:32:33.401 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:32:33.402+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:32:33.475 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:32:33.476+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:32:33.526 INFO:teuthology.orchestra.run.smithi096.stdout:1165 read 0x53444d1 thru 0x534ac19 (0x6749 bytes) 2024-04-24T15:32:33.527 INFO:teuthology.orchestra.run.smithi096.stdout:1167 read 0x5196048 thru 0x519aa2b (0x49e4 bytes) 2024-04-24T15:32:33.528 INFO:teuthology.orchestra.run.smithi096.stdout:1168 write 0x573f316 thru 0x5745b1f (0x680a bytes) 2024-04-24T15:32:33.530 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:32:33.530 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:32:33.530 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-04-24T15:32:33.531 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:32:33.545 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-24T15:32:33.586 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:32:33.587+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:32:33.615 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:33.615 INFO:teuthology.orchestra.run.smithi096.stdout:1172 trunc from 0xd67e7f4 to 0xd72793b 2024-04-24T15:32:33.620 INFO:teuthology.orchestra.run.smithi096.stdout:1173 punch from 0x391aacf to 0x391aee4, (0x415 bytes) 2024-04-24T15:32:33.620 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:33.626 INFO:teuthology.orchestra.run.smithi096.stdout:1175 trunc from 0xd72793b to 0x8c95d60 2024-04-24T15:32:33.639 INFO:teuthology.orchestra.run.smithi096.stdout:1176 read 0x47dbacc thru 0x47eb958 (0xfe8d bytes) 2024-04-24T15:32:33.640 INFO:teuthology.orchestra.run.smithi096.stdout:1177 punch from 0x4a415e9 to 0x4a4a81f, (0x9236 bytes) 2024-04-24T15:32:33.640 INFO:teuthology.orchestra.run.smithi096.stdout:1179 read 0x4cfd437 thru 0x4d0ab0a (0xd6d4 bytes) 2024-04-24T15:32:33.653 INFO:teuthology.orchestra.run.smithi096.stdout:1180 writesame 0xe99f4e0 thru 0xe9a263d data_size 0x2(0x315e bytes) 2024-04-24T15:32:33.653 INFO:teuthology.orchestra.run.smithi096.stdout:1181 writesame 0xcc66771 thru 0xcc7595e data_size 0xad(0xf1ee bytes) 2024-04-24T15:32:33.661 INFO:teuthology.orchestra.run.smithi096.stdout:1182 read 0x3125674 thru 0x3129d32 (0x46bf bytes) 2024-04-24T15:32:33.661 INFO:teuthology.orchestra.run.smithi096.stdout:1183 write 0x499c0f3 thru 0x49ac080 (0xff8e bytes) 2024-04-24T15:32:33.676 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:32:33.678+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:32:33.729 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:32:33.729 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:32:33.729 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-04-24T15:32:33.729 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-04-24T15:32:33.730 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:32:33.740 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-24T15:32:33.765 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:33.765 INFO:teuthology.orchestra.run.smithi096.stdout:1188 punch from 0xdac24f2 to 0xdac6d4d, (0x485b bytes) 2024-04-24T15:32:33.765 INFO:teuthology.orchestra.run.smithi096.stdout:1189 writesame 0x103946a thru 0x104650f data_size 0x2(0xd0a6 bytes) 2024-04-24T15:32:33.777 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:32:33.779+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:32:33.878 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:32:33.879+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:32:33.885 INFO:teuthology.orchestra.run.smithi096.stdout:1190 write 0x41ccd88 thru 0x41d013a (0x33b3 bytes) 2024-04-24T15:32:33.908 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:32:33.908 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-04-24T15:32:33.909 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-04-24T15:32:33.909 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-04-24T15:32:33.909 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.909 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4426.0:2434 4.d e32/26 4:b7840674:::rbd_data.114a1f60cb66.00000000000001a7:head v 32'12, mlcod=32'10)", 2024-04-24T15:32:33.909 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:23.024172+0000", 2024-04-24T15:32:33.909 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 10.883542641, 2024-04-24T15:32:33.909 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057919520000000004, 2024-04-24T15:32:33.909 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:33.909 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:33.909 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:33.909 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.909 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:33.909 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.024172+0000", 2024-04-24T15:32:33.909 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:33.910 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.910 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.910 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:33.910 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.024172+0000", 2024-04-24T15:32:33.910 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:33.910 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.910 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.910 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:33.910 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.024171+0000", 2024-04-24T15:32:33.910 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:33.910 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.910 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.910 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:33.910 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.024178+0000", 2024-04-24T15:32:33.910 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2420000000000003e-06 2024-04-24T15:32:33.910 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.911 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.911 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:33.911 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.024191+0000", 2024-04-24T15:32:33.911 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2444e-05 2024-04-24T15:32:33.911 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.911 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.911 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:33.911 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.024197+0000", 2024-04-24T15:32:33.911 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2249999999999997e-06 2024-04-24T15:32:33.911 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.911 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.911 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:33.911 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.024251+0000", 2024-04-24T15:32:33.911 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4221000000000001e-05 2024-04-24T15:32:33.911 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.911 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.912 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:33.912 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.024270+0000", 2024-04-24T15:32:33.912 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9151999999999999e-05 2024-04-24T15:32:33.912 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.912 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.912 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:33.912 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.029925+0000", 2024-04-24T15:32:33.912 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0056546160000000003 2024-04-24T15:32:33.912 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.912 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.912 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:33.912 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.029964+0000", 2024-04-24T15:32:33.912 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8855e-05 2024-04-24T15:32:33.913 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:33.913 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:33.913 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:33.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.913 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.913 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:2450 4.1c 4:3f9dbb33:::rbd_data.114a1f60cb66.00000000000001f3:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-24T15:32:33.913 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:23.027406+0000", 2024-04-24T15:32:33.913 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 10.880308268, 2024-04-24T15:32:33.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.005672379, 2024-04-24T15:32:33.913 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:33.913 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:33.913 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:33.913 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:33.913 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:33.913 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2450 2024-04-24T15:32:33.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.914 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:33.914 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.914 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:33.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027406+0000", 2024-04-24T15:32:33.914 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:33.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.914 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.914 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:33.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027406+0000", 2024-04-24T15:32:33.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:33.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.915 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:33.915 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027405+0000", 2024-04-24T15:32:33.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:33.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.915 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:33.915 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027411+0000", 2024-04-24T15:32:33.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7610000000000004e-06 2024-04-24T15:32:33.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.916 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:33.916 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027422+0000", 2024-04-24T15:32:33.916 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1462e-05 2024-04-24T15:32:33.916 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.916 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.916 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:33.916 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027429+0000", 2024-04-24T15:32:33.916 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3339999999999998e-06 2024-04-24T15:32:33.916 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.916 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.916 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:33.916 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027478+0000", 2024-04-24T15:32:33.916 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9804e-05 2024-04-24T15:32:33.916 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.916 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.916 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:33.917 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027559+0000", 2024-04-24T15:32:33.917 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1068000000000004e-05 2024-04-24T15:32:33.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.917 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.917 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:33.917 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027672+0000", 2024-04-24T15:32:33.917 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000112632 2024-04-24T15:32:33.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.917 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.917 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:33.917 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.032737+0000", 2024-04-24T15:32:33.917 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0050646429999999997 2024-04-24T15:32:33.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.917 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.917 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:33.918 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.033029+0000", 2024-04-24T15:32:33.918 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00029226300000000002 2024-04-24T15:32:33.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.918 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.918 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:33.918 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.033057+0000", 2024-04-24T15:32:33.918 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7872e-05 2024-04-24T15:32:33.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.918 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.918 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:33.918 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.033078+0000", 2024-04-24T15:32:33.918 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.158e-05 2024-04-24T15:32:33.918 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:33.918 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:33.918 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:33.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.919 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4426.0:2447 4.6 e32/27 4:668081fa:::rbd_data.114a1f60cb66.00000000000001f0:head v 32'16, mlcod=32'13)", 2024-04-24T15:32:33.919 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:23.027434+0000", 2024-04-24T15:32:33.919 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 10.880280386000001, 2024-04-24T15:32:33.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0052559449999999997, 2024-04-24T15:32:33.919 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:33.919 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:33.919 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:33.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.919 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:33.919 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027434+0000", 2024-04-24T15:32:33.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:33.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.919 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:33.919 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027434+0000", 2024-04-24T15:32:33.920 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:33.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.920 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.920 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:33.920 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027433+0000", 2024-04-24T15:32:33.920 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:33.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.920 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.920 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:33.920 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027440+0000", 2024-04-24T15:32:33.920 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2099999999999998e-06 2024-04-24T15:32:33.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.920 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.920 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:33.920 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027450+0000", 2024-04-24T15:32:33.921 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0278000000000001e-05 2024-04-24T15:32:33.921 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.921 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.921 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:33.921 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027456+0000", 2024-04-24T15:32:33.921 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3720000000000001e-06 2024-04-24T15:32:33.921 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.921 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.921 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:33.921 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027488+0000", 2024-04-24T15:32:33.921 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1427000000000003e-05 2024-04-24T15:32:33.921 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.921 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.921 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:33.921 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027510+0000", 2024-04-24T15:32:33.921 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.243e-05 2024-04-24T15:32:33.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.922 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.922 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:33.922 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.032654+0000", 2024-04-24T15:32:33.922 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0051436210000000001 2024-04-24T15:32:33.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.922 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.922 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:33.922 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.032690+0000", 2024-04-24T15:32:33.922 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6188e-05 2024-04-24T15:32:33.922 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:33.922 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:33.922 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:33.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.922 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.923 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:2454 4.4 4:21c2614c:::rbd_data.114a1f60cb66.00000000000001f7:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-24T15:32:33.923 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:23.030259+0000", 2024-04-24T15:32:33.923 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 10.877455700000001, 2024-04-24T15:32:33.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0056217740000000004, 2024-04-24T15:32:33.923 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:33.923 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:33.923 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:33.923 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:33.923 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:33.923 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2454 2024-04-24T15:32:33.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.923 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:33.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:33.923 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.030259+0000", 2024-04-24T15:32:33.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:33.924 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.924 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.924 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:33.924 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.030259+0000", 2024-04-24T15:32:33.924 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:33.924 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.924 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.924 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:33.924 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.030258+0000", 2024-04-24T15:32:33.924 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:33.924 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.924 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.924 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:33.924 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.030263+0000", 2024-04-24T15:32:33.924 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1869999999999996e-06 2024-04-24T15:32:33.924 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.925 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.925 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:33.925 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.030275+0000", 2024-04-24T15:32:33.925 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.199e-05 2024-04-24T15:32:33.925 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.925 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.925 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:33.925 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.030281+0000", 2024-04-24T15:32:33.925 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1680000000000001e-06 2024-04-24T15:32:33.925 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.925 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.925 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:33.925 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.030311+0000", 2024-04-24T15:32:33.925 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0008000000000001e-05 2024-04-24T15:32:33.925 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.926 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:33.926 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.030355+0000", 2024-04-24T15:32:33.926 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3489999999999999e-05 2024-04-24T15:32:33.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.926 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:33.926 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.030408+0000", 2024-04-24T15:32:33.926 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3737999999999999e-05 2024-04-24T15:32:33.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.926 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:33.926 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.035580+0000", 2024-04-24T15:32:33.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0051716560000000002 2024-04-24T15:32:33.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.927 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:33.927 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.035820+0000", 2024-04-24T15:32:33.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00023958900000000001 2024-04-24T15:32:33.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.927 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:33.927 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.035847+0000", 2024-04-24T15:32:33.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7495999999999999e-05 2024-04-24T15:32:33.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.927 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:33.928 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.035880+0000", 2024-04-24T15:32:33.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3367000000000001e-05 2024-04-24T15:32:33.928 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:33.928 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:33.928 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:33.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.928 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4426.0:2510 4.12 e33/27 4:4fee0517:::rbd_data.114a1f60cb66.000000000000029d:head v 33'19, mlcod=32'17)", 2024-04-24T15:32:33.928 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:23.078201+0000", 2024-04-24T15:32:33.928 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 10.829513142, 2024-04-24T15:32:33.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0060097689999999999, 2024-04-24T15:32:33.928 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:33.928 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:33.928 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:33.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.929 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:33.929 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.078201+0000", 2024-04-24T15:32:33.929 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:33.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.929 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.929 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:33.929 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.078201+0000", 2024-04-24T15:32:33.929 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:33.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.929 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.929 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:33.929 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.078201+0000", 2024-04-24T15:32:33.929 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:33.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.930 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.930 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:33.930 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.078213+0000", 2024-04-24T15:32:33.930 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2261e-05 2024-04-24T15:32:33.930 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.930 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.930 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:33.930 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.078223+0000", 2024-04-24T15:32:33.930 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4709999999999996e-06 2024-04-24T15:32:33.930 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.930 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.930 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:33.930 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.078230+0000", 2024-04-24T15:32:33.930 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8979999999999999e-06 2024-04-24T15:32:33.930 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.931 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:33.931 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.079842+0000", 2024-04-24T15:32:33.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001612048 2024-04-24T15:32:33.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.931 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:33.931 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.079870+0000", 2024-04-24T15:32:33.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8487e-05 2024-04-24T15:32:33.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.931 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:33.931 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.084169+0000", 2024-04-24T15:32:33.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0042991490000000004 2024-04-24T15:32:33.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.932 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.932 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:33.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.084211+0000", 2024-04-24T15:32:33.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1792000000000002e-05 2024-04-24T15:32:33.932 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:33.932 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:33.932 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:33.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.932 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.932 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:2522 4.6 4:60ceedf0:::rbd_data.114a1f60cb66.000000000000011d:head [set-alloc-hint object_size 262144 write_size 262144,write 110592~2048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-24T15:32:33.932 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:23.088240+0000", 2024-04-24T15:32:33.932 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 10.819474228000001, 2024-04-24T15:32:33.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.988232797, 2024-04-24T15:32:33.932 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:33.932 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:33.932 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:33.933 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:33.933 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:33.933 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2522 2024-04-24T15:32:33.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.933 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:33.933 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.933 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:33.933 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088240+0000", 2024-04-24T15:32:33.933 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:33.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.933 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.933 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:33.933 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088240+0000", 2024-04-24T15:32:33.933 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:33.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.933 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.933 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:33.934 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088238+0000", 2024-04-24T15:32:33.934 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999981 2024-04-24T15:32:33.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.934 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.934 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:33.934 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088254+0000", 2024-04-24T15:32:33.934 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5562000000000001e-05 2024-04-24T15:32:33.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.934 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.934 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:33.934 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088275+0000", 2024-04-24T15:32:33.934 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1256000000000001e-05 2024-04-24T15:32:33.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.934 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.935 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:33.935 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088283+0000", 2024-04-24T15:32:33.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3219999999999993e-06 2024-04-24T15:32:33.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.935 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:33.935 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088351+0000", 2024-04-24T15:32:33.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7595000000000006e-05 2024-04-24T15:32:33.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.935 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-04-24T15:32:33.935 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088362+0000", 2024-04-24T15:32:33.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1764e-05 2024-04-24T15:32:33.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.936 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:33.936 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.073748+0000", 2024-04-24T15:32:33.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98538541899999998 2024-04-24T15:32:33.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.936 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:33.936 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.073925+0000", 2024-04-24T15:32:33.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017747600000000001 2024-04-24T15:32:33.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.936 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:33.936 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.074092+0000", 2024-04-24T15:32:33.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000166622 2024-04-24T15:32:33.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:33.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.074774+0000", 2024-04-24T15:32:33.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00068225000000000002 2024-04-24T15:32:33.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:33.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.076412+0000", 2024-04-24T15:32:33.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001638189 2024-04-24T15:32:33.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:33.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.076446+0000", 2024-04-24T15:32:33.938 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3587000000000002e-05 2024-04-24T15:32:33.938 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.938 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.938 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:33.938 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.076473+0000", 2024-04-24T15:32:33.938 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6886999999999999e-05 2024-04-24T15:32:33.938 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:33.938 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:33.938 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:33.938 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.938 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.938 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:2755 4.1c 4:3c8baac7:::rbd_data.114a1f60cb66.00000000000000a8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:33.938 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.252831+0000", 2024-04-24T15:32:33.939 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.6548834140000004, 2024-04-24T15:32:33.939 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057169500000000002, 2024-04-24T15:32:33.939 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:33.939 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:33.939 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:33.939 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:33.939 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:33.939 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2755 2024-04-24T15:32:33.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.939 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:33.939 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.939 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:33.939 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.252831+0000", 2024-04-24T15:32:33.940 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:33.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.940 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:33.940 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.252831+0000", 2024-04-24T15:32:33.940 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:33.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.940 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:33.940 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.252830+0000", 2024-04-24T15:32:33.940 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:33.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.940 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:33.940 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.252835+0000", 2024-04-24T15:32:33.941 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3739999999999997e-06 2024-04-24T15:32:33.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.941 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:33.941 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.252846+0000", 2024-04-24T15:32:33.941 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.039e-05 2024-04-24T15:32:33.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.941 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:33.941 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.252850+0000", 2024-04-24T15:32:33.941 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4100000000000001e-06 2024-04-24T15:32:33.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.941 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:33.941 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.252885+0000", 2024-04-24T15:32:33.941 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5213999999999999e-05 2024-04-24T15:32:33.942 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.942 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.942 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:33.942 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.252957+0000", 2024-04-24T15:32:33.942 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2285000000000003e-05 2024-04-24T15:32:33.942 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.942 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.942 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:33.942 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.253053+0000", 2024-04-24T15:32:33.942 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5841999999999995e-05 2024-04-24T15:32:33.942 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.942 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.942 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:33.942 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.255905+0000", 2024-04-24T15:32:33.943 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028514539999999998 2024-04-24T15:32:33.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.943 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.943 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:33.943 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.258474+0000", 2024-04-24T15:32:33.943 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0025695679999999999 2024-04-24T15:32:33.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.943 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.943 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:33.943 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.258518+0000", 2024-04-24T15:32:33.943 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3242999999999999e-05 2024-04-24T15:32:33.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.943 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.943 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:33.943 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.258548+0000", 2024-04-24T15:32:33.943 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0318999999999999e-05 2024-04-24T15:32:33.943 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:33.944 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:33.944 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:33.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.944 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4426.0:2761 4.19 e34/26 4:9e865ec5:::rbd_data.114a1f60cb66.00000000000000c0:head v 34'14, mlcod=34'12)", 2024-04-24T15:32:33.944 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.255783+0000", 2024-04-24T15:32:33.944 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.6519318169999995, 2024-04-24T15:32:33.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057401149999999996, 2024-04-24T15:32:33.944 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:33.944 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:33.944 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:33.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.944 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:33.944 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.255783+0000", 2024-04-24T15:32:33.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:33.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.945 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.945 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:33.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.255783+0000", 2024-04-24T15:32:33.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:33.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.945 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.945 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:33.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.255782+0000", 2024-04-24T15:32:33.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:33.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.945 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.945 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:33.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.255794+0000", 2024-04-24T15:32:33.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2429000000000001e-05 2024-04-24T15:32:33.946 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.946 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.946 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:33.946 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.255802+0000", 2024-04-24T15:32:33.946 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2139999999999999e-06 2024-04-24T15:32:33.946 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.946 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.946 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:33.946 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.255807+0000", 2024-04-24T15:32:33.946 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6500000000000001e-06 2024-04-24T15:32:33.946 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.946 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.946 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:33.946 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.255846+0000", 2024-04-24T15:32:33.947 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8497000000000001e-05 2024-04-24T15:32:33.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.947 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.947 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:33.947 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.255867+0000", 2024-04-24T15:32:33.947 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0733999999999998e-05 2024-04-24T15:32:33.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.947 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.947 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:33.947 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.261440+0000", 2024-04-24T15:32:33.947 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0055733099999999997 2024-04-24T15:32:33.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.947 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.947 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:33.947 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.261523+0000", 2024-04-24T15:32:33.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2764999999999998e-05 2024-04-24T15:32:33.948 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:33.948 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:33.948 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:33.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.948 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:2767 4.6 4:6297db09:::rbd_data.114a1f60cb66.00000000000000e1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:33.948 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.256201+0000", 2024-04-24T15:32:33.948 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.6515132779999995, 2024-04-24T15:32:33.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0056859040000000003, 2024-04-24T15:32:33.948 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:33.948 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:33.948 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:33.948 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:33.948 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:33.949 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2767 2024-04-24T15:32:33.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.949 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:33.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:33.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.256201+0000", 2024-04-24T15:32:33.949 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:33.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:33.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.256201+0000", 2024-04-24T15:32:33.949 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:33.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.950 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.950 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:33.950 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.256200+0000", 2024-04-24T15:32:33.950 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:33.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.950 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.950 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:33.950 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.256205+0000", 2024-04-24T15:32:33.950 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.587e-06 2024-04-24T15:32:33.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.950 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.950 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:33.950 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.256218+0000", 2024-04-24T15:32:33.950 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2362e-05 2024-04-24T15:32:33.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.951 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.951 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:33.951 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.256224+0000", 2024-04-24T15:32:33.951 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6120000000000004e-06 2024-04-24T15:32:33.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.951 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.951 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:33.951 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.256274+0000", 2024-04-24T15:32:33.951 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9084e-05 2024-04-24T15:32:33.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.951 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.951 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:33.951 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.256392+0000", 2024-04-24T15:32:33.952 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000118127 2024-04-24T15:32:33.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.952 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:33.952 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.256508+0000", 2024-04-24T15:32:33.952 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000116777 2024-04-24T15:32:33.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.952 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:33.952 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.261537+0000", 2024-04-24T15:32:33.952 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0050290380000000004 2024-04-24T15:32:33.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.952 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:33.952 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.261801+0000", 2024-04-24T15:32:33.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00026362400000000001 2024-04-24T15:32:33.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.953 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:33.953 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.261848+0000", 2024-04-24T15:32:33.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6700999999999999e-05 2024-04-24T15:32:33.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.953 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:33.953 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.261887+0000", 2024-04-24T15:32:33.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9159999999999998e-05 2024-04-24T15:32:33.953 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:33.953 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:33.953 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:33.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.954 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.954 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:3772 4.4 4:24a7cfbc:::rbd_data.114a1f60cb66.0000000000000020:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:33.954 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.699611+0000", 2024-04-24T15:32:33.954 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.2081033980000004, 2024-04-24T15:32:33.954 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0060871340000000001, 2024-04-24T15:32:33.954 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:33.954 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:33.954 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:33.954 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:33.954 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:33.954 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3772 2024-04-24T15:32:33.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.954 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:33.954 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.955 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:33.955 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.699611+0000", 2024-04-24T15:32:33.955 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:33.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.955 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.955 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:33.955 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.699611+0000", 2024-04-24T15:32:33.955 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:33.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.955 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.955 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:33.955 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.699609+0000", 2024-04-24T15:32:33.955 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999981 2024-04-24T15:32:33.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.955 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.956 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:33.956 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.699618+0000", 2024-04-24T15:32:33.956 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3859999999999996e-06 2024-04-24T15:32:33.956 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.956 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.956 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:33.956 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.699637+0000", 2024-04-24T15:32:33.956 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8688000000000001e-05 2024-04-24T15:32:33.956 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.956 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.956 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:33.956 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.699647+0000", 2024-04-24T15:32:33.956 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.029e-05 2024-04-24T15:32:33.956 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.956 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.957 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:33.957 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.699692+0000", 2024-04-24T15:32:33.957 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4400000000000002e-05 2024-04-24T15:32:33.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.957 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.957 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:33.957 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.699751+0000", 2024-04-24T15:32:33.957 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8953999999999998e-05 2024-04-24T15:32:33.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.957 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.957 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:33.957 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.700249+0000", 2024-04-24T15:32:33.957 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00049869900000000002 2024-04-24T15:32:33.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.958 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.958 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:33.958 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.705560+0000", 2024-04-24T15:32:33.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0053110550000000003 2024-04-24T15:32:33.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.958 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.958 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:33.958 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.705602+0000", 2024-04-24T15:32:33.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1689e-05 2024-04-24T15:32:33.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.958 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.958 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:33.958 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.705652+0000", 2024-04-24T15:32:33.959 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9740000000000001e-05 2024-04-24T15:32:33.959 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.959 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.959 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:33.959 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.705698+0000", 2024-04-24T15:32:33.959 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6329000000000003e-05 2024-04-24T15:32:33.959 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:33.959 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:33.959 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:33.959 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.959 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.959 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:3774 4.4 4:25b094fe:::rbd_data.114a1f60cb66.0000000000000022:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:33.959 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.699669+0000", 2024-04-24T15:32:33.959 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.2080452669999993, 2024-04-24T15:32:33.960 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.006296678, 2024-04-24T15:32:33.960 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:33.960 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:33.960 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:33.960 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:33.960 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:33.960 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3774 2024-04-24T15:32:33.960 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.960 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:33.960 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.960 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:33.960 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.699669+0000", 2024-04-24T15:32:33.960 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:33.960 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.960 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.961 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:33.961 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.699669+0000", 2024-04-24T15:32:33.961 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:33.961 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.961 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.961 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:33.961 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.699669+0000", 2024-04-24T15:32:33.961 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:33.961 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.961 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.961 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:33.961 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.699672+0000", 2024-04-24T15:32:33.961 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2830000000000002e-06 2024-04-24T15:32:33.961 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.962 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.962 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:33.962 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.699677+0000", 2024-04-24T15:32:33.962 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7530000000000001e-06 2024-04-24T15:32:33.962 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.962 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.962 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:33.962 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.699680+0000", 2024-04-24T15:32:33.962 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0130000000000002e-06 2024-04-24T15:32:33.962 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.962 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.962 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:33.962 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.702736+0000", 2024-04-24T15:32:33.963 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0030567379999999998 2024-04-24T15:32:33.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.963 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.963 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:33.963 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.702797+0000", 2024-04-24T15:32:33.963 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0563000000000003e-05 2024-04-24T15:32:33.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.963 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.963 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:33.963 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.703297+0000", 2024-04-24T15:32:33.963 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00049955300000000002 2024-04-24T15:32:33.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.963 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.963 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:33.964 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.705595+0000", 2024-04-24T15:32:33.964 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002298076 2024-04-24T15:32:33.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.964 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.964 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:33.964 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.705870+0000", 2024-04-24T15:32:33.964 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000275285 2024-04-24T15:32:33.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.964 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.964 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:33.964 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.705923+0000", 2024-04-24T15:32:33.964 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3229999999999997e-05 2024-04-24T15:32:33.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.964 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.965 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:33.965 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.705966+0000", 2024-04-24T15:32:33.965 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2636999999999999e-05 2024-04-24T15:32:33.965 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:33.965 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:33.965 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:33.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.965 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.965 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:3795 4.1c 4:3a2027c5:::rbd_data.114a1f60cb66.0000000000000025:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:33.965 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.723461+0000", 2024-04-24T15:32:33.965 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.1842534100000002, 2024-04-24T15:32:33.965 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0066452719999999998, 2024-04-24T15:32:33.965 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:33.965 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:33.966 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:33.966 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:33.966 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:33.966 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3795 2024-04-24T15:32:33.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.966 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:33.966 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.966 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:33.966 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.723461+0000", 2024-04-24T15:32:33.966 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:33.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.966 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.966 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:33.966 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.723461+0000", 2024-04-24T15:32:33.967 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:33.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.967 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.967 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:33.967 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.723459+0000", 2024-04-24T15:32:33.967 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999986 2024-04-24T15:32:33.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.967 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.967 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:33.967 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.723470+0000", 2024-04-24T15:32:33.967 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0097e-05 2024-04-24T15:32:33.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.967 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.967 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:33.968 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.723484+0000", 2024-04-24T15:32:33.968 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4338000000000001e-05 2024-04-24T15:32:33.968 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.968 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.968 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:33.968 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.723496+0000", 2024-04-24T15:32:33.968 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2125e-05 2024-04-24T15:32:33.968 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.968 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.968 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:33.968 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.723547+0000", 2024-04-24T15:32:33.968 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0909000000000003e-05 2024-04-24T15:32:33.968 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.968 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.968 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:33.968 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.723639+0000", 2024-04-24T15:32:33.968 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1844999999999999e-05 2024-04-24T15:32:33.969 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.969 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.969 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:33.969 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.724463+0000", 2024-04-24T15:32:33.969 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00082389099999999999 2024-04-24T15:32:33.969 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.969 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.969 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:33.969 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.728863+0000", 2024-04-24T15:32:33.969 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0044007610000000004 2024-04-24T15:32:33.969 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.969 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.969 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:33.969 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.730017+0000", 2024-04-24T15:32:33.969 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0011534410000000001 2024-04-24T15:32:33.969 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.970 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.970 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:33.970 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.730067+0000", 2024-04-24T15:32:33.970 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0430000000000003e-05 2024-04-24T15:32:33.970 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.970 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.970 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:33.970 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.730106+0000", 2024-04-24T15:32:33.970 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8902000000000001e-05 2024-04-24T15:32:33.970 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:33.970 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:33.970 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:33.970 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.970 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.971 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:3929 4.6 4:640a4fea:::rbd_data.114a1f60cb66.00000000000000dc:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:33.971 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.809952+0000", 2024-04-24T15:32:33.971 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.0977622260000004, 2024-04-24T15:32:33.971 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0053076180000000001, 2024-04-24T15:32:33.971 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:33.971 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:33.971 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:33.971 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:33.971 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:33.971 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3929 2024-04-24T15:32:33.971 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.971 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:33.971 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.971 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:33.971 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.809952+0000", 2024-04-24T15:32:33.972 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:33.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.972 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:33.972 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.809952+0000", 2024-04-24T15:32:33.972 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:33.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.972 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:33.972 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.809951+0000", 2024-04-24T15:32:33.972 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999986 2024-04-24T15:32:33.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.972 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:33.973 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.809957+0000", 2024-04-24T15:32:33.973 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2879999999999998e-06 2024-04-24T15:32:33.973 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.973 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.973 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:33.973 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.809971+0000", 2024-04-24T15:32:33.973 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4258e-05 2024-04-24T15:32:33.973 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.973 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.973 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:33.973 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.809978+0000", 2024-04-24T15:32:33.973 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5400000000000001e-06 2024-04-24T15:32:33.973 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.973 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.974 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:33.974 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.810029+0000", 2024-04-24T15:32:33.974 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1289000000000003e-05 2024-04-24T15:32:33.974 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.974 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.974 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:33.974 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.810130+0000", 2024-04-24T15:32:33.974 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000100301 2024-04-24T15:32:33.974 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.974 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.974 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:33.974 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.810260+0000", 2024-04-24T15:32:33.974 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013075700000000001 2024-04-24T15:32:33.975 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.975 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.975 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:33.975 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.810856+0000", 2024-04-24T15:32:33.975 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00059576900000000001 2024-04-24T15:32:33.975 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.975 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.975 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:33.975 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.815165+0000", 2024-04-24T15:32:33.975 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0043092169999999997 2024-04-24T15:32:33.975 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.975 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.975 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:33.975 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.815217+0000", 2024-04-24T15:32:33.976 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1539999999999998e-05 2024-04-24T15:32:33.976 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.976 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.976 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:33.976 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.815260+0000", 2024-04-24T15:32:33.976 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2883999999999998e-05 2024-04-24T15:32:33.976 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:33.976 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:33.976 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:33.976 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.976 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.976 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:3934 4.6 4:6297db09:::rbd_data.114a1f60cb66.00000000000000e1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:33.976 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.812623+0000", 2024-04-24T15:32:33.976 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.0950910710000006, 2024-04-24T15:32:33.976 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.005653712, 2024-04-24T15:32:33.976 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:33.977 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:33.977 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:33.977 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:33.977 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:33.977 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3934 2024-04-24T15:32:33.977 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.977 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:33.977 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.977 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:33.977 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.812623+0000", 2024-04-24T15:32:33.977 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:33.977 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.977 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.977 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:33.977 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.812623+0000", 2024-04-24T15:32:33.977 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:33.978 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.978 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.978 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:33.978 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.812622+0000", 2024-04-24T15:32:33.978 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:33.978 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.978 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.978 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:33.978 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.812627+0000", 2024-04-24T15:32:33.978 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1150000000000002e-06 2024-04-24T15:32:33.978 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.978 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.978 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:33.978 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.812640+0000", 2024-04-24T15:32:33.978 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3095000000000001e-05 2024-04-24T15:32:33.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.979 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.979 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:33.979 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.812647+0000", 2024-04-24T15:32:33.979 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5069999999999997e-06 2024-04-24T15:32:33.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.979 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.979 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:33.979 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.812685+0000", 2024-04-24T15:32:33.979 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8359000000000003e-05 2024-04-24T15:32:33.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.979 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.979 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:33.979 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.812748+0000", 2024-04-24T15:32:33.979 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2389000000000002e-05 2024-04-24T15:32:33.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.980 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.980 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:33.980 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.812808+0000", 2024-04-24T15:32:33.980 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0817999999999999e-05 2024-04-24T15:32:33.980 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.980 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.980 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:33.980 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.813294+0000", 2024-04-24T15:32:33.980 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000485763 2024-04-24T15:32:33.980 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.980 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.980 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:33.980 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.818204+0000", 2024-04-24T15:32:33.980 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0049093119999999999 2024-04-24T15:32:33.980 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.980 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.981 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:33.981 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.818240+0000", 2024-04-24T15:32:33.981 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6550000000000001e-05 2024-04-24T15:32:33.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.981 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.981 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:33.981 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.818277+0000", 2024-04-24T15:32:33.981 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6918999999999997e-05 2024-04-24T15:32:33.981 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:33.981 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:33.981 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:33.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.981 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.981 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:3943 4.6 4:639bc13c:::rbd_data.114a1f60cb66.00000000000000ea:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:33.981 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.815303+0000", 2024-04-24T15:32:33.981 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.0924108770000007, 2024-04-24T15:32:33.982 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0055918529999999999, 2024-04-24T15:32:33.982 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:33.982 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:33.982 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:33.982 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:33.982 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:33.982 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3943 2024-04-24T15:32:33.982 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.982 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:33.982 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.982 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:33.982 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.815303+0000", 2024-04-24T15:32:33.982 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:33.982 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.982 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.983 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:33.983 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.815303+0000", 2024-04-24T15:32:33.983 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:33.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.983 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.983 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:33.983 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.815303+0000", 2024-04-24T15:32:33.983 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:33.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.983 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.983 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:33.983 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.815308+0000", 2024-04-24T15:32:33.983 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0180000000000001e-06 2024-04-24T15:32:33.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.983 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.983 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:33.984 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.815320+0000", 2024-04-24T15:32:33.984 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.272e-05 2024-04-24T15:32:33.984 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.984 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.984 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:33.984 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.815327+0000", 2024-04-24T15:32:33.984 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7889999999999997e-06 2024-04-24T15:32:33.984 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.984 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.984 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:33.984 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.815370+0000", 2024-04-24T15:32:33.984 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2608000000000003e-05 2024-04-24T15:32:33.984 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.984 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.984 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:33.984 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.815450+0000", 2024-04-24T15:32:33.984 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9944999999999994e-05 2024-04-24T15:32:33.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.985 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.985 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:33.985 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.815560+0000", 2024-04-24T15:32:33.985 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000110153 2024-04-24T15:32:33.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.985 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.985 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:33.985 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.817736+0000", 2024-04-24T15:32:33.985 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0021763889999999999 2024-04-24T15:32:33.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.985 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.985 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:33.985 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.820810+0000", 2024-04-24T15:32:33.985 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0030733449999999999 2024-04-24T15:32:33.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.986 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:33.986 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.820857+0000", 2024-04-24T15:32:33.986 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6928000000000002e-05 2024-04-24T15:32:33.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.986 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:33.986 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.820895+0000", 2024-04-24T15:32:33.986 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8705999999999998e-05 2024-04-24T15:32:33.986 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:33.986 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:33.986 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:33.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.987 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:3976 4.1c 4:396ab59e:::rbd_data.114a1f60cb66.0000000000000160:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:33.987 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.827920+0000", 2024-04-24T15:32:33.987 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.0797945290000008, 2024-04-24T15:32:33.987 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057086940000000003, 2024-04-24T15:32:33.987 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:33.987 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:33.987 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:33.987 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:33.987 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:33.987 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3976 2024-04-24T15:32:33.987 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.987 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:33.987 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.987 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:33.987 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.827920+0000", 2024-04-24T15:32:33.988 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:33.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.988 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.988 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:33.988 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.827920+0000", 2024-04-24T15:32:33.988 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:33.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.988 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.988 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:33.988 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.827919+0000", 2024-04-24T15:32:33.988 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:33.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.988 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.988 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:33.988 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.827924+0000", 2024-04-24T15:32:33.988 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4270000000000001e-06 2024-04-24T15:32:33.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.989 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:33.989 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.827934+0000", 2024-04-24T15:32:33.989 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0606e-05 2024-04-24T15:32:33.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.989 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:33.989 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.827939+0000", 2024-04-24T15:32:33.989 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.792e-06 2024-04-24T15:32:33.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.989 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:33.989 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.827967+0000", 2024-04-24T15:32:33.989 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7919000000000002e-05 2024-04-24T15:32:33.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.990 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:33.990 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.828011+0000", 2024-04-24T15:32:33.990 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3896000000000001e-05 2024-04-24T15:32:33.990 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.990 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.990 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:33.990 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.828070+0000", 2024-04-24T15:32:33.990 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9348999999999997e-05 2024-04-24T15:32:33.990 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.990 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.990 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:33.990 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.830513+0000", 2024-04-24T15:32:33.990 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024429669999999999 2024-04-24T15:32:33.990 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.991 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:33.991 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.833567+0000", 2024-04-24T15:32:33.991 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0030542590000000001 2024-04-24T15:32:33.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.991 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:33.991 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.833608+0000", 2024-04-24T15:32:33.991 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1148e-05 2024-04-24T15:32:33.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.991 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:33.991 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.833629+0000", 2024-04-24T15:32:33.991 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0066999999999999e-05 2024-04-24T15:32:33.991 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:33.991 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:33.991 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:33.992 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.992 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.992 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:3987 4.1c 4:3a30a8f2:::rbd_data.114a1f60cb66.0000000000000174:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:33.992 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.833638+0000", 2024-04-24T15:32:33.992 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.0740762490000009, 2024-04-24T15:32:33.992 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057825430000000002, 2024-04-24T15:32:33.992 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:33.992 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:33.992 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:33.992 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:33.992 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:33.992 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3987 2024-04-24T15:32:33.992 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.992 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:33.992 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.992 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:33.993 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.833638+0000", 2024-04-24T15:32:33.993 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:33.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.993 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.993 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:33.993 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.833638+0000", 2024-04-24T15:32:33.993 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:33.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.993 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.993 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:33.993 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.833637+0000", 2024-04-24T15:32:33.993 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:33.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.993 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.993 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:33.993 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.833646+0000", 2024-04-24T15:32:33.993 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0180000000000007e-06 2024-04-24T15:32:33.994 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.994 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.994 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:33.994 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.833659+0000", 2024-04-24T15:32:33.994 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.313e-05 2024-04-24T15:32:33.994 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.994 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.994 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:33.994 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.833664+0000", 2024-04-24T15:32:33.994 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.006e-06 2024-04-24T15:32:33.994 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.994 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.994 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:33.994 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.833697+0000", 2024-04-24T15:32:33.994 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2978000000000002e-05 2024-04-24T15:32:33.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.995 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.995 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:33.995 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.833756+0000", 2024-04-24T15:32:33.995 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8811000000000003e-05 2024-04-24T15:32:33.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.995 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.995 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:33.995 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.833823+0000", 2024-04-24T15:32:33.995 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7225999999999996e-05 2024-04-24T15:32:33.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.995 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.995 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:33.996 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.838767+0000", 2024-04-24T15:32:33.996 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0049435809999999998 2024-04-24T15:32:33.996 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.996 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.996 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:33.996 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.839341+0000", 2024-04-24T15:32:33.996 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00057366800000000005 2024-04-24T15:32:33.996 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.996 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.996 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:33.996 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.839385+0000", 2024-04-24T15:32:33.996 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4597e-05 2024-04-24T15:32:33.996 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.996 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.996 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:33.996 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.839421+0000", 2024-04-24T15:32:33.996 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5516999999999999e-05 2024-04-24T15:32:33.997 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:33.997 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:33.997 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:33.997 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.997 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.997 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:4005 4.6 4:64968d7a:::rbd_data.114a1f60cb66.00000000000001bc:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:33.997 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.841940+0000", 2024-04-24T15:32:33.997 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.0657739900000003, 2024-04-24T15:32:33.997 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.005717481, 2024-04-24T15:32:33.997 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:33.997 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:33.997 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:33.997 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:33.997 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:33.997 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4005 2024-04-24T15:32:33.997 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.997 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:33.998 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.998 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:33.998 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.841940+0000", 2024-04-24T15:32:33.998 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:33.998 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.998 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.998 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:33.998 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.841940+0000", 2024-04-24T15:32:33.998 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:33.998 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.998 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.998 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:33.998 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.841940+0000", 2024-04-24T15:32:33.998 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:33.998 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.999 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:33.999 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.841944+0000", 2024-04-24T15:32:33.999 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.904e-06 2024-04-24T15:32:33.999 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.999 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:33.999 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.841954+0000", 2024-04-24T15:32:33.999 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7499999999999998e-06 2024-04-24T15:32:33.999 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:33.999 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:33.999 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.841960+0000", 2024-04-24T15:32:33.999 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4399999999999996e-06 2024-04-24T15:32:33.999 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:33.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.000 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:34.000 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.841991+0000", 2024-04-24T15:32:34.000 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1615000000000001e-05 2024-04-24T15:32:34.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.000 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.000 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:34.000 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.842067+0000", 2024-04-24T15:32:34.000 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6002999999999997e-05 2024-04-24T15:32:34.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.000 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.000 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:34.000 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.842168+0000", 2024-04-24T15:32:34.000 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000100474 2024-04-24T15:32:34.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.001 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:34.001 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.847470+0000", 2024-04-24T15:32:34.001 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0053021830000000002 2024-04-24T15:32:34.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.001 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:34.001 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.847577+0000", 2024-04-24T15:32:34.001 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000107425 2024-04-24T15:32:34.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.001 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:34.001 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.847622+0000", 2024-04-24T15:32:34.001 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5015999999999997e-05 2024-04-24T15:32:34.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.002 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.002 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:34.002 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.847658+0000", 2024-04-24T15:32:34.002 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5509999999999997e-05 2024-04-24T15:32:34.002 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:34.002 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:34.002 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:34.002 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.002 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.002 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:4119 4.6 4:6588fd86:::rbd_data.114a1f60cb66.0000000000000179:head [set-alloc-hint object_size 262144 write_size 262144,write 151552~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:34.002 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.935980+0000", 2024-04-24T15:32:34.002 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 8.9717341210000008, 2024-04-24T15:32:34.003 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.006739391, 2024-04-24T15:32:34.003 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:34.003 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:34.003 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:34.003 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:34.003 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:34.003 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4119 2024-04-24T15:32:34.003 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.003 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:34.003 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.003 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:34.003 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.935980+0000", 2024-04-24T15:32:34.003 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:34.003 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.004 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.004 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:34.004 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.935980+0000", 2024-04-24T15:32:34.004 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:34.004 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.004 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.004 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:34.004 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.935980+0000", 2024-04-24T15:32:34.004 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:34.004 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.004 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.004 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:34.004 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.935987+0000", 2024-04-24T15:32:34.005 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1590000000000004e-06 2024-04-24T15:32:34.005 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.005 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.005 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:34.005 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.935996+0000", 2024-04-24T15:32:34.005 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8389999999999996e-06 2024-04-24T15:32:34.005 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.005 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.005 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:34.005 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.936001+0000", 2024-04-24T15:32:34.005 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6890000000000001e-06 2024-04-24T15:32:34.005 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.005 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.006 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:34.006 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.936456+0000", 2024-04-24T15:32:34.006 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00045432600000000001 2024-04-24T15:32:34.006 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.006 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.006 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:34.006 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.936530+0000", 2024-04-24T15:32:34.006 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4550999999999998e-05 2024-04-24T15:32:34.006 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.006 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.006 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:34.006 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.936663+0000", 2024-04-24T15:32:34.006 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000133225 2024-04-24T15:32:34.007 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.007 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.007 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:34.007 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.938066+0000", 2024-04-24T15:32:34.007 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0014031969999999999 2024-04-24T15:32:34.007 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.007 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.007 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:34.007 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.942636+0000", 2024-04-24T15:32:34.007 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0045691270000000001 2024-04-24T15:32:34.007 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.007 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.007 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:34.007 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.942684+0000", 2024-04-24T15:32:34.007 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8123e-05 2024-04-24T15:32:34.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.008 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.008 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:34.008 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.942720+0000", 2024-04-24T15:32:34.008 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5880000000000002e-05 2024-04-24T15:32:34.008 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:34.008 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:34.008 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:34.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.008 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.008 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:9497 4.1c 4:3e8473f6:::rbd_header.114a1f60cb66:head [watch reconnect cookie 140672528495600 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:34.008 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.312983+0000", 2024-04-24T15:32:34.008 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.59473131099999998, 2024-04-24T15:32:34.008 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0070173939999999997, 2024-04-24T15:32:34.009 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:34.009 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:34.009 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:34.009 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:34.009 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:34.009 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 9497 2024-04-24T15:32:34.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.009 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:34.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.009 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:34.009 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.312983+0000", 2024-04-24T15:32:34.009 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:34.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.010 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:34.010 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.312983+0000", 2024-04-24T15:32:34.010 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:34.010 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.010 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.010 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:34.010 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.312983+0000", 2024-04-24T15:32:34.010 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:34.010 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.010 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.010 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:34.010 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.312989+0000", 2024-04-24T15:32:34.010 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3090000000000001e-06 2024-04-24T15:32:34.010 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.010 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.011 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:34.011 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.312995+0000", 2024-04-24T15:32:34.011 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.477e-06 2024-04-24T15:32:34.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.011 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.011 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:34.011 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.312999+0000", 2024-04-24T15:32:34.011 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8489999999999999e-06 2024-04-24T15:32:34.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.011 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.011 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:34.011 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.313327+0000", 2024-04-24T15:32:34.011 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00032772399999999999 2024-04-24T15:32:34.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.011 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.011 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:34.011 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.313391+0000", 2024-04-24T15:32:34.012 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4388999999999996e-05 2024-04-24T15:32:34.012 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.012 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.012 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:34.012 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.320000+0000", 2024-04-24T15:32:34.012 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0066090660000000002 2024-04-24T15:32:34.012 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:34.012 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:34.012 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:34.012 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:34.012 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:34.012 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:32:34.013 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:32:34.015+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:32:34.014 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-24T15:32:34.015 INFO:teuthology.orchestra.run.smithi096.stdout:1192 punch from 0xbb7bf23 to 0xbb7d427, (0x1504 bytes) 2024-04-24T15:32:34.015 INFO:teuthology.orchestra.run.smithi096.stdout:1193 trunc from 0xe9a263e to 0x6740e08 2024-04-24T15:32:34.058 INFO:teuthology.orchestra.run.smithi096.stdout:1194 write 0x6ce6f85 thru 0x6cedce3 (0x6d5f bytes) 2024-04-24T15:32:34.070 INFO:teuthology.orchestra.run.smithi096.stdout:1197 writesame 0x966ea28 thru 0x9675ea7 data_size 0x20(0x7480 bytes) 2024-04-24T15:32:34.070 INFO:teuthology.orchestra.run.smithi096.stdout:1198 write 0x3eed7eb thru 0x3efc3f1 (0xec07 bytes) 2024-04-24T15:32:34.080 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:32:34.082+0000 7fc897291700 -1 reset not still connected to 0x55c1a191a680 2024-04-24T15:32:34.115 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:32:34.116+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:32:34.121 INFO:teuthology.orchestra.run.smithi096.stdout:1200 writesame 0xe36ecbb thru 0xe3794c3 data_size 0xd(0xa809 bytes) 2024-04-24T15:32:34.121 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:34.121 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:34.121 INFO:teuthology.orchestra.run.smithi096.stdout:1203 punch from 0xb020ccb to 0xb02c12e, (0xb463 bytes) 2024-04-24T15:32:34.121 INFO:teuthology.orchestra.run.smithi096.stdout:1204 flatten 2024-04-24T15:32:34.133 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 5, 6, 7] out_osds: [1, 4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-04-24T15:32:34.133 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-24T15:32:34.133 INFO:tasks.thrashosds.thrasher:Removing osd 5, in_osds are: [0, 2, 3, 5, 6, 7] 2024-04-24T15:32:34.133 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 5 2024-04-24T15:32:34.157 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-24T15:32:34.215 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:32:34.216 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:32:34.216 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-04-24T15:32:34.216 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:32:34.217 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:32:34.217+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:32:34.229 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-24T15:32:34.316 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:32:34.317+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:32:34.417 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:32:34.418+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:32:34.423 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:32:34.423 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:32:34.423 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-04-24T15:32:34.423 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-04-24T15:32:34.423 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:32:34.434 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-24T15:32:34.517 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:32:34.519+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:32:34.521 INFO:tasks.ceph.mon.a.smithi096.stderr:2024-04-24T15:32:34.522+0000 7fa88ddc8700 -1 mon.a@0(leader).osd e42 definitely_dead 0 2024-04-24T15:32:34.557 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-24T15:32:34.612 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:32:34.612 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-04-24T15:32:34.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-04-24T15:32:34.612 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-04-24T15:32:34.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.612 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14043 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2383872~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:34.612 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892347+0000", 2024-04-24T15:32:34.612 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.71945904500000002, 2024-04-24T15:32:34.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038537279000000001, 2024-04-24T15:32:34.613 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:34.613 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:34.613 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:34.613 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:34.613 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:34.613 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14043 2024-04-24T15:32:34.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.613 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:34.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:34.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892347+0000", 2024-04-24T15:32:34.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:34.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.614 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:34.614 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892347+0000", 2024-04-24T15:32:34.614 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:34.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.614 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:34.614 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892347+0000", 2024-04-24T15:32:34.614 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:34.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.614 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:34.614 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892349+0000", 2024-04-24T15:32:34.614 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3400000000000001e-06 2024-04-24T15:32:34.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:34.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892357+0000", 2024-04-24T15:32:34.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1540000000000002e-06 2024-04-24T15:32:34.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:34.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892360+0000", 2024-04-24T15:32:34.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5829999999999998e-06 2024-04-24T15:32:34.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:34.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930795+0000", 2024-04-24T15:32:34.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038434590999999997 2024-04-24T15:32:34.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:34.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930836+0000", 2024-04-24T15:32:34.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0827000000000001e-05 2024-04-24T15:32:34.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:34.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930885+0000", 2024-04-24T15:32:34.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9029000000000001e-05 2024-04-24T15:32:34.616 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:34.616 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:34.616 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:34.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.617 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14044 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2424832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:34.617 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892372+0000", 2024-04-24T15:32:34.617 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.719434401, 2024-04-24T15:32:34.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038624890000000002, 2024-04-24T15:32:34.617 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:34.617 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:34.617 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:34.617 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:34.617 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:34.617 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14044 2024-04-24T15:32:34.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.617 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:34.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.618 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:34.618 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892372+0000", 2024-04-24T15:32:34.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:34.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.618 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:34.618 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892372+0000", 2024-04-24T15:32:34.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:34.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.618 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:34.618 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892372+0000", 2024-04-24T15:32:34.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:34.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:34.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892374+0000", 2024-04-24T15:32:34.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4899999999999999e-06 2024-04-24T15:32:34.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:34.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892380+0000", 2024-04-24T15:32:34.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.631e-06 2024-04-24T15:32:34.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:34.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892383+0000", 2024-04-24T15:32:34.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7400000000000002e-06 2024-04-24T15:32:34.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:34.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930905+0000", 2024-04-24T15:32:34.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038521396999999999 2024-04-24T15:32:34.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:34.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930949+0000", 2024-04-24T15:32:34.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4153999999999998e-05 2024-04-24T15:32:34.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:34.621 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930997+0000", 2024-04-24T15:32:34.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7998999999999999e-05 2024-04-24T15:32:34.621 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:34.621 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:34.621 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:34.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.621 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14045 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2465792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:34.621 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892387+0000", 2024-04-24T15:32:34.621 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.71941913400000002, 2024-04-24T15:32:34.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038720933999999999, 2024-04-24T15:32:34.621 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:34.621 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:34.621 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:34.621 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:34.622 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:34.622 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14045 2024-04-24T15:32:34.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.622 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:34.622 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.622 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:34.622 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892387+0000", 2024-04-24T15:32:34.622 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:34.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.622 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.622 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:34.622 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892387+0000", 2024-04-24T15:32:34.622 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:34.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.623 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:34.623 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892387+0000", 2024-04-24T15:32:34.623 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:34.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.623 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:34.623 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892392+0000", 2024-04-24T15:32:34.623 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0039999999999999e-06 2024-04-24T15:32:34.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.623 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:34.623 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892398+0000", 2024-04-24T15:32:34.623 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0900000000000001e-06 2024-04-24T15:32:34.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.624 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:34.624 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892402+0000", 2024-04-24T15:32:34.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.399e-06 2024-04-24T15:32:34.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.624 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:34.624 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931018+0000", 2024-04-24T15:32:34.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038616551999999998 2024-04-24T15:32:34.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.624 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:34.624 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931059+0000", 2024-04-24T15:32:34.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1205000000000003e-05 2024-04-24T15:32:34.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.625 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:34.625 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931108+0000", 2024-04-24T15:32:34.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8996000000000003e-05 2024-04-24T15:32:34.625 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:34.625 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:34.625 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:34.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.625 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14046 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2506752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:34.625 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892406+0000", 2024-04-24T15:32:34.625 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.71940054399999998, 2024-04-24T15:32:34.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038814507999999998, 2024-04-24T15:32:34.626 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:34.626 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:34.626 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:34.626 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:34.626 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:34.626 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14046 2024-04-24T15:32:34.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.626 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:34.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.626 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:34.626 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892406+0000", 2024-04-24T15:32:34.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:34.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:34.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892406+0000", 2024-04-24T15:32:34.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:34.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:34.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892406+0000", 2024-04-24T15:32:34.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:34.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:34.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892413+0000", 2024-04-24T15:32:34.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7449999999999995e-06 2024-04-24T15:32:34.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:34.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892419+0000", 2024-04-24T15:32:34.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5790000000000003e-06 2024-04-24T15:32:34.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:34.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892423+0000", 2024-04-24T15:32:34.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8199999999999998e-06 2024-04-24T15:32:34.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:34.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931128+0000", 2024-04-24T15:32:34.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038705502000000003 2024-04-24T15:32:34.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:34.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931172+0000", 2024-04-24T15:32:34.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4066999999999997e-05 2024-04-24T15:32:34.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:34.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931220+0000", 2024-04-24T15:32:34.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8167999999999997e-05 2024-04-24T15:32:34.629 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:34.629 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:34.629 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:34.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.630 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14047 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2547712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:34.630 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892427+0000", 2024-04-24T15:32:34.630 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.71937931499999996, 2024-04-24T15:32:34.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038904506999999998, 2024-04-24T15:32:34.630 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:34.630 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:34.630 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:34.630 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:34.630 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:34.630 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14047 2024-04-24T15:32:34.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.630 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:34.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:34.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892427+0000", 2024-04-24T15:32:34.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:34.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:34.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892427+0000", 2024-04-24T15:32:34.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:34.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:34.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892427+0000", 2024-04-24T15:32:34.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:34.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:34.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892429+0000", 2024-04-24T15:32:34.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3099999999999999e-06 2024-04-24T15:32:34.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:34.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892435+0000", 2024-04-24T15:32:34.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5520000000000004e-06 2024-04-24T15:32:34.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.633 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:34.633 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892438+0000", 2024-04-24T15:32:34.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4240000000000002e-06 2024-04-24T15:32:34.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.633 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:34.633 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931242+0000", 2024-04-24T15:32:34.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038804162000000003 2024-04-24T15:32:34.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.633 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:34.633 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931284+0000", 2024-04-24T15:32:34.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1414e-05 2024-04-24T15:32:34.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:34.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931332+0000", 2024-04-24T15:32:34.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8024e-05 2024-04-24T15:32:34.634 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:34.634 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:34.634 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:34.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.634 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14048 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2588672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:34.634 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892442+0000", 2024-04-24T15:32:34.634 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.71936435200000004, 2024-04-24T15:32:34.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039003132000000003, 2024-04-24T15:32:34.634 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:34.635 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:34.635 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:34.635 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:34.635 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:34.635 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14048 2024-04-24T15:32:34.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.635 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:34.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:34.635 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892442+0000", 2024-04-24T15:32:34.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:34.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:34.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892442+0000", 2024-04-24T15:32:34.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:34.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:34.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892442+0000", 2024-04-24T15:32:34.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:34.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:34.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892444+0000", 2024-04-24T15:32:34.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1169999999999998e-06 2024-04-24T15:32:34.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:34.637 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892449+0000", 2024-04-24T15:32:34.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1599999999999997e-06 2024-04-24T15:32:34.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.637 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:34.637 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892453+0000", 2024-04-24T15:32:34.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5930000000000002e-06 2024-04-24T15:32:34.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.637 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:34.637 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931352+0000", 2024-04-24T15:32:34.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038899361 2024-04-24T15:32:34.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.638 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:34.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931398+0000", 2024-04-24T15:32:34.638 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6315e-05 2024-04-24T15:32:34.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.638 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:34.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931445+0000", 2024-04-24T15:32:34.638 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6950000000000003e-05 2024-04-24T15:32:34.638 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:34.638 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:34.638 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:34.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.639 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14049 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2629632~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:34.639 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892458+0000", 2024-04-24T15:32:34.639 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.71934893200000005, 2024-04-24T15:32:34.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039103322000000003, 2024-04-24T15:32:34.639 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:34.639 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:34.639 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:34.639 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:34.639 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:34.639 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14049 2024-04-24T15:32:34.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.639 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:34.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:34.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892458+0000", 2024-04-24T15:32:34.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:34.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:34.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892458+0000", 2024-04-24T15:32:34.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:34.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:34.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892457+0000", 2024-04-24T15:32:34.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:34.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.641 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:34.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892459+0000", 2024-04-24T15:32:34.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.305e-06 2024-04-24T15:32:34.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.641 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:34.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892465+0000", 2024-04-24T15:32:34.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4360000000000001e-06 2024-04-24T15:32:34.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.641 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:34.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892474+0000", 2024-04-24T15:32:34.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7639999999999994e-06 2024-04-24T15:32:34.642 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.642 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.642 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:34.642 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931466+0000", 2024-04-24T15:32:34.642 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038992644 2024-04-24T15:32:34.642 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.642 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.642 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:34.642 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931512+0000", 2024-04-24T15:32:34.642 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5200000000000001e-05 2024-04-24T15:32:34.642 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.642 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.642 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:34.642 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931561+0000", 2024-04-24T15:32:34.643 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9357999999999997e-05 2024-04-24T15:32:34.643 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:34.643 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:34.643 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:34.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.643 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14050 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2670592~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:34.643 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892478+0000", 2024-04-24T15:32:34.643 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.71932834499999998, 2024-04-24T15:32:34.643 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039195102000000002, 2024-04-24T15:32:34.643 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:34.643 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:34.643 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:34.643 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:34.644 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:34.644 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14050 2024-04-24T15:32:34.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.644 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:34.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:34.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892478+0000", 2024-04-24T15:32:34.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:34.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:34.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892478+0000", 2024-04-24T15:32:34.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:34.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:34.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892478+0000", 2024-04-24T15:32:34.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:34.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:34.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892480+0000", 2024-04-24T15:32:34.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3300000000000001e-06 2024-04-24T15:32:34.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:34.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892486+0000", 2024-04-24T15:32:34.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9200000000000001e-06 2024-04-24T15:32:34.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.646 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:34.646 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892494+0000", 2024-04-24T15:32:34.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5159999999999997e-06 2024-04-24T15:32:34.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.646 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:34.646 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931581+0000", 2024-04-24T15:32:34.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039087894999999998 2024-04-24T15:32:34.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.646 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:34.647 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931626+0000", 2024-04-24T15:32:34.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4943e-05 2024-04-24T15:32:34.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.647 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:34.647 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931673+0000", 2024-04-24T15:32:34.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6906000000000001e-05 2024-04-24T15:32:34.647 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:34.647 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:34.647 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:34.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.647 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14051 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2711552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:34.647 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892507+0000", 2024-04-24T15:32:34.648 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.71929964599999996, 2024-04-24T15:32:34.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039274393999999997, 2024-04-24T15:32:34.648 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:34.648 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:34.648 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:34.648 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:34.648 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:34.648 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14051 2024-04-24T15:32:34.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.648 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:34.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.648 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:34.648 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892507+0000", 2024-04-24T15:32:34.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:34.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:34.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892507+0000", 2024-04-24T15:32:34.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:34.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:34.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892506+0000", 2024-04-24T15:32:34.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:34.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:34.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892509+0000", 2024-04-24T15:32:34.650 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8760000000000001e-06 2024-04-24T15:32:34.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.650 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:34.650 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892515+0000", 2024-04-24T15:32:34.650 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1840000000000004e-06 2024-04-24T15:32:34.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.650 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:34.650 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892519+0000", 2024-04-24T15:32:34.650 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7570000000000002e-06 2024-04-24T15:32:34.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.651 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:34.651 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931694+0000", 2024-04-24T15:32:34.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039174960000000002 2024-04-24T15:32:34.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.651 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:34.651 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931734+0000", 2024-04-24T15:32:34.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0284000000000003e-05 2024-04-24T15:32:34.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.651 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:34.651 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931781+0000", 2024-04-24T15:32:34.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6813000000000001e-05 2024-04-24T15:32:34.651 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:34.651 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:34.652 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:34.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.652 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14052 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2752512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:34.652 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892524+0000", 2024-04-24T15:32:34.652 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.71928249899999996, 2024-04-24T15:32:34.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039371122000000001, 2024-04-24T15:32:34.652 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:34.652 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:34.652 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:34.652 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:34.652 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:34.652 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14052 2024-04-24T15:32:34.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.653 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:34.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:34.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892524+0000", 2024-04-24T15:32:34.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:34.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:34.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892524+0000", 2024-04-24T15:32:34.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:34.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:34.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892524+0000", 2024-04-24T15:32:34.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:34.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.654 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:34.654 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892526+0000", 2024-04-24T15:32:34.654 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3630000000000001e-06 2024-04-24T15:32:34.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.654 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:34.654 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892538+0000", 2024-04-24T15:32:34.654 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1677e-05 2024-04-24T15:32:34.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.655 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:34.655 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892541+0000", 2024-04-24T15:32:34.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.822e-06 2024-04-24T15:32:34.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.655 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:34.655 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931802+0000", 2024-04-24T15:32:34.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039260911000000003 2024-04-24T15:32:34.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.655 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:34.655 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931847+0000", 2024-04-24T15:32:34.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4450999999999998e-05 2024-04-24T15:32:34.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.656 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:34.656 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931895+0000", 2024-04-24T15:32:34.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8257999999999998e-05 2024-04-24T15:32:34.656 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:34.656 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:34.656 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:34.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.656 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14053 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2793472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:34.656 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892546+0000", 2024-04-24T15:32:34.656 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.71926052100000004, 2024-04-24T15:32:34.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039457712999999998, 2024-04-24T15:32:34.656 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:34.656 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:34.657 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:34.657 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:34.657 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:34.657 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14053 2024-04-24T15:32:34.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.657 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:34.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:34.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892546+0000", 2024-04-24T15:32:34.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:34.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:34.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892546+0000", 2024-04-24T15:32:34.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:34.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:34.658 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892546+0000", 2024-04-24T15:32:34.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:34.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:34.658 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892548+0000", 2024-04-24T15:32:34.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3120000000000001e-06 2024-04-24T15:32:34.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:34.659 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892553+0000", 2024-04-24T15:32:34.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2320000000000001e-06 2024-04-24T15:32:34.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.659 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:34.659 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892559+0000", 2024-04-24T15:32:34.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5480000000000001e-06 2024-04-24T15:32:34.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.659 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:34.659 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931916+0000", 2024-04-24T15:32:34.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039357180999999998 2024-04-24T15:32:34.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.659 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:34.660 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931956+0000", 2024-04-24T15:32:34.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0565000000000002e-05 2024-04-24T15:32:34.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.660 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:34.660 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932004+0000", 2024-04-24T15:32:34.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7224000000000001e-05 2024-04-24T15:32:34.660 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:34.660 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:34.660 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:34.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.660 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14054 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2834432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:34.660 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892563+0000", 2024-04-24T15:32:34.661 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.71924386900000004, 2024-04-24T15:32:34.661 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039552406999999998, 2024-04-24T15:32:34.661 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:34.661 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:34.661 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:34.661 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:34.661 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:34.661 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14054 2024-04-24T15:32:34.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.661 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:34.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:34.661 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892563+0000", 2024-04-24T15:32:34.661 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:34.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.662 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:34.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892563+0000", 2024-04-24T15:32:34.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:34.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.662 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:34.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892562+0000", 2024-04-24T15:32:34.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:34.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.662 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:34.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892564+0000", 2024-04-24T15:32:34.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2029999999999999e-06 2024-04-24T15:32:34.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.663 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:34.663 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892570+0000", 2024-04-24T15:32:34.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1460000000000004e-06 2024-04-24T15:32:34.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.663 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:34.663 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892573+0000", 2024-04-24T15:32:34.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2030000000000001e-06 2024-04-24T15:32:34.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.663 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:34.663 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932024+0000", 2024-04-24T15:32:34.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039451696000000001 2024-04-24T15:32:34.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.664 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:34.664 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932068+0000", 2024-04-24T15:32:34.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3970000000000001e-05 2024-04-24T15:32:34.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.664 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:34.664 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932115+0000", 2024-04-24T15:32:34.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6569000000000001e-05 2024-04-24T15:32:34.664 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:34.664 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:34.664 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:34.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.665 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14055 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2875392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:34.665 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892591+0000", 2024-04-24T15:32:34.665 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.71921589600000002, 2024-04-24T15:32:34.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039634309, 2024-04-24T15:32:34.665 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:34.665 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:34.665 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:34.665 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:34.665 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:34.665 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14055 2024-04-24T15:32:34.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.665 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:34.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.666 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:34.666 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892591+0000", 2024-04-24T15:32:34.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:34.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.666 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:34.666 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892591+0000", 2024-04-24T15:32:34.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:34.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.666 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:34.666 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892590+0000", 2024-04-24T15:32:34.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:34.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.667 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:34.667 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892592+0000", 2024-04-24T15:32:34.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7519999999999999e-06 2024-04-24T15:32:34.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.667 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:34.667 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892596+0000", 2024-04-24T15:32:34.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.608e-06 2024-04-24T15:32:34.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.667 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:34.667 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892598+0000", 2024-04-24T15:32:34.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.531e-06 2024-04-24T15:32:34.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.668 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:34.668 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932136+0000", 2024-04-24T15:32:34.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039537668999999998 2024-04-24T15:32:34.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.668 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:34.668 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932177+0000", 2024-04-24T15:32:34.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0880000000000002e-05 2024-04-24T15:32:34.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.668 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:34.668 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932225+0000", 2024-04-24T15:32:34.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8223999999999998e-05 2024-04-24T15:32:34.669 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:34.669 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:34.669 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:34.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.669 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14056 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2916352~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:34.669 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892600+0000", 2024-04-24T15:32:34.669 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.71920606200000003, 2024-04-24T15:32:34.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039736447000000001, 2024-04-24T15:32:34.669 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:34.669 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:34.669 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:34.669 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:34.669 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:34.670 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14056 2024-04-24T15:32:34.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.670 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:34.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:34.670 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892600+0000", 2024-04-24T15:32:34.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:34.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:34.670 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892600+0000", 2024-04-24T15:32:34.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:34.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.671 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:34.671 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892600+0000", 2024-04-24T15:32:34.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967296 2024-04-24T15:32:34.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.671 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:34.671 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892602+0000", 2024-04-24T15:32:34.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.576e-06 2024-04-24T15:32:34.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.671 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:34.671 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892605+0000", 2024-04-24T15:32:34.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.574e-06 2024-04-24T15:32:34.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.672 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:34.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892608+0000", 2024-04-24T15:32:34.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3800000000000001e-06 2024-04-24T15:32:34.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.672 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:34.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932245+0000", 2024-04-24T15:32:34.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039637378000000001 2024-04-24T15:32:34.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.672 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:34.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932290+0000", 2024-04-24T15:32:34.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4734999999999998e-05 2024-04-24T15:32:34.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.673 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:34.673 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932337+0000", 2024-04-24T15:32:34.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7033000000000002e-05 2024-04-24T15:32:34.673 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:34.673 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:34.673 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:34.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.673 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14062 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 3162112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:34.673 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.894556+0000", 2024-04-24T15:32:34.673 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.71725049200000002, 2024-04-24T15:32:34.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.03845316, 2024-04-24T15:32:34.673 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:34.673 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:34.674 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:34.674 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:34.674 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:34.674 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14062 2024-04-24T15:32:34.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.674 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:34.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.674 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:34.674 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894556+0000", 2024-04-24T15:32:34.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:34.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.674 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:34.674 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894556+0000", 2024-04-24T15:32:34.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:34.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:34.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894556+0000", 2024-04-24T15:32:34.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967296 2024-04-24T15:32:34.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:34.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894557+0000", 2024-04-24T15:32:34.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3230000000000001e-06 2024-04-24T15:32:34.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:34.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894560+0000", 2024-04-24T15:32:34.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7719999999999999e-06 2024-04-24T15:32:34.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:34.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894562+0000", 2024-04-24T15:32:34.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9479999999999998e-06 2024-04-24T15:32:34.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:34.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932917+0000", 2024-04-24T15:32:34.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038355490999999999 2024-04-24T15:32:34.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.677 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:34.677 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932961+0000", 2024-04-24T15:32:34.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3302e-05 2024-04-24T15:32:34.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.677 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:34.677 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933009+0000", 2024-04-24T15:32:34.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8535000000000002e-05 2024-04-24T15:32:34.677 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:34.677 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:34.677 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:34.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.677 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14063 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 3203072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:34.677 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.894564+0000", 2024-04-24T15:32:34.678 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.71724247100000005, 2024-04-24T15:32:34.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038554995000000002, 2024-04-24T15:32:34.678 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:34.678 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:34.678 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:34.678 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:34.678 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:34.678 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14063 2024-04-24T15:32:34.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.678 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:34.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:34.678 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894564+0000", 2024-04-24T15:32:34.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:34.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.679 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:34.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894564+0000", 2024-04-24T15:32:34.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:34.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.679 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:34.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894564+0000", 2024-04-24T15:32:34.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967296 2024-04-24T15:32:34.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.679 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:34.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894565+0000", 2024-04-24T15:32:34.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2640000000000001e-06 2024-04-24T15:32:34.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.680 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:34.680 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894569+0000", 2024-04-24T15:32:34.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1710000000000004e-06 2024-04-24T15:32:34.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.680 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:34.680 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894573+0000", 2024-04-24T15:32:34.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4410000000000002e-06 2024-04-24T15:32:34.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.680 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:34.680 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933030+0000", 2024-04-24T15:32:34.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038457709999999999 2024-04-24T15:32:34.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:34.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933072+0000", 2024-04-24T15:32:34.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.193e-05 2024-04-24T15:32:34.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:34.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933119+0000", 2024-04-24T15:32:34.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6715000000000003e-05 2024-04-24T15:32:34.681 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:34.681 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:34.681 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:34.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.682 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.682 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14064 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 3244032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:34.682 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.894591+0000", 2024-04-24T15:32:34.682 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.71721582900000003, 2024-04-24T15:32:34.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038639806999999998, 2024-04-24T15:32:34.682 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:34.682 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:34.682 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:34.682 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:34.682 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:34.682 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14064 2024-04-24T15:32:34.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.682 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:34.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.683 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:34.683 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894591+0000", 2024-04-24T15:32:34.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:34.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.683 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:34.683 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894591+0000", 2024-04-24T15:32:34.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:34.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.683 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:34.683 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894590+0000", 2024-04-24T15:32:34.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:34.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:34.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894593+0000", 2024-04-24T15:32:34.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4719999999999998e-06 2024-04-24T15:32:34.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:34.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894598+0000", 2024-04-24T15:32:34.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.57e-06 2024-04-24T15:32:34.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:34.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894602+0000", 2024-04-24T15:32:34.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.614e-06 2024-04-24T15:32:34.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:34.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933139+0000", 2024-04-24T15:32:34.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038536929999999997 2024-04-24T15:32:34.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:34.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933183+0000", 2024-04-24T15:32:34.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4217e-05 2024-04-24T15:32:34.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:34.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933230+0000", 2024-04-24T15:32:34.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7434e-05 2024-04-24T15:32:34.686 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:34.686 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:34.686 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:34.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.686 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14065 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 3284992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:34.686 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.894606+0000", 2024-04-24T15:32:34.686 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.71720044699999996, 2024-04-24T15:32:34.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038734126000000001, 2024-04-24T15:32:34.686 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:34.686 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:34.686 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:34.687 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:34.687 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:34.687 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14065 2024-04-24T15:32:34.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.687 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:34.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.687 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:34.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894606+0000", 2024-04-24T15:32:34.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:34.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.687 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:34.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894606+0000", 2024-04-24T15:32:34.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:34.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:34.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894606+0000", 2024-04-24T15:32:34.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:34.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:34.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894608+0000", 2024-04-24T15:32:34.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0880000000000002e-06 2024-04-24T15:32:34.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:34.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894614+0000", 2024-04-24T15:32:34.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7830000000000004e-06 2024-04-24T15:32:34.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:34.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894621+0000", 2024-04-24T15:32:34.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5809999999999999e-06 2024-04-24T15:32:34.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:34.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933251+0000", 2024-04-24T15:32:34.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038629995 2024-04-24T15:32:34.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:34.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933292+0000", 2024-04-24T15:32:34.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0380000000000003e-05 2024-04-24T15:32:34.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.692 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:34.692 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933340+0000", 2024-04-24T15:32:34.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8662999999999998e-05 2024-04-24T15:32:34.692 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:34.692 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:34.692 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:34.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.692 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14066 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 3325952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:34.692 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.894625+0000", 2024-04-24T15:32:34.692 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.71718100799999995, 2024-04-24T15:32:34.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038825539999999999, 2024-04-24T15:32:34.692 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:34.692 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:34.693 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:34.693 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:34.693 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:34.693 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14066 2024-04-24T15:32:34.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.693 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:34.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:34.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894625+0000", 2024-04-24T15:32:34.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:34.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:34.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894625+0000", 2024-04-24T15:32:34.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:34.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.694 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:34.694 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894625+0000", 2024-04-24T15:32:34.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:34.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.694 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:34.694 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894627+0000", 2024-04-24T15:32:34.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.092e-06 2024-04-24T15:32:34.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.694 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:34.694 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894635+0000", 2024-04-24T15:32:34.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6660000000000008e-06 2024-04-24T15:32:34.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:34.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894639+0000", 2024-04-24T15:32:34.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.799e-06 2024-04-24T15:32:34.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:34.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933360+0000", 2024-04-24T15:32:34.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038721569999999997 2024-04-24T15:32:34.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:34.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933403+0000", 2024-04-24T15:32:34.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2605999999999998e-05 2024-04-24T15:32:34.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.696 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:34.696 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933451+0000", 2024-04-24T15:32:34.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8149999999999998e-05 2024-04-24T15:32:34.696 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:34.696 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:34.696 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:34.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.696 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14067 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 3366912~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:34.696 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.894644+0000", 2024-04-24T15:32:34.697 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.71716294999999997, 2024-04-24T15:32:34.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038926068000000001, 2024-04-24T15:32:34.697 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:34.697 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:34.697 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:34.697 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:34.697 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:34.697 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14067 2024-04-24T15:32:34.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.697 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:34.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.697 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:34.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894644+0000", 2024-04-24T15:32:34.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:34.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.698 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:34.698 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894644+0000", 2024-04-24T15:32:34.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:34.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.698 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:34.698 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894643+0000", 2024-04-24T15:32:34.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:34.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.698 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:34.698 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894646+0000", 2024-04-24T15:32:34.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3379999999999999e-06 2024-04-24T15:32:34.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:34.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894651+0000", 2024-04-24T15:32:34.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.485e-06 2024-04-24T15:32:34.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:34.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894655+0000", 2024-04-24T15:32:34.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.709e-06 2024-04-24T15:32:34.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:34.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933472+0000", 2024-04-24T15:32:34.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038816992000000002 2024-04-24T15:32:34.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:34.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933521+0000", 2024-04-24T15:32:34.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8841999999999998e-05 2024-04-24T15:32:34.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:34.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:34.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:34.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933570+0000", 2024-04-24T15:32:34.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9063000000000001e-05 2024-04-24T15:32:34.700 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:34.700 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:34.701 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:34.701 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:34.701 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:34.701 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:32:34.702 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-24T15:32:34.703 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:32:34.703+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:32:34.709 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-04-24T15:32:34.738 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-04-24T15:32:34.763 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-04-24T15:32:34.799 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-04-24T15:32:34.803 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:32:34.804+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:32:34.820 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-04-24T15:32:34.839 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-04-24T15:32:34.860 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-04-24T15:32:34.879 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-04-24T15:32:34.881 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T15:32:34.881 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T15:32:34.881 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T15:32:34.881 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T15:32:34.881 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T15:32:34.881 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T15:32:34.881 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T15:32:34.881 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T15:32:34.886 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:34.886 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:34.898 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-24T15:32:34.901 INFO:teuthology.orchestra.run.smithi096.stdout:1205 writesame 0x195a674 thru 0x1964ef6 data_size 0x1(0xa883 bytes) 2024-04-24T15:32:34.904 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:32:34.905+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:32:34.916 INFO:teuthology.orchestra.run.smithi096.stdout:1206 write 0x800123b thru 0x800a1e3 (0x8fa9 bytes) 2024-04-24T15:32:34.927 INFO:teuthology.orchestra.run.smithi096.stdout:1207 read 0xc4e6f34 thru 0xc4f65ba (0xf687 bytes) 2024-04-24T15:32:34.927 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:34.927 INFO:teuthology.orchestra.run.smithi096.stdout:1209 write 0xad5193f thru 0xad567f5 (0x4eb7 bytes) 2024-04-24T15:32:34.938 INFO:teuthology.orchestra.run.smithi096.stdout:1210 read 0x96b8059 thru 0x96bd8f1 (0x5899 bytes) 2024-04-24T15:32:34.938 INFO:teuthology.orchestra.run.smithi096.stdout:1212 read 0x9b0d727 thru 0x9b130c0 (0x599a bytes) 2024-04-24T15:32:34.938 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:34.947 INFO:teuthology.orchestra.run.smithi096.stdout:1215 trunc from 0xe3794c4 to 0x61a63c9 2024-04-24T15:32:34.962 INFO:teuthology.orchestra.run.smithi096.stdout:1219 read 0x5c05538 thru 0x5c0f448 (0x9f11 bytes) 2024-04-24T15:32:34.962 INFO:teuthology.orchestra.run.smithi096.stdout:1220 punch from 0xa0f03c to 0xa14cb4, (0x5c78 bytes) 2024-04-24T15:32:34.963 INFO:teuthology.orchestra.run.smithi096.stdout:1221 punch from 0x23013b6 to 0x230c637, (0xb281 bytes) 2024-04-24T15:32:34.963 INFO:teuthology.orchestra.run.smithi096.stdout:1222 punch from 0x1a4089a to 0x1a4d9ea, (0xd150 bytes) 2024-04-24T15:32:34.963 INFO:teuthology.orchestra.run.smithi096.stdout:1223 punch from 0x52a3ea1 to 0x52a680c, (0x296b bytes) 2024-04-24T15:32:34.964 INFO:teuthology.orchestra.run.smithi096.stdout:1224 trunc from 0x61a63c9 to 0x411cb39 2024-04-24T15:32:34.980 INFO:teuthology.orchestra.run.smithi096.stdout:1226 punch from 0x2bafbb to 0x2bd5a2, (0x25e7 bytes) 2024-04-24T15:32:34.980 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:34.980 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:34.980 INFO:teuthology.orchestra.run.smithi096.stdout:1232 read 0x40fcb91 thru 0x41045d3 (0x7a43 bytes) 2024-04-24T15:32:34.980 INFO:teuthology.orchestra.run.smithi096.stdout:1233 write 0x3bae54c thru 0x3bbe51b (0xffd0 bytes) 2024-04-24T15:32:34.982 INFO:teuthology.orchestra.run.smithi096.stdout:1235 write 0x11c6a7a thru 0x11cb7b3 (0x4d3a bytes) 2024-04-24T15:32:34.986 INFO:teuthology.orchestra.run.smithi096.stdout:1236 trunc from 0x411cb39 to 0x5a4171f 2024-04-24T15:32:34.998 INFO:teuthology.orchestra.run.smithi096.stdout:1238 trunc from 0x5a4171f to 0xb412f7a 2024-04-24T15:32:35.004 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:32:35.006+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:32:35.008 INFO:teuthology.orchestra.run.smithi096.stdout:1239 trunc from 0xb412f7a to 0xed4cbfd 2024-04-24T15:32:35.014 INFO:teuthology.orchestra.run.smithi096.stdout:1240 punch from 0x69033bb to 0x6906f5a, (0x3b9f bytes) 2024-04-24T15:32:35.014 INFO:teuthology.orchestra.run.smithi096.stdout:1241 punch from 0x52d4db2 to 0x52da320, (0x556e bytes) 2024-04-24T15:32:35.014 INFO:teuthology.orchestra.run.smithi096.stdout:1243 writesame 0x99f971f thru 0x99fc33a data_size 0x6(0x2c1c bytes) 2024-04-24T15:32:35.027 INFO:teuthology.orchestra.run.smithi096.stdout:1244 trunc from 0xed4cbfd to 0x70e166b 2024-04-24T15:32:35.038 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:35.038 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:35.044 INFO:teuthology.orchestra.run.smithi096.stdout:1245 write 0x980dd12 thru 0x980f552 (0x1841 bytes) 2024-04-24T15:32:35.050 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-24T15:32:35.054 INFO:teuthology.orchestra.run.smithi096.stdout:1246 trunc from 0x980f553 to 0x5429f01 2024-04-24T15:32:35.074 INFO:teuthology.orchestra.run.smithi096.stdout:1248 writesame 0x854f8f4 thru 0x8559a39 data_size 0x15(0xa146 bytes) 2024-04-24T15:32:35.074 INFO:teuthology.orchestra.run.smithi096.stdout:1249 writesame 0x61fe0e0 thru 0x6208cac data_size 0x3d(0xabcd bytes) 2024-04-24T15:32:35.088 INFO:teuthology.orchestra.run.smithi096.stdout:1252 writesame 0x9b4c4ce thru 0x9b5b7b3 data_size 0x1(0xf2e6 bytes) 2024-04-24T15:32:35.088 INFO:teuthology.orchestra.run.smithi096.stdout:1253 writesame 0xa0ecbb thru 0xa16ec3 data_size 0x1(0x8209 bytes) 2024-04-24T15:32:35.096 INFO:teuthology.orchestra.run.smithi096.stdout:1255 trunc from 0x9b5b7b4 to 0x8a6ca71 2024-04-24T15:32:35.102 INFO:teuthology.orchestra.run.smithi096.stdout:1256 read 0x6be9b5d thru 0x6bef3bf (0x5863 bytes) 2024-04-24T15:32:35.106 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:32:35.107+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:32:35.112 INFO:teuthology.orchestra.run.smithi096.stdout:1260 write 0xc8e554e thru 0xc8eab34 (0x55e7 bytes) 2024-04-24T15:32:35.121 INFO:teuthology.orchestra.run.smithi096.stdout:1261 read 0x257880 thru 0x25df11 (0x6692 bytes) 2024-04-24T15:32:35.122 INFO:teuthology.orchestra.run.smithi096.stdout:1262 write 0x2110c90 thru 0x21207c7 (0xfb38 bytes) 2024-04-24T15:32:35.132 INFO:teuthology.orchestra.run.smithi096.stdout:1263 write 0x1f9e75 thru 0x1fcf9a (0x3126 bytes) 2024-04-24T15:32:35.133 INFO:teuthology.orchestra.run.smithi096.stdout:1264 read 0x6d33d94 thru 0x6d380fb (0x4368 bytes) 2024-04-24T15:32:35.133 INFO:teuthology.orchestra.run.smithi096.stdout:1265 punch from 0x7bbf99a to 0x7bc07c1, (0xe27 bytes) 2024-04-24T15:32:35.134 INFO:teuthology.orchestra.run.smithi096.stdout:1266 read 0x6ff6d54 thru 0x6ff8ebc (0x2169 bytes) 2024-04-24T15:32:35.134 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:35.134 INFO:teuthology.orchestra.run.smithi096.stdout:1268 write 0x4670e84 thru 0x4673f62 (0x30df bytes) 2024-04-24T15:32:35.139 INFO:teuthology.orchestra.run.smithi096.stdout:1269 write 0x808820e thru 0x8092e76 (0xac69 bytes) 2024-04-24T15:32:35.149 INFO:teuthology.orchestra.run.smithi096.stdout:1270 writesame 0xc68b471 thru 0xc695ddc data_size 0xe(0xa96c bytes) 2024-04-24T15:32:35.158 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:35.158 INFO:teuthology.orchestra.run.smithi096.stdout:1272 write 0x9088b5 thru 0x90c7b9 (0x3f05 bytes) 2024-04-24T15:32:35.165 INFO:teuthology.orchestra.run.smithi096.stderr:marked out osd.5. 2024-04-24T15:32:35.166 INFO:teuthology.orchestra.run.smithi096.stderr:noscrub is unset 2024-04-24T15:32:35.167 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:35.168 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:35.168 INFO:teuthology.orchestra.run.smithi096.stdout:1275 writesame 0x3647109 thru 0x3656981 data_size 0x27(0xf879 bytes) 2024-04-24T15:32:35.207 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:32:35.208+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:32:35.226 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:35.226 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:35.235 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-24T15:32:35.307 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:32:35.309+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:32:35.400 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:35.400 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:35.408 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:32:35.410+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:32:35.410 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-24T15:32:35.509 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:32:35.511+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:32:35.532 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:35.533 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:35.543 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-24T15:32:35.626 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:32:35.628+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:32:35.706 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:35.706 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:35.711 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:32:35.712+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:32:35.715 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-24T15:32:35.812 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:32:35.813+0000 7f23abcf6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 44021) UID: 0 2024-04-24T15:32:35.879 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:35.879 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:35.889 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-24T15:32:35.912 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:32:35.913+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:32:36.013 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:32:36.014+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:32:36.051 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:36.051 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:36.061 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-24T15:32:36.114 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:32:36.115+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:32:36.195 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:36.195 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:36.206 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-24T15:32:36.215 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:32:36.216+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:32:36.316 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:32:36.317+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:32:36.372 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:36.372 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:36.381 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-24T15:32:36.417 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:32:36.418+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:32:36.518 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:32:36.519+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:32:36.544 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:36.545 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:36.555 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-24T15:32:36.619 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:32:36.620+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:32:36.678 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:36.678 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:36.688 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-24T15:32:36.733 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:32:36.734+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:32:36.821 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:32:36.823+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:32:36.822 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:36.822 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:36.832 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-24T15:32:36.892 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-24T15:32:36.922 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:32:36.924+0000 7f23abcf6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 44021) UID: 0 2024-04-24T15:32:36.999 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:36.999 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:37.010 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-24T15:32:37.054 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:32:37.055+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:32:37.057 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-04-24T15:32:37.075 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-04-24T15:32:37.093 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-04-24T15:32:37.111 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-04-24T15:32:37.124 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:32:37.125+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:32:37.130 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-04-24T15:32:37.138 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:32:37.139 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-04-24T15:32:37.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-04-24T15:32:37.139 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-04-24T15:32:37.139 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.139 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:18 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'8, mlcod=15'5)", 2024-04-24T15:32:37.139 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:00.973932+0000", 2024-04-24T15:32:37.139 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.164785936000001, 2024-04-24T15:32:37.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0064286839999999996, 2024-04-24T15:32:37.139 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:37.139 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:37.139 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:37.139 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.140 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:37.140 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.973932+0000", 2024-04-24T15:32:37.140 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:37.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.140 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.140 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:37.140 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.973932+0000", 2024-04-24T15:32:37.140 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:37.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.140 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.140 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:37.140 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.973931+0000", 2024-04-24T15:32:37.140 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:37.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.141 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:37.141 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.973938+0000", 2024-04-24T15:32:37.141 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.029e-06 2024-04-24T15:32:37.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.141 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:37.141 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.973948+0000", 2024-04-24T15:32:37.141 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9129999999999998e-06 2024-04-24T15:32:37.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.141 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:37.141 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.973952+0000", 2024-04-24T15:32:37.142 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3069999999999996e-06 2024-04-24T15:32:37.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.142 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.142 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:37.142 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.973986+0000", 2024-04-24T15:32:37.142 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3643000000000003e-05 2024-04-24T15:32:37.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.142 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.142 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:37.142 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.974006+0000", 2024-04-24T15:32:37.142 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9658999999999999e-05 2024-04-24T15:32:37.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.142 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.143 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:37.143 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.980330+0000", 2024-04-24T15:32:37.143 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0063243479999999996 2024-04-24T15:32:37.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.143 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.143 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:37.143 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.980361+0000", 2024-04-24T15:32:37.143 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0527000000000001e-05 2024-04-24T15:32:37.143 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:37.143 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:37.143 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:37.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.143 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.143 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:21 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'11, mlcod=15'8)", 2024-04-24T15:32:37.144 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:00.986502+0000", 2024-04-24T15:32:37.144 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.152216002999999, 2024-04-24T15:32:37.144 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0062008819999999996, 2024-04-24T15:32:37.144 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:37.144 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:37.144 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:37.144 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.144 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:37.144 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.986502+0000", 2024-04-24T15:32:37.144 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:37.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.144 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.144 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:37.144 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.986502+0000", 2024-04-24T15:32:37.144 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:37.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.145 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.145 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:37.145 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.986501+0000", 2024-04-24T15:32:37.145 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:37.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.145 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.145 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:37.145 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.986639+0000", 2024-04-24T15:32:37.145 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013731400000000001 2024-04-24T15:32:37.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.145 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.145 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:37.145 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.986649+0000", 2024-04-24T15:32:37.145 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.041e-05 2024-04-24T15:32:37.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.146 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.146 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:37.146 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.986654+0000", 2024-04-24T15:32:37.146 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2000000000000002e-06 2024-04-24T15:32:37.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.146 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.146 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:37.146 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.986678+0000", 2024-04-24T15:32:37.146 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3731e-05 2024-04-24T15:32:37.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.146 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.146 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:37.146 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.986691+0000", 2024-04-24T15:32:37.146 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3078000000000001e-05 2024-04-24T15:32:37.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.147 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:37.147 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.992666+0000", 2024-04-24T15:32:37.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0059751650000000002 2024-04-24T15:32:37.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.147 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:37.147 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.992703+0000", 2024-04-24T15:32:37.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6622000000000003e-05 2024-04-24T15:32:37.147 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:37.147 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:37.147 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:37.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.148 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.148 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:22 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'12, mlcod=15'8)", 2024-04-24T15:32:37.148 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:00.987102+0000", 2024-04-24T15:32:37.148 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.151616156999999, 2024-04-24T15:32:37.148 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0056291650000000002, 2024-04-24T15:32:37.148 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:37.148 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:37.148 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:37.148 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.148 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:37.148 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.987102+0000", 2024-04-24T15:32:37.148 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:37.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.149 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.149 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:37.149 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.987102+0000", 2024-04-24T15:32:37.149 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:37.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.149 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.149 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:37.149 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.987101+0000", 2024-04-24T15:32:37.149 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:37.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.149 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.149 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:37.149 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.987106+0000", 2024-04-24T15:32:37.149 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0969999999999996e-06 2024-04-24T15:32:37.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.149 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.150 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:37.150 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.987113+0000", 2024-04-24T15:32:37.150 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6789999999999999e-06 2024-04-24T15:32:37.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.150 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.150 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:37.150 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.987117+0000", 2024-04-24T15:32:37.150 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6519999999999999e-06 2024-04-24T15:32:37.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.150 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.150 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:37.150 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.987143+0000", 2024-04-24T15:32:37.150 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6098999999999999e-05 2024-04-24T15:32:37.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.151 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:37.151 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.987157+0000", 2024-04-24T15:32:37.151 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4100999999999999e-05 2024-04-24T15:32:37.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.151 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:37.151 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.992706+0000", 2024-04-24T15:32:37.151 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0055495630000000004 2024-04-24T15:32:37.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.151 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:37.151 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.992731+0000", 2024-04-24T15:32:37.151 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4519e-05 2024-04-24T15:32:37.151 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:37.151 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:37.152 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:37.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.152 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:69 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'43, mlcod=15'39)", 2024-04-24T15:32:37.152 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.058504+0000", 2024-04-24T15:32:37.152 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.080213682999997, 2024-04-24T15:32:37.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.006494606, 2024-04-24T15:32:37.152 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:37.152 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:37.152 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:37.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.152 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:37.152 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.058504+0000", 2024-04-24T15:32:37.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:37.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:37.153 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.058504+0000", 2024-04-24T15:32:37.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:37.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:37.153 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.058503+0000", 2024-04-24T15:32:37.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:37.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:37.153 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.058588+0000", 2024-04-24T15:32:37.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4087999999999999e-05 2024-04-24T15:32:37.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.154 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.154 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:37.154 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.058597+0000", 2024-04-24T15:32:37.154 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7459999999999995e-06 2024-04-24T15:32:37.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.154 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.154 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:37.154 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.058602+0000", 2024-04-24T15:32:37.154 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6929999999999998e-06 2024-04-24T15:32:37.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.154 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.154 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:37.155 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.060983+0000", 2024-04-24T15:32:37.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023812690000000001 2024-04-24T15:32:37.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.155 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:37.155 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.060996+0000", 2024-04-24T15:32:37.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2826999999999999e-05 2024-04-24T15:32:37.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.155 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:37.155 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.064972+0000", 2024-04-24T15:32:37.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0039757890000000004 2024-04-24T15:32:37.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.155 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:37.155 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.064999+0000", 2024-04-24T15:32:37.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6917999999999999e-05 2024-04-24T15:32:37.156 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:37.156 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:37.156 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:37.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.156 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:70 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'44, mlcod=15'39)", 2024-04-24T15:32:37.156 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.059180+0000", 2024-04-24T15:32:37.156 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.079537930999997, 2024-04-24T15:32:37.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0058429709999999998, 2024-04-24T15:32:37.156 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:37.156 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:37.156 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:37.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.156 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:37.157 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.059180+0000", 2024-04-24T15:32:37.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:37.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.157 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.157 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:37.157 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.059180+0000", 2024-04-24T15:32:37.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:37.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.157 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.157 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:37.157 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.059179+0000", 2024-04-24T15:32:37.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:37.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.157 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.158 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:37.158 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.059184+0000", 2024-04-24T15:32:37.158 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.074e-06 2024-04-24T15:32:37.158 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.158 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.158 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:37.158 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.059194+0000", 2024-04-24T15:32:37.158 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6050000000000004e-06 2024-04-24T15:32:37.158 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.158 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.158 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:37.158 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.059199+0000", 2024-04-24T15:32:37.158 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2639999999999999e-06 2024-04-24T15:32:37.158 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.159 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.159 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:37.159 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.061424+0000", 2024-04-24T15:32:37.159 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0022244880000000002 2024-04-24T15:32:37.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.159 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.159 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:37.159 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.061444+0000", 2024-04-24T15:32:37.159 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0633e-05 2024-04-24T15:32:37.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.159 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.159 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:37.159 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.065001+0000", 2024-04-24T15:32:37.159 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035567670000000002 2024-04-24T15:32:37.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:37.160 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.065023+0000", 2024-04-24T15:32:37.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1724000000000001e-05 2024-04-24T15:32:37.160 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:37.160 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:37.160 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:37.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.160 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:154 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'108, mlcod=15'105)", 2024-04-24T15:32:37.160 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.929033+0000", 2024-04-24T15:32:37.160 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 35.209684549999999, 2024-04-24T15:32:37.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0068887990000000001, 2024-04-24T15:32:37.160 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:37.161 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:37.161 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:37.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.161 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:37.161 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929033+0000", 2024-04-24T15:32:37.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:37.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.161 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:37.161 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929033+0000", 2024-04-24T15:32:37.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:37.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.161 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:37.161 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929033+0000", 2024-04-24T15:32:37.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:37.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:37.162 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929042+0000", 2024-04-24T15:32:37.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5440000000000004e-06 2024-04-24T15:32:37.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:37.162 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929051+0000", 2024-04-24T15:32:37.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0580000000000004e-06 2024-04-24T15:32:37.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.163 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:37.163 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929055+0000", 2024-04-24T15:32:37.163 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.337e-06 2024-04-24T15:32:37.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.163 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.163 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:37.163 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929074+0000", 2024-04-24T15:32:37.163 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9724999999999998e-05 2024-04-24T15:32:37.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.163 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.163 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:37.163 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929084+0000", 2024-04-24T15:32:37.163 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7049999999999994e-06 2024-04-24T15:32:37.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.163 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.164 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:37.164 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.935895+0000", 2024-04-24T15:32:37.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0068104359999999996 2024-04-24T15:32:37.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.164 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:37.164 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.935922+0000", 2024-04-24T15:32:37.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7538999999999998e-05 2024-04-24T15:32:37.164 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:37.164 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:37.164 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:37.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.164 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:155 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'109, mlcod=15'105)", 2024-04-24T15:32:37.165 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.929398+0000", 2024-04-24T15:32:37.165 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 35.209319569000002, 2024-04-24T15:32:37.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0098754700000000008, 2024-04-24T15:32:37.165 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:37.165 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:37.165 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:37.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.165 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:37.165 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929398+0000", 2024-04-24T15:32:37.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:37.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.165 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:37.165 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929398+0000", 2024-04-24T15:32:37.166 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:37.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.166 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:37.166 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929398+0000", 2024-04-24T15:32:37.166 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:37.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.166 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:37.166 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929571+0000", 2024-04-24T15:32:37.166 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017281999999999999 2024-04-24T15:32:37.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.166 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:37.166 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929578+0000", 2024-04-24T15:32:37.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8929999999999999e-06 2024-04-24T15:32:37.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.167 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:37.167 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929581+0000", 2024-04-24T15:32:37.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1709999999999998e-06 2024-04-24T15:32:37.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.167 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:37.167 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929605+0000", 2024-04-24T15:32:37.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3938e-05 2024-04-24T15:32:37.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.168 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:37.168 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929618+0000", 2024-04-24T15:32:37.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3653e-05 2024-04-24T15:32:37.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.168 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:37.168 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939186+0000", 2024-04-24T15:32:37.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0095674260000000004 2024-04-24T15:32:37.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.168 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:37.168 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939274+0000", 2024-04-24T15:32:37.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7928000000000002e-05 2024-04-24T15:32:37.168 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:37.168 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:37.169 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:37.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.169 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.169 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:156 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'110, mlcod=15'105)", 2024-04-24T15:32:37.169 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.930050+0000", 2024-04-24T15:32:37.169 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 35.208668252999999, 2024-04-24T15:32:37.169 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0092911650000000005, 2024-04-24T15:32:37.169 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:37.169 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:37.169 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:37.169 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.169 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:37.169 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930050+0000", 2024-04-24T15:32:37.169 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:37.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.169 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.170 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:37.170 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930050+0000", 2024-04-24T15:32:37.170 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:37.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.170 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:37.170 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930049+0000", 2024-04-24T15:32:37.170 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:37.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.170 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:37.170 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930053+0000", 2024-04-24T15:32:37.170 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.98e-06 2024-04-24T15:32:37.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:37.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930058+0000", 2024-04-24T15:32:37.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3530000000000002e-06 2024-04-24T15:32:37.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:37.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930061+0000", 2024-04-24T15:32:37.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6970000000000002e-06 2024-04-24T15:32:37.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:37.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930082+0000", 2024-04-24T15:32:37.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0815e-05 2024-04-24T15:32:37.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:37.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930093+0000", 2024-04-24T15:32:37.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0699999999999999e-05 2024-04-24T15:32:37.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:37.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939280+0000", 2024-04-24T15:32:37.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0091874880000000006 2024-04-24T15:32:37.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:37.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939341+0000", 2024-04-24T15:32:37.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0608e-05 2024-04-24T15:32:37.172 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:37.172 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:37.172 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:37.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.173 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:157 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'111, mlcod=15'106)", 2024-04-24T15:32:37.173 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.930506+0000", 2024-04-24T15:32:37.173 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 35.208211394999999, 2024-04-24T15:32:37.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.008889246, 2024-04-24T15:32:37.173 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:37.173 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:37.173 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:37.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.173 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:37.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930506+0000", 2024-04-24T15:32:37.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:37.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.174 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:37.174 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930506+0000", 2024-04-24T15:32:37.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:37.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.174 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:37.174 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930506+0000", 2024-04-24T15:32:37.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:37.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.174 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:37.174 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930510+0000", 2024-04-24T15:32:37.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1810000000000003e-06 2024-04-24T15:32:37.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:37.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930516+0000", 2024-04-24T15:32:37.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1920000000000002e-06 2024-04-24T15:32:37.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:37.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930520+0000", 2024-04-24T15:32:37.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.501e-06 2024-04-24T15:32:37.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:37.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930539+0000", 2024-04-24T15:32:37.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8929000000000001e-05 2024-04-24T15:32:37.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:37.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930550+0000", 2024-04-24T15:32:37.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1168999999999999e-05 2024-04-24T15:32:37.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:37.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939343+0000", 2024-04-24T15:32:37.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0087932900000000005 2024-04-24T15:32:37.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:37.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939396+0000", 2024-04-24T15:32:37.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2454999999999998e-05 2024-04-24T15:32:37.176 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:37.176 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:37.177 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:37.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.177 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:158 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'112, mlcod=15'106)", 2024-04-24T15:32:37.177 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.930877+0000", 2024-04-24T15:32:37.177 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 35.207840703999999, 2024-04-24T15:32:37.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.008572741, 2024-04-24T15:32:37.177 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:37.177 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:37.177 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:37.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:37.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930877+0000", 2024-04-24T15:32:37.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:37.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:37.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930877+0000", 2024-04-24T15:32:37.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:37.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:37.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930877+0000", 2024-04-24T15:32:37.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:37.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:37.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930968+0000", 2024-04-24T15:32:37.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1638999999999995e-05 2024-04-24T15:32:37.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.179 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:37.179 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930975+0000", 2024-04-24T15:32:37.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2169999999999999e-06 2024-04-24T15:32:37.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.179 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:37.179 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930977+0000", 2024-04-24T15:32:37.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.791e-06 2024-04-24T15:32:37.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.179 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:37.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931002+0000", 2024-04-24T15:32:37.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4491e-05 2024-04-24T15:32:37.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:37.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931012+0000", 2024-04-24T15:32:37.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0217000000000001e-05 2024-04-24T15:32:37.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:37.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939398+0000", 2024-04-24T15:32:37.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.008385366 2024-04-24T15:32:37.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:37.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939450+0000", 2024-04-24T15:32:37.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2326e-05 2024-04-24T15:32:37.181 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:37.181 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:37.181 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:37.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.181 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:159 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'113, mlcod=15'106)", 2024-04-24T15:32:37.181 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.931313+0000", 2024-04-24T15:32:37.181 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 35.207405352000002, 2024-04-24T15:32:37.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0081994159999999993, 2024-04-24T15:32:37.181 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:37.181 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:37.181 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:37.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:37.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931313+0000", 2024-04-24T15:32:37.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:37.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:37.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931313+0000", 2024-04-24T15:32:37.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:37.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:37.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931312+0000", 2024-04-24T15:32:37.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:37.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:37.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931316+0000", 2024-04-24T15:32:37.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0589999999999996e-06 2024-04-24T15:32:37.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:37.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931322+0000", 2024-04-24T15:32:37.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2020000000000001e-06 2024-04-24T15:32:37.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:37.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931327+0000", 2024-04-24T15:32:37.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2440000000000004e-06 2024-04-24T15:32:37.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:37.184 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931348+0000", 2024-04-24T15:32:37.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1464e-05 2024-04-24T15:32:37.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.184 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:37.184 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931359+0000", 2024-04-24T15:32:37.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0699e-05 2024-04-24T15:32:37.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.184 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:37.184 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939452+0000", 2024-04-24T15:32:37.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0080931749999999993 2024-04-24T15:32:37.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.184 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:37.185 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939512+0000", 2024-04-24T15:32:37.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0050000000000003e-05 2024-04-24T15:32:37.185 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:37.185 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:37.185 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:37.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.185 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:2447 4.6 4:668081fa:::rbd_data.114a1f60cb66.00000000000001f0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-24T15:32:37.185 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:23.027189+0000", 2024-04-24T15:32:37.185 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.111529328, 2024-04-24T15:32:37.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0056741200000000004, 2024-04-24T15:32:37.185 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:37.185 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:37.185 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:37.186 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:37.186 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:37.186 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2447 2024-04-24T15:32:37.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.186 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:37.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:37.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027189+0000", 2024-04-24T15:32:37.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:37.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:37.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027189+0000", 2024-04-24T15:32:37.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:37.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:37.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027188+0000", 2024-04-24T15:32:37.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:37.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:37.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027194+0000", 2024-04-24T15:32:37.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0499999999999997e-06 2024-04-24T15:32:37.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:37.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027205+0000", 2024-04-24T15:32:37.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1619e-05 2024-04-24T15:32:37.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:37.188 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027212+0000", 2024-04-24T15:32:37.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4289999999999998e-06 2024-04-24T15:32:37.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:37.188 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027243+0000", 2024-04-24T15:32:37.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1510000000000002e-05 2024-04-24T15:32:37.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:37.188 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027288+0000", 2024-04-24T15:32:37.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4484000000000003e-05 2024-04-24T15:32:37.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-24T15:32:37.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027351+0000", 2024-04-24T15:32:37.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3311000000000004e-05 2024-04-24T15:32:37.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:37.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.029874+0000", 2024-04-24T15:32:37.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00252315 2024-04-24T15:32:37.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:37.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.032786+0000", 2024-04-24T15:32:37.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0029115370000000001 2024-04-24T15:32:37.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:37.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.032828+0000", 2024-04-24T15:32:37.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2132000000000003e-05 2024-04-24T15:32:37.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:37.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.032863+0000", 2024-04-24T15:32:37.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4912e-05 2024-04-24T15:32:37.190 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:37.190 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:37.190 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:37.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.190 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:2449 4.6 4:61283b7c:::rbd_data.114a1f60cb66.00000000000001f2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-24T15:32:37.190 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:23.027620+0000", 2024-04-24T15:32:37.190 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.111097915, 2024-04-24T15:32:37.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0053273959999999999, 2024-04-24T15:32:37.191 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:37.191 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:37.191 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:37.191 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:37.191 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:37.191 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2449 2024-04-24T15:32:37.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.191 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:37.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:37.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027620+0000", 2024-04-24T15:32:37.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:37.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:37.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027620+0000", 2024-04-24T15:32:37.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:37.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:37.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027619+0000", 2024-04-24T15:32:37.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:37.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:37.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027623+0000", 2024-04-24T15:32:37.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7100000000000001e-06 2024-04-24T15:32:37.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:37.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027636+0000", 2024-04-24T15:32:37.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2609000000000001e-05 2024-04-24T15:32:37.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:37.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027643+0000", 2024-04-24T15:32:37.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.182e-06 2024-04-24T15:32:37.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:37.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027678+0000", 2024-04-24T15:32:37.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5641e-05 2024-04-24T15:32:37.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:37.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027765+0000", 2024-04-24T15:32:37.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6691000000000001e-05 2024-04-24T15:32:37.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-24T15:32:37.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027864+0000", 2024-04-24T15:32:37.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9257000000000002e-05 2024-04-24T15:32:37.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:37.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.032638+0000", 2024-04-24T15:32:37.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004773429 2024-04-24T15:32:37.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:37.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.032896+0000", 2024-04-24T15:32:37.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000258352 2024-04-24T15:32:37.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:37.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.032924+0000", 2024-04-24T15:32:37.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8337999999999999e-05 2024-04-24T15:32:37.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:37.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.032947+0000", 2024-04-24T15:32:37.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2849000000000001e-05 2024-04-24T15:32:37.195 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:37.195 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:37.195 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:37.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.196 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4426.0:2767 4.6 e34/33 4:6297db09:::rbd_data.114a1f60cb66.00000000000000e1:head v 34'25, mlcod=34'23)", 2024-04-24T15:32:37.196 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.256572+0000", 2024-04-24T15:32:37.196 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 12.882145938000001, 2024-04-24T15:32:37.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0050987539999999996, 2024-04-24T15:32:37.196 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:37.196 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:37.196 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:37.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:37.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.256572+0000", 2024-04-24T15:32:37.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:37.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:37.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.256572+0000", 2024-04-24T15:32:37.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:37.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:37.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.256571+0000", 2024-04-24T15:32:37.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:37.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:37.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.256578+0000", 2024-04-24T15:32:37.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.161e-06 2024-04-24T15:32:37.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:37.198 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.256585+0000", 2024-04-24T15:32:37.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.391e-06 2024-04-24T15:32:37.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.198 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:37.198 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.256590+0000", 2024-04-24T15:32:37.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8189999999999999e-06 2024-04-24T15:32:37.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.198 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:37.198 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.256623+0000", 2024-04-24T15:32:37.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3618000000000002e-05 2024-04-24T15:32:37.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:37.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.256642+0000", 2024-04-24T15:32:37.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8479999999999999e-05 2024-04-24T15:32:37.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:37.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.261622+0000", 2024-04-24T15:32:37.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0049796959999999996 2024-04-24T15:32:37.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:37.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.261671+0000", 2024-04-24T15:32:37.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9147000000000002e-05 2024-04-24T15:32:37.199 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:37.199 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:37.199 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:37.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.200 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4426.0:3917 4.3 e34/33 4:c23f5710:::rbd_data.114a1f60cb66.0000000000000091:head v 34'28, mlcod=34'26)", 2024-04-24T15:32:37.200 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.804899+0000", 2024-04-24T15:32:37.200 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 12.333819289000001, 2024-04-24T15:32:37.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0050025249999999999, 2024-04-24T15:32:37.200 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:37.200 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:37.200 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:37.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:37.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.804899+0000", 2024-04-24T15:32:37.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:37.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:37.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.804899+0000", 2024-04-24T15:32:37.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:37.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:37.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.804898+0000", 2024-04-24T15:32:37.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:37.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:37.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.804903+0000", 2024-04-24T15:32:37.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9679999999999997e-06 2024-04-24T15:32:37.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:37.202 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.804911+0000", 2024-04-24T15:32:37.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8490000000000006e-06 2024-04-24T15:32:37.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:37.202 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.804916+0000", 2024-04-24T15:32:37.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5220000000000001e-06 2024-04-24T15:32:37.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:37.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.804958+0000", 2024-04-24T15:32:37.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2607000000000001e-05 2024-04-24T15:32:37.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:37.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.804975+0000", 2024-04-24T15:32:37.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6433999999999999e-05 2024-04-24T15:32:37.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:37.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.809868+0000", 2024-04-24T15:32:37.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0048933839999999998 2024-04-24T15:32:37.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:37.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.809901+0000", 2024-04-24T15:32:37.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3158999999999999e-05 2024-04-24T15:32:37.204 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:37.204 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:37.204 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:37.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.204 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4426.0:3934 4.6 e34/33 4:6297db09:::rbd_data.114a1f60cb66.00000000000000e1:head v 34'40, mlcod=34'37)", 2024-04-24T15:32:37.204 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.812857+0000", 2024-04-24T15:32:37.204 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 12.325861230999999, 2024-04-24T15:32:37.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0052230590000000004, 2024-04-24T15:32:37.204 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:37.204 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:37.204 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:37.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.205 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:37.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.812857+0000", 2024-04-24T15:32:37.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:37.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.205 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:37.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.812857+0000", 2024-04-24T15:32:37.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:37.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.205 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:37.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.812856+0000", 2024-04-24T15:32:37.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:37.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.206 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.206 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:37.206 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.812861+0000", 2024-04-24T15:32:37.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4e-06 2024-04-24T15:32:37.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.206 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.206 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:37.206 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.812871+0000", 2024-04-24T15:32:37.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8679999999999994e-06 2024-04-24T15:32:37.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.206 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.206 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:37.206 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.812877+0000", 2024-04-24T15:32:37.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5310000000000001e-06 2024-04-24T15:32:37.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.207 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:37.207 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.812913+0000", 2024-04-24T15:32:37.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6548999999999999e-05 2024-04-24T15:32:37.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.207 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:37.207 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.812934+0000", 2024-04-24T15:32:37.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0590000000000001e-05 2024-04-24T15:32:37.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.207 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:37.207 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.818047+0000", 2024-04-24T15:32:37.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0051134090000000002 2024-04-24T15:32:37.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:37.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.818080+0000", 2024-04-24T15:32:37.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2292000000000002e-05 2024-04-24T15:32:37.208 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:37.208 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:37.208 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:37.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.208 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4426.0:3943 4.6 e34/33 4:639bc13c:::rbd_data.114a1f60cb66.00000000000000ea:head v 34'41, mlcod=34'38)", 2024-04-24T15:32:37.208 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.815615+0000", 2024-04-24T15:32:37.208 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 12.323102939, 2024-04-24T15:32:37.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0050425490000000003, 2024-04-24T15:32:37.208 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:37.208 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:37.208 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:37.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:37.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.815615+0000", 2024-04-24T15:32:37.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:37.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:37.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.815615+0000", 2024-04-24T15:32:37.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:37.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:37.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.815614+0000", 2024-04-24T15:32:37.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:37.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.210 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:37.210 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.815620+0000", 2024-04-24T15:32:37.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1460000000000001e-06 2024-04-24T15:32:37.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.210 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:37.210 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.815632+0000", 2024-04-24T15:32:37.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2026e-05 2024-04-24T15:32:37.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.210 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:37.210 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.815638+0000", 2024-04-24T15:32:37.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8479999999999997e-06 2024-04-24T15:32:37.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.211 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:37.211 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.815678+0000", 2024-04-24T15:32:37.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9981000000000002e-05 2024-04-24T15:32:37.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.211 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:37.211 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.815702+0000", 2024-04-24T15:32:37.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3917000000000002e-05 2024-04-24T15:32:37.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.211 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:37.211 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.820633+0000", 2024-04-24T15:32:37.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0049306380000000002 2024-04-24T15:32:37.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:37.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.820657+0000", 2024-04-24T15:32:37.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.476e-05 2024-04-24T15:32:37.212 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:37.212 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:37.212 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:37.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.212 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4426.0:4005 4.6 e34/33 4:64968d7a:::rbd_data.114a1f60cb66.00000000000001bc:head v 34'45, mlcod=34'43)", 2024-04-24T15:32:37.212 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.842218+0000", 2024-04-24T15:32:37.212 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 12.296499684, 2024-04-24T15:32:37.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0052518060000000004, 2024-04-24T15:32:37.212 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:37.213 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:37.213 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:37.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:37.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.842218+0000", 2024-04-24T15:32:37.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:37.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:37.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.842218+0000", 2024-04-24T15:32:37.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:37.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:37.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.842218+0000", 2024-04-24T15:32:37.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:37.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:37.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.842223+0000", 2024-04-24T15:32:37.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0579999999999997e-06 2024-04-24T15:32:37.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:37.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.842231+0000", 2024-04-24T15:32:37.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5259999999999993e-06 2024-04-24T15:32:37.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:37.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.842235+0000", 2024-04-24T15:32:37.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0990000000000001e-06 2024-04-24T15:32:37.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:37.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.842271+0000", 2024-04-24T15:32:37.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5716000000000002e-05 2024-04-24T15:32:37.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:37.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.842291+0000", 2024-04-24T15:32:37.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9581000000000001e-05 2024-04-24T15:32:37.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:37.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.847434+0000", 2024-04-24T15:32:37.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0051435480000000004 2024-04-24T15:32:37.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:37.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.847470+0000", 2024-04-24T15:32:37.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5929000000000001e-05 2024-04-24T15:32:37.216 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:37.216 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:37.216 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:37.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.216 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4426.0:4119 4.6 e34/33 4:6588fd86:::rbd_data.114a1f60cb66.0000000000000179:head v 34'48, mlcod=34'45)", 2024-04-24T15:32:37.216 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.936723+0000", 2024-04-24T15:32:37.217 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 12.201994887, 2024-04-24T15:32:37.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0058208640000000002, 2024-04-24T15:32:37.217 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:37.217 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:37.217 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:37.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:37.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.936723+0000", 2024-04-24T15:32:37.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:37.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:37.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.936723+0000", 2024-04-24T15:32:37.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:37.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:37.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.936722+0000", 2024-04-24T15:32:37.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:37.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:37.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.936735+0000", 2024-04-24T15:32:37.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2153999999999999e-05 2024-04-24T15:32:37.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:37.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.936743+0000", 2024-04-24T15:32:37.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9360000000000005e-06 2024-04-24T15:32:37.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:37.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.936749+0000", 2024-04-24T15:32:37.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5040000000000002e-06 2024-04-24T15:32:37.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:37.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.936783+0000", 2024-04-24T15:32:37.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3689999999999998e-05 2024-04-24T15:32:37.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:37.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.936803+0000", 2024-04-24T15:32:37.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0047999999999998e-05 2024-04-24T15:32:37.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:37.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.942517+0000", 2024-04-24T15:32:37.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057144220000000003 2024-04-24T15:32:37.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:37.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.942544+0000", 2024-04-24T15:32:37.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6735999999999999e-05 2024-04-24T15:32:37.220 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:37.221 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:37.221 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:37.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.221 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4426.0:4353 4.3 e34/33 4:c38b6006:::rbd_data.114a1f60cb66.0000000000000175:head v 34'35, mlcod=34'33)", 2024-04-24T15:32:37.221 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:25.154297+0000", 2024-04-24T15:32:37.221 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 11.984420815, 2024-04-24T15:32:37.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0051580410000000004, 2024-04-24T15:32:37.221 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:37.221 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:37.221 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:37.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:37.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:25.154297+0000", 2024-04-24T15:32:37.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:37.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:37.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:25.154297+0000", 2024-04-24T15:32:37.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:37.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:37.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:25.154296+0000", 2024-04-24T15:32:37.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:37.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:37.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:25.154312+0000", 2024-04-24T15:32:37.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6271000000000001e-05 2024-04-24T15:32:37.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:37.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:25.154330+0000", 2024-04-24T15:32:37.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8076000000000001e-05 2024-04-24T15:32:37.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:37.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:25.154340+0000", 2024-04-24T15:32:37.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7210000000000006e-06 2024-04-24T15:32:37.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:37.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:25.154399+0000", 2024-04-24T15:32:37.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9082000000000001e-05 2024-04-24T15:32:37.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:37.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:25.154422+0000", 2024-04-24T15:32:37.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2569e-05 2024-04-24T15:32:37.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:37.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:25.159411+0000", 2024-04-24T15:32:37.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0049891609999999998 2024-04-24T15:32:37.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:37.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:37.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:37.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:25.159455+0000", 2024-04-24T15:32:37.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4288e-05 2024-04-24T15:32:37.225 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:37.225 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:37.225 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:37.225 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:37.225 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:37.225 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:32:37.225 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-24T15:32:37.227 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-24T15:32:37.227 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-04-24T15:32:37.227 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-04-24T15:32:37.228 INFO:teuthology.orchestra.run.smithi096.stdout:1276 write 0x7e2572c thru 0x7e2cb5e (0x7433 bytes) 2024-04-24T15:32:37.228 INFO:teuthology.orchestra.run.smithi096.stdout:1278 read 0x327d287 thru 0x328005e (0x2dd8 bytes) 2024-04-24T15:32:37.228 INFO:teuthology.orchestra.run.smithi096.stdout:1281 read 0x934b6f3 thru 0x934d562 (0x1e70 bytes) 2024-04-24T15:32:37.228 INFO:teuthology.orchestra.run.smithi096.stdout:1282 punch from 0x278a580 to 0x2796402, (0xbe82 bytes) 2024-04-24T15:32:37.228 INFO:teuthology.orchestra.run.smithi096.stdout:1283 trunc from 0xc8eab35 to 0x95b0b19 2024-04-24T15:32:37.229 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-04-24T15:32:37.229 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T15:32:37.230 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T15:32:37.230 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T15:32:37.230 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T15:32:37.230 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T15:32:37.230 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T15:32:37.230 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T15:32:37.230 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T15:32:37.231 INFO:teuthology.orchestra.run.smithi096.stdout:1284 read 0x7fda33f thru 0x7fdaf9c (0xc5e bytes) 2024-04-24T15:32:37.231 INFO:teuthology.orchestra.run.smithi096.stdout:1285 writesame 0x37e907c thru 0x37e9b4a data_size 0x1(0xacf bytes) 2024-04-24T15:32:37.231 INFO:teuthology.orchestra.run.smithi096.stdout:1286 read 0x44f2f0 thru 0x45dd4e (0xea5f bytes) 2024-04-24T15:32:37.231 INFO:teuthology.orchestra.run.smithi096.stdout:1287 write 0x43e8cab thru 0x43f1639 (0x898f bytes) 2024-04-24T15:32:37.231 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:37.232 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:32:37.228+0000 7f23abcf6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 44021) UID: 0 2024-04-24T15:32:37.232 INFO:teuthology.orchestra.run.smithi096.stdout:1289 writesame 0xcdcf6fa thru 0xcdd051f data_size 0x1(0xe26 bytes) 2024-04-24T15:32:37.232 INFO:teuthology.orchestra.run.smithi096.stdout:1292 read 0x695f231 thru 0x69661ec (0x6fbc bytes) 2024-04-24T15:32:37.232 INFO:teuthology.orchestra.run.smithi096.stdout:1294 write 0x59430db thru 0x59527a4 (0xf6ca bytes) 2024-04-24T15:32:37.239 INFO:teuthology.orchestra.run.smithi096.stdout:1296 write 0x46b6f56 thru 0x46c0f2d (0x9fd8 bytes) 2024-04-24T15:32:37.244 INFO:teuthology.orchestra.run.smithi096.stdout:1297 read 0x3717635 thru 0x3727587 (0xff53 bytes) 2024-04-24T15:32:37.245 INFO:teuthology.orchestra.run.smithi096.stdout:1298 write 0x7aeab50 thru 0x7af73c3 (0xc874 bytes) 2024-04-24T15:32:37.247 INFO:teuthology.orchestra.run.smithi096.stdout:1299 writesame 0x99a511a thru 0x99b3718 data_size 0x1(0xe5ff bytes) 2024-04-24T15:32:37.263 INFO:teuthology.orchestra.run.smithi096.stdout:1301 write 0xc82b5e9 thru 0xc82f025 (0x3a3d bytes) 2024-04-24T15:32:37.285 INFO:teuthology.orchestra.run.smithi096.stdout:1302 trunc from 0xcdd0520 to 0x1572f40 2024-04-24T15:32:37.327 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:32:37.328+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:32:37.329 INFO:teuthology.orchestra.run.smithi096.stdout:1303 write 0x9a22fcf thru 0x9a23d13 (0xd45 bytes) 2024-04-24T15:32:37.332 INFO:teuthology.orchestra.run.smithi096.stdout:1306 read 0x15bfc3d thru 0x15c1f69 (0x232d bytes) 2024-04-24T15:32:37.332 INFO:teuthology.orchestra.run.smithi096.stdout:1308 writesame 0x5225d29 thru 0x522841d data_size 0x1(0x26f5 bytes) 2024-04-24T15:32:37.345 INFO:teuthology.orchestra.run.smithi096.stdout:1309 punch from 0x6c85868 to 0x6c8a05a, (0x47f2 bytes) 2024-04-24T15:32:37.351 INFO:teuthology.orchestra.run.smithi096.stdout:1310 trunc from 0x9a23d14 to 0x5467695 2024-04-24T15:32:37.380 INFO:teuthology.orchestra.run.smithi096.stdout:1311 write 0xa42c088 thru 0xa435b65 (0x9ade bytes) 2024-04-24T15:32:37.381 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:32:37.381 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:32:37.381 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-04-24T15:32:37.381 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:32:37.390 INFO:teuthology.orchestra.run.smithi096.stdout:1312 write 0x61205ae thru 0x6125a4e (0x54a1 bytes) 2024-04-24T15:32:37.392 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-24T15:32:37.408 INFO:teuthology.orchestra.run.smithi096.stdout:1313 writesame 0xeb28c14 thru 0xeb31412 data_size 0x37(0x87ff bytes) 2024-04-24T15:32:37.408 INFO:teuthology.orchestra.run.smithi096.stdout:1315 punch from 0x640dbe9 to 0x641be5d, (0xe274 bytes) 2024-04-24T15:32:37.409 INFO:teuthology.orchestra.run.smithi096.stdout:1319 read 0xc015c43 thru 0xc02119e (0xb55c bytes) 2024-04-24T15:32:37.409 INFO:teuthology.orchestra.run.smithi096.stdout:1320 write 0x89671d thru 0x8a27a3 (0xc087 bytes) 2024-04-24T15:32:37.410 INFO:teuthology.orchestra.run.smithi096.stdout:1321 write 0x6a79077 thru 0x6a84530 (0xb4ba bytes) 2024-04-24T15:32:37.419 INFO:teuthology.orchestra.run.smithi096.stdout:1322 punch from 0x4097a3b to 0x409b0a3, (0x3668 bytes) 2024-04-24T15:32:37.425 INFO:teuthology.orchestra.run.smithi096.stdout:1324 trunc from 0xeb31413 to 0xa011006 2024-04-24T15:32:37.428 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:32:37.429+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:32:37.432 INFO:teuthology.orchestra.run.smithi096.stdout:1325 punch from 0x13e9dc6 to 0x13eab00, (0xd3a bytes) 2024-04-24T15:32:37.432 INFO:teuthology.orchestra.run.smithi096.stdout:1326 write 0x9b4a8f4 thru 0x9b4c8c2 (0x1fcf bytes) 2024-04-24T15:32:37.435 INFO:teuthology.orchestra.run.smithi096.stdout:1328 writesame 0x2eff9a0 thru 0x2f053f0 data_size 0x15(0x5a51 bytes) 2024-04-24T15:32:37.440 INFO:teuthology.orchestra.run.smithi096.stdout:1329 read 0x418a707 thru 0x418ea37 (0x4331 bytes) 2024-04-24T15:32:37.440 INFO:teuthology.orchestra.run.smithi096.stdout:1330 writesame 0x98e219 thru 0x98f25a data_size 0x1(0x1042 bytes) 2024-04-24T15:32:37.442 INFO:teuthology.orchestra.run.smithi096.stdout:1331 punch from 0x1641518 to 0x164d307, (0xbdef bytes) 2024-04-24T15:32:37.453 INFO:teuthology.orchestra.run.smithi096.stdout:1332 write 0xdfe5ee0 thru 0xdfee247 (0x8368 bytes) 2024-04-24T15:32:37.463 INFO:teuthology.orchestra.run.smithi096.stdout:1333 trunc from 0xdfee248 to 0x9dd0279 2024-04-24T15:32:37.483 INFO:teuthology.orchestra.run.smithi096.stdout:1334 write 0xbafd079 thru 0xbb0270f (0x5697 bytes) 2024-04-24T15:32:37.489 INFO:teuthology.orchestra.run.smithi096.stdout:1335 write 0x51e682f thru 0x51f4320 (0xdaf2 bytes) 2024-04-24T15:32:37.494 INFO:teuthology.orchestra.run.smithi096.stdout:1336 writesame 0x417a7b2 thru 0x417e5ba data_size 0x1(0x3e09 bytes) 2024-04-24T15:32:37.502 INFO:teuthology.orchestra.run.smithi096.stdout:1338 punch from 0x449ec6b to 0x449fe2d, (0x11c2 bytes) 2024-04-24T15:32:37.502 INFO:teuthology.orchestra.run.smithi096.stdout:1339 read 0x5e63bc5 thru 0x5e66794 (0x2bd0 bytes) 2024-04-24T15:32:37.503 INFO:teuthology.orchestra.run.smithi096.stdout:1340 read 0x979b9c9 thru 0x97a5b80 (0xa1b8 bytes) 2024-04-24T15:32:37.503 INFO:teuthology.orchestra.run.smithi096.stdout:1341 read 0x1fc1b5 thru 0x2058c6 (0x9712 bytes) 2024-04-24T15:32:37.512 INFO:teuthology.orchestra.run.smithi096.stdout:1343 write 0xc7241a3 thru 0xc732c6d (0xeacb bytes) 2024-04-24T15:32:37.522 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:37.522 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:37.523 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:37.523 INFO:teuthology.orchestra.run.smithi096.stdout:1347 read 0xa10ffbd thru 0xa116100 (0x6144 bytes) 2024-04-24T15:32:37.529 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:32:37.530+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:32:37.531 INFO:teuthology.orchestra.run.smithi096.stdout:1348 trunc from 0xc732c6e to 0x5e90b93 2024-04-24T15:32:37.545 INFO:teuthology.orchestra.run.smithi096.stdout:1349 write 0x18e8f98 thru 0x18ede6a (0x4ed3 bytes) 2024-04-24T15:32:37.557 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:32:37.557 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:32:37.558 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-04-24T15:32:37.558 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-04-24T15:32:37.558 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:32:37.561 INFO:teuthology.orchestra.run.smithi096.stdout:1350 write 0xc0af622 thru 0xc0bba99 (0xc478 bytes) 2024-04-24T15:32:37.569 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-24T15:32:37.572 INFO:teuthology.orchestra.run.smithi096.stdout:1351 write 0xba1dfe4 thru 0xba2a92f (0xc94c bytes) 2024-04-24T15:32:37.578 INFO:teuthology.orchestra.run.smithi096.stdout:1352 punch from 0x8033d38 to 0x8036ef0, (0x31b8 bytes) 2024-04-24T15:32:37.582 INFO:teuthology.orchestra.run.smithi096.stdout:1354 trunc from 0xc0bba9a to 0x8c2b2ae 2024-04-24T15:32:37.594 INFO:teuthology.orchestra.run.smithi096.stdout:1355 punch from 0x47d2eef to 0x47d7420, (0x4531 bytes) 2024-04-24T15:32:37.594 INFO:teuthology.orchestra.run.smithi096.stdout:1356 write 0x4122038 thru 0x4130f33 (0xeefc bytes) 2024-04-24T15:32:37.605 INFO:teuthology.orchestra.run.smithi096.stdout:1358 write 0x9b6baab thru 0x9b6c1d5 (0x72b bytes) 2024-04-24T15:32:37.608 INFO:teuthology.orchestra.run.smithi096.stdout:1359 read 0x79bf3cd thru 0x79cbf93 (0xcbc7 bytes) 2024-04-24T15:32:37.609 INFO:teuthology.orchestra.run.smithi096.stdout:1360 punch from 0x30b1e34 to 0x30bf527, (0xd6f3 bytes) 2024-04-24T15:32:37.609 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:37.609 INFO:teuthology.orchestra.run.smithi096.stdout:1363 writesame 0x49fcaa3 thru 0x4a05a9c data_size 0x3(0x8ffa bytes) 2024-04-24T15:32:37.616 INFO:teuthology.orchestra.run.smithi096.stdout:1365 punch from 0x63d8e64 to 0x63db837, (0x29d3 bytes) 2024-04-24T15:32:37.630 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:32:37.631+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:32:37.704 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:32:37.704 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-04-24T15:32:37.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-04-24T15:32:37.704 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [] 2024-04-24T15:32:37.704 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:32:37.713 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-24T15:32:37.755 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:32:37.756+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:32:37.832 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:32:37.834+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:32:37.879 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:32:37.879 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:32:37.879 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-04-24T15:32:37.879 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:32:37.889 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-24T15:32:37.933 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:32:37.934+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:32:38.011 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:32:38.011 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:32:38.011 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-04-24T15:32:38.011 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-04-24T15:32:38.011 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:32:38.021 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-24T15:32:38.062 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:32:38.063+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:32:38.135 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:32:38.136+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:32:38.188 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:32:38.188 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-04-24T15:32:38.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-04-24T15:32:38.188 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-04-24T15:32:38.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.188 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4426.0:2434 4.d e32/26 4:b7840674:::rbd_data.114a1f60cb66.00000000000001a7:head v 32'12, mlcod=32'10)", 2024-04-24T15:32:38.188 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:23.024172+0000", 2024-04-24T15:32:38.189 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 15.163454132, 2024-04-24T15:32:38.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057919520000000004, 2024-04-24T15:32:38.189 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:38.189 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:38.189 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:38.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:38.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.024172+0000", 2024-04-24T15:32:38.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:38.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.024172+0000", 2024-04-24T15:32:38.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:38.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.024171+0000", 2024-04-24T15:32:38.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:38.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:38.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.024178+0000", 2024-04-24T15:32:38.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2420000000000003e-06 2024-04-24T15:32:38.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:38.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.024191+0000", 2024-04-24T15:32:38.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2444e-05 2024-04-24T15:32:38.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:38.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.024197+0000", 2024-04-24T15:32:38.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2249999999999997e-06 2024-04-24T15:32:38.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:38.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.024251+0000", 2024-04-24T15:32:38.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4221000000000001e-05 2024-04-24T15:32:38.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:38.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.024270+0000", 2024-04-24T15:32:38.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9151999999999999e-05 2024-04-24T15:32:38.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:38.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.029925+0000", 2024-04-24T15:32:38.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0056546160000000003 2024-04-24T15:32:38.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:38.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.029964+0000", 2024-04-24T15:32:38.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8855e-05 2024-04-24T15:32:38.192 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.192 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:38.192 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.193 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:2450 4.1c 4:3f9dbb33:::rbd_data.114a1f60cb66.00000000000001f3:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-24T15:32:38.193 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:23.027406+0000", 2024-04-24T15:32:38.193 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 15.160219759, 2024-04-24T15:32:38.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.005672379, 2024-04-24T15:32:38.193 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:38.193 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:38.193 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:38.193 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:38.193 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:38.193 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2450 2024-04-24T15:32:38.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.193 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:38.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:38.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027406+0000", 2024-04-24T15:32:38.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:38.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027406+0000", 2024-04-24T15:32:38.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:38.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027405+0000", 2024-04-24T15:32:38.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:38.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:38.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027411+0000", 2024-04-24T15:32:38.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7610000000000004e-06 2024-04-24T15:32:38.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:38.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027422+0000", 2024-04-24T15:32:38.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1462e-05 2024-04-24T15:32:38.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:38.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027429+0000", 2024-04-24T15:32:38.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3339999999999998e-06 2024-04-24T15:32:38.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:38.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027478+0000", 2024-04-24T15:32:38.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9804e-05 2024-04-24T15:32:38.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:38.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027559+0000", 2024-04-24T15:32:38.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1068000000000004e-05 2024-04-24T15:32:38.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:38.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027672+0000", 2024-04-24T15:32:38.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000112632 2024-04-24T15:32:38.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:38.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.032737+0000", 2024-04-24T15:32:38.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0050646429999999997 2024-04-24T15:32:38.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:38.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.033029+0000", 2024-04-24T15:32:38.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00029226300000000002 2024-04-24T15:32:38.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:38.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.033057+0000", 2024-04-24T15:32:38.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7872e-05 2024-04-24T15:32:38.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:38.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.033078+0000", 2024-04-24T15:32:38.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.158e-05 2024-04-24T15:32:38.198 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.198 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:38.198 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.198 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4426.0:2510 4.12 e33/27 4:4fee0517:::rbd_data.114a1f60cb66.000000000000029d:head v 33'19, mlcod=32'17)", 2024-04-24T15:32:38.198 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:23.078201+0000", 2024-04-24T15:32:38.198 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 15.109424633, 2024-04-24T15:32:38.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0060097689999999999, 2024-04-24T15:32:38.198 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:38.198 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:38.198 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:38.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.198 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:38.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.078201+0000", 2024-04-24T15:32:38.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:38.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.078201+0000", 2024-04-24T15:32:38.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:38.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.078201+0000", 2024-04-24T15:32:38.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:38.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:38.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.078213+0000", 2024-04-24T15:32:38.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2261e-05 2024-04-24T15:32:38.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:38.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.078223+0000", 2024-04-24T15:32:38.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4709999999999996e-06 2024-04-24T15:32:38.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:38.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.078230+0000", 2024-04-24T15:32:38.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8979999999999999e-06 2024-04-24T15:32:38.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:38.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.079842+0000", 2024-04-24T15:32:38.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001612048 2024-04-24T15:32:38.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:38.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.079870+0000", 2024-04-24T15:32:38.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8487e-05 2024-04-24T15:32:38.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:38.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.084169+0000", 2024-04-24T15:32:38.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0042991490000000004 2024-04-24T15:32:38.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:38.202 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.084211+0000", 2024-04-24T15:32:38.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1792000000000002e-05 2024-04-24T15:32:38.202 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.202 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:38.202 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.202 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:2522 4.6 4:60ceedf0:::rbd_data.114a1f60cb66.000000000000011d:head [set-alloc-hint object_size 262144 write_size 262144,write 110592~2048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-24T15:32:38.202 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:23.088240+0000", 2024-04-24T15:32:38.203 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 15.099385719000001, 2024-04-24T15:32:38.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.988232797, 2024-04-24T15:32:38.203 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:38.203 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:38.203 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:38.203 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:38.203 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:38.203 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2522 2024-04-24T15:32:38.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.203 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:38.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:38.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088240+0000", 2024-04-24T15:32:38.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:38.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088240+0000", 2024-04-24T15:32:38.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:38.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088238+0000", 2024-04-24T15:32:38.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999981 2024-04-24T15:32:38.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:38.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088254+0000", 2024-04-24T15:32:38.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5562000000000001e-05 2024-04-24T15:32:38.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.205 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:38.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088275+0000", 2024-04-24T15:32:38.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1256000000000001e-05 2024-04-24T15:32:38.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.205 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:38.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088283+0000", 2024-04-24T15:32:38.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3219999999999993e-06 2024-04-24T15:32:38.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.206 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.206 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:38.206 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088351+0000", 2024-04-24T15:32:38.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7595000000000006e-05 2024-04-24T15:32:38.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.206 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.206 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-04-24T15:32:38.206 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088362+0000", 2024-04-24T15:32:38.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1764e-05 2024-04-24T15:32:38.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.206 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.206 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:38.206 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.073748+0000", 2024-04-24T15:32:38.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98538541899999998 2024-04-24T15:32:38.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.207 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:38.207 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.073925+0000", 2024-04-24T15:32:38.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017747600000000001 2024-04-24T15:32:38.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.207 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:38.207 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.074092+0000", 2024-04-24T15:32:38.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000166622 2024-04-24T15:32:38.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.207 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:38.207 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.074774+0000", 2024-04-24T15:32:38.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00068225000000000002 2024-04-24T15:32:38.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:38.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.076412+0000", 2024-04-24T15:32:38.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001638189 2024-04-24T15:32:38.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:38.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.076446+0000", 2024-04-24T15:32:38.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3587000000000002e-05 2024-04-24T15:32:38.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:38.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.076473+0000", 2024-04-24T15:32:38.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6886999999999999e-05 2024-04-24T15:32:38.209 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.209 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:38.209 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.209 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:2755 4.1c 4:3c8baac7:::rbd_data.114a1f60cb66.00000000000000a8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:38.209 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.252831+0000", 2024-04-24T15:32:38.209 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.934794905, 2024-04-24T15:32:38.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057169500000000002, 2024-04-24T15:32:38.209 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:38.209 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:38.209 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:38.209 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:38.210 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:38.210 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2755 2024-04-24T15:32:38.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.210 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:38.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.210 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:38.210 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.252831+0000", 2024-04-24T15:32:38.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.210 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:38.210 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.252831+0000", 2024-04-24T15:32:38.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.210 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:38.211 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.252830+0000", 2024-04-24T15:32:38.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:38.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.211 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:38.211 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.252835+0000", 2024-04-24T15:32:38.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3739999999999997e-06 2024-04-24T15:32:38.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.211 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:38.211 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.252846+0000", 2024-04-24T15:32:38.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.039e-05 2024-04-24T15:32:38.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.211 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:38.211 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.252850+0000", 2024-04-24T15:32:38.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4100000000000001e-06 2024-04-24T15:32:38.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:38.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.252885+0000", 2024-04-24T15:32:38.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5213999999999999e-05 2024-04-24T15:32:38.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:38.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.252957+0000", 2024-04-24T15:32:38.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2285000000000003e-05 2024-04-24T15:32:38.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:38.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.253053+0000", 2024-04-24T15:32:38.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5841999999999995e-05 2024-04-24T15:32:38.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:38.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.255905+0000", 2024-04-24T15:32:38.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028514539999999998 2024-04-24T15:32:38.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:38.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.258474+0000", 2024-04-24T15:32:38.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0025695679999999999 2024-04-24T15:32:38.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:38.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.258518+0000", 2024-04-24T15:32:38.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3242999999999999e-05 2024-04-24T15:32:38.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:38.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.258548+0000", 2024-04-24T15:32:38.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0318999999999999e-05 2024-04-24T15:32:38.214 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.214 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:38.214 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.214 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4426.0:2761 4.19 e34/26 4:9e865ec5:::rbd_data.114a1f60cb66.00000000000000c0:head v 34'14, mlcod=34'12)", 2024-04-24T15:32:38.214 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.255783+0000", 2024-04-24T15:32:38.215 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.931843307999999, 2024-04-24T15:32:38.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057401149999999996, 2024-04-24T15:32:38.215 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:38.215 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:38.215 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:38.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:38.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.255783+0000", 2024-04-24T15:32:38.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:38.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.255783+0000", 2024-04-24T15:32:38.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:38.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.255782+0000", 2024-04-24T15:32:38.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:38.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:38.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.255794+0000", 2024-04-24T15:32:38.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2429000000000001e-05 2024-04-24T15:32:38.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:38.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.255802+0000", 2024-04-24T15:32:38.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2139999999999999e-06 2024-04-24T15:32:38.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:38.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.255807+0000", 2024-04-24T15:32:38.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6500000000000001e-06 2024-04-24T15:32:38.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:38.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.255846+0000", 2024-04-24T15:32:38.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8497000000000001e-05 2024-04-24T15:32:38.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:38.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.255867+0000", 2024-04-24T15:32:38.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0733999999999998e-05 2024-04-24T15:32:38.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:38.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.261440+0000", 2024-04-24T15:32:38.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0055733099999999997 2024-04-24T15:32:38.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:38.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.261523+0000", 2024-04-24T15:32:38.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2764999999999998e-05 2024-04-24T15:32:38.218 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.219 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:38.219 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.219 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:2767 4.6 4:6297db09:::rbd_data.114a1f60cb66.00000000000000e1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:38.219 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.256201+0000", 2024-04-24T15:32:38.219 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.931424768999999, 2024-04-24T15:32:38.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0056859040000000003, 2024-04-24T15:32:38.219 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:38.219 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:38.219 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:38.219 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:38.219 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:38.219 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2767 2024-04-24T15:32:38.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.220 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:38.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:38.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.256201+0000", 2024-04-24T15:32:38.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:38.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.256201+0000", 2024-04-24T15:32:38.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:38.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.256200+0000", 2024-04-24T15:32:38.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:38.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:38.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.256205+0000", 2024-04-24T15:32:38.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.587e-06 2024-04-24T15:32:38.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:38.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.256218+0000", 2024-04-24T15:32:38.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2362e-05 2024-04-24T15:32:38.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:38.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.256224+0000", 2024-04-24T15:32:38.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6120000000000004e-06 2024-04-24T15:32:38.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:38.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.256274+0000", 2024-04-24T15:32:38.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9084e-05 2024-04-24T15:32:38.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:38.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.256392+0000", 2024-04-24T15:32:38.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000118127 2024-04-24T15:32:38.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:38.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.256508+0000", 2024-04-24T15:32:38.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000116777 2024-04-24T15:32:38.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:38.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.261537+0000", 2024-04-24T15:32:38.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0050290380000000004 2024-04-24T15:32:38.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:38.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.261801+0000", 2024-04-24T15:32:38.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00026362400000000001 2024-04-24T15:32:38.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:38.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.261848+0000", 2024-04-24T15:32:38.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6700999999999999e-05 2024-04-24T15:32:38.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:38.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.261887+0000", 2024-04-24T15:32:38.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9159999999999998e-05 2024-04-24T15:32:38.224 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.224 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:38.224 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.224 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:3772 4.4 4:24a7cfbc:::rbd_data.114a1f60cb66.0000000000000020:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:38.224 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.699611+0000", 2024-04-24T15:32:38.224 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.488014889, 2024-04-24T15:32:38.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0060871340000000001, 2024-04-24T15:32:38.225 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:38.225 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:38.225 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:38.225 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:38.225 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:38.225 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3772 2024-04-24T15:32:38.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.225 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:38.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:38.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.699611+0000", 2024-04-24T15:32:38.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:38.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.699611+0000", 2024-04-24T15:32:38.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:38.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.699609+0000", 2024-04-24T15:32:38.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999981 2024-04-24T15:32:38.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:38.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.699618+0000", 2024-04-24T15:32:38.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3859999999999996e-06 2024-04-24T15:32:38.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:38.227 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.699637+0000", 2024-04-24T15:32:38.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8688000000000001e-05 2024-04-24T15:32:38.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:38.227 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.699647+0000", 2024-04-24T15:32:38.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.029e-05 2024-04-24T15:32:38.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:38.227 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.699692+0000", 2024-04-24T15:32:38.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4400000000000002e-05 2024-04-24T15:32:38.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.228 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:38.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.699751+0000", 2024-04-24T15:32:38.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8953999999999998e-05 2024-04-24T15:32:38.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.228 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:38.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.700249+0000", 2024-04-24T15:32:38.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00049869900000000002 2024-04-24T15:32:38.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.228 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:38.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.705560+0000", 2024-04-24T15:32:38.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0053110550000000003 2024-04-24T15:32:38.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:38.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.705602+0000", 2024-04-24T15:32:38.229 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1689e-05 2024-04-24T15:32:38.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:38.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.705652+0000", 2024-04-24T15:32:38.229 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9740000000000001e-05 2024-04-24T15:32:38.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:38.230 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.705698+0000", 2024-04-24T15:32:38.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6329000000000003e-05 2024-04-24T15:32:38.230 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.230 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:38.230 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.230 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:3774 4.4 4:25b094fe:::rbd_data.114a1f60cb66.0000000000000022:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:38.230 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.699669+0000", 2024-04-24T15:32:38.230 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.487956757999999, 2024-04-24T15:32:38.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.006296678, 2024-04-24T15:32:38.230 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:38.230 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:38.230 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:38.231 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:38.231 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:38.231 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3774 2024-04-24T15:32:38.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.231 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:38.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.231 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:38.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.699669+0000", 2024-04-24T15:32:38.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.231 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:38.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.699669+0000", 2024-04-24T15:32:38.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.232 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:38.232 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.699669+0000", 2024-04-24T15:32:38.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:38.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.232 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:38.232 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.699672+0000", 2024-04-24T15:32:38.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2830000000000002e-06 2024-04-24T15:32:38.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.232 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:38.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.699677+0000", 2024-04-24T15:32:38.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7530000000000001e-06 2024-04-24T15:32:38.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:38.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.699680+0000", 2024-04-24T15:32:38.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0130000000000002e-06 2024-04-24T15:32:38.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:38.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.702736+0000", 2024-04-24T15:32:38.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0030567379999999998 2024-04-24T15:32:38.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:38.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.702797+0000", 2024-04-24T15:32:38.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0563000000000003e-05 2024-04-24T15:32:38.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:38.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.703297+0000", 2024-04-24T15:32:38.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00049955300000000002 2024-04-24T15:32:38.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:38.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.705595+0000", 2024-04-24T15:32:38.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002298076 2024-04-24T15:32:38.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:38.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.705870+0000", 2024-04-24T15:32:38.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000275285 2024-04-24T15:32:38.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:38.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.705923+0000", 2024-04-24T15:32:38.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3229999999999997e-05 2024-04-24T15:32:38.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:38.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.705966+0000", 2024-04-24T15:32:38.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2636999999999999e-05 2024-04-24T15:32:38.235 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.236 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:38.236 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.236 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:3795 4.1c 4:3a2027c5:::rbd_data.114a1f60cb66.0000000000000025:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:38.236 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.723461+0000", 2024-04-24T15:32:38.236 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.464164901, 2024-04-24T15:32:38.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0066452719999999998, 2024-04-24T15:32:38.236 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:38.236 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:38.236 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:38.236 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:38.236 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:38.236 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3795 2024-04-24T15:32:38.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.237 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:38.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:38.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.723461+0000", 2024-04-24T15:32:38.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:38.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.723461+0000", 2024-04-24T15:32:38.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:38.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.723459+0000", 2024-04-24T15:32:38.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999986 2024-04-24T15:32:38.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.238 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:38.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.723470+0000", 2024-04-24T15:32:38.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0097e-05 2024-04-24T15:32:38.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.238 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:38.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.723484+0000", 2024-04-24T15:32:38.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4338000000000001e-05 2024-04-24T15:32:38.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.238 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:38.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.723496+0000", 2024-04-24T15:32:38.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2125e-05 2024-04-24T15:32:38.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:38.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.723547+0000", 2024-04-24T15:32:38.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0909000000000003e-05 2024-04-24T15:32:38.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:38.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.723639+0000", 2024-04-24T15:32:38.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1844999999999999e-05 2024-04-24T15:32:38.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:38.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.724463+0000", 2024-04-24T15:32:38.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00082389099999999999 2024-04-24T15:32:38.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:38.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.728863+0000", 2024-04-24T15:32:38.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0044007610000000004 2024-04-24T15:32:38.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:38.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.730017+0000", 2024-04-24T15:32:38.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0011534410000000001 2024-04-24T15:32:38.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:38.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.730067+0000", 2024-04-24T15:32:38.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0430000000000003e-05 2024-04-24T15:32:38.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:38.241 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.730106+0000", 2024-04-24T15:32:38.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8902000000000001e-05 2024-04-24T15:32:38.241 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.241 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:38.241 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.241 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:3976 4.1c 4:396ab59e:::rbd_data.114a1f60cb66.0000000000000160:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:38.241 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.827920+0000", 2024-04-24T15:32:38.241 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.359706020000001, 2024-04-24T15:32:38.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057086940000000003, 2024-04-24T15:32:38.242 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:38.242 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:38.242 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:38.242 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:38.242 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:38.242 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3976 2024-04-24T15:32:38.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.242 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:38.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:38.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.827920+0000", 2024-04-24T15:32:38.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:38.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.827920+0000", 2024-04-24T15:32:38.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:38.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.827919+0000", 2024-04-24T15:32:38.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:38.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:38.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.827924+0000", 2024-04-24T15:32:38.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4270000000000001e-06 2024-04-24T15:32:38.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:38.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.827934+0000", 2024-04-24T15:32:38.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0606e-05 2024-04-24T15:32:38.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:38.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.827939+0000", 2024-04-24T15:32:38.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.792e-06 2024-04-24T15:32:38.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:38.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.827967+0000", 2024-04-24T15:32:38.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7919000000000002e-05 2024-04-24T15:32:38.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:38.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.828011+0000", 2024-04-24T15:32:38.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3896000000000001e-05 2024-04-24T15:32:38.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:38.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.828070+0000", 2024-04-24T15:32:38.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9348999999999997e-05 2024-04-24T15:32:38.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:38.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.830513+0000", 2024-04-24T15:32:38.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024429669999999999 2024-04-24T15:32:38.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.246 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:38.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.833567+0000", 2024-04-24T15:32:38.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0030542590000000001 2024-04-24T15:32:38.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.246 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:38.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.833608+0000", 2024-04-24T15:32:38.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1148e-05 2024-04-24T15:32:38.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.246 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:38.247 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.833629+0000", 2024-04-24T15:32:38.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0066999999999999e-05 2024-04-24T15:32:38.247 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.247 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:38.247 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.247 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:3987 4.1c 4:3a30a8f2:::rbd_data.114a1f60cb66.0000000000000174:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:38.247 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.833638+0000", 2024-04-24T15:32:38.247 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.353987740000001, 2024-04-24T15:32:38.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057825430000000002, 2024-04-24T15:32:38.247 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:38.247 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:38.247 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:38.247 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:38.248 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:38.248 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3987 2024-04-24T15:32:38.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.248 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:38.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:38.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.833638+0000", 2024-04-24T15:32:38.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:38.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.833638+0000", 2024-04-24T15:32:38.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:38.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.833637+0000", 2024-04-24T15:32:38.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:38.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:38.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.833646+0000", 2024-04-24T15:32:38.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0180000000000007e-06 2024-04-24T15:32:38.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:38.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.833659+0000", 2024-04-24T15:32:38.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.313e-05 2024-04-24T15:32:38.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:38.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.833664+0000", 2024-04-24T15:32:38.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.006e-06 2024-04-24T15:32:38.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:38.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.833697+0000", 2024-04-24T15:32:38.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2978000000000002e-05 2024-04-24T15:32:38.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:38.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.833756+0000", 2024-04-24T15:32:38.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8811000000000003e-05 2024-04-24T15:32:38.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:38.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.833823+0000", 2024-04-24T15:32:38.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7225999999999996e-05 2024-04-24T15:32:38.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:38.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.838767+0000", 2024-04-24T15:32:38.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0049435809999999998 2024-04-24T15:32:38.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:38.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.839341+0000", 2024-04-24T15:32:38.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00057366800000000005 2024-04-24T15:32:38.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:38.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.839385+0000", 2024-04-24T15:32:38.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4597e-05 2024-04-24T15:32:38.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:38.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.839421+0000", 2024-04-24T15:32:38.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5516999999999999e-05 2024-04-24T15:32:38.252 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.253 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:38.253 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.253 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:4005 4.6 4:64968d7a:::rbd_data.114a1f60cb66.00000000000001bc:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:38.253 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.841940+0000", 2024-04-24T15:32:38.253 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.345685481, 2024-04-24T15:32:38.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.005717481, 2024-04-24T15:32:38.253 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:38.253 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:38.253 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:38.253 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:38.253 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:38.254 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4005 2024-04-24T15:32:38.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.254 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:38.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:38.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.841940+0000", 2024-04-24T15:32:38.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:38.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.841940+0000", 2024-04-24T15:32:38.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:38.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.841940+0000", 2024-04-24T15:32:38.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:38.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:38.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.841944+0000", 2024-04-24T15:32:38.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.904e-06 2024-04-24T15:32:38.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:38.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.841954+0000", 2024-04-24T15:32:38.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7499999999999998e-06 2024-04-24T15:32:38.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:38.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.841960+0000", 2024-04-24T15:32:38.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4399999999999996e-06 2024-04-24T15:32:38.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:38.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.841991+0000", 2024-04-24T15:32:38.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1615000000000001e-05 2024-04-24T15:32:38.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:38.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.842067+0000", 2024-04-24T15:32:38.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6002999999999997e-05 2024-04-24T15:32:38.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:38.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.842168+0000", 2024-04-24T15:32:38.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000100474 2024-04-24T15:32:38.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:38.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.847470+0000", 2024-04-24T15:32:38.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0053021830000000002 2024-04-24T15:32:38.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:38.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.847577+0000", 2024-04-24T15:32:38.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000107425 2024-04-24T15:32:38.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:38.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.847622+0000", 2024-04-24T15:32:38.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5015999999999997e-05 2024-04-24T15:32:38.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:38.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.847658+0000", 2024-04-24T15:32:38.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5509999999999997e-05 2024-04-24T15:32:38.258 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.258 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:38.259 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.259 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:4119 4.6 4:6588fd86:::rbd_data.114a1f60cb66.0000000000000179:head [set-alloc-hint object_size 262144 write_size 262144,write 151552~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:38.259 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.935980+0000", 2024-04-24T15:32:38.259 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.251645612000001, 2024-04-24T15:32:38.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.006739391, 2024-04-24T15:32:38.259 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:38.259 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:38.259 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:38.259 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:38.259 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:38.260 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4119 2024-04-24T15:32:38.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.260 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:38.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:38.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.935980+0000", 2024-04-24T15:32:38.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:38.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.935980+0000", 2024-04-24T15:32:38.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:38.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.935980+0000", 2024-04-24T15:32:38.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:38.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:38.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.935987+0000", 2024-04-24T15:32:38.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1590000000000004e-06 2024-04-24T15:32:38.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:38.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.935996+0000", 2024-04-24T15:32:38.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8389999999999996e-06 2024-04-24T15:32:38.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:38.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.936001+0000", 2024-04-24T15:32:38.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6890000000000001e-06 2024-04-24T15:32:38.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:38.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.936456+0000", 2024-04-24T15:32:38.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00045432600000000001 2024-04-24T15:32:38.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:38.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.936530+0000", 2024-04-24T15:32:38.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4550999999999998e-05 2024-04-24T15:32:38.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:38.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.936663+0000", 2024-04-24T15:32:38.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000133225 2024-04-24T15:32:38.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:38.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.938066+0000", 2024-04-24T15:32:38.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0014031969999999999 2024-04-24T15:32:38.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:38.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.942636+0000", 2024-04-24T15:32:38.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0045691270000000001 2024-04-24T15:32:38.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:38.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.942684+0000", 2024-04-24T15:32:38.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8123e-05 2024-04-24T15:32:38.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:38.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.942720+0000", 2024-04-24T15:32:38.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5880000000000002e-05 2024-04-24T15:32:38.264 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.264 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:38.264 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.264 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:9497 4.1c 4:3e8473f6:::rbd_header.114a1f60cb66:head [watch reconnect cookie 140672528495600 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:38.264 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.312983+0000", 2024-04-24T15:32:38.265 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8746428020000003, 2024-04-24T15:32:38.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0070173939999999997, 2024-04-24T15:32:38.265 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:38.265 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:38.265 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:38.265 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:38.265 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:38.265 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 9497 2024-04-24T15:32:38.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.265 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:38.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.265 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:38.265 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.312983+0000", 2024-04-24T15:32:38.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:38.266 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.312983+0000", 2024-04-24T15:32:38.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:38.266 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.312983+0000", 2024-04-24T15:32:38.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:38.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:38.266 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.312989+0000", 2024-04-24T15:32:38.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3090000000000001e-06 2024-04-24T15:32:38.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:38.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.312995+0000", 2024-04-24T15:32:38.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.477e-06 2024-04-24T15:32:38.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:38.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.312999+0000", 2024-04-24T15:32:38.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8489999999999999e-06 2024-04-24T15:32:38.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:38.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.313327+0000", 2024-04-24T15:32:38.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00032772399999999999 2024-04-24T15:32:38.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:38.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.313391+0000", 2024-04-24T15:32:38.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4388999999999996e-05 2024-04-24T15:32:38.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:38.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.320000+0000", 2024-04-24T15:32:38.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0066090660000000002 2024-04-24T15:32:38.268 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.268 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:38.268 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.268 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:18726 4.4 4:21862cbe:::rbd_data.114ae15b9a20.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:38.268 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:34.274457+0000", 2024-04-24T15:32:38.269 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.9131689939999998, 2024-04-24T15:32:38.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0056902769999999997, 2024-04-24T15:32:38.269 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:38.269 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:38.269 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:38.269 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:38.269 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:38.269 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 18726 2024-04-24T15:32:38.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.269 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:38.269 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.269 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:38.269 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:34.274457+0000", 2024-04-24T15:32:38.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.269 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.270 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:38.270 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:34.274457+0000", 2024-04-24T15:32:38.270 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.270 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:38.270 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:34.274454+0000", 2024-04-24T15:32:38.270 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999976 2024-04-24T15:32:38.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.270 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:38.270 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:34.275538+0000", 2024-04-24T15:32:38.270 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0010832089999999999 2024-04-24T15:32:38.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.271 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:38.271 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:34.275565+0000", 2024-04-24T15:32:38.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7936000000000002e-05 2024-04-24T15:32:38.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.271 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:38.271 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:34.275577+0000", 2024-04-24T15:32:38.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1579999999999999e-05 2024-04-24T15:32:38.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.271 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:38.271 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:34.275633+0000", 2024-04-24T15:32:38.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5809e-05 2024-04-24T15:32:38.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:38.272 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:34.275794+0000", 2024-04-24T15:32:38.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000161111 2024-04-24T15:32:38.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:38.272 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:34.276180+0000", 2024-04-24T15:32:38.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00038595300000000002 2024-04-24T15:32:38.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:38.272 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:34.279950+0000", 2024-04-24T15:32:38.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0037696959999999999 2024-04-24T15:32:38.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.273 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:38.273 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:34.280043+0000", 2024-04-24T15:32:38.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3681e-05 2024-04-24T15:32:38.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.273 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.273 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:38.273 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:34.280102+0000", 2024-04-24T15:32:38.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8587e-05 2024-04-24T15:32:38.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.273 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.273 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:38.273 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:34.280147+0000", 2024-04-24T15:32:38.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5237000000000001e-05 2024-04-24T15:32:38.273 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.273 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:38.273 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.274 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.274 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:19861 4.d 4:b2ab6102:::rbd_data.114ae15b9a20.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:38.274 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:34.404476+0000", 2024-04-24T15:32:38.274 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.7831499869999998, 2024-04-24T15:32:38.274 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0072684150000000003, 2024-04-24T15:32:38.274 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:38.274 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:38.274 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:38.274 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:38.274 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:38.274 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 19861 2024-04-24T15:32:38.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.274 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:38.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.275 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:38.275 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:34.404476+0000", 2024-04-24T15:32:38.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.275 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:38.275 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:34.404476+0000", 2024-04-24T15:32:38.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.275 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:38.275 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:34.404473+0000", 2024-04-24T15:32:38.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999971 2024-04-24T15:32:38.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:38.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:34.405518+0000", 2024-04-24T15:32:38.276 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0010449490000000001 2024-04-24T15:32:38.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:38.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:34.405557+0000", 2024-04-24T15:32:38.276 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8427999999999999e-05 2024-04-24T15:32:38.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:38.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:34.405569+0000", 2024-04-24T15:32:38.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2129e-05 2024-04-24T15:32:38.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.277 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.277 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:38.277 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:34.405625+0000", 2024-04-24T15:32:38.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5980999999999998e-05 2024-04-24T15:32:38.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.277 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.277 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:38.277 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:34.405788+0000", 2024-04-24T15:32:38.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016350499999999999 2024-04-24T15:32:38.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.277 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.277 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 6", 2024-04-24T15:32:38.277 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:34.406065+0000", 2024-04-24T15:32:38.278 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000276631 2024-04-24T15:32:38.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.278 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:38.278 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:34.409212+0000", 2024-04-24T15:32:38.278 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003147066 2024-04-24T15:32:38.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.278 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:38.278 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:34.411631+0000", 2024-04-24T15:32:38.278 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024194960000000001 2024-04-24T15:32:38.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.278 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:38.279 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:34.411693+0000", 2024-04-24T15:32:38.279 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1383999999999994e-05 2024-04-24T15:32:38.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.279 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.279 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:38.279 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:34.411744+0000", 2024-04-24T15:32:38.279 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1474000000000002e-05 2024-04-24T15:32:38.279 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.279 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:38.279 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.279 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.279 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:24347 4.6 4:628d3169:::rbd_data.114ae15b9a20.000000000000004b:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-04-24T15:32:38.279 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:37.251701+0000", 2024-04-24T15:32:38.280 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.93592438099999997, 2024-04-24T15:32:38.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0087394719999999999, 2024-04-24T15:32:38.280 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:38.280 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:38.280 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:38.280 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:38.280 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:38.280 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 24347 2024-04-24T15:32:38.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.280 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:38.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.280 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:38.280 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.251701+0000", 2024-04-24T15:32:38.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:38.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.251701+0000", 2024-04-24T15:32:38.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:38.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.251700+0000", 2024-04-24T15:32:38.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999986 2024-04-24T15:32:38.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:38.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.251708+0000", 2024-04-24T15:32:38.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.49e-06 2024-04-24T15:32:38.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.282 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.282 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:38.282 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.251724+0000", 2024-04-24T15:32:38.282 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5803000000000001e-05 2024-04-24T15:32:38.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.282 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.282 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:38.282 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.251732+0000", 2024-04-24T15:32:38.282 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8460000000000008e-06 2024-04-24T15:32:38.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.282 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.282 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:38.283 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.251775+0000", 2024-04-24T15:32:38.283 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3102000000000002e-05 2024-04-24T15:32:38.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.283 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:38.283 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.251847+0000", 2024-04-24T15:32:38.283 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1482000000000004e-05 2024-04-24T15:32:38.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.283 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:38.283 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.252681+0000", 2024-04-24T15:32:38.283 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00083451599999999999 2024-04-24T15:32:38.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.283 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:38.284 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.260266+0000", 2024-04-24T15:32:38.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0075852599999999999 2024-04-24T15:32:38.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.284 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:38.284 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.260345+0000", 2024-04-24T15:32:38.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8824000000000003e-05 2024-04-24T15:32:38.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.284 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:38.284 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.260397+0000", 2024-04-24T15:32:38.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1573999999999998e-05 2024-04-24T15:32:38.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:38.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.260441+0000", 2024-04-24T15:32:38.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4195e-05 2024-04-24T15:32:38.285 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.285 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:38.285 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.285 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:24351 4.6 4:61b5c231:::rbd_data.114ae15b9a20.000000000000004f:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-04-24T15:32:38.285 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:37.254276+0000", 2024-04-24T15:32:38.285 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.93335030399999996, 2024-04-24T15:32:38.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0064981930000000002, 2024-04-24T15:32:38.285 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:38.285 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:38.286 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:38.286 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:38.286 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:38.286 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 24351 2024-04-24T15:32:38.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.286 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:38.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:38.286 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.254276+0000", 2024-04-24T15:32:38.286 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:38.286 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.254276+0000", 2024-04-24T15:32:38.286 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.287 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:38.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.254274+0000", 2024-04-24T15:32:38.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999981 2024-04-24T15:32:38.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.287 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:38.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.254283+0000", 2024-04-24T15:32:38.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.346e-06 2024-04-24T15:32:38.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.287 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:38.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.254298+0000", 2024-04-24T15:32:38.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5318000000000001e-05 2024-04-24T15:32:38.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.288 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:38.288 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.254306+0000", 2024-04-24T15:32:38.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.622e-06 2024-04-24T15:32:38.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.288 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:38.288 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.255950+0000", 2024-04-24T15:32:38.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001644473 2024-04-24T15:32:38.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.288 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:38.288 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.256055+0000", 2024-04-24T15:32:38.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104273 2024-04-24T15:32:38.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:38.289 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.256982+0000", 2024-04-24T15:32:38.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00092704699999999996 2024-04-24T15:32:38.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:38.289 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.260520+0000", 2024-04-24T15:32:38.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035386979999999998 2024-04-24T15:32:38.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:38.289 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.260692+0000", 2024-04-24T15:32:38.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017125300000000001 2024-04-24T15:32:38.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:38.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.260738+0000", 2024-04-24T15:32:38.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6820999999999998e-05 2024-04-24T15:32:38.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:38.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.260774+0000", 2024-04-24T15:32:38.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5237999999999997e-05 2024-04-24T15:32:38.290 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.290 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:38.291 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.291 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:24393 4.4 4:2764ffd7:::rbd_data.114ae15b9a20.0000000000000031:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-04-24T15:32:38.291 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:37.296111+0000", 2024-04-24T15:32:38.291 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.891515171, 2024-04-24T15:32:38.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0058595019999999999, 2024-04-24T15:32:38.291 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:38.291 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:38.291 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:38.291 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:38.291 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:38.291 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 24393 2024-04-24T15:32:38.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.292 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:38.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.292 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:38.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.296111+0000", 2024-04-24T15:32:38.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.292 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:38.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.296111+0000", 2024-04-24T15:32:38.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.293 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:38.293 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.296110+0000", 2024-04-24T15:32:38.293 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:38.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.293 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:38.293 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.296114+0000", 2024-04-24T15:32:38.293 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4240000000000003e-06 2024-04-24T15:32:38.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.293 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:38.293 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.296125+0000", 2024-04-24T15:32:38.293 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0869999999999999e-05 2024-04-24T15:32:38.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.294 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.294 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:38.294 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.296130+0000", 2024-04-24T15:32:38.294 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7859999999999996e-06 2024-04-24T15:32:38.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.294 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.294 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:38.294 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.296174+0000", 2024-04-24T15:32:38.294 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3492999999999998e-05 2024-04-24T15:32:38.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.294 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.294 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:38.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.296258+0000", 2024-04-24T15:32:38.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3981999999999997e-05 2024-04-24T15:32:38.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:38.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.296383+0000", 2024-04-24T15:32:38.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000125143 2024-04-24T15:32:38.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:38.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.298229+0000", 2024-04-24T15:32:38.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018462260000000001 2024-04-24T15:32:38.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:38.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.301889+0000", 2024-04-24T15:32:38.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003660122 2024-04-24T15:32:38.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:38.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.301936+0000", 2024-04-24T15:32:38.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7284999999999999e-05 2024-04-24T15:32:38.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:38.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.301970+0000", 2024-04-24T15:32:38.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3892e-05 2024-04-24T15:32:38.296 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.297 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:38.297 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.297 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.297 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:38.297 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:32:38.298 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-24T15:32:38.299 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:32:38.299+0000 7f23abcf6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 44021) UID: 0 2024-04-24T15:32:38.399 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:32:38.400+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:32:38.462 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:32:38.462 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:32:38.462 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-04-24T15:32:38.462 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:32:38.471 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-24T15:32:38.500 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:32:38.501+0000 7f23abcf6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 44021) UID: 0 2024-04-24T15:32:38.593 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:32:38.593 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:32:38.594 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-04-24T15:32:38.594 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-04-24T15:32:38.594 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:32:38.601 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:32:38.602+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:32:38.603 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-24T15:32:38.637 INFO:teuthology.orchestra.run.smithi096.stderr:nodeep-scrub is unset 2024-04-24T15:32:38.641 INFO:teuthology.orchestra.run.smithi096.stdout:1366 write 0xee42b15 thru 0xee48e6a (0x6356 bytes) 2024-04-24T15:32:38.654 INFO:teuthology.orchestra.run.smithi096.stdout:1367 read 0x698e188 thru 0x699d887 (0xf700 bytes) 2024-04-24T15:32:38.655 INFO:teuthology.orchestra.run.smithi096.stdout:1371 write 0xc53cc3a thru 0xc53e2bc (0x1683 bytes) 2024-04-24T15:32:38.660 INFO:teuthology.orchestra.run.smithi096.stdout:1372 write 0x7d219d2 thru 0x7d260ae (0x46dd bytes) 2024-04-24T15:32:38.669 INFO:teuthology.orchestra.run.smithi096.stdout:1373 read 0x2ed0eec thru 0x2ed2ef5 (0x200a bytes) 2024-04-24T15:32:38.670 INFO:teuthology.orchestra.run.smithi096.stdout:1374 clone 5 order 23 su 2097152 sc 14 2024-04-24T15:32:38.702 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:32:38.703+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:32:38.738 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:32:38.739 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-04-24T15:32:38.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-04-24T15:32:38.739 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-04-24T15:32:38.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.739 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14043 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2383872~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:38.739 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892347+0000", 2024-04-24T15:32:38.739 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8460722790000004, 2024-04-24T15:32:38.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038537279000000001, 2024-04-24T15:32:38.739 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:38.739 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:38.739 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:38.739 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:38.739 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:38.740 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14043 2024-04-24T15:32:38.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.740 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:38.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:38.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892347+0000", 2024-04-24T15:32:38.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:38.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892347+0000", 2024-04-24T15:32:38.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:38.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892347+0000", 2024-04-24T15:32:38.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:38.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:38.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892349+0000", 2024-04-24T15:32:38.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3400000000000001e-06 2024-04-24T15:32:38.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:38.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892357+0000", 2024-04-24T15:32:38.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1540000000000002e-06 2024-04-24T15:32:38.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.742 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:38.742 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892360+0000", 2024-04-24T15:32:38.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5829999999999998e-06 2024-04-24T15:32:38.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.742 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:38.742 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930795+0000", 2024-04-24T15:32:38.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038434590999999997 2024-04-24T15:32:38.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.742 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:38.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930836+0000", 2024-04-24T15:32:38.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0827000000000001e-05 2024-04-24T15:32:38.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:38.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930885+0000", 2024-04-24T15:32:38.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9029000000000001e-05 2024-04-24T15:32:38.743 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.743 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:38.743 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.743 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14044 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2424832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:38.743 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892372+0000", 2024-04-24T15:32:38.744 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8460476349999997, 2024-04-24T15:32:38.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038624890000000002, 2024-04-24T15:32:38.744 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:38.744 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:38.744 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:38.744 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:38.744 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:38.744 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14044 2024-04-24T15:32:38.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.744 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:38.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:38.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892372+0000", 2024-04-24T15:32:38.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:38.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892372+0000", 2024-04-24T15:32:38.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:38.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892372+0000", 2024-04-24T15:32:38.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:38.746 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.746 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.746 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:38.746 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892374+0000", 2024-04-24T15:32:38.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4899999999999999e-06 2024-04-24T15:32:38.746 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.746 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.746 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:38.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892380+0000", 2024-04-24T15:32:38.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.631e-06 2024-04-24T15:32:38.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:38.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892383+0000", 2024-04-24T15:32:38.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7400000000000002e-06 2024-04-24T15:32:38.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:38.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930905+0000", 2024-04-24T15:32:38.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038521396999999999 2024-04-24T15:32:38.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:38.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930949+0000", 2024-04-24T15:32:38.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4153999999999998e-05 2024-04-24T15:32:38.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:38.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930997+0000", 2024-04-24T15:32:38.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7998999999999999e-05 2024-04-24T15:32:38.748 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.748 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:38.748 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.749 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14045 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2465792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:38.749 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892387+0000", 2024-04-24T15:32:38.749 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8460323680000004, 2024-04-24T15:32:38.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038720933999999999, 2024-04-24T15:32:38.749 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:38.749 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:38.749 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:38.749 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:38.749 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:38.749 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14045 2024-04-24T15:32:38.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.749 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:38.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:38.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892387+0000", 2024-04-24T15:32:38.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:38.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892387+0000", 2024-04-24T15:32:38.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:38.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892387+0000", 2024-04-24T15:32:38.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:38.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:38.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892392+0000", 2024-04-24T15:32:38.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0039999999999999e-06 2024-04-24T15:32:38.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:38.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892398+0000", 2024-04-24T15:32:38.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0900000000000001e-06 2024-04-24T15:32:38.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:38.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892402+0000", 2024-04-24T15:32:38.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.399e-06 2024-04-24T15:32:38.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:38.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931018+0000", 2024-04-24T15:32:38.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038616551999999998 2024-04-24T15:32:38.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:38.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931059+0000", 2024-04-24T15:32:38.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1205000000000003e-05 2024-04-24T15:32:38.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:38.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931108+0000", 2024-04-24T15:32:38.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8996000000000003e-05 2024-04-24T15:32:38.754 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.754 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:38.754 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.755 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14046 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2506752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:38.755 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892406+0000", 2024-04-24T15:32:38.755 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8460137779999997, 2024-04-24T15:32:38.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038814507999999998, 2024-04-24T15:32:38.755 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:38.755 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:38.755 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:38.755 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:38.755 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:38.755 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14046 2024-04-24T15:32:38.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.756 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:38.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:38.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892406+0000", 2024-04-24T15:32:38.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:38.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892406+0000", 2024-04-24T15:32:38.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:38.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892406+0000", 2024-04-24T15:32:38.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:38.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:38.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892413+0000", 2024-04-24T15:32:38.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7449999999999995e-06 2024-04-24T15:32:38.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:38.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892419+0000", 2024-04-24T15:32:38.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5790000000000003e-06 2024-04-24T15:32:38.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:38.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892423+0000", 2024-04-24T15:32:38.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8199999999999998e-06 2024-04-24T15:32:38.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:38.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931128+0000", 2024-04-24T15:32:38.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038705502000000003 2024-04-24T15:32:38.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:38.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931172+0000", 2024-04-24T15:32:38.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4066999999999997e-05 2024-04-24T15:32:38.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:38.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931220+0000", 2024-04-24T15:32:38.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8167999999999997e-05 2024-04-24T15:32:38.759 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.759 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:38.759 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.760 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14047 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2547712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:38.760 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892427+0000", 2024-04-24T15:32:38.760 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.845992549, 2024-04-24T15:32:38.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038904506999999998, 2024-04-24T15:32:38.760 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:38.760 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:38.760 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:38.760 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:38.760 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:38.760 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14047 2024-04-24T15:32:38.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.760 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:38.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:38.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892427+0000", 2024-04-24T15:32:38.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:38.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892427+0000", 2024-04-24T15:32:38.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:38.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892427+0000", 2024-04-24T15:32:38.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:38.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:38.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892429+0000", 2024-04-24T15:32:38.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3099999999999999e-06 2024-04-24T15:32:38.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:38.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892435+0000", 2024-04-24T15:32:38.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5520000000000004e-06 2024-04-24T15:32:38.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:38.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892438+0000", 2024-04-24T15:32:38.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4240000000000002e-06 2024-04-24T15:32:38.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:38.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931242+0000", 2024-04-24T15:32:38.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038804162000000003 2024-04-24T15:32:38.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:38.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931284+0000", 2024-04-24T15:32:38.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1414e-05 2024-04-24T15:32:38.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:38.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931332+0000", 2024-04-24T15:32:38.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8024e-05 2024-04-24T15:32:38.764 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.764 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:38.764 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.764 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14048 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2588672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:38.764 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892442+0000", 2024-04-24T15:32:38.764 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8459775860000001, 2024-04-24T15:32:38.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039003132000000003, 2024-04-24T15:32:38.764 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:38.764 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:38.765 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:38.765 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:38.765 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:38.765 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14048 2024-04-24T15:32:38.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.765 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:38.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.765 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:38.765 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892442+0000", 2024-04-24T15:32:38.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.765 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:38.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892442+0000", 2024-04-24T15:32:38.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:38.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892442+0000", 2024-04-24T15:32:38.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:38.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:38.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892444+0000", 2024-04-24T15:32:38.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1169999999999998e-06 2024-04-24T15:32:38.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:38.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892449+0000", 2024-04-24T15:32:38.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1599999999999997e-06 2024-04-24T15:32:38.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:38.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892453+0000", 2024-04-24T15:32:38.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5930000000000002e-06 2024-04-24T15:32:38.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:38.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931352+0000", 2024-04-24T15:32:38.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038899361 2024-04-24T15:32:38.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:38.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931398+0000", 2024-04-24T15:32:38.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6315e-05 2024-04-24T15:32:38.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:38.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931445+0000", 2024-04-24T15:32:38.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6950000000000003e-05 2024-04-24T15:32:38.768 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.768 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:38.768 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.769 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14049 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2629632~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:38.769 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892458+0000", 2024-04-24T15:32:38.769 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8459621659999996, 2024-04-24T15:32:38.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039103322000000003, 2024-04-24T15:32:38.769 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:38.769 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:38.769 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:38.769 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:38.769 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:38.769 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14049 2024-04-24T15:32:38.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.769 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:38.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:38.770 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892458+0000", 2024-04-24T15:32:38.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:38.770 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892458+0000", 2024-04-24T15:32:38.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:38.770 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892457+0000", 2024-04-24T15:32:38.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:38.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:38.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892459+0000", 2024-04-24T15:32:38.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.305e-06 2024-04-24T15:32:38.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:38.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892465+0000", 2024-04-24T15:32:38.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4360000000000001e-06 2024-04-24T15:32:38.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:38.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892474+0000", 2024-04-24T15:32:38.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7639999999999994e-06 2024-04-24T15:32:38.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:38.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931466+0000", 2024-04-24T15:32:38.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038992644 2024-04-24T15:32:38.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:38.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931512+0000", 2024-04-24T15:32:38.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5200000000000001e-05 2024-04-24T15:32:38.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:38.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931561+0000", 2024-04-24T15:32:38.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9357999999999997e-05 2024-04-24T15:32:38.773 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.773 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:38.773 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.773 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14050 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2670592~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:38.773 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892478+0000", 2024-04-24T15:32:38.773 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8459415789999998, 2024-04-24T15:32:38.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039195102000000002, 2024-04-24T15:32:38.773 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:38.774 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:38.774 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:38.774 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:38.774 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:38.774 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14050 2024-04-24T15:32:38.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.774 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:38.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:38.774 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892478+0000", 2024-04-24T15:32:38.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:38.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892478+0000", 2024-04-24T15:32:38.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:38.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892478+0000", 2024-04-24T15:32:38.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:38.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:38.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892480+0000", 2024-04-24T15:32:38.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3300000000000001e-06 2024-04-24T15:32:38.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:38.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892486+0000", 2024-04-24T15:32:38.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9200000000000001e-06 2024-04-24T15:32:38.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:38.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892494+0000", 2024-04-24T15:32:38.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5159999999999997e-06 2024-04-24T15:32:38.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:38.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931581+0000", 2024-04-24T15:32:38.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039087894999999998 2024-04-24T15:32:38.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:38.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931626+0000", 2024-04-24T15:32:38.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4943e-05 2024-04-24T15:32:38.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:38.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931673+0000", 2024-04-24T15:32:38.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6906000000000001e-05 2024-04-24T15:32:38.777 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.777 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:38.777 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.778 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14051 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2711552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:38.778 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892507+0000", 2024-04-24T15:32:38.778 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8459128800000002, 2024-04-24T15:32:38.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039274393999999997, 2024-04-24T15:32:38.778 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:38.778 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:38.778 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:38.778 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:38.778 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:38.778 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14051 2024-04-24T15:32:38.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.778 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:38.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:38.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892507+0000", 2024-04-24T15:32:38.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:38.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892507+0000", 2024-04-24T15:32:38.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:38.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892506+0000", 2024-04-24T15:32:38.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:38.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:38.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892509+0000", 2024-04-24T15:32:38.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8760000000000001e-06 2024-04-24T15:32:38.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:38.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892515+0000", 2024-04-24T15:32:38.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1840000000000004e-06 2024-04-24T15:32:38.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:38.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892519+0000", 2024-04-24T15:32:38.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7570000000000002e-06 2024-04-24T15:32:38.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:38.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931694+0000", 2024-04-24T15:32:38.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039174960000000002 2024-04-24T15:32:38.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:38.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931734+0000", 2024-04-24T15:32:38.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0284000000000003e-05 2024-04-24T15:32:38.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:38.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931781+0000", 2024-04-24T15:32:38.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6813000000000001e-05 2024-04-24T15:32:38.782 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.782 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:38.782 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.782 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14052 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2752512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:38.782 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892524+0000", 2024-04-24T15:32:38.782 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8458957329999999, 2024-04-24T15:32:38.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039371122000000001, 2024-04-24T15:32:38.782 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:38.782 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:38.782 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:38.783 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:38.783 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:38.783 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14052 2024-04-24T15:32:38.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.783 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:38.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:38.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892524+0000", 2024-04-24T15:32:38.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:38.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892524+0000", 2024-04-24T15:32:38.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:38.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892524+0000", 2024-04-24T15:32:38.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:38.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:38.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892526+0000", 2024-04-24T15:32:38.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3630000000000001e-06 2024-04-24T15:32:38.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:38.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892538+0000", 2024-04-24T15:32:38.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1677e-05 2024-04-24T15:32:38.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:38.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892541+0000", 2024-04-24T15:32:38.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.822e-06 2024-04-24T15:32:38.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:38.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931802+0000", 2024-04-24T15:32:38.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039260911000000003 2024-04-24T15:32:38.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:38.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931847+0000", 2024-04-24T15:32:38.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4450999999999998e-05 2024-04-24T15:32:38.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:38.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931895+0000", 2024-04-24T15:32:38.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8257999999999998e-05 2024-04-24T15:32:38.786 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.786 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:38.786 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.786 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14053 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2793472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:38.786 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892546+0000", 2024-04-24T15:32:38.787 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8458737549999995, 2024-04-24T15:32:38.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039457712999999998, 2024-04-24T15:32:38.787 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:38.787 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:38.787 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:38.787 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:38.787 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:38.787 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14053 2024-04-24T15:32:38.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.787 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:38.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:38.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892546+0000", 2024-04-24T15:32:38.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:38.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892546+0000", 2024-04-24T15:32:38.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:38.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892546+0000", 2024-04-24T15:32:38.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:38.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:38.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892548+0000", 2024-04-24T15:32:38.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3120000000000001e-06 2024-04-24T15:32:38.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:38.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892553+0000", 2024-04-24T15:32:38.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2320000000000001e-06 2024-04-24T15:32:38.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:38.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892559+0000", 2024-04-24T15:32:38.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5480000000000001e-06 2024-04-24T15:32:38.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:38.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931916+0000", 2024-04-24T15:32:38.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039357180999999998 2024-04-24T15:32:38.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:38.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931956+0000", 2024-04-24T15:32:38.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0565000000000002e-05 2024-04-24T15:32:38.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:38.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932004+0000", 2024-04-24T15:32:38.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7224000000000001e-05 2024-04-24T15:32:38.790 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.790 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:38.790 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.791 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14054 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2834432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:38.791 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892563+0000", 2024-04-24T15:32:38.791 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8458571030000002, 2024-04-24T15:32:38.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039552406999999998, 2024-04-24T15:32:38.791 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:38.791 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:38.791 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:38.791 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:38.791 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:38.791 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14054 2024-04-24T15:32:38.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.791 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:38.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:38.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892563+0000", 2024-04-24T15:32:38.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:38.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892563+0000", 2024-04-24T15:32:38.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:38.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892562+0000", 2024-04-24T15:32:38.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:38.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:38.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892564+0000", 2024-04-24T15:32:38.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2029999999999999e-06 2024-04-24T15:32:38.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:38.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892570+0000", 2024-04-24T15:32:38.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1460000000000004e-06 2024-04-24T15:32:38.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:38.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892573+0000", 2024-04-24T15:32:38.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2030000000000001e-06 2024-04-24T15:32:38.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:38.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932024+0000", 2024-04-24T15:32:38.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039451696000000001 2024-04-24T15:32:38.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:38.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932068+0000", 2024-04-24T15:32:38.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3970000000000001e-05 2024-04-24T15:32:38.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:38.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932115+0000", 2024-04-24T15:32:38.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6569000000000001e-05 2024-04-24T15:32:38.796 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.796 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:38.796 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.796 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14055 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2875392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:38.796 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892591+0000", 2024-04-24T15:32:38.796 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8458291300000003, 2024-04-24T15:32:38.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039634309, 2024-04-24T15:32:38.796 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:38.797 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:38.797 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:38.797 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:38.797 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:38.797 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14055 2024-04-24T15:32:38.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.797 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:38.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:38.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892591+0000", 2024-04-24T15:32:38.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:38.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892591+0000", 2024-04-24T15:32:38.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:38.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892590+0000", 2024-04-24T15:32:38.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:38.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:38.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892592+0000", 2024-04-24T15:32:38.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7519999999999999e-06 2024-04-24T15:32:38.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:38.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892596+0000", 2024-04-24T15:32:38.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.608e-06 2024-04-24T15:32:38.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:38.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892598+0000", 2024-04-24T15:32:38.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.531e-06 2024-04-24T15:32:38.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:38.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932136+0000", 2024-04-24T15:32:38.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039537668999999998 2024-04-24T15:32:38.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:38.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932177+0000", 2024-04-24T15:32:38.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0880000000000002e-05 2024-04-24T15:32:38.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:38.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932225+0000", 2024-04-24T15:32:38.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8223999999999998e-05 2024-04-24T15:32:38.800 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.800 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:38.800 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.801 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14056 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2916352~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:38.801 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892600+0000", 2024-04-24T15:32:38.801 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8458192960000002, 2024-04-24T15:32:38.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039736447000000001, 2024-04-24T15:32:38.801 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:38.801 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:38.801 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:38.801 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:38.801 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:38.801 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14056 2024-04-24T15:32:38.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.801 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:38.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:38.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892600+0000", 2024-04-24T15:32:38.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.802 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:38.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892600+0000", 2024-04-24T15:32:38.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.802 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:38.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892600+0000", 2024-04-24T15:32:38.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967296 2024-04-24T15:32:38.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:38.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892602+0000", 2024-04-24T15:32:38.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.576e-06 2024-04-24T15:32:38.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:38.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892605+0000", 2024-04-24T15:32:38.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.574e-06 2024-04-24T15:32:38.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:38.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892608+0000", 2024-04-24T15:32:38.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3800000000000001e-06 2024-04-24T15:32:38.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:38.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932245+0000", 2024-04-24T15:32:38.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039637378000000001 2024-04-24T15:32:38.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:38.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932290+0000", 2024-04-24T15:32:38.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4734999999999998e-05 2024-04-24T15:32:38.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:38.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932337+0000", 2024-04-24T15:32:38.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7033000000000002e-05 2024-04-24T15:32:38.805 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.805 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:38.805 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.805 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14062 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 3162112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:38.805 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.894556+0000", 2024-04-24T15:32:38.805 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8438637260000004, 2024-04-24T15:32:38.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.03845316, 2024-04-24T15:32:38.805 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:38.805 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:38.805 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:38.805 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:38.805 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:38.806 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14062 2024-04-24T15:32:38.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.806 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:38.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.806 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:38.806 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894556+0000", 2024-04-24T15:32:38.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.806 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:38.806 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894556+0000", 2024-04-24T15:32:38.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:38.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894556+0000", 2024-04-24T15:32:38.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967296 2024-04-24T15:32:38.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:38.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894557+0000", 2024-04-24T15:32:38.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3230000000000001e-06 2024-04-24T15:32:38.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:38.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894560+0000", 2024-04-24T15:32:38.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7719999999999999e-06 2024-04-24T15:32:38.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:38.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894562+0000", 2024-04-24T15:32:38.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9479999999999998e-06 2024-04-24T15:32:38.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:38.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932917+0000", 2024-04-24T15:32:38.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038355490999999999 2024-04-24T15:32:38.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:38.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932961+0000", 2024-04-24T15:32:38.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3302e-05 2024-04-24T15:32:38.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:38.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933009+0000", 2024-04-24T15:32:38.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8535000000000002e-05 2024-04-24T15:32:38.809 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.809 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:38.809 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.809 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14063 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 3203072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:38.809 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.894564+0000", 2024-04-24T15:32:38.809 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8438557050000002, 2024-04-24T15:32:38.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038554995000000002, 2024-04-24T15:32:38.809 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:38.810 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:38.810 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:38.810 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:38.810 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:38.810 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14063 2024-04-24T15:32:38.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.810 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:38.810 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.810 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:38.810 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894564+0000", 2024-04-24T15:32:38.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.810 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.810 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:38.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894564+0000", 2024-04-24T15:32:38.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:38.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894564+0000", 2024-04-24T15:32:38.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967296 2024-04-24T15:32:38.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:38.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894565+0000", 2024-04-24T15:32:38.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2640000000000001e-06 2024-04-24T15:32:38.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:38.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894569+0000", 2024-04-24T15:32:38.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1710000000000004e-06 2024-04-24T15:32:38.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:38.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894573+0000", 2024-04-24T15:32:38.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4410000000000002e-06 2024-04-24T15:32:38.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:38.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933030+0000", 2024-04-24T15:32:38.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038457709999999999 2024-04-24T15:32:38.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:38.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933072+0000", 2024-04-24T15:32:38.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.193e-05 2024-04-24T15:32:38.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:38.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933119+0000", 2024-04-24T15:32:38.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6715000000000003e-05 2024-04-24T15:32:38.813 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.813 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:38.813 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.814 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14064 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 3244032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:38.814 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.894591+0000", 2024-04-24T15:32:38.814 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8438290630000003, 2024-04-24T15:32:38.814 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038639806999999998, 2024-04-24T15:32:38.814 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:38.814 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:38.814 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:38.814 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:38.814 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:38.814 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14064 2024-04-24T15:32:38.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.814 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:38.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:38.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894591+0000", 2024-04-24T15:32:38.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:38.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894591+0000", 2024-04-24T15:32:38.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:38.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894590+0000", 2024-04-24T15:32:38.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:38.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:38.816 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894593+0000", 2024-04-24T15:32:38.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4719999999999998e-06 2024-04-24T15:32:38.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:38.816 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894598+0000", 2024-04-24T15:32:38.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.57e-06 2024-04-24T15:32:38.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:38.816 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894602+0000", 2024-04-24T15:32:38.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.614e-06 2024-04-24T15:32:38.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:38.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933139+0000", 2024-04-24T15:32:38.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038536929999999997 2024-04-24T15:32:38.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:38.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933183+0000", 2024-04-24T15:32:38.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4217e-05 2024-04-24T15:32:38.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:38.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933230+0000", 2024-04-24T15:32:38.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7434e-05 2024-04-24T15:32:38.818 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.818 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:38.818 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.818 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14065 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 3284992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:38.818 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.894606+0000", 2024-04-24T15:32:38.818 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8438136810000003, 2024-04-24T15:32:38.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038734126000000001, 2024-04-24T15:32:38.818 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:38.818 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:38.818 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:38.819 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:38.819 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:38.819 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14065 2024-04-24T15:32:38.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.819 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:38.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.819 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:38.819 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894606+0000", 2024-04-24T15:32:38.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.819 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:38.819 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894606+0000", 2024-04-24T15:32:38.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.820 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:38.820 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894606+0000", 2024-04-24T15:32:38.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:38.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.820 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:38.820 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894608+0000", 2024-04-24T15:32:38.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0880000000000002e-06 2024-04-24T15:32:38.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.820 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:38.820 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894614+0000", 2024-04-24T15:32:38.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7830000000000004e-06 2024-04-24T15:32:38.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:38.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894621+0000", 2024-04-24T15:32:38.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5809999999999999e-06 2024-04-24T15:32:38.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:38.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933251+0000", 2024-04-24T15:32:38.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038629995 2024-04-24T15:32:38.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:38.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933292+0000", 2024-04-24T15:32:38.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0380000000000003e-05 2024-04-24T15:32:38.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.822 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:38.822 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933340+0000", 2024-04-24T15:32:38.822 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8662999999999998e-05 2024-04-24T15:32:38.822 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.822 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:38.822 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.822 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14066 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 3325952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:38.822 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.894625+0000", 2024-04-24T15:32:38.822 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8437942419999995, 2024-04-24T15:32:38.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038825539999999999, 2024-04-24T15:32:38.823 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:38.823 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:38.823 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:38.823 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:38.823 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:38.823 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14066 2024-04-24T15:32:38.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.823 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:38.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.823 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:38.823 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894625+0000", 2024-04-24T15:32:38.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.824 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:38.824 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894625+0000", 2024-04-24T15:32:38.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.824 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.824 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:38.824 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894625+0000", 2024-04-24T15:32:38.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:38.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.824 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.824 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:38.824 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894627+0000", 2024-04-24T15:32:38.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.092e-06 2024-04-24T15:32:38.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.824 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.825 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:38.825 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894635+0000", 2024-04-24T15:32:38.825 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6660000000000008e-06 2024-04-24T15:32:38.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.825 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.825 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:38.825 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894639+0000", 2024-04-24T15:32:38.825 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.799e-06 2024-04-24T15:32:38.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.825 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.825 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:38.825 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933360+0000", 2024-04-24T15:32:38.825 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038721569999999997 2024-04-24T15:32:38.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.825 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.826 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:38.826 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933403+0000", 2024-04-24T15:32:38.826 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2605999999999998e-05 2024-04-24T15:32:38.826 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.826 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:38.826 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933451+0000", 2024-04-24T15:32:38.826 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8149999999999998e-05 2024-04-24T15:32:38.826 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.826 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:38.826 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.826 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.827 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14067 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 3366912~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:38.827 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.894644+0000", 2024-04-24T15:32:38.827 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8437761840000002, 2024-04-24T15:32:38.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038926068000000001, 2024-04-24T15:32:38.827 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:38.827 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:38.827 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:38.827 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:38.827 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:38.827 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14067 2024-04-24T15:32:38.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.827 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:38.827 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.827 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:38.827 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894644+0000", 2024-04-24T15:32:38.828 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.828 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.828 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:38.828 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894644+0000", 2024-04-24T15:32:38.828 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:38.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.828 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.828 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:38.828 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894643+0000", 2024-04-24T15:32:38.828 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:38.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.828 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.828 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:38.829 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894646+0000", 2024-04-24T15:32:38.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3379999999999999e-06 2024-04-24T15:32:38.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.829 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:38.829 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894651+0000", 2024-04-24T15:32:38.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.485e-06 2024-04-24T15:32:38.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.829 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:38.829 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894655+0000", 2024-04-24T15:32:38.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.709e-06 2024-04-24T15:32:38.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.830 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:38.830 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933472+0000", 2024-04-24T15:32:38.830 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038816992000000002 2024-04-24T15:32:38.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.830 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:38.830 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933521+0000", 2024-04-24T15:32:38.830 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8841999999999998e-05 2024-04-24T15:32:38.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:38.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:38.830 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:38.830 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933570+0000", 2024-04-24T15:32:38.830 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9063000000000001e-05 2024-04-24T15:32:38.831 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.831 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:38.831 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.831 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:38.831 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:38.831 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:32:38.832 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-24T15:32:38.833 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:32:38.833+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:32:38.935 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:32:38.936+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:32:38.966 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:32:38.966 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:32:38.966 INFO:teuthology.orchestra.run.smithi151.stdout: "num_ops": 0 2024-04-24T15:32:38.966 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:32:38.976 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-24T15:32:39.060 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:32:39.062+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:32:39.136 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:32:39.137+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:32:39.139 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:32:39.139 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:32:39.140 INFO:teuthology.orchestra.run.smithi151.stdout: "complaint_time": 30, 2024-04-24T15:32:39.140 INFO:teuthology.orchestra.run.smithi151.stdout: "num_blocked_ops": 0 2024-04-24T15:32:39.140 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:32:39.149 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-24T15:32:39.231 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-24T15:32:39.236 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:32:39.238+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:32:39.315 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:32:39.315 INFO:teuthology.orchestra.run.smithi151.stdout: "size": 20, 2024-04-24T15:32:39.315 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 600, 2024-04-24T15:32:39.316 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [ 2024-04-24T15:32:39.316 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.316 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:2507 4.e 4:729aaa1b:::rbd_data.114a1f60cb66.000000000000029a:head [set-alloc-hint object_size 262144 write_size 262144,write 233472~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-24T15:32:39.316 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:23.072715+0000", 2024-04-24T15:32:39.316 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 16.241826160999999, 2024-04-24T15:32:39.316 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0065961370000000002, 2024-04-24T15:32:39.316 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:39.316 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:39.316 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:39.316 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:39.316 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:39.316 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 2507 2024-04-24T15:32:39.316 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.317 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:39.317 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.317 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:39.317 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.072715+0000", 2024-04-24T15:32:39.317 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:39.317 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.317 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.317 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:39.317 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.072715+0000", 2024-04-24T15:32:39.317 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:39.317 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.317 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.317 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:39.317 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.072713+0000", 2024-04-24T15:32:39.318 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999981 2024-04-24T15:32:39.318 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.318 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.318 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:39.318 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.072727+0000", 2024-04-24T15:32:39.318 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3549e-05 2024-04-24T15:32:39.318 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.318 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.318 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:39.318 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.072739+0000", 2024-04-24T15:32:39.318 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.1973999999999999e-05 2024-04-24T15:32:39.318 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.319 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.319 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:39.319 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.072746+0000", 2024-04-24T15:32:39.319 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2849999999999997e-06 2024-04-24T15:32:39.319 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.319 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.319 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:39.319 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.072798+0000", 2024-04-24T15:32:39.319 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.2289e-05 2024-04-24T15:32:39.319 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.319 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.319 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op must wait for map", 2024-04-24T15:32:39.319 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.072806+0000", 2024-04-24T15:32:39.320 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.8469999999999996e-06 2024-04-24T15:32:39.320 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.320 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.320 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:39.320 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.074503+0000", 2024-04-24T15:32:39.320 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0016967029999999999 2024-04-24T15:32:39.320 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.320 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.320 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:39.320 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.074593+0000", 2024-04-24T15:32:39.320 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.0313000000000001e-05 2024-04-24T15:32:39.320 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.320 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.321 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 6", 2024-04-24T15:32:39.321 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.074732+0000", 2024-04-24T15:32:39.321 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000138376 2024-04-24T15:32:39.321 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.321 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.321 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:39.321 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.078538+0000", 2024-04-24T15:32:39.321 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0038068749999999999 2024-04-24T15:32:39.321 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.321 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.321 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:39.321 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.079232+0000", 2024-04-24T15:32:39.321 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00069326900000000005 2024-04-24T15:32:39.321 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.321 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.321 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:39.322 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.079273+0000", 2024-04-24T15:32:39.322 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.1236000000000003e-05 2024-04-24T15:32:39.322 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.322 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.322 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:39.322 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.079311+0000", 2024-04-24T15:32:39.322 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8497000000000001e-05 2024-04-24T15:32:39.322 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:39.322 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:39.322 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:39.322 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.322 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.322 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_repop(client.4426.0:2563 4.1d e34/27 4:b841be81:::rbd_data.114a1f60cb66.00000000000001d7:head v 34'17, mlcod=32'15)", 2024-04-24T15:32:39.323 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.108404+0000", 2024-04-24T15:32:39.323 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 15.206137629000001, 2024-04-24T15:32:39.323 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0073539469999999996, 2024-04-24T15:32:39.323 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:39.323 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:39.323 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:39.323 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.323 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:39.323 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.108404+0000", 2024-04-24T15:32:39.323 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:39.323 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.323 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.323 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:39.323 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.108404+0000", 2024-04-24T15:32:39.324 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:39.324 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.324 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.324 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:39.324 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.108403+0000", 2024-04-24T15:32:39.324 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:39.324 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.324 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.324 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:39.324 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.108420+0000", 2024-04-24T15:32:39.324 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.6980999999999999e-05 2024-04-24T15:32:39.324 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.324 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.324 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:39.324 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.108428+0000", 2024-04-24T15:32:39.325 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0560000000000005e-06 2024-04-24T15:32:39.325 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.325 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.325 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:39.325 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.108432+0000", 2024-04-24T15:32:39.325 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.4060000000000001e-06 2024-04-24T15:32:39.325 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.325 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.325 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:39.325 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.108462+0000", 2024-04-24T15:32:39.325 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.008e-05 2024-04-24T15:32:39.325 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.325 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.325 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:39.326 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.108475+0000", 2024-04-24T15:32:39.326 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3018e-05 2024-04-24T15:32:39.326 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.326 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.326 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:39.326 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.115726+0000", 2024-04-24T15:32:39.326 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0072510040000000001 2024-04-24T15:32:39.326 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.326 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.326 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:39.326 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.115758+0000", 2024-04-24T15:32:39.326 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.2092999999999999e-05 2024-04-24T15:32:39.326 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:39.327 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:39.327 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:39.327 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.327 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.327 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:2620 4.18 4:1e938922:::rbd_data.114a1f60cb66.0000000000000028:head [stat,write 163840~6144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:39.327 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.151073+0000", 2024-04-24T15:32:39.327 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 15.163468915999999, 2024-04-24T15:32:39.327 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0063989210000000001, 2024-04-24T15:32:39.327 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:39.327 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:39.327 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:39.327 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:39.327 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:39.327 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 2620 2024-04-24T15:32:39.327 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.328 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:39.328 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.328 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:39.328 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.151073+0000", 2024-04-24T15:32:39.328 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:39.328 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.328 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.328 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:39.328 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.151073+0000", 2024-04-24T15:32:39.328 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:39.328 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.328 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.328 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:39.328 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.151072+0000", 2024-04-24T15:32:39.328 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:39.329 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.329 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.329 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:39.329 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.151083+0000", 2024-04-24T15:32:39.329 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0801999999999999e-05 2024-04-24T15:32:39.329 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.329 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.329 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:39.329 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.151086+0000", 2024-04-24T15:32:39.329 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.3900000000000002e-06 2024-04-24T15:32:39.329 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.329 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.329 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:39.329 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.151089+0000", 2024-04-24T15:32:39.330 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.294e-06 2024-04-24T15:32:39.330 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.330 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.330 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:39.330 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.152198+0000", 2024-04-24T15:32:39.330 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.001109407 2024-04-24T15:32:39.330 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.330 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.330 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:39.330 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.152273+0000", 2024-04-24T15:32:39.330 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.4419999999999996e-05 2024-04-24T15:32:39.330 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.330 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.331 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 6", 2024-04-24T15:32:39.331 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.152396+0000", 2024-04-24T15:32:39.331 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00012322400000000001 2024-04-24T15:32:39.331 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.331 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.331 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:39.331 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.157242+0000", 2024-04-24T15:32:39.331 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0048462369999999998 2024-04-24T15:32:39.331 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.331 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.331 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:39.331 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.157414+0000", 2024-04-24T15:32:39.331 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000172411 2024-04-24T15:32:39.331 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.332 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.332 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:39.332 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.157451+0000", 2024-04-24T15:32:39.332 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.6269000000000001e-05 2024-04-24T15:32:39.332 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.332 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.332 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:39.332 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.157471+0000", 2024-04-24T15:32:39.332 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.0741e-05 2024-04-24T15:32:39.332 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:39.332 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:39.332 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:39.332 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.332 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.332 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:2623 4.18 4:1b4f38fa:::rbd_data.114a1f60cb66.000000000000002b:head [stat,write 161792~6144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:39.333 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.151108+0000", 2024-04-24T15:32:39.333 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 15.163433654, 2024-04-24T15:32:39.333 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0064461170000000003, 2024-04-24T15:32:39.333 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:39.333 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:39.333 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:39.333 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:39.333 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:39.333 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 2623 2024-04-24T15:32:39.333 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.333 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:39.333 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.333 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:39.333 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.151108+0000", 2024-04-24T15:32:39.334 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:39.334 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.334 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.334 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:39.334 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.151108+0000", 2024-04-24T15:32:39.334 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:39.334 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.334 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.334 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:39.334 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.151108+0000", 2024-04-24T15:32:39.334 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:39.334 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.334 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.334 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:39.335 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.151117+0000", 2024-04-24T15:32:39.335 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.8560000000000002e-06 2024-04-24T15:32:39.335 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.335 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.335 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:39.335 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.151122+0000", 2024-04-24T15:32:39.335 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5390000000000001e-06 2024-04-24T15:32:39.335 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.335 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.335 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:39.335 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.151124+0000", 2024-04-24T15:32:39.335 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.2800000000000002e-06 2024-04-24T15:32:39.335 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.335 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.336 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:39.336 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.152676+0000", 2024-04-24T15:32:39.336 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.001552124 2024-04-24T15:32:39.336 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.336 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.336 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:39.336 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.152759+0000", 2024-04-24T15:32:39.336 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.2688000000000005e-05 2024-04-24T15:32:39.336 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.336 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.336 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 6", 2024-04-24T15:32:39.336 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.152867+0000", 2024-04-24T15:32:39.336 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00010779400000000001 2024-04-24T15:32:39.336 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.337 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.337 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:39.337 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.157259+0000", 2024-04-24T15:32:39.337 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0043918230000000004 2024-04-24T15:32:39.337 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.337 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.337 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:39.337 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.157498+0000", 2024-04-24T15:32:39.337 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000239829 2024-04-24T15:32:39.337 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.337 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.337 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:39.337 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.157525+0000", 2024-04-24T15:32:39.337 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.6868000000000001e-05 2024-04-24T15:32:39.338 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.338 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.338 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:39.338 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.157554+0000", 2024-04-24T15:32:39.338 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8602000000000002e-05 2024-04-24T15:32:39.338 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:39.338 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:39.338 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:39.338 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.338 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.338 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:3618 4.18 4:1f7d02ce:::rbd_data.114a1f60cb66.0000000000000161:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:39.338 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.581260+0000", 2024-04-24T15:32:39.338 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 14.733281297, 2024-04-24T15:32:39.338 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0068184120000000003, 2024-04-24T15:32:39.339 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:39.339 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:39.339 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:39.339 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:39.339 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:39.339 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 3618 2024-04-24T15:32:39.339 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.339 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:39.339 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.339 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:39.339 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.581260+0000", 2024-04-24T15:32:39.339 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:39.339 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.340 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.340 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:39.340 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.581260+0000", 2024-04-24T15:32:39.340 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:39.340 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.340 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.340 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:39.340 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.581259+0000", 2024-04-24T15:32:39.340 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999999 2024-04-24T15:32:39.340 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.340 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.340 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:39.340 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.581265+0000", 2024-04-24T15:32:39.340 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.003e-06 2024-04-24T15:32:39.341 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.341 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.341 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:39.341 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.581278+0000", 2024-04-24T15:32:39.341 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.2483000000000001e-05 2024-04-24T15:32:39.341 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.341 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.341 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:39.341 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.581284+0000", 2024-04-24T15:32:39.341 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.7399999999999998e-06 2024-04-24T15:32:39.341 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.341 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.341 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:39.341 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.581783+0000", 2024-04-24T15:32:39.341 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00049892199999999995 2024-04-24T15:32:39.342 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.342 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.342 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:39.342 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.581902+0000", 2024-04-24T15:32:39.342 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000118272 2024-04-24T15:32:39.342 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.342 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.342 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 6", 2024-04-24T15:32:39.342 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.583068+0000", 2024-04-24T15:32:39.342 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0011665759999999999 2024-04-24T15:32:39.342 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.342 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.342 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:39.343 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.587081+0000", 2024-04-24T15:32:39.343 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.004012781 2024-04-24T15:32:39.343 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.343 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.343 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:39.343 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.587997+0000", 2024-04-24T15:32:39.343 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00091637700000000001 2024-04-24T15:32:39.343 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.343 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.343 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:39.343 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.588041+0000", 2024-04-24T15:32:39.343 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3619999999999999e-05 2024-04-24T15:32:39.343 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.344 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.344 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:39.344 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.588079+0000", 2024-04-24T15:32:39.344 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.7654000000000002e-05 2024-04-24T15:32:39.344 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:39.344 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:39.344 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:39.344 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.344 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.344 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:3767 4.9 4:908f8f18:::rbd_data.114a1f60cb66.000000000000001b:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:39.344 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.699557+0000", 2024-04-24T15:32:39.344 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 14.614984572000001, 2024-04-24T15:32:39.344 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0088231639999999997, 2024-04-24T15:32:39.344 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:39.345 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:39.345 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:39.345 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:39.345 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:39.345 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 3767 2024-04-24T15:32:39.345 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.345 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:39.345 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.345 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:39.345 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699557+0000", 2024-04-24T15:32:39.345 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:39.345 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.345 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.345 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:39.346 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699557+0000", 2024-04-24T15:32:39.346 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:39.346 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.346 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.346 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:39.346 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699556+0000", 2024-04-24T15:32:39.346 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999999 2024-04-24T15:32:39.346 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.346 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.346 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:39.346 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699561+0000", 2024-04-24T15:32:39.346 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.0440000000000003e-06 2024-04-24T15:32:39.346 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.346 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.347 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:39.347 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699571+0000", 2024-04-24T15:32:39.347 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0237999999999999e-05 2024-04-24T15:32:39.347 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.347 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.347 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:39.347 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699582+0000", 2024-04-24T15:32:39.347 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.1009000000000001e-05 2024-04-24T15:32:39.347 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.347 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.347 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:39.347 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699607+0000", 2024-04-24T15:32:39.347 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.5024999999999998e-05 2024-04-24T15:32:39.347 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.348 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.348 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:39.348 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699671+0000", 2024-04-24T15:32:39.348 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.3219999999999994e-05 2024-04-24T15:32:39.348 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.348 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.348 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:39.348 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.700952+0000", 2024-04-24T15:32:39.348 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.001281952 2024-04-24T15:32:39.348 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.348 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.348 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:39.348 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.705912+0000", 2024-04-24T15:32:39.348 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0049599889999999997 2024-04-24T15:32:39.349 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.349 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.349 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:39.349 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.708300+0000", 2024-04-24T15:32:39.349 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.002387934 2024-04-24T15:32:39.349 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.349 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.349 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:39.349 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.708345+0000", 2024-04-24T15:32:39.349 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4175000000000003e-05 2024-04-24T15:32:39.349 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.349 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.349 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:39.349 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.708380+0000", 2024-04-24T15:32:39.349 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5460999999999998e-05 2024-04-24T15:32:39.350 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:39.350 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:39.350 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:39.350 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.350 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.350 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:3770 4.13 4:cb3c841f:::rbd_data.114a1f60cb66.000000000000001e:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:39.350 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.699597+0000", 2024-04-24T15:32:39.350 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 14.614944108, 2024-04-24T15:32:39.350 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0083605450000000005, 2024-04-24T15:32:39.350 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:39.350 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:39.350 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:39.350 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:39.351 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:39.351 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 3770 2024-04-24T15:32:39.351 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.351 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:39.351 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.351 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:39.351 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699597+0000", 2024-04-24T15:32:39.351 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:39.351 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.351 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.352 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:39.352 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699597+0000", 2024-04-24T15:32:39.352 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:39.352 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.352 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.352 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:39.352 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699597+0000", 2024-04-24T15:32:39.352 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:39.352 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.352 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.352 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:39.352 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699600+0000", 2024-04-24T15:32:39.352 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.371e-06 2024-04-24T15:32:39.352 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.353 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.353 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:39.353 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699611+0000", 2024-04-24T15:32:39.353 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0552e-05 2024-04-24T15:32:39.353 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.353 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.353 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:39.353 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699615+0000", 2024-04-24T15:32:39.353 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5689999999999998e-06 2024-04-24T15:32:39.353 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.353 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.353 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:39.353 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699692+0000", 2024-04-24T15:32:39.354 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.6664999999999999e-05 2024-04-24T15:32:39.354 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.354 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.354 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:39.354 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699740+0000", 2024-04-24T15:32:39.354 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8032999999999999e-05 2024-04-24T15:32:39.354 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.354 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.354 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 6", 2024-04-24T15:32:39.354 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.700940+0000", 2024-04-24T15:32:39.354 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.001199978 2024-04-24T15:32:39.354 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.354 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.354 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:39.355 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.706397+0000", 2024-04-24T15:32:39.355 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0054569459999999998 2024-04-24T15:32:39.355 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.355 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.355 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:39.355 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.707870+0000", 2024-04-24T15:32:39.355 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.001473294 2024-04-24T15:32:39.355 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.355 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.355 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:39.355 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.707920+0000", 2024-04-24T15:32:39.355 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.994e-05 2024-04-24T15:32:39.355 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.356 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.356 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:39.356 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.707958+0000", 2024-04-24T15:32:39.356 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.7761e-05 2024-04-24T15:32:39.356 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:39.356 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:39.356 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:39.356 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.356 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.356 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:3773 4.e 4:7173e872:::rbd_data.114a1f60cb66.0000000000000021:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:39.356 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.699632+0000", 2024-04-24T15:32:39.356 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 14.614909545, 2024-04-24T15:32:39.356 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0068017260000000001, 2024-04-24T15:32:39.357 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:39.357 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:39.357 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:39.357 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:39.357 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:39.357 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 3773 2024-04-24T15:32:39.357 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.357 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:39.357 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.357 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:39.357 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699632+0000", 2024-04-24T15:32:39.357 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:39.357 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.358 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.358 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:39.358 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699632+0000", 2024-04-24T15:32:39.358 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:39.358 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.358 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.358 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:39.358 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699631+0000", 2024-04-24T15:32:39.358 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:39.358 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.358 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.358 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:39.359 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699639+0000", 2024-04-24T15:32:39.359 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7319999999999998e-06 2024-04-24T15:32:39.359 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.359 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.359 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:39.359 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699646+0000", 2024-04-24T15:32:39.359 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1330000000000001e-06 2024-04-24T15:32:39.359 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.359 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.359 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:39.359 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699651+0000", 2024-04-24T15:32:39.359 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.9200000000000003e-06 2024-04-24T15:32:39.359 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.360 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.360 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:39.360 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699689+0000", 2024-04-24T15:32:39.360 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.7979999999999999e-05 2024-04-24T15:32:39.360 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.360 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.360 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:39.360 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699759+0000", 2024-04-24T15:32:39.360 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.9468000000000006e-05 2024-04-24T15:32:39.360 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.360 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.360 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 6", 2024-04-24T15:32:39.360 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.701164+0000", 2024-04-24T15:32:39.361 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0014056629999999999 2024-04-24T15:32:39.361 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.361 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.361 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:39.361 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.705975+0000", 2024-04-24T15:32:39.361 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0048109340000000002 2024-04-24T15:32:39.361 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.361 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.361 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:39.361 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.706359+0000", 2024-04-24T15:32:39.361 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00038331699999999999 2024-04-24T15:32:39.361 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.361 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.362 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:39.362 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.706407+0000", 2024-04-24T15:32:39.362 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8544999999999997e-05 2024-04-24T15:32:39.362 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.362 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.362 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:39.362 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.706434+0000", 2024-04-24T15:32:39.362 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.6553000000000001e-05 2024-04-24T15:32:39.362 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:39.362 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:39.362 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:39.362 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.362 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.363 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:3794 4.18 4:1efc862d:::rbd_data.114a1f60cb66.0000000000000024:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:39.363 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.723446+0000", 2024-04-24T15:32:39.363 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 14.591095026, 2024-04-24T15:32:39.363 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.007572957, 2024-04-24T15:32:39.363 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:39.363 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:39.363 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:39.363 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:39.363 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:39.363 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 3794 2024-04-24T15:32:39.363 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.363 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:39.363 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.363 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:39.364 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723446+0000", 2024-04-24T15:32:39.364 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:39.364 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.364 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.364 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:39.364 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723446+0000", 2024-04-24T15:32:39.364 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:39.364 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.364 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.364 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:39.364 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723446+0000", 2024-04-24T15:32:39.364 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999999 2024-04-24T15:32:39.364 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.364 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.365 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:39.365 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723450+0000", 2024-04-24T15:32:39.365 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.1729999999999997e-06 2024-04-24T15:32:39.365 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.365 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.365 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:39.365 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723458+0000", 2024-04-24T15:32:39.365 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.653e-06 2024-04-24T15:32:39.365 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.365 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.365 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:39.365 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723465+0000", 2024-04-24T15:32:39.365 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.4450000000000001e-06 2024-04-24T15:32:39.365 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.365 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.366 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:39.366 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723501+0000", 2024-04-24T15:32:39.366 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5726000000000003e-05 2024-04-24T15:32:39.366 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.366 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.366 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:39.366 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723547+0000", 2024-04-24T15:32:39.366 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6479e-05 2024-04-24T15:32:39.366 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.366 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.366 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 6", 2024-04-24T15:32:39.366 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.724384+0000", 2024-04-24T15:32:39.366 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00083723800000000002 2024-04-24T15:32:39.366 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.367 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.367 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:39.367 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.730925+0000", 2024-04-24T15:32:39.367 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0065409179999999997 2024-04-24T15:32:39.367 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.367 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.367 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:39.367 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.730969+0000", 2024-04-24T15:32:39.367 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3869999999999998e-05 2024-04-24T15:32:39.367 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.367 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.367 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:39.367 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.730997+0000", 2024-04-24T15:32:39.367 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7783000000000001e-05 2024-04-24T15:32:39.367 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.368 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.368 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:39.368 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.731019+0000", 2024-04-24T15:32:39.368 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.2427999999999999e-05 2024-04-24T15:32:39.368 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:39.368 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:39.368 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:39.368 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.368 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.368 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:3797 4.18 4:1d8bdb81:::rbd_data.114a1f60cb66.0000000000000027:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:39.368 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.723479+0000", 2024-04-24T15:32:39.368 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 14.591062016, 2024-04-24T15:32:39.368 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.013378915, 2024-04-24T15:32:39.368 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:39.368 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:39.369 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:39.369 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:39.369 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:39.369 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 3797 2024-04-24T15:32:39.369 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.369 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:39.369 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.369 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:39.369 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723479+0000", 2024-04-24T15:32:39.369 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:39.369 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.369 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.369 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:39.369 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723479+0000", 2024-04-24T15:32:39.370 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:39.370 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.370 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.370 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:39.370 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723479+0000", 2024-04-24T15:32:39.370 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:39.370 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.370 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.370 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:39.370 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723482+0000", 2024-04-24T15:32:39.370 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5410000000000001e-06 2024-04-24T15:32:39.370 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.370 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.370 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:39.371 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723492+0000", 2024-04-24T15:32:39.371 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.6879999999999994e-06 2024-04-24T15:32:39.371 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.371 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.371 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:39.371 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723496+0000", 2024-04-24T15:32:39.371 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.34e-06 2024-04-24T15:32:39.371 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.371 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.371 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:39.371 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.728008+0000", 2024-04-24T15:32:39.371 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0045115049999999999 2024-04-24T15:32:39.372 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.372 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.372 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:39.372 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.728058+0000", 2024-04-24T15:32:39.372 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.0201000000000003e-05 2024-04-24T15:32:39.372 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.372 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.372 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 6", 2024-04-24T15:32:39.372 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.728499+0000", 2024-04-24T15:32:39.372 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00044129299999999999 2024-04-24T15:32:39.372 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.372 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.372 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:39.372 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.730962+0000", 2024-04-24T15:32:39.373 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0024628430000000002 2024-04-24T15:32:39.373 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.373 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.373 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:39.373 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.736810+0000", 2024-04-24T15:32:39.373 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0058474490000000002 2024-04-24T15:32:39.373 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.373 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.373 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:39.373 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.736837+0000", 2024-04-24T15:32:39.373 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7251999999999999e-05 2024-04-24T15:32:39.373 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.373 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.373 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:39.374 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.736858+0000", 2024-04-24T15:32:39.374 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.1387999999999999e-05 2024-04-24T15:32:39.374 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:39.374 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:39.374 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:39.374 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.374 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.374 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:3798 4.18 4:1e938922:::rbd_data.114a1f60cb66.0000000000000028:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:39.374 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.723506+0000", 2024-04-24T15:32:39.374 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 14.591035602, 2024-04-24T15:32:39.374 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.013398188, 2024-04-24T15:32:39.374 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:39.374 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:39.375 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:39.375 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:39.375 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:39.375 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 3798 2024-04-24T15:32:39.375 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.375 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:39.375 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.375 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:39.375 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723506+0000", 2024-04-24T15:32:39.375 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:39.375 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.375 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.375 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:39.375 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723506+0000", 2024-04-24T15:32:39.376 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:39.376 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.376 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.376 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:39.376 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723505+0000", 2024-04-24T15:32:39.376 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:39.376 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.376 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.376 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:39.376 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723509+0000", 2024-04-24T15:32:39.376 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.2349999999999999e-06 2024-04-24T15:32:39.376 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.376 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.376 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:39.377 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723515+0000", 2024-04-24T15:32:39.377 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.3239999999999999e-06 2024-04-24T15:32:39.377 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.377 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.377 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:39.377 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723519+0000", 2024-04-24T15:32:39.377 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8580000000000002e-06 2024-04-24T15:32:39.377 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.377 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.377 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:39.377 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.731045+0000", 2024-04-24T15:32:39.377 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0075259309999999996 2024-04-24T15:32:39.377 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.377 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.377 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:39.378 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.731090+0000", 2024-04-24T15:32:39.378 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4805999999999998e-05 2024-04-24T15:32:39.378 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.378 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.378 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 6", 2024-04-24T15:32:39.378 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.731571+0000", 2024-04-24T15:32:39.378 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00048136400000000002 2024-04-24T15:32:39.378 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.378 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.378 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:39.378 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.736784+0000", 2024-04-24T15:32:39.378 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0052135269999999999 2024-04-24T15:32:39.378 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.379 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.379 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:39.379 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.736865+0000", 2024-04-24T15:32:39.379 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0141000000000004e-05 2024-04-24T15:32:39.379 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.379 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.379 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:39.379 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.736887+0000", 2024-04-24T15:32:39.379 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.2194000000000001e-05 2024-04-24T15:32:39.379 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.379 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.379 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:39.379 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.736904+0000", 2024-04-24T15:32:39.380 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.7302000000000001e-05 2024-04-24T15:32:39.380 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:39.380 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:39.380 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:39.380 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.380 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.380 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:3801 4.18 4:1b4f38fa:::rbd_data.114a1f60cb66.000000000000002b:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:39.380 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.723629+0000", 2024-04-24T15:32:39.380 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 14.590912458, 2024-04-24T15:32:39.380 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.013474534, 2024-04-24T15:32:39.380 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:39.380 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:39.380 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:39.381 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:39.381 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:39.381 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 3801 2024-04-24T15:32:39.381 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.381 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:39.381 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.381 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:39.381 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723629+0000", 2024-04-24T15:32:39.381 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:39.381 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.381 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.381 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:39.381 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723629+0000", 2024-04-24T15:32:39.381 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:39.382 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.382 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.382 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:39.382 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723628+0000", 2024-04-24T15:32:39.382 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:39.382 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.382 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.382 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:39.382 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723632+0000", 2024-04-24T15:32:39.382 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.4690000000000002e-06 2024-04-24T15:32:39.382 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.382 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.383 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:39.383 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723640+0000", 2024-04-24T15:32:39.383 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3990000000000004e-06 2024-04-24T15:32:39.383 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.383 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.383 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:39.383 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723645+0000", 2024-04-24T15:32:39.383 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3699999999999997e-06 2024-04-24T15:32:39.383 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.383 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.383 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:39.383 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.733996+0000", 2024-04-24T15:32:39.383 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.010351111 2024-04-24T15:32:39.383 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.384 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.384 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:39.384 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.734045+0000", 2024-04-24T15:32:39.384 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.9166000000000003e-05 2024-04-24T15:32:39.384 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.384 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.384 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 6", 2024-04-24T15:32:39.384 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.734461+0000", 2024-04-24T15:32:39.384 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00041584 2024-04-24T15:32:39.384 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.384 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.384 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:39.384 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.736806+0000", 2024-04-24T15:32:39.385 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.002345232 2024-04-24T15:32:39.385 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.385 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.385 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:39.385 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.737065+0000", 2024-04-24T15:32:39.385 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00025913100000000001 2024-04-24T15:32:39.385 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.385 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.385 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:39.385 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.737086+0000", 2024-04-24T15:32:39.385 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.1180000000000001e-05 2024-04-24T15:32:39.385 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.385 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.386 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:39.386 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.737104+0000", 2024-04-24T15:32:39.386 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.7238999999999999e-05 2024-04-24T15:32:39.386 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:39.386 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:39.386 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:39.386 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.386 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.386 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:3896 4.1b 4:dd750b71:::rbd_data.114a1f60cb66.0000000000000058:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:39.386 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.797438+0000", 2024-04-24T15:32:39.386 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 14.517103233, 2024-04-24T15:32:39.386 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0070008420000000002, 2024-04-24T15:32:39.387 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:39.387 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:39.387 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:39.387 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:39.387 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:39.387 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 3896 2024-04-24T15:32:39.387 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.387 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:39.387 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.387 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:39.387 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797438+0000", 2024-04-24T15:32:39.387 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:39.387 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.388 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.388 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:39.388 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797438+0000", 2024-04-24T15:32:39.388 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:39.388 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.388 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.388 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:39.388 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797438+0000", 2024-04-24T15:32:39.388 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:39.388 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.388 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.388 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:39.388 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797441+0000", 2024-04-24T15:32:39.388 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8779999999999998e-06 2024-04-24T15:32:39.389 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.389 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.389 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:39.389 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797447+0000", 2024-04-24T15:32:39.389 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.6989999999999997e-06 2024-04-24T15:32:39.389 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.389 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.389 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:39.389 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797451+0000", 2024-04-24T15:32:39.389 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9689999999999996e-06 2024-04-24T15:32:39.389 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.389 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.389 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:39.389 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797915+0000", 2024-04-24T15:32:39.389 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00046327400000000001 2024-04-24T15:32:39.390 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.390 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.390 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:39.390 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797993+0000", 2024-04-24T15:32:39.390 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.8584000000000005e-05 2024-04-24T15:32:39.390 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.390 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.390 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:39.390 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.798090+0000", 2024-04-24T15:32:39.390 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.6638000000000005e-05 2024-04-24T15:32:39.390 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.390 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.390 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:39.390 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.799021+0000", 2024-04-24T15:32:39.391 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00093112299999999996 2024-04-24T15:32:39.391 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.391 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.391 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:39.391 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.804372+0000", 2024-04-24T15:32:39.391 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0053506789999999997 2024-04-24T15:32:39.391 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.391 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.391 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:39.391 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.804405+0000", 2024-04-24T15:32:39.391 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.3015999999999997e-05 2024-04-24T15:32:39.391 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.391 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.391 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:39.392 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.804439+0000", 2024-04-24T15:32:39.392 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.4431000000000003e-05 2024-04-24T15:32:39.392 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:39.392 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:39.392 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:39.392 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.392 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.392 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:3900 4.1a 4:5b94e8b4:::rbd_data.114a1f60cb66.000000000000006e:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:39.392 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.797456+0000", 2024-04-24T15:32:39.392 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 14.517085532999999, 2024-04-24T15:32:39.392 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.006973524, 2024-04-24T15:32:39.392 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:39.392 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:39.392 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:39.393 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:39.393 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:39.393 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 3900 2024-04-24T15:32:39.393 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.393 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:39.393 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.393 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:39.393 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797456+0000", 2024-04-24T15:32:39.393 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:39.393 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.393 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.393 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:39.393 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797456+0000", 2024-04-24T15:32:39.393 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:39.393 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.394 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.394 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:39.394 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797456+0000", 2024-04-24T15:32:39.394 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:39.394 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.394 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.394 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:39.394 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797458+0000", 2024-04-24T15:32:39.394 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.3209999999999999e-06 2024-04-24T15:32:39.394 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.394 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.394 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:39.394 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797464+0000", 2024-04-24T15:32:39.395 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.7579999999999998e-06 2024-04-24T15:32:39.395 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.395 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.395 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:39.395 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797467+0000", 2024-04-24T15:32:39.395 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.585e-06 2024-04-24T15:32:39.395 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.395 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.395 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:39.395 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797504+0000", 2024-04-24T15:32:39.395 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.7120999999999999e-05 2024-04-24T15:32:39.395 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.395 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.395 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:39.396 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797578+0000", 2024-04-24T15:32:39.396 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.3693000000000001e-05 2024-04-24T15:32:39.396 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.396 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.396 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:39.396 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797675+0000", 2024-04-24T15:32:39.396 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.7454000000000006e-05 2024-04-24T15:32:39.396 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.396 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.396 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:39.396 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.799033+0000", 2024-04-24T15:32:39.396 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.001357104 2024-04-24T15:32:39.396 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.396 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.397 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:39.397 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.804357+0000", 2024-04-24T15:32:39.397 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0053247490000000001 2024-04-24T15:32:39.397 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.397 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.397 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:39.397 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.804397+0000", 2024-04-24T15:32:39.397 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9573000000000002e-05 2024-04-24T15:32:39.397 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.397 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.397 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:39.397 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.804429+0000", 2024-04-24T15:32:39.397 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.2529e-05 2024-04-24T15:32:39.397 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:39.398 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:39.398 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:39.398 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.398 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.398 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:3923 4.9 4:94be6b6c:::rbd_data.114a1f60cb66.0000000000000097:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:39.398 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.806292+0000", 2024-04-24T15:32:39.398 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 14.508249671, 2024-04-24T15:32:39.398 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0062496729999999999, 2024-04-24T15:32:39.398 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:39.398 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:39.398 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:39.398 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:39.398 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:39.399 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 3923 2024-04-24T15:32:39.399 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.399 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:39.399 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.399 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:39.399 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.806292+0000", 2024-04-24T15:32:39.399 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:39.399 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.399 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.399 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:39.399 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.806292+0000", 2024-04-24T15:32:39.399 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:39.399 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.400 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.400 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:39.400 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.806291+0000", 2024-04-24T15:32:39.400 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999999 2024-04-24T15:32:39.400 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.400 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.400 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:39.400 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.806296+0000", 2024-04-24T15:32:39.400 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5730000000000001e-06 2024-04-24T15:32:39.400 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.400 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.400 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:39.400 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.806306+0000", 2024-04-24T15:32:39.400 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0409e-05 2024-04-24T15:32:39.401 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.401 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.401 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:39.401 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.806311+0000", 2024-04-24T15:32:39.401 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.7129999999999996e-06 2024-04-24T15:32:39.401 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.401 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.401 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:39.401 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.806350+0000", 2024-04-24T15:32:39.401 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9351999999999999e-05 2024-04-24T15:32:39.401 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.401 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.401 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:39.402 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.806431+0000", 2024-04-24T15:32:39.402 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.1050000000000005e-05 2024-04-24T15:32:39.402 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.402 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.402 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:39.402 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.806539+0000", 2024-04-24T15:32:39.402 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000107723 2024-04-24T15:32:39.402 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.402 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.402 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:39.402 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.809607+0000", 2024-04-24T15:32:39.402 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.003068468 2024-04-24T15:32:39.402 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.403 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.403 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:39.403 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.812493+0000", 2024-04-24T15:32:39.403 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0028862689999999999 2024-04-24T15:32:39.403 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.403 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.403 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:39.403 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.812521+0000", 2024-04-24T15:32:39.403 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7226e-05 2024-04-24T15:32:39.403 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.403 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.403 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:39.403 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.812541+0000", 2024-04-24T15:32:39.403 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.0729000000000001e-05 2024-04-24T15:32:39.404 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:39.404 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:39.404 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:39.404 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.404 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.404 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:3975 4.1b 4:d9227084:::rbd_data.114a1f60cb66.000000000000015f:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:39.404 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.827427+0000", 2024-04-24T15:32:39.404 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 14.487114226999999, 2024-04-24T15:32:39.404 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0061324140000000001, 2024-04-24T15:32:39.404 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:39.404 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:39.404 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:39.404 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:39.404 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:39.405 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 3975 2024-04-24T15:32:39.405 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.405 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:39.405 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.405 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:39.405 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.827427+0000", 2024-04-24T15:32:39.405 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:39.405 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.405 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.405 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:39.405 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.827427+0000", 2024-04-24T15:32:39.405 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:39.405 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.405 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.406 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:39.406 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.827427+0000", 2024-04-24T15:32:39.406 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:39.406 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.406 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.406 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:39.406 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.827431+0000", 2024-04-24T15:32:39.406 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2509999999999996e-06 2024-04-24T15:32:39.406 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.406 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.406 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:39.406 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.827440+0000", 2024-04-24T15:32:39.406 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.816e-06 2024-04-24T15:32:39.406 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.407 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.407 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:39.407 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.827445+0000", 2024-04-24T15:32:39.407 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.2059999999999998e-06 2024-04-24T15:32:39.407 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.407 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.407 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:39.407 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.827872+0000", 2024-04-24T15:32:39.407 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00042740299999999997 2024-04-24T15:32:39.407 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.407 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.407 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:39.407 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.827941+0000", 2024-04-24T15:32:39.407 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.8652000000000006e-05 2024-04-24T15:32:39.408 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.408 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.408 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:39.408 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.828048+0000", 2024-04-24T15:32:39.408 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000106941 2024-04-24T15:32:39.408 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.408 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.408 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:39.408 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.831351+0000", 2024-04-24T15:32:39.408 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0033031010000000001 2024-04-24T15:32:39.408 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.408 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.408 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:39.408 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.833495+0000", 2024-04-24T15:32:39.409 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0021436379999999998 2024-04-24T15:32:39.409 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.409 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.409 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:39.409 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.833526+0000", 2024-04-24T15:32:39.409 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.1436999999999997e-05 2024-04-24T15:32:39.409 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.409 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.409 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:39.409 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.833560+0000", 2024-04-24T15:32:39.409 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.3560999999999999e-05 2024-04-24T15:32:39.409 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:39.409 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:39.409 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:39.410 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.410 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.410 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:4010 4.1 4:813d513a:::rbd_data.114a1f60cb66.00000000000001c1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:39.410 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.845178+0000", 2024-04-24T15:32:39.410 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 14.469363878999999, 2024-04-24T15:32:39.410 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.007702108, 2024-04-24T15:32:39.410 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:39.410 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:39.410 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:39.410 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:39.410 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:39.410 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 4010 2024-04-24T15:32:39.411 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.411 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:39.411 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.411 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:39.411 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.845178+0000", 2024-04-24T15:32:39.411 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:39.411 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.411 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.411 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:39.411 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.845178+0000", 2024-04-24T15:32:39.411 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:39.411 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.411 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.411 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:39.411 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.845177+0000", 2024-04-24T15:32:39.412 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:39.412 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.412 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.412 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:39.412 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.845180+0000", 2024-04-24T15:32:39.412 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8119999999999999e-06 2024-04-24T15:32:39.412 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.412 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.412 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:39.412 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.845186+0000", 2024-04-24T15:32:39.412 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.9649999999999996e-06 2024-04-24T15:32:39.412 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.412 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.412 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:39.413 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.845189+0000", 2024-04-24T15:32:39.413 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.6040000000000001e-06 2024-04-24T15:32:39.413 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.413 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.413 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:39.413 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.845220+0000", 2024-04-24T15:32:39.413 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.0207000000000001e-05 2024-04-24T15:32:39.413 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.413 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.413 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:39.413 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.845262+0000", 2024-04-24T15:32:39.413 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2716000000000002e-05 2024-04-24T15:32:39.413 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.414 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.414 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 5", 2024-04-24T15:32:39.414 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.845326+0000", 2024-04-24T15:32:39.414 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.3490000000000004e-05 2024-04-24T15:32:39.414 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.414 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.414 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:39.414 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.849620+0000", 2024-04-24T15:32:39.414 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0042944769999999997 2024-04-24T15:32:39.414 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.414 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.414 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:39.414 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.852802+0000", 2024-04-24T15:32:39.414 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.003181798 2024-04-24T15:32:39.415 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.415 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.415 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:39.415 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.852846+0000", 2024-04-24T15:32:39.415 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3606000000000002e-05 2024-04-24T15:32:39.415 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.415 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.415 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:39.415 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.852880+0000", 2024-04-24T15:32:39.415 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.3908999999999997e-05 2024-04-24T15:32:39.415 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:39.415 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:39.415 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:39.415 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.415 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.416 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:4256 4.13 4:c8361c81:::rbd_data.114a1f60cb66.0000000000000316:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:39.416 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:25.054726+0000", 2024-04-24T15:32:39.416 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 14.259815435, 2024-04-24T15:32:39.416 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0079439720000000005, 2024-04-24T15:32:39.416 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:39.416 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:39.416 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:39.416 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:39.416 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:39.416 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 4256 2024-04-24T15:32:39.416 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.416 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:39.416 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.416 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:39.417 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.054726+0000", 2024-04-24T15:32:39.417 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:39.417 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.417 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.417 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:39.417 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.054726+0000", 2024-04-24T15:32:39.417 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:39.417 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.417 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.417 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:39.417 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.054725+0000", 2024-04-24T15:32:39.417 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999999 2024-04-24T15:32:39.417 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.417 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.417 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:39.418 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.054730+0000", 2024-04-24T15:32:39.418 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.4729999999999999e-06 2024-04-24T15:32:39.418 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.418 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.418 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:39.418 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.054742+0000", 2024-04-24T15:32:39.418 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.1489e-05 2024-04-24T15:32:39.418 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.418 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.418 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:39.418 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.054748+0000", 2024-04-24T15:32:39.418 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.879e-06 2024-04-24T15:32:39.418 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.419 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.419 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:39.419 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.054791+0000", 2024-04-24T15:32:39.419 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2673999999999998e-05 2024-04-24T15:32:39.419 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.419 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.419 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:39.419 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.054872+0000", 2024-04-24T15:32:39.419 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.1254999999999994e-05 2024-04-24T15:32:39.419 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.419 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.419 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 6", 2024-04-24T15:32:39.419 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.054978+0000", 2024-04-24T15:32:39.419 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00010624 2024-04-24T15:32:39.420 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.420 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.420 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:39.420 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.055893+0000", 2024-04-24T15:32:39.420 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00091453300000000003 2024-04-24T15:32:39.420 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.420 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.420 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:39.420 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.062620+0000", 2024-04-24T15:32:39.420 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0067271839999999998 2024-04-24T15:32:39.420 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.420 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.420 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:39.420 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.062645+0000", 2024-04-24T15:32:39.421 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.5491e-05 2024-04-24T15:32:39.421 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.421 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.421 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:39.421 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.062670+0000", 2024-04-24T15:32:39.421 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.474e-05 2024-04-24T15:32:39.421 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:39.421 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:39.421 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:39.421 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.421 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.421 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:7398 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-04-24T15:32:39.421 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:27.771937+0000", 2024-04-24T15:32:39.421 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 11.542604084000001, 2024-04-24T15:32:39.421 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.015783228, 2024-04-24T15:32:39.422 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:39.422 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:39.422 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:39.422 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:39.422 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:39.422 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 7398 2024-04-24T15:32:39.422 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.422 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:39.422 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.422 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:39.422 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:27.771937+0000", 2024-04-24T15:32:39.422 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:39.422 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.422 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.423 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:39.423 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:27.771937+0000", 2024-04-24T15:32:39.423 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:39.423 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.423 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.423 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:39.423 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:27.771936+0000", 2024-04-24T15:32:39.423 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999986 2024-04-24T15:32:39.423 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.423 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.423 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:39.423 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:27.771944+0000", 2024-04-24T15:32:39.423 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.8960000000000003e-06 2024-04-24T15:32:39.423 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.423 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.423 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:39.424 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:27.771955+0000", 2024-04-24T15:32:39.424 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0951000000000001e-05 2024-04-24T15:32:39.424 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.424 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.424 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:39.424 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:27.771959+0000", 2024-04-24T15:32:39.424 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.69e-06 2024-04-24T15:32:39.424 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.424 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.424 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:39.424 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:27.771998+0000", 2024-04-24T15:32:39.424 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8705000000000003e-05 2024-04-24T15:32:39.424 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.424 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.424 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:39.425 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:27.772054+0000", 2024-04-24T15:32:39.425 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.5420000000000001e-05 2024-04-24T15:32:39.425 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.425 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.425 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:39.425 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:27.774882+0000", 2024-04-24T15:32:39.425 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.002828344 2024-04-24T15:32:39.425 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.425 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.425 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:39.425 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:27.787163+0000", 2024-04-24T15:32:39.425 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.012281556000000001 2024-04-24T15:32:39.425 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.425 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.426 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:39.426 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:27.787651+0000", 2024-04-24T15:32:39.426 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00048708600000000002 2024-04-24T15:32:39.426 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.426 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.426 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:39.426 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:27.787691+0000", 2024-04-24T15:32:39.426 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0531999999999998e-05 2024-04-24T15:32:39.426 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.426 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.426 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:39.426 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:27.787721+0000", 2024-04-24T15:32:39.427 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.9518000000000001e-05 2024-04-24T15:32:39.427 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:39.427 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:39.427 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:39.427 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.427 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.427 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:7825 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-04-24T15:32:39.427 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:28.311523+0000", 2024-04-24T15:32:39.427 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 11.003018024999999, 2024-04-24T15:32:39.427 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.019381513, 2024-04-24T15:32:39.427 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:39.427 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:39.427 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:39.427 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:39.428 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:39.428 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 7825 2024-04-24T15:32:39.428 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.428 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:39.428 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.428 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:39.428 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:28.311523+0000", 2024-04-24T15:32:39.428 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:39.428 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.428 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.428 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:39.428 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:28.311523+0000", 2024-04-24T15:32:39.428 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:39.428 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.429 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.429 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:39.429 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:28.311523+0000", 2024-04-24T15:32:39.429 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:39.429 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.429 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.429 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:39.429 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:28.311527+0000", 2024-04-24T15:32:39.429 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6290000000000001e-06 2024-04-24T15:32:39.429 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.429 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.429 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:39.429 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:28.311538+0000", 2024-04-24T15:32:39.429 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0239e-05 2024-04-24T15:32:39.430 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.430 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.430 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:39.430 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:28.311542+0000", 2024-04-24T15:32:39.430 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3059999999999999e-06 2024-04-24T15:32:39.430 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.430 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.430 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:39.430 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:28.311621+0000", 2024-04-24T15:32:39.430 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.8671999999999995e-05 2024-04-24T15:32:39.430 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.430 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.430 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:39.431 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:28.311667+0000", 2024-04-24T15:32:39.431 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6842999999999999e-05 2024-04-24T15:32:39.431 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.431 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.431 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:39.431 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:28.315077+0000", 2024-04-24T15:32:39.431 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0034100889999999998 2024-04-24T15:32:39.431 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.431 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.431 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:39.431 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:28.330557+0000", 2024-04-24T15:32:39.431 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.015479100000000001 2024-04-24T15:32:39.431 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.431 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.431 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:39.432 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:28.330843+0000", 2024-04-24T15:32:39.432 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000286526 2024-04-24T15:32:39.432 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.432 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.432 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:39.432 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:28.330877+0000", 2024-04-24T15:32:39.432 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.4119999999999999e-05 2024-04-24T15:32:39.432 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:39.432 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:39.432 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:39.432 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:28.330905+0000", 2024-04-24T15:32:39.432 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7688999999999999e-05 2024-04-24T15:32:39.432 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:39.432 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:39.433 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:39.433 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:39.433 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:39.433 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:32:39.434 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-04-24T15:32:39.434 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-04-24T15:32:39.434 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-04-24T15:32:39.434 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-04-24T15:32:39.435 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-24T15:32:39.435 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:32:39.435+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:32:39.446 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-04-24T15:32:39.464 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-04-24T15:32:39.482 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-04-24T15:32:39.500 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-04-24T15:32:39.501 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T15:32:39.502 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T15:32:39.502 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T15:32:39.502 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T15:32:39.502 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T15:32:39.502 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T15:32:39.502 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T15:32:39.502 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T15:32:39.535 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:32:39.536+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:32:39.599 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:39.600 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:39.609 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-24T15:32:39.635 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:32:39.636+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:32:39.738 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:32:39.740+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:32:39.772 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:39.772 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:39.781 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-24T15:32:39.823 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #3, image image_client.0-clone3 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent4 2024-04-24T15:32:39.824 INFO:teuthology.orchestra.run.smithi096.stdout:1375 writesame 0x1fd362c thru 0x1fdabde data_size 0x1(0x75b3 bytes) 2024-04-24T15:32:39.839 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:32:39.841+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:32:39.856 INFO:teuthology.orchestra.run.smithi096.stdout:1376 write 0x2d8f7b thru 0x2df154 (0x61da bytes) 2024-04-24T15:32:39.858 INFO:teuthology.orchestra.run.smithi096.stdout:1378 write 0x7129f43 thru 0x713270d (0x87cb bytes) 2024-04-24T15:32:39.862 INFO:teuthology.orchestra.run.smithi096.stdout:1380 read 0xa72cc1e thru 0xa738d2f (0xc112 bytes) 2024-04-24T15:32:39.862 INFO:teuthology.orchestra.run.smithi096.stdout:1386 write 0x80dd2f2 thru 0x80e8763 (0xb472 bytes) 2024-04-24T15:32:39.865 INFO:teuthology.orchestra.run.smithi096.stdout:1387 write 0xd78437 thru 0xd78d78 (0x942 bytes) 2024-04-24T15:32:39.869 INFO:teuthology.orchestra.run.smithi096.stdout:1388 write 0x582042e thru 0x5824d87 (0x495a bytes) 2024-04-24T15:32:39.874 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:39.874 INFO:teuthology.orchestra.run.smithi096.stdout:1390 write 0xbbc5453 thru 0xbbcee91 (0x9a3f bytes) 2024-04-24T15:32:39.877 INFO:teuthology.orchestra.run.smithi096.stdout:1391 write 0xb196702 thru 0xb199cfa (0x35f9 bytes) 2024-04-24T15:32:39.880 INFO:teuthology.orchestra.run.smithi096.stdout:1392 read 0x8ee8f52 thru 0x8ef4667 (0xb716 bytes) 2024-04-24T15:32:39.881 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:39.881 INFO:teuthology.orchestra.run.smithi096.stdout:1394 read 0xaec45b3 thru 0xaec8b1a (0x4568 bytes) 2024-04-24T15:32:39.909 INFO:teuthology.orchestra.run.smithi096.stdout:1395 write 0x17a69c8 thru 0x17a9029 (0x2662 bytes) 2024-04-24T15:32:39.919 INFO:teuthology.orchestra.run.smithi096.stdout:1397 trunc from 0xee48e6b to 0xa0ce83f 2024-04-24T15:32:39.940 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:32:39.942+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:32:39.947 INFO:teuthology.orchestra.run.smithi096.stdout:1398 write 0xdaaee3d thru 0xdabcef4 (0xe0b8 bytes) 2024-04-24T15:32:39.948 INFO:teuthology.orchestra.run.smithi096.stdout:1399 write 0xa083295 thru 0xa088ab2 (0x581e bytes) 2024-04-24T15:32:39.949 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:39.949 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:39.950 INFO:teuthology.orchestra.run.smithi096.stdout:1400 punch from 0xa4f870e to 0xa501983, (0x9275 bytes) 2024-04-24T15:32:39.950 INFO:teuthology.orchestra.run.smithi096.stdout:1401 read 0x7c2e8dc thru 0x7c33c18 (0x533d bytes) 2024-04-24T15:32:39.950 INFO:teuthology.orchestra.run.smithi096.stdout:1403 write 0x2424d02 thru 0x24316ed (0xc9ec bytes) 2024-04-24T15:32:39.961 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-24T15:32:39.967 INFO:teuthology.orchestra.run.smithi096.stdout:1404 writesame 0x1f0b22c thru 0x1f10790 data_size 0x15(0x5565 bytes) 2024-04-24T15:32:39.969 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:39.969 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:39.969 INFO:teuthology.orchestra.run.smithi096.stdout:1408 write 0xbed4406 thru 0xbee0a9b (0xc696 bytes) 2024-04-24T15:32:39.971 INFO:teuthology.orchestra.run.smithi096.stdout:1409 write 0x92d945 thru 0x932414 (0x4ad0 bytes) 2024-04-24T15:32:39.979 INFO:teuthology.orchestra.run.smithi096.stdout:1410 trunc from 0xdabcef5 to 0x89422e9 2024-04-24T15:32:39.990 INFO:teuthology.orchestra.run.smithi096.stdout:1415 write 0xe6e9db thru 0xe7958d (0xabb3 bytes) 2024-04-24T15:32:39.995 INFO:teuthology.orchestra.run.smithi096.stdout:1416 punch from 0x447c76a to 0x4488f48, (0xc7de bytes) 2024-04-24T15:32:39.995 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:39.999 INFO:teuthology.orchestra.run.smithi096.stdout:1420 write 0x8d0400d thru 0x8d1338c (0xf380 bytes) 2024-04-24T15:32:40.001 INFO:teuthology.orchestra.run.smithi096.stdout:1421 read 0x375f4c5 thru 0x37631a0 (0x3cdc bytes) 2024-04-24T15:32:40.001 INFO:teuthology.orchestra.run.smithi096.stdout:1422 write 0x8672de4 thru 0x8677e93 (0x50b0 bytes) 2024-04-24T15:32:40.002 INFO:teuthology.orchestra.run.smithi096.stdout:1423 write 0xd49424 thru 0xd4d668 (0x4245 bytes) 2024-04-24T15:32:40.004 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:40.007 INFO:teuthology.orchestra.run.smithi096.stdout:1425 trunc from 0x8d1338d to 0xbbb38f7 2024-04-24T15:32:40.011 INFO:teuthology.orchestra.run.smithi096.stdout:1426 write 0x536e5c5 thru 0x5370330 (0x1d6c bytes) 2024-04-24T15:32:40.016 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:40.016 INFO:teuthology.orchestra.run.smithi096.stdout:1429 writesame 0xa84ec88 thru 0xa85323f data_size 0x2e(0x45b8 bytes) 2024-04-24T15:32:40.020 INFO:teuthology.orchestra.run.smithi096.stdout:1431 trunc from 0xbbb38f7 to 0xe103e7c 2024-04-24T15:32:40.025 INFO:teuthology.orchestra.run.smithi096.stdout:1432 read 0x6d073c5 thru 0x6d10c69 (0x98a5 bytes) 2024-04-24T15:32:40.025 INFO:teuthology.orchestra.run.smithi096.stdout:1433 read 0x9261d11 thru 0x926a8c5 (0x8bb5 bytes) 2024-04-24T15:32:40.025 INFO:teuthology.orchestra.run.smithi096.stdout:1435 read 0xcf5eacf thru 0xcf63a33 (0x4f65 bytes) 2024-04-24T15:32:40.026 INFO:teuthology.orchestra.run.smithi096.stdout:1436 write 0xd790bf9 thru 0xd79f5f3 (0xe9fb bytes) 2024-04-24T15:32:40.028 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:40.028 INFO:teuthology.orchestra.run.smithi096.stdout:1440 writesame 0xaf32ff2 thru 0xaf35a40 data_size 0x1(0x2a4f bytes) 2024-04-24T15:32:40.038 INFO:teuthology.orchestra.run.smithi096.stdout:1442 write 0xe3c96ab thru 0xe3d8a53 (0xf3a9 bytes) 2024-04-24T15:32:40.041 INFO:teuthology.orchestra.run.smithi096.stdout:1444 punch from 0x1a8a9fc to 0x1a8ba26, (0x102a bytes) 2024-04-24T15:32:40.041 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:32:40.042+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:32:40.056 INFO:teuthology.orchestra.run.smithi096.stdout:1445 trunc from 0xe3d8a54 to 0x164a5a7 2024-04-24T15:32:40.094 INFO:teuthology.orchestra.run.smithi096.stdout:1446 write 0x30806bb thru 0x308c7d9 (0xc11f bytes) 2024-04-24T15:32:40.095 INFO:teuthology.orchestra.run.smithi096.stdout:1448 punch from 0x15d3561 to 0x15db196, (0x7c35 bytes) 2024-04-24T15:32:40.095 INFO:teuthology.orchestra.run.smithi096.stdout:1450 read 0x2acf245 thru 0x2add08c (0xde48 bytes) 2024-04-24T15:32:40.095 INFO:teuthology.orchestra.run.smithi096.stdout:1451 read 0x1f44b42 thru 0x1f4729b (0x275a bytes) 2024-04-24T15:32:40.096 INFO:teuthology.orchestra.run.smithi096.stdout:1452 write 0x2ea4190 thru 0x2eb2c17 (0xea88 bytes) 2024-04-24T15:32:40.097 INFO:teuthology.orchestra.run.smithi096.stdout:1454 read 0x2e05c1c thru 0x2e101e8 (0xa5cd bytes) 2024-04-24T15:32:40.097 INFO:teuthology.orchestra.run.smithi096.stdout:1455 write 0x2bb276e thru 0x2bbd48b (0xad1e bytes) 2024-04-24T15:32:40.098 INFO:teuthology.orchestra.run.smithi096.stdout:1456 punch from 0x4abb to 0xfe25, (0xb36a bytes) 2024-04-24T15:32:40.115 INFO:teuthology.orchestra.run.smithi096.stdout:1457 write 0xd6c14fa thru 0xd6c1ba2 (0x6a9 bytes) 2024-04-24T15:32:40.117 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:40.118 INFO:teuthology.orchestra.run.smithi096.stdout:1461 writesame 0x28a8779 thru 0x28aeb81 data_size 0x1(0x6409 bytes) 2024-04-24T15:32:40.127 INFO:teuthology.orchestra.run.smithi096.stdout:1462 writesame 0x751998a thru 0x7528a05 data_size 0x2(0xf07c bytes) 2024-04-24T15:32:40.132 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:40.133 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:40.142 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:32:40.143+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:32:40.143 INFO:teuthology.orchestra.run.smithi096.stdout:1463 read 0xaf5773d thru 0xaf58d40 (0x1604 bytes) 2024-04-24T15:32:40.143 INFO:teuthology.orchestra.run.smithi096.stdout:1464 read 0xd34fbd0 thru 0xd35a883 (0xacb4 bytes) 2024-04-24T15:32:40.143 INFO:teuthology.orchestra.run.smithi096.stdout:1465 read 0x9bac8a7 thru 0x9bb7d5f (0xb4b9 bytes) 2024-04-24T15:32:40.143 INFO:teuthology.orchestra.run.smithi096.stdout:1466 write 0x5f68f75 thru 0x5f727a2 (0x982e bytes) 2024-04-24T15:32:40.144 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-24T15:32:40.145 INFO:teuthology.orchestra.run.smithi096.stdout:1468 punch from 0x4314d40 to 0x4320f73, (0xc233 bytes) 2024-04-24T15:32:40.145 INFO:teuthology.orchestra.run.smithi096.stdout:1470 read 0x81b592f thru 0x81bbc95 (0x6367 bytes) 2024-04-24T15:32:40.145 INFO:teuthology.orchestra.run.smithi096.stdout:1471 punch from 0x8e0d063 to 0x8e15691, (0x862e bytes) 2024-04-24T15:32:40.145 INFO:teuthology.orchestra.run.smithi096.stdout:1472 write 0x5177ed6 thru 0x517c6f2 (0x481d bytes) 2024-04-24T15:32:40.150 INFO:teuthology.orchestra.run.smithi096.stdout:1474 read 0x7da6fac thru 0x7daf6a5 (0x86fa bytes) 2024-04-24T15:32:40.150 INFO:teuthology.orchestra.run.smithi096.stdout:1475 write 0x862edc6 thru 0x863695d (0x7b98 bytes) 2024-04-24T15:32:40.150 INFO:teuthology.orchestra.run.smithi096.stdout:1476 punch from 0xc9f72f3 to 0xca03f26, (0xcc33 bytes) 2024-04-24T15:32:40.151 INFO:teuthology.orchestra.run.smithi096.stdout:1478 read 0x56b2feb thru 0x56ba31b (0x7331 bytes) 2024-04-24T15:32:40.151 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:40.151 INFO:teuthology.orchestra.run.smithi096.stdout:1480 read 0x8fc6d32 thru 0x8fc82f7 (0x15c6 bytes) 2024-04-24T15:32:40.151 INFO:teuthology.orchestra.run.smithi096.stdout:1481 punch from 0x384c5b1 to 0x3856b5e, (0xa5ad bytes) 2024-04-24T15:32:40.151 INFO:teuthology.orchestra.run.smithi096.stdout:1482 write 0x99821a4 thru 0x9985859 (0x36b6 bytes) 2024-04-24T15:32:40.152 INFO:teuthology.orchestra.run.smithi096.stdout:1483 writesame 0xa632da7 thru 0xa63bfa3 data_size 0x13(0x91fd bytes) 2024-04-24T15:32:40.155 INFO:teuthology.orchestra.run.smithi096.stdout:1485 read 0x982ea0 thru 0x986b5d (0x3cbe bytes) 2024-04-24T15:32:40.156 INFO:teuthology.orchestra.run.smithi096.stdout:1486 read 0x921465c thru 0x921c41c (0x7dc1 bytes) 2024-04-24T15:32:40.156 INFO:teuthology.orchestra.run.smithi096.stdout:1487 punch from 0x1dbfce2 to 0x1dcc383, (0xc6a1 bytes) 2024-04-24T15:32:40.156 INFO:teuthology.orchestra.run.smithi096.stdout:1489 read 0x67f5047 thru 0x6804421 (0xf3db bytes) 2024-04-24T15:32:40.156 INFO:teuthology.orchestra.run.smithi096.stdout:1490 read 0x8adfba1 thru 0x8ae221a (0x267a bytes) 2024-04-24T15:32:40.157 INFO:teuthology.orchestra.run.smithi096.stdout:1491 write 0xcbcb669 thru 0xcbcbe55 (0x7ed bytes) 2024-04-24T15:32:40.159 INFO:teuthology.orchestra.run.smithi096.stdout:1493 write 0x6654a15 thru 0x6660557 (0xbb43 bytes) 2024-04-24T15:32:40.160 INFO:teuthology.orchestra.run.smithi096.stdout:1494 read 0xcda434d thru 0xcda4468 (0x11c bytes) 2024-04-24T15:32:40.160 INFO:teuthology.orchestra.run.smithi096.stdout:1495 write 0x1aab16e thru 0x1ab54ae (0xa341 bytes) 2024-04-24T15:32:40.163 INFO:teuthology.orchestra.run.smithi096.stdout:1496 read 0xd2b455c thru 0xd2c1ab1 (0xd556 bytes) 2024-04-24T15:32:40.163 INFO:teuthology.orchestra.run.smithi096.stdout:1497 writesame 0x6ed2a79 thru 0x6ed9dfb data_size 0x1(0x7383 bytes) 2024-04-24T15:32:40.176 INFO:teuthology.orchestra.run.smithi096.stdout:1498 write 0x7267ec1 thru 0x726c349 (0x4489 bytes) 2024-04-24T15:32:40.178 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 6, 7] out_osds: [1, 4, 5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-04-24T15:32:40.178 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-24T15:32:40.178 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-04-24T15:32:40.179 INFO:tasks.thrashosds.thrasher:Setting osd 2 primary_affinity to 0.375504 2024-04-24T15:32:40.179 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 0.3755038884266936 2024-04-24T15:32:40.179 INFO:teuthology.orchestra.run.smithi096.stdout:1499 punch from 0x15163e5 to 0x1518636, (0x2251 bytes) 2024-04-24T15:32:40.182 INFO:teuthology.orchestra.run.smithi096.stdout:1500 trunc from 0xd6c1ba3 to 0xb51d2e7 2024-04-24T15:32:40.192 INFO:teuthology.orchestra.run.smithi096.stdout:1501 write 0x3142acc thru 0x314983e (0x6d73 bytes) 2024-04-24T15:32:40.193 INFO:teuthology.orchestra.run.smithi096.stdout:1503 write 0x84f6c3a thru 0x84f8617 (0x19de bytes) 2024-04-24T15:32:40.196 INFO:teuthology.orchestra.run.smithi096.stdout:1505 write 0x367bff4 thru 0x367d8b6 (0x18c3 bytes) 2024-04-24T15:32:40.197 INFO:teuthology.orchestra.run.smithi096.stdout:1506 writesame 0x82441d4 thru 0x824e5b7 data_size 0x22(0xa3e4 bytes) 2024-04-24T15:32:40.211 INFO:teuthology.orchestra.run.smithi096.stdout:1509 trunc from 0xb51d2e7 to 0x12c12d3 2024-04-24T15:32:40.241 INFO:teuthology.orchestra.run.smithi096.stdout:1510 read 0x5ca987 thru 0x5d7470 (0xcaea bytes) 2024-04-24T15:32:40.242 INFO:teuthology.orchestra.run.smithi096.stdout:1511 read 0x984cd1 thru 0x98f109 (0xa439 bytes) 2024-04-24T15:32:40.243 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:32:40.244+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:32:40.252 INFO:teuthology.orchestra.run.smithi096.stdout:1512 writesame 0x5277e76 thru 0x52844b8 data_size 0x1(0xc643 bytes) 2024-04-24T15:32:40.252 INFO:teuthology.orchestra.run.smithi096.stdout:1513 punch from 0x2ece9b8 to 0x2ed7855, (0x8e9d bytes) 2024-04-24T15:32:40.260 INFO:teuthology.orchestra.run.smithi096.stdout:1516 writesame 0x81380d2 thru 0x813d151 data_size 0x70(0x5080 bytes) 2024-04-24T15:32:40.267 INFO:teuthology.orchestra.run.smithi096.stdout:1518 writesame 0x90ca596 thru 0x90d4a96 data_size 0x1(0xa501 bytes) 2024-04-24T15:32:40.267 INFO:teuthology.orchestra.run.smithi096.stdout:1519 read 0x55dd3f0 thru 0x55ea1c1 (0xcdd2 bytes) 2024-04-24T15:32:40.267 INFO:teuthology.orchestra.run.smithi096.stdout:1520 write 0x481412b thru 0x481587c (0x1752 bytes) 2024-04-24T15:32:40.268 INFO:teuthology.orchestra.run.smithi096.stdout:1521 punch from 0x3605502 to 0x36133df, (0xdedd bytes) 2024-04-24T15:32:40.268 INFO:teuthology.orchestra.run.smithi096.stdout:1522 write 0x26510b9 thru 0x2657402 (0x634a bytes) 2024-04-24T15:32:40.269 INFO:teuthology.orchestra.run.smithi096.stdout:1523 writesame 0x12492d3 thru 0x12506b0 data_size 0x1(0x73de bytes) 2024-04-24T15:32:40.291 INFO:teuthology.orchestra.run.smithi096.stdout:1525 write 0xd25b99e thru 0xd2619d0 (0x6033 bytes) 2024-04-24T15:32:40.293 INFO:teuthology.orchestra.run.smithi096.stdout:1526 write 0x2e09f00 thru 0x2e0b2a0 (0x13a1 bytes) 2024-04-24T15:32:40.306 INFO:teuthology.orchestra.run.smithi096.stdout:1527 trunc from 0xd2619d1 to 0x484aa91 2024-04-24T15:32:40.308 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:40.308 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:40.319 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-24T15:32:40.323 INFO:teuthology.orchestra.run.smithi096.stdout:1528 punch from 0x456dd02 to 0x456ed9c, (0x109a bytes) 2024-04-24T15:32:40.323 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:40.331 INFO:teuthology.orchestra.run.smithi096.stdout:1532 write 0x696eb2a thru 0x69733ae (0x4885 bytes) 2024-04-24T15:32:40.334 INFO:teuthology.orchestra.run.smithi096.stdout:1533 punch from 0x56f38dd to 0x570263b, (0xed5e bytes) 2024-04-24T15:32:40.334 INFO:teuthology.orchestra.run.smithi096.stdout:1534 read 0x47bdf84 thru 0x47c2a12 (0x4a8f bytes) 2024-04-24T15:32:40.334 INFO:teuthology.orchestra.run.smithi096.stdout:1535 clone 6 order 24 su 1048576 sc 6 2024-04-24T15:32:40.361 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:32:40.362+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:32:40.445 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:32:40.446+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:32:40.483 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:40.483 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:40.493 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-24T15:32:40.546 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:32:40.547+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:32:40.647 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:32:40.648+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:32:40.653 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-24T15:32:40.656 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:40.656 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:40.666 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-24T15:32:40.748 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:32:40.749+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:32:40.830 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:40.830 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:40.839 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-24T15:32:40.884 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:32:40.885+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:32:40.949 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:32:40.951+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:32:40.962 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:40.962 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:40.973 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-24T15:32:41.050 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:32:41.050+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:32:41.096 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:41.096 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:41.106 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-24T15:32:41.190 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:32:41.190+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:32:41.199 INFO:teuthology.orchestra.run.smithi096.stderr:noscrub is set 2024-04-24T15:32:41.200 INFO:teuthology.orchestra.run.smithi096.stderr:set osd.2 primary-affinity to 0.375504 (8246092) 2024-04-24T15:32:41.257 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:32:41.257+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:32:41.279 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:41.279 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:41.289 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-24T15:32:41.374 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:32:41.374+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:32:41.453 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:41.453 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:41.454 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:32:41.456+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:32:41.463 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-24T15:32:41.513 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-24T15:32:41.556 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:32:41.557+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:32:41.631 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:41.631 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:41.642 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-24T15:32:41.656 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:32:41.657+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:32:41.689 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-04-24T15:32:41.707 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-04-24T15:32:41.726 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-04-24T15:32:41.745 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-04-24T15:32:41.757 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:32:41.757+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:32:41.764 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-04-24T15:32:41.769 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:32:41.769 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:32:41.769 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-04-24T15:32:41.769 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-04-24T15:32:41.769 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:32:41.779 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-24T15:32:41.783 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-04-24T15:32:41.805 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-04-24T15:32:41.825 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-04-24T15:32:41.826 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T15:32:41.826 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T15:32:41.826 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T15:32:41.826 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T15:32:41.826 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T15:32:41.826 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T15:32:41.826 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T15:32:41.826 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T15:32:41.858 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:32:41.860+0000 7f23abcf6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 44021) UID: 0 2024-04-24T15:32:41.910 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:32:41.910 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-04-24T15:32:41.910 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-04-24T15:32:41.910 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [] 2024-04-24T15:32:41.910 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:32:41.920 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-24T15:32:41.959 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:32:41.961+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:32:42.043 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:32:42.043 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:32:42.043 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-04-24T15:32:42.043 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:32:42.053 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-24T15:32:42.096 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:32:42.096+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:32:42.161 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:32:42.162+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:32:42.174 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:32:42.174 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:32:42.174 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-04-24T15:32:42.174 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-04-24T15:32:42.175 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:32:42.184 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-24T15:32:42.252 INFO:teuthology.orchestra.run.smithi096.stdout:truncating image image_client.0-clone5 from 0x69733af (overlap 0x12c12d3) to 0x960969 2024-04-24T15:32:42.261 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:32:42.263+0000 7f23abcf6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 44021) UID: 0 2024-04-24T15:32:42.317 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #4, image image_client.0-clone4 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent5 2024-04-24T15:32:42.362 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:32:42.364+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:32:42.368 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:32:42.368 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-04-24T15:32:42.368 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-04-24T15:32:42.368 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-04-24T15:32:42.368 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.368 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4426.0:2434 4.d e32/26 4:b7840674:::rbd_data.114a1f60cb66.00000000000001a7:head v 32'12, mlcod=32'10)", 2024-04-24T15:32:42.369 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:23.024172+0000", 2024-04-24T15:32:42.369 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.343121517, 2024-04-24T15:32:42.369 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057919520000000004, 2024-04-24T15:32:42.369 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:42.369 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:42.369 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:42.369 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.369 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:42.369 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.024172+0000", 2024-04-24T15:32:42.369 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:42.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.369 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.369 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:42.369 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.024172+0000", 2024-04-24T15:32:42.370 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:42.370 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.370 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.370 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:42.370 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.024171+0000", 2024-04-24T15:32:42.370 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:42.370 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.370 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.370 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:42.370 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.024178+0000", 2024-04-24T15:32:42.370 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2420000000000003e-06 2024-04-24T15:32:42.370 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.371 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.371 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:42.371 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.024191+0000", 2024-04-24T15:32:42.371 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2444e-05 2024-04-24T15:32:42.371 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.371 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.371 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:42.371 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.024197+0000", 2024-04-24T15:32:42.371 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2249999999999997e-06 2024-04-24T15:32:42.371 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.371 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.371 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:42.371 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.024251+0000", 2024-04-24T15:32:42.371 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4221000000000001e-05 2024-04-24T15:32:42.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.372 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.372 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:42.372 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.024270+0000", 2024-04-24T15:32:42.372 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9151999999999999e-05 2024-04-24T15:32:42.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.372 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.372 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:42.372 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.029925+0000", 2024-04-24T15:32:42.372 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0056546160000000003 2024-04-24T15:32:42.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.372 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.372 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:42.373 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.029964+0000", 2024-04-24T15:32:42.373 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8855e-05 2024-04-24T15:32:42.373 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:42.373 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:42.373 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:42.373 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.373 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.373 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:2450 4.1c 4:3f9dbb33:::rbd_data.114a1f60cb66.00000000000001f3:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-24T15:32:42.373 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:23.027406+0000", 2024-04-24T15:32:42.373 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.339887143999999, 2024-04-24T15:32:42.373 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.005672379, 2024-04-24T15:32:42.373 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:42.373 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:42.373 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:42.374 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:42.374 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:42.374 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2450 2024-04-24T15:32:42.374 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.374 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:42.374 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.374 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:42.374 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027406+0000", 2024-04-24T15:32:42.374 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:42.374 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.374 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.374 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:42.374 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027406+0000", 2024-04-24T15:32:42.375 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:42.375 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.375 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.375 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:42.375 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027405+0000", 2024-04-24T15:32:42.375 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:42.375 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.375 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.375 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:42.375 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027411+0000", 2024-04-24T15:32:42.375 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7610000000000004e-06 2024-04-24T15:32:42.375 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.376 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.376 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:42.376 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027422+0000", 2024-04-24T15:32:42.376 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1462e-05 2024-04-24T15:32:42.376 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.376 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.376 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:42.376 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027429+0000", 2024-04-24T15:32:42.376 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3339999999999998e-06 2024-04-24T15:32:42.376 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.376 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.376 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:42.377 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027478+0000", 2024-04-24T15:32:42.377 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9804e-05 2024-04-24T15:32:42.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.377 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.377 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:42.377 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027559+0000", 2024-04-24T15:32:42.377 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1068000000000004e-05 2024-04-24T15:32:42.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.377 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.377 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:42.377 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027672+0000", 2024-04-24T15:32:42.377 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000112632 2024-04-24T15:32:42.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.377 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.378 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:42.378 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.032737+0000", 2024-04-24T15:32:42.378 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0050646429999999997 2024-04-24T15:32:42.378 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.378 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.378 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:42.378 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.033029+0000", 2024-04-24T15:32:42.378 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00029226300000000002 2024-04-24T15:32:42.378 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.378 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.378 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:42.378 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.033057+0000", 2024-04-24T15:32:42.378 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7872e-05 2024-04-24T15:32:42.378 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.379 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.379 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:42.379 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.033078+0000", 2024-04-24T15:32:42.379 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.158e-05 2024-04-24T15:32:42.379 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:42.379 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:42.379 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:42.379 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.379 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.379 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4426.0:2510 4.12 e33/27 4:4fee0517:::rbd_data.114a1f60cb66.000000000000029d:head v 33'19, mlcod=32'17)", 2024-04-24T15:32:42.379 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:23.078201+0000", 2024-04-24T15:32:42.379 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.289092018000002, 2024-04-24T15:32:42.380 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0060097689999999999, 2024-04-24T15:32:42.380 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:42.380 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:42.380 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:42.380 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.380 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:42.380 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.078201+0000", 2024-04-24T15:32:42.380 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:42.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.380 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.380 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:42.380 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.078201+0000", 2024-04-24T15:32:42.380 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:42.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.381 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:42.381 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.078201+0000", 2024-04-24T15:32:42.381 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:42.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.381 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:42.381 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.078213+0000", 2024-04-24T15:32:42.381 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2261e-05 2024-04-24T15:32:42.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.381 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:42.381 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.078223+0000", 2024-04-24T15:32:42.382 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4709999999999996e-06 2024-04-24T15:32:42.382 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.382 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.382 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:42.382 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.078230+0000", 2024-04-24T15:32:42.382 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8979999999999999e-06 2024-04-24T15:32:42.382 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.382 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.382 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:42.382 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.079842+0000", 2024-04-24T15:32:42.382 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001612048 2024-04-24T15:32:42.382 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.382 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.383 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:42.383 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.079870+0000", 2024-04-24T15:32:42.383 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8487e-05 2024-04-24T15:32:42.383 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.383 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.383 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:42.383 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.084169+0000", 2024-04-24T15:32:42.383 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0042991490000000004 2024-04-24T15:32:42.383 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.384 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.384 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:42.384 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.084211+0000", 2024-04-24T15:32:42.384 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1792000000000002e-05 2024-04-24T15:32:42.384 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:42.384 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:42.384 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:42.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.384 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.384 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:2522 4.6 4:60ceedf0:::rbd_data.114a1f60cb66.000000000000011d:head [set-alloc-hint object_size 262144 write_size 262144,write 110592~2048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-24T15:32:42.384 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:23.088240+0000", 2024-04-24T15:32:42.385 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.279053103999999, 2024-04-24T15:32:42.385 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.988232797, 2024-04-24T15:32:42.385 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:42.385 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:42.385 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:42.385 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:42.385 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:42.385 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2522 2024-04-24T15:32:42.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.385 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:42.385 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.386 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:42.386 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088240+0000", 2024-04-24T15:32:42.386 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:42.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.386 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.386 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:42.386 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088240+0000", 2024-04-24T15:32:42.386 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:42.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.386 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.386 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:42.386 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088238+0000", 2024-04-24T15:32:42.387 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999981 2024-04-24T15:32:42.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.387 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.387 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:42.387 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088254+0000", 2024-04-24T15:32:42.387 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5562000000000001e-05 2024-04-24T15:32:42.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.387 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.387 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:42.387 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088275+0000", 2024-04-24T15:32:42.387 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1256000000000001e-05 2024-04-24T15:32:42.388 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.388 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.388 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:42.388 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088283+0000", 2024-04-24T15:32:42.388 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3219999999999993e-06 2024-04-24T15:32:42.388 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.388 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.388 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:42.388 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088351+0000", 2024-04-24T15:32:42.388 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7595000000000006e-05 2024-04-24T15:32:42.388 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.388 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.389 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-04-24T15:32:42.389 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088362+0000", 2024-04-24T15:32:42.389 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1764e-05 2024-04-24T15:32:42.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.389 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.389 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:42.389 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.073748+0000", 2024-04-24T15:32:42.389 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98538541899999998 2024-04-24T15:32:42.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.389 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.389 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:42.389 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.073925+0000", 2024-04-24T15:32:42.389 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017747600000000001 2024-04-24T15:32:42.390 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.390 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.390 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:42.390 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.074092+0000", 2024-04-24T15:32:42.390 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000166622 2024-04-24T15:32:42.390 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.390 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.390 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:42.390 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.074774+0000", 2024-04-24T15:32:42.390 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00068225000000000002 2024-04-24T15:32:42.390 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.390 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.391 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:42.391 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.076412+0000", 2024-04-24T15:32:42.391 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001638189 2024-04-24T15:32:42.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.391 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.391 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:42.391 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.076446+0000", 2024-04-24T15:32:42.391 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3587000000000002e-05 2024-04-24T15:32:42.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.391 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.391 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:42.391 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.076473+0000", 2024-04-24T15:32:42.391 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6886999999999999e-05 2024-04-24T15:32:42.392 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:42.392 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:42.392 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:42.392 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.392 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.392 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:2755 4.1c 4:3c8baac7:::rbd_data.114a1f60cb66.00000000000000a8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:42.392 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.252831+0000", 2024-04-24T15:32:42.392 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.114462289999999, 2024-04-24T15:32:42.392 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057169500000000002, 2024-04-24T15:32:42.392 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:42.392 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:42.392 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:42.393 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:42.393 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:42.393 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2755 2024-04-24T15:32:42.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.393 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:42.393 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.393 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:42.393 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.252831+0000", 2024-04-24T15:32:42.393 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:42.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.393 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.393 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:42.393 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.252831+0000", 2024-04-24T15:32:42.394 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:42.394 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.394 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.394 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:42.394 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.252830+0000", 2024-04-24T15:32:42.394 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:42.394 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.394 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.394 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:42.394 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.252835+0000", 2024-04-24T15:32:42.394 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3739999999999997e-06 2024-04-24T15:32:42.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.395 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.395 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:42.395 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.252846+0000", 2024-04-24T15:32:42.395 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.039e-05 2024-04-24T15:32:42.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.395 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.395 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:42.395 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.252850+0000", 2024-04-24T15:32:42.395 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4100000000000001e-06 2024-04-24T15:32:42.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.395 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.396 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:42.396 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.252885+0000", 2024-04-24T15:32:42.396 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5213999999999999e-05 2024-04-24T15:32:42.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.396 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.396 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:42.396 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.252957+0000", 2024-04-24T15:32:42.396 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2285000000000003e-05 2024-04-24T15:32:42.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.396 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.396 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:42.396 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.253053+0000", 2024-04-24T15:32:42.396 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5841999999999995e-05 2024-04-24T15:32:42.397 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.397 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.397 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:42.397 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.255905+0000", 2024-04-24T15:32:42.397 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028514539999999998 2024-04-24T15:32:42.397 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.397 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.397 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:42.397 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.258474+0000", 2024-04-24T15:32:42.397 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0025695679999999999 2024-04-24T15:32:42.397 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.397 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.397 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:42.397 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.258518+0000", 2024-04-24T15:32:42.397 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3242999999999999e-05 2024-04-24T15:32:42.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.398 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.398 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:42.398 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.258548+0000", 2024-04-24T15:32:42.398 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0318999999999999e-05 2024-04-24T15:32:42.398 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:42.398 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:42.398 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:42.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.398 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.398 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4426.0:2761 4.19 e34/26 4:9e865ec5:::rbd_data.114a1f60cb66.00000000000000c0:head v 34'14, mlcod=34'12)", 2024-04-24T15:32:42.398 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.255783+0000", 2024-04-24T15:32:42.399 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.111510693, 2024-04-24T15:32:42.399 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057401149999999996, 2024-04-24T15:32:42.399 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:42.399 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:42.399 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:42.399 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.399 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:42.399 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.255783+0000", 2024-04-24T15:32:42.399 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:42.399 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.399 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.399 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:42.399 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.255783+0000", 2024-04-24T15:32:42.399 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:42.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.400 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.400 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:42.400 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.255782+0000", 2024-04-24T15:32:42.400 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:42.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.400 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.400 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:42.400 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.255794+0000", 2024-04-24T15:32:42.400 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2429000000000001e-05 2024-04-24T15:32:42.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.400 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.401 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:42.401 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.255802+0000", 2024-04-24T15:32:42.401 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2139999999999999e-06 2024-04-24T15:32:42.401 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.401 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.401 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:42.401 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.255807+0000", 2024-04-24T15:32:42.401 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6500000000000001e-06 2024-04-24T15:32:42.401 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.401 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.401 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:42.401 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.255846+0000", 2024-04-24T15:32:42.402 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8497000000000001e-05 2024-04-24T15:32:42.402 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.402 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.402 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:42.402 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.255867+0000", 2024-04-24T15:32:42.402 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0733999999999998e-05 2024-04-24T15:32:42.402 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.402 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.402 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:42.402 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.261440+0000", 2024-04-24T15:32:42.402 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0055733099999999997 2024-04-24T15:32:42.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.403 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.403 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:42.403 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.261523+0000", 2024-04-24T15:32:42.403 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2764999999999998e-05 2024-04-24T15:32:42.403 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:42.403 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:42.403 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:42.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.403 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.403 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:2767 4.6 4:6297db09:::rbd_data.114a1f60cb66.00000000000000e1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:42.403 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.256201+0000", 2024-04-24T15:32:42.403 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.111092154000001, 2024-04-24T15:32:42.403 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0056859040000000003, 2024-04-24T15:32:42.404 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:42.404 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:42.404 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:42.404 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:42.404 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:42.404 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2767 2024-04-24T15:32:42.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.404 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:42.404 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.404 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:42.404 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.256201+0000", 2024-04-24T15:32:42.404 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:42.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.405 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.405 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:42.405 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.256201+0000", 2024-04-24T15:32:42.405 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:42.405 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.405 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.405 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:42.405 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.256200+0000", 2024-04-24T15:32:42.405 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:42.405 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.405 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.405 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:42.405 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.256205+0000", 2024-04-24T15:32:42.406 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.587e-06 2024-04-24T15:32:42.406 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.406 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.406 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:42.406 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.256218+0000", 2024-04-24T15:32:42.406 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2362e-05 2024-04-24T15:32:42.406 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.406 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.406 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:42.406 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.256224+0000", 2024-04-24T15:32:42.406 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6120000000000004e-06 2024-04-24T15:32:42.406 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.407 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.407 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:42.407 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.256274+0000", 2024-04-24T15:32:42.407 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9084e-05 2024-04-24T15:32:42.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.407 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.407 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:42.407 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.256392+0000", 2024-04-24T15:32:42.407 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000118127 2024-04-24T15:32:42.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.407 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.407 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:42.407 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.256508+0000", 2024-04-24T15:32:42.407 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000116777 2024-04-24T15:32:42.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.408 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.408 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:42.408 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.261537+0000", 2024-04-24T15:32:42.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0050290380000000004 2024-04-24T15:32:42.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.408 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.408 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:42.408 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.261801+0000", 2024-04-24T15:32:42.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00026362400000000001 2024-04-24T15:32:42.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.408 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.408 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:42.408 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.261848+0000", 2024-04-24T15:32:42.409 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6700999999999999e-05 2024-04-24T15:32:42.409 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.409 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.409 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:42.409 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.261887+0000", 2024-04-24T15:32:42.409 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9159999999999998e-05 2024-04-24T15:32:42.409 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:42.409 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:42.409 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:42.409 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.409 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.409 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:3772 4.4 4:24a7cfbc:::rbd_data.114a1f60cb66.0000000000000020:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:42.409 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.699611+0000", 2024-04-24T15:32:42.409 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 17.667682274000001, 2024-04-24T15:32:42.410 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0060871340000000001, 2024-04-24T15:32:42.410 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:42.410 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:42.410 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:42.410 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:42.410 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:42.410 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3772 2024-04-24T15:32:42.410 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.410 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:42.410 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.410 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:42.410 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.699611+0000", 2024-04-24T15:32:42.410 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:42.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:42.411 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.699611+0000", 2024-04-24T15:32:42.411 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:42.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:42.411 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.699609+0000", 2024-04-24T15:32:42.411 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999981 2024-04-24T15:32:42.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:42.411 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.699618+0000", 2024-04-24T15:32:42.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3859999999999996e-06 2024-04-24T15:32:42.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:42.412 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.699637+0000", 2024-04-24T15:32:42.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8688000000000001e-05 2024-04-24T15:32:42.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:42.412 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.699647+0000", 2024-04-24T15:32:42.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.029e-05 2024-04-24T15:32:42.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:42.413 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.699692+0000", 2024-04-24T15:32:42.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4400000000000002e-05 2024-04-24T15:32:42.413 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.413 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.413 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:42.413 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.699751+0000", 2024-04-24T15:32:42.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8953999999999998e-05 2024-04-24T15:32:42.413 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.413 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.413 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:42.413 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.700249+0000", 2024-04-24T15:32:42.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00049869900000000002 2024-04-24T15:32:42.414 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.414 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.414 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:42.414 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.705560+0000", 2024-04-24T15:32:42.414 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0053110550000000003 2024-04-24T15:32:42.414 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.414 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.414 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:42.414 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.705602+0000", 2024-04-24T15:32:42.414 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1689e-05 2024-04-24T15:32:42.414 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.415 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.415 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:42.415 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.705652+0000", 2024-04-24T15:32:42.415 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9740000000000001e-05 2024-04-24T15:32:42.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.415 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.415 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:42.415 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.705698+0000", 2024-04-24T15:32:42.415 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6329000000000003e-05 2024-04-24T15:32:42.415 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:42.415 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:42.415 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:42.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.416 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.416 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:3774 4.4 4:25b094fe:::rbd_data.114a1f60cb66.0000000000000022:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:42.416 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.699669+0000", 2024-04-24T15:32:42.416 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 17.667624143000001, 2024-04-24T15:32:42.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.006296678, 2024-04-24T15:32:42.416 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:42.416 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:42.416 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:42.416 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:42.416 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:42.417 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3774 2024-04-24T15:32:42.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.417 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:42.417 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.417 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:42.417 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.699669+0000", 2024-04-24T15:32:42.417 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:42.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.417 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.417 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:42.417 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.699669+0000", 2024-04-24T15:32:42.418 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:42.418 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.418 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.418 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:42.418 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.699669+0000", 2024-04-24T15:32:42.418 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:42.418 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.418 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.418 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:42.418 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.699672+0000", 2024-04-24T15:32:42.418 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2830000000000002e-06 2024-04-24T15:32:42.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.419 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.419 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:42.419 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.699677+0000", 2024-04-24T15:32:42.419 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7530000000000001e-06 2024-04-24T15:32:42.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.419 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.419 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:42.419 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.699680+0000", 2024-04-24T15:32:42.419 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0130000000000002e-06 2024-04-24T15:32:42.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.420 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:42.420 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.702736+0000", 2024-04-24T15:32:42.420 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0030567379999999998 2024-04-24T15:32:42.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.420 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:42.420 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.702797+0000", 2024-04-24T15:32:42.420 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0563000000000003e-05 2024-04-24T15:32:42.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.420 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:42.421 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.703297+0000", 2024-04-24T15:32:42.421 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00049955300000000002 2024-04-24T15:32:42.421 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.421 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.421 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:42.421 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.705595+0000", 2024-04-24T15:32:42.421 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002298076 2024-04-24T15:32:42.421 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.421 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.421 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:42.421 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.705870+0000", 2024-04-24T15:32:42.421 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000275285 2024-04-24T15:32:42.422 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.422 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.422 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:42.422 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.705923+0000", 2024-04-24T15:32:42.422 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3229999999999997e-05 2024-04-24T15:32:42.422 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.422 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.422 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:42.422 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.705966+0000", 2024-04-24T15:32:42.422 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2636999999999999e-05 2024-04-24T15:32:42.423 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:42.423 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:42.423 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:42.423 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.423 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.423 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:3795 4.1c 4:3a2027c5:::rbd_data.114a1f60cb66.0000000000000025:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:42.423 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.723461+0000", 2024-04-24T15:32:42.423 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 17.643832285999999, 2024-04-24T15:32:42.423 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0066452719999999998, 2024-04-24T15:32:42.423 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:42.423 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:42.423 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:42.424 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:42.424 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:42.424 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3795 2024-04-24T15:32:42.424 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.424 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:42.424 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.424 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:42.424 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.723461+0000", 2024-04-24T15:32:42.424 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:42.424 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.424 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.424 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:42.425 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.723461+0000", 2024-04-24T15:32:42.425 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:42.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.425 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.425 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:42.425 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.723459+0000", 2024-04-24T15:32:42.425 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999986 2024-04-24T15:32:42.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.425 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.425 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:42.425 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.723470+0000", 2024-04-24T15:32:42.425 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0097e-05 2024-04-24T15:32:42.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.426 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.426 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:42.426 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.723484+0000", 2024-04-24T15:32:42.426 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4338000000000001e-05 2024-04-24T15:32:42.426 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.426 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.426 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:42.426 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.723496+0000", 2024-04-24T15:32:42.426 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2125e-05 2024-04-24T15:32:42.426 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.426 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.427 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:42.427 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.723547+0000", 2024-04-24T15:32:42.427 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0909000000000003e-05 2024-04-24T15:32:42.427 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.427 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.427 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:42.427 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.723639+0000", 2024-04-24T15:32:42.427 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1844999999999999e-05 2024-04-24T15:32:42.427 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.427 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.427 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:42.427 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.724463+0000", 2024-04-24T15:32:42.428 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00082389099999999999 2024-04-24T15:32:42.428 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.428 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.428 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:42.428 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.728863+0000", 2024-04-24T15:32:42.428 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0044007610000000004 2024-04-24T15:32:42.428 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.428 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.428 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:42.428 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.730017+0000", 2024-04-24T15:32:42.428 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0011534410000000001 2024-04-24T15:32:42.428 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.428 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.428 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:42.429 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.730067+0000", 2024-04-24T15:32:42.429 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0430000000000003e-05 2024-04-24T15:32:42.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.429 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:42.429 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.730106+0000", 2024-04-24T15:32:42.429 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8902000000000001e-05 2024-04-24T15:32:42.429 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:42.429 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:42.429 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:42.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.429 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.430 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:3976 4.1c 4:396ab59e:::rbd_data.114a1f60cb66.0000000000000160:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:42.430 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.827920+0000", 2024-04-24T15:32:42.430 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 17.539373404999999, 2024-04-24T15:32:42.430 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057086940000000003, 2024-04-24T15:32:42.430 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:42.430 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:42.430 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:42.430 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:42.430 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:42.430 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3976 2024-04-24T15:32:42.430 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.430 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:42.431 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.431 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:42.431 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.827920+0000", 2024-04-24T15:32:42.431 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:42.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.431 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.431 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:42.431 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.827920+0000", 2024-04-24T15:32:42.431 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:42.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.431 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.431 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:42.431 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.827919+0000", 2024-04-24T15:32:42.431 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:42.432 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.432 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.432 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:42.432 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.827924+0000", 2024-04-24T15:32:42.432 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4270000000000001e-06 2024-04-24T15:32:42.432 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.432 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.432 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:42.432 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.827934+0000", 2024-04-24T15:32:42.432 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0606e-05 2024-04-24T15:32:42.432 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.432 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.432 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:42.433 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.827939+0000", 2024-04-24T15:32:42.433 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.792e-06 2024-04-24T15:32:42.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.433 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.433 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:42.433 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.827967+0000", 2024-04-24T15:32:42.433 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7919000000000002e-05 2024-04-24T15:32:42.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.433 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.433 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:42.433 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.828011+0000", 2024-04-24T15:32:42.433 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3896000000000001e-05 2024-04-24T15:32:42.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.433 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.434 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:42.434 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.828070+0000", 2024-04-24T15:32:42.434 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9348999999999997e-05 2024-04-24T15:32:42.434 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.434 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.434 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:42.434 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.830513+0000", 2024-04-24T15:32:42.434 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024429669999999999 2024-04-24T15:32:42.434 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.434 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.434 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:42.434 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.833567+0000", 2024-04-24T15:32:42.435 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0030542590000000001 2024-04-24T15:32:42.435 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.435 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.435 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:42.435 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.833608+0000", 2024-04-24T15:32:42.435 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1148e-05 2024-04-24T15:32:42.435 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.435 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.435 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:42.435 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.833629+0000", 2024-04-24T15:32:42.435 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0066999999999999e-05 2024-04-24T15:32:42.435 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:42.436 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:42.436 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:42.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.436 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:3987 4.1c 4:3a30a8f2:::rbd_data.114a1f60cb66.0000000000000174:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:42.436 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.833638+0000", 2024-04-24T15:32:42.436 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 17.533655124999999, 2024-04-24T15:32:42.436 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057825430000000002, 2024-04-24T15:32:42.436 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:42.436 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:42.436 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:42.436 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:42.436 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:42.436 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3987 2024-04-24T15:32:42.437 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.437 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:42.437 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.437 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:42.437 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.833638+0000", 2024-04-24T15:32:42.437 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:42.437 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.437 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.437 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:42.437 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.833638+0000", 2024-04-24T15:32:42.437 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:42.437 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.438 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.438 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:42.438 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.833637+0000", 2024-04-24T15:32:42.438 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:42.438 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.438 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.438 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:42.438 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.833646+0000", 2024-04-24T15:32:42.438 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0180000000000007e-06 2024-04-24T15:32:42.438 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.438 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.438 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:42.439 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.833659+0000", 2024-04-24T15:32:42.439 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.313e-05 2024-04-24T15:32:42.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.439 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:42.439 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.833664+0000", 2024-04-24T15:32:42.439 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.006e-06 2024-04-24T15:32:42.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.439 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:42.440 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.833697+0000", 2024-04-24T15:32:42.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2978000000000002e-05 2024-04-24T15:32:42.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.440 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.440 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:42.440 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.833756+0000", 2024-04-24T15:32:42.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8811000000000003e-05 2024-04-24T15:32:42.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.440 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.440 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:42.440 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.833823+0000", 2024-04-24T15:32:42.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7225999999999996e-05 2024-04-24T15:32:42.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.441 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:42.441 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.838767+0000", 2024-04-24T15:32:42.441 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0049435809999999998 2024-04-24T15:32:42.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.441 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:42.441 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.839341+0000", 2024-04-24T15:32:42.441 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00057366800000000005 2024-04-24T15:32:42.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.441 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:42.441 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.839385+0000", 2024-04-24T15:32:42.442 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4597e-05 2024-04-24T15:32:42.442 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.442 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.442 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:42.442 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.839421+0000", 2024-04-24T15:32:42.442 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5516999999999999e-05 2024-04-24T15:32:42.442 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:42.442 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:42.442 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:42.442 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.442 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.442 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:4005 4.6 4:64968d7a:::rbd_data.114a1f60cb66.00000000000001bc:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:42.442 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.841940+0000", 2024-04-24T15:32:42.442 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 17.525352865999999, 2024-04-24T15:32:42.443 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.005717481, 2024-04-24T15:32:42.443 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:42.443 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:42.443 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:42.443 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:42.443 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:42.443 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4005 2024-04-24T15:32:42.443 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.443 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:42.443 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.443 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:42.443 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.841940+0000", 2024-04-24T15:32:42.443 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:42.443 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.444 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.444 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:42.444 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.841940+0000", 2024-04-24T15:32:42.444 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:42.444 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.444 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.444 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:42.444 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.841940+0000", 2024-04-24T15:32:42.444 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:42.444 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.444 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.444 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:42.444 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.841944+0000", 2024-04-24T15:32:42.444 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.904e-06 2024-04-24T15:32:42.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.445 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.445 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:42.445 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.841954+0000", 2024-04-24T15:32:42.445 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7499999999999998e-06 2024-04-24T15:32:42.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.445 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.445 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:42.445 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.841960+0000", 2024-04-24T15:32:42.445 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4399999999999996e-06 2024-04-24T15:32:42.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.445 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.446 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:42.446 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.841991+0000", 2024-04-24T15:32:42.446 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1615000000000001e-05 2024-04-24T15:32:42.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.446 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.446 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:42.446 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.842067+0000", 2024-04-24T15:32:42.446 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6002999999999997e-05 2024-04-24T15:32:42.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.446 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.446 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:42.446 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.842168+0000", 2024-04-24T15:32:42.447 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000100474 2024-04-24T15:32:42.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.447 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:42.447 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.847470+0000", 2024-04-24T15:32:42.447 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0053021830000000002 2024-04-24T15:32:42.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.447 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:42.447 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.847577+0000", 2024-04-24T15:32:42.447 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000107425 2024-04-24T15:32:42.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.448 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.448 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:42.448 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.847622+0000", 2024-04-24T15:32:42.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5015999999999997e-05 2024-04-24T15:32:42.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.448 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.448 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:42.448 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.847658+0000", 2024-04-24T15:32:42.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5509999999999997e-05 2024-04-24T15:32:42.448 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:42.448 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:42.449 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:42.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.449 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:4119 4.6 4:6588fd86:::rbd_data.114a1f60cb66.0000000000000179:head [set-alloc-hint object_size 262144 write_size 262144,write 151552~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:42.449 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.935980+0000", 2024-04-24T15:32:42.449 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 17.431312996999999, 2024-04-24T15:32:42.449 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.006739391, 2024-04-24T15:32:42.449 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:42.449 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:42.449 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:42.449 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:42.449 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:42.449 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4119 2024-04-24T15:32:42.450 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.450 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:42.450 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.450 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:42.450 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.935980+0000", 2024-04-24T15:32:42.450 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:42.450 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.450 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.450 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:42.450 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.935980+0000", 2024-04-24T15:32:42.450 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:42.450 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.450 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.451 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:42.451 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.935980+0000", 2024-04-24T15:32:42.451 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:42.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.451 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.451 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:42.451 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.935987+0000", 2024-04-24T15:32:42.451 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1590000000000004e-06 2024-04-24T15:32:42.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.451 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.451 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:42.451 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.935996+0000", 2024-04-24T15:32:42.451 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8389999999999996e-06 2024-04-24T15:32:42.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.452 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.452 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:42.452 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.936001+0000", 2024-04-24T15:32:42.452 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6890000000000001e-06 2024-04-24T15:32:42.452 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.452 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.452 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:42.452 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.936456+0000", 2024-04-24T15:32:42.452 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00045432600000000001 2024-04-24T15:32:42.452 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.452 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.452 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:42.452 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.936530+0000", 2024-04-24T15:32:42.452 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4550999999999998e-05 2024-04-24T15:32:42.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.453 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.453 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:42.453 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.936663+0000", 2024-04-24T15:32:42.453 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000133225 2024-04-24T15:32:42.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.453 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.453 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:42.453 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.938066+0000", 2024-04-24T15:32:42.453 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0014031969999999999 2024-04-24T15:32:42.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.453 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.453 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:42.453 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.942636+0000", 2024-04-24T15:32:42.454 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0045691270000000001 2024-04-24T15:32:42.454 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.454 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.454 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:42.454 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.942684+0000", 2024-04-24T15:32:42.454 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8123e-05 2024-04-24T15:32:42.454 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.454 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.454 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:42.454 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.942720+0000", 2024-04-24T15:32:42.454 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5880000000000002e-05 2024-04-24T15:32:42.454 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:42.454 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:42.455 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:42.455 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.455 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.455 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:9497 4.1c 4:3e8473f6:::rbd_header.114a1f60cb66:head [watch reconnect cookie 140672528495600 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:42.455 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.312983+0000", 2024-04-24T15:32:42.455 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.0543101870000005, 2024-04-24T15:32:42.455 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0070173939999999997, 2024-04-24T15:32:42.455 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:42.455 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:42.455 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:42.455 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:42.456 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:42.456 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 9497 2024-04-24T15:32:42.456 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.456 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:42.456 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.456 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:42.456 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.312983+0000", 2024-04-24T15:32:42.456 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:42.456 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.456 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.456 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:42.456 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.312983+0000", 2024-04-24T15:32:42.456 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:42.457 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.457 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:42.457 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.312983+0000", 2024-04-24T15:32:42.457 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:42.457 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.457 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:42.457 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.312989+0000", 2024-04-24T15:32:42.457 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3090000000000001e-06 2024-04-24T15:32:42.457 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.457 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:42.457 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.312995+0000", 2024-04-24T15:32:42.458 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.477e-06 2024-04-24T15:32:42.458 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.458 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.458 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:42.458 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.312999+0000", 2024-04-24T15:32:42.458 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8489999999999999e-06 2024-04-24T15:32:42.458 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.458 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.458 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:42.458 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.313327+0000", 2024-04-24T15:32:42.458 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00032772399999999999 2024-04-24T15:32:42.458 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.459 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.459 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:42.459 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.313391+0000", 2024-04-24T15:32:42.459 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4388999999999996e-05 2024-04-24T15:32:42.459 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.459 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.459 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:42.459 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.320000+0000", 2024-04-24T15:32:42.459 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0066090660000000002 2024-04-24T15:32:42.459 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:42.459 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:42.459 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:42.459 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.459 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.460 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:18726 4.4 4:21862cbe:::rbd_data.114ae15b9a20.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:42.460 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:34.274457+0000", 2024-04-24T15:32:42.460 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 8.0928363789999995, 2024-04-24T15:32:42.460 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0056902769999999997, 2024-04-24T15:32:42.460 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:42.460 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:42.460 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:42.460 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:42.460 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:42.460 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 18726 2024-04-24T15:32:42.460 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.460 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:42.460 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.461 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:42.461 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:34.274457+0000", 2024-04-24T15:32:42.461 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:42.461 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.461 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.461 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:42.461 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:34.274457+0000", 2024-04-24T15:32:42.461 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:42.461 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.461 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.461 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:42.462 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:34.274454+0000", 2024-04-24T15:32:42.462 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999976 2024-04-24T15:32:42.462 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.462 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.462 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:42.462 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:34.275538+0000", 2024-04-24T15:32:42.462 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0010832089999999999 2024-04-24T15:32:42.462 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.462 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.462 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:42.462 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:34.275565+0000", 2024-04-24T15:32:42.462 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7936000000000002e-05 2024-04-24T15:32:42.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.463 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.463 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:42.463 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:34.275577+0000", 2024-04-24T15:32:42.463 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1579999999999999e-05 2024-04-24T15:32:42.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.463 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.463 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:42.463 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:34.275633+0000", 2024-04-24T15:32:42.463 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5809e-05 2024-04-24T15:32:42.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.463 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.464 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:42.464 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:34.275794+0000", 2024-04-24T15:32:42.464 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000161111 2024-04-24T15:32:42.464 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.464 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.464 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:42.464 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:34.276180+0000", 2024-04-24T15:32:42.464 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00038595300000000002 2024-04-24T15:32:42.464 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.464 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.464 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:42.464 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:34.279950+0000", 2024-04-24T15:32:42.464 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0037696959999999999 2024-04-24T15:32:42.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.465 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.465 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:42.465 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:34.280043+0000", 2024-04-24T15:32:42.465 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3681e-05 2024-04-24T15:32:42.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.465 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.465 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:42.465 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:34.280102+0000", 2024-04-24T15:32:42.465 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8587e-05 2024-04-24T15:32:42.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.465 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.465 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:42.465 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:34.280147+0000", 2024-04-24T15:32:42.466 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5237000000000001e-05 2024-04-24T15:32:42.466 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:42.466 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:42.466 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:42.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.466 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.466 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:19861 4.d 4:b2ab6102:::rbd_data.114ae15b9a20.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:42.466 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:34.404476+0000", 2024-04-24T15:32:42.466 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 7.9628173719999999, 2024-04-24T15:32:42.466 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0072684150000000003, 2024-04-24T15:32:42.466 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:42.466 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:42.467 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:42.467 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:42.467 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:42.467 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 19861 2024-04-24T15:32:42.467 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.467 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:42.467 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.467 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:42.467 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:34.404476+0000", 2024-04-24T15:32:42.467 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:42.467 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.467 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.467 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:42.467 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:34.404476+0000", 2024-04-24T15:32:42.468 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:42.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.468 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.468 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:42.468 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:34.404473+0000", 2024-04-24T15:32:42.468 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999971 2024-04-24T15:32:42.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.468 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.468 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:42.468 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:34.405518+0000", 2024-04-24T15:32:42.468 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0010449490000000001 2024-04-24T15:32:42.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.469 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.469 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:42.469 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:34.405557+0000", 2024-04-24T15:32:42.469 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8427999999999999e-05 2024-04-24T15:32:42.469 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.469 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.469 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:42.469 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:34.405569+0000", 2024-04-24T15:32:42.469 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2129e-05 2024-04-24T15:32:42.469 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.469 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.469 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:42.469 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:34.405625+0000", 2024-04-24T15:32:42.470 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5980999999999998e-05 2024-04-24T15:32:42.470 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.470 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.470 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:42.470 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:34.405788+0000", 2024-04-24T15:32:42.470 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016350499999999999 2024-04-24T15:32:42.470 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.470 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.470 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 6", 2024-04-24T15:32:42.470 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:34.406065+0000", 2024-04-24T15:32:42.470 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000276631 2024-04-24T15:32:42.470 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.471 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.471 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:42.471 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:34.409212+0000", 2024-04-24T15:32:42.471 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003147066 2024-04-24T15:32:42.471 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.471 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.471 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:42.471 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:34.411631+0000", 2024-04-24T15:32:42.471 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024194960000000001 2024-04-24T15:32:42.471 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.471 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.471 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:42.471 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:34.411693+0000", 2024-04-24T15:32:42.472 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1383999999999994e-05 2024-04-24T15:32:42.472 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.472 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.472 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:42.472 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:34.411744+0000", 2024-04-24T15:32:42.472 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1474000000000002e-05 2024-04-24T15:32:42.472 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:42.472 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:42.472 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:42.472 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.473 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.473 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:24347 4.6 4:628d3169:::rbd_data.114ae15b9a20.000000000000004b:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-04-24T15:32:42.473 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:37.251701+0000", 2024-04-24T15:32:42.473 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 5.1155917659999997, 2024-04-24T15:32:42.473 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0087394719999999999, 2024-04-24T15:32:42.473 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:42.473 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:42.473 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:42.473 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:42.473 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:42.473 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 24347 2024-04-24T15:32:42.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.474 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:42.474 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.474 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:42.474 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.251701+0000", 2024-04-24T15:32:42.474 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:42.474 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.474 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.474 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:42.474 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.251701+0000", 2024-04-24T15:32:42.474 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:42.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.475 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.475 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:42.475 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.251700+0000", 2024-04-24T15:32:42.475 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999986 2024-04-24T15:32:42.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.475 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.475 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:42.475 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.251708+0000", 2024-04-24T15:32:42.475 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.49e-06 2024-04-24T15:32:42.476 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.476 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.476 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:42.476 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.251724+0000", 2024-04-24T15:32:42.476 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5803000000000001e-05 2024-04-24T15:32:42.476 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.476 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.476 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:42.476 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.251732+0000", 2024-04-24T15:32:42.476 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8460000000000008e-06 2024-04-24T15:32:42.476 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.476 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.477 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:42.477 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.251775+0000", 2024-04-24T15:32:42.477 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3102000000000002e-05 2024-04-24T15:32:42.477 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.477 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.477 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:42.477 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.251847+0000", 2024-04-24T15:32:42.477 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1482000000000004e-05 2024-04-24T15:32:42.477 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.477 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.477 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:42.477 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.252681+0000", 2024-04-24T15:32:42.478 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00083451599999999999 2024-04-24T15:32:42.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.478 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.478 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:42.478 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.260266+0000", 2024-04-24T15:32:42.478 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0075852599999999999 2024-04-24T15:32:42.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.478 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.478 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:42.478 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.260345+0000", 2024-04-24T15:32:42.478 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8824000000000003e-05 2024-04-24T15:32:42.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.479 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.479 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:42.479 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.260397+0000", 2024-04-24T15:32:42.479 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1573999999999998e-05 2024-04-24T15:32:42.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.479 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.479 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:42.479 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.260441+0000", 2024-04-24T15:32:42.479 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4195e-05 2024-04-24T15:32:42.479 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:42.479 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:42.480 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:42.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.480 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.480 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:24351 4.6 4:61b5c231:::rbd_data.114ae15b9a20.000000000000004f:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-04-24T15:32:42.480 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:37.254276+0000", 2024-04-24T15:32:42.480 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 5.1130176890000003, 2024-04-24T15:32:42.480 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0064981930000000002, 2024-04-24T15:32:42.480 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:42.480 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:42.480 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:42.480 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:42.480 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:42.481 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 24351 2024-04-24T15:32:42.481 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.481 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:42.481 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.481 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:42.481 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.254276+0000", 2024-04-24T15:32:42.481 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:42.481 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.481 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.481 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:42.481 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.254276+0000", 2024-04-24T15:32:42.481 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:42.481 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.482 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.482 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:42.482 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.254274+0000", 2024-04-24T15:32:42.482 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999981 2024-04-24T15:32:42.482 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.482 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.482 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:42.482 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.254283+0000", 2024-04-24T15:32:42.482 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.346e-06 2024-04-24T15:32:42.482 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.482 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.482 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:42.483 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.254298+0000", 2024-04-24T15:32:42.483 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5318000000000001e-05 2024-04-24T15:32:42.483 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.483 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.483 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:42.483 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.254306+0000", 2024-04-24T15:32:42.483 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.622e-06 2024-04-24T15:32:42.483 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.483 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.483 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:42.483 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.255950+0000", 2024-04-24T15:32:42.483 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001644473 2024-04-24T15:32:42.483 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.484 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.484 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:42.484 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.256055+0000", 2024-04-24T15:32:42.484 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104273 2024-04-24T15:32:42.484 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.484 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.484 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:42.484 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.256982+0000", 2024-04-24T15:32:42.484 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00092704699999999996 2024-04-24T15:32:42.484 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.484 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.484 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:42.485 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.260520+0000", 2024-04-24T15:32:42.485 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035386979999999998 2024-04-24T15:32:42.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.485 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.485 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:42.485 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.260692+0000", 2024-04-24T15:32:42.485 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017125300000000001 2024-04-24T15:32:42.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.485 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.485 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:42.485 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.260738+0000", 2024-04-24T15:32:42.485 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6820999999999998e-05 2024-04-24T15:32:42.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.486 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.486 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:42.486 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.260774+0000", 2024-04-24T15:32:42.486 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5237999999999997e-05 2024-04-24T15:32:42.486 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:42.486 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:42.486 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:42.486 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.486 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.486 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:24393 4.4 4:2764ffd7:::rbd_data.114ae15b9a20.0000000000000031:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-04-24T15:32:42.486 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:37.296111+0000", 2024-04-24T15:32:42.486 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 5.0711825560000001, 2024-04-24T15:32:42.487 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0058595019999999999, 2024-04-24T15:32:42.487 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:42.487 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:42.487 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:42.487 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:42.487 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:42.487 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 24393 2024-04-24T15:32:42.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.487 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:42.487 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.487 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:42.487 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.296111+0000", 2024-04-24T15:32:42.488 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:42.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.488 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.488 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:42.488 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.296111+0000", 2024-04-24T15:32:42.488 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:42.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.488 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.488 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:42.488 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.296110+0000", 2024-04-24T15:32:42.488 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:42.489 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.489 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.489 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:42.489 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.296114+0000", 2024-04-24T15:32:42.489 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4240000000000003e-06 2024-04-24T15:32:42.489 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.489 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.489 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:42.489 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.296125+0000", 2024-04-24T15:32:42.489 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0869999999999999e-05 2024-04-24T15:32:42.489 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.490 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.490 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:42.490 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.296130+0000", 2024-04-24T15:32:42.490 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7859999999999996e-06 2024-04-24T15:32:42.490 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.490 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.490 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:42.490 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.296174+0000", 2024-04-24T15:32:42.490 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3492999999999998e-05 2024-04-24T15:32:42.490 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.490 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.490 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:42.491 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.296258+0000", 2024-04-24T15:32:42.491 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3981999999999997e-05 2024-04-24T15:32:42.491 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.491 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.491 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:42.491 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.296383+0000", 2024-04-24T15:32:42.491 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000125143 2024-04-24T15:32:42.491 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.491 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.491 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:42.491 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.298229+0000", 2024-04-24T15:32:42.492 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018462260000000001 2024-04-24T15:32:42.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.492 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.492 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:42.492 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.301889+0000", 2024-04-24T15:32:42.492 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003660122 2024-04-24T15:32:42.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.492 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.492 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:42.492 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.301936+0000", 2024-04-24T15:32:42.492 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7284999999999999e-05 2024-04-24T15:32:42.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.493 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.493 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:42.493 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.301970+0000", 2024-04-24T15:32:42.493 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3892e-05 2024-04-24T15:32:42.493 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:42.493 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:42.493 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:42.493 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:42.493 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:42.493 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:32:42.495 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-24T15:32:42.495 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:32:42.496+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:32:42.595 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:32:42.597+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:32:42.597 INFO:teuthology.orchestra.run.smithi096.stdout:1536 trunc from 0x69733af to 0x944956e 2024-04-24T15:32:42.606 INFO:teuthology.orchestra.run.smithi096.stdout:1538 read 0x9f0483 thru 0x9f2f37 (0x2ab5 bytes) 2024-04-24T15:32:42.611 INFO:teuthology.orchestra.run.smithi096.stdout:1540 trunc from 0x944956e to 0x5f07696 2024-04-24T15:32:42.621 INFO:teuthology.orchestra.run.smithi096.stdout:1541 writesame 0x3a214e2 thru 0x3a2dc81 data_size 0x10(0xc7a0 bytes) 2024-04-24T15:32:42.627 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:42.627 INFO:teuthology.orchestra.run.smithi096.stdout:1544 write 0x5dc50eb thru 0x5dd3c37 (0xeb4d bytes) 2024-04-24T15:32:42.638 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:42.638 INFO:teuthology.orchestra.run.smithi096.stdout:1546 read 0x2c413fa thru 0x2c43cc6 (0x28cd bytes) 2024-04-24T15:32:42.639 INFO:teuthology.orchestra.run.smithi096.stdout:1548 punch from 0x489ee02 to 0x48a0309, (0x1507 bytes) 2024-04-24T15:32:42.639 INFO:teuthology.orchestra.run.smithi096.stdout:1550 read 0x15f6075 thru 0x15ff34b (0x92d7 bytes) 2024-04-24T15:32:42.639 INFO:teuthology.orchestra.run.smithi096.stdout:1551 punch from 0x349ac91 to 0x349ebd4, (0x3f43 bytes) 2024-04-24T15:32:42.639 INFO:teuthology.orchestra.run.smithi096.stdout:1552 punch from 0x42adab7 to 0x42ae3e5, (0x92e bytes) 2024-04-24T15:32:42.648 INFO:teuthology.orchestra.run.smithi096.stdout:1553 trunc from 0x5f07696 to 0xd8e3740 2024-04-24T15:32:42.654 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:42.654 INFO:teuthology.orchestra.run.smithi096.stdout:1555 write 0x393042b thru 0x393f961 (0xf537 bytes) 2024-04-24T15:32:42.663 INFO:teuthology.orchestra.run.smithi096.stdout:1556 trunc from 0xd8e3740 to 0x74d1344 2024-04-24T15:32:42.669 INFO:teuthology.orchestra.run.smithi096.stdout:1557 trunc from 0x74d1344 to 0x77b2826 2024-04-24T15:32:42.675 INFO:teuthology.orchestra.run.smithi096.stdout:1558 writesame 0x2067a7f thru 0x206a0f0 data_size 0x26(0x2672 bytes) 2024-04-24T15:32:42.678 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:32:42.678 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:32:42.678 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-04-24T15:32:42.678 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:32:42.690 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-24T15:32:42.701 INFO:teuthology.orchestra.run.smithi096.stdout:1559 write 0x9ffac38 thru 0xa006049 (0xb412 bytes) 2024-04-24T15:32:42.701 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:32:42.701+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:32:42.706 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:42.706 INFO:teuthology.orchestra.run.smithi096.stdout:1561 read 0x7cab598 thru 0x7cb71f0 (0xbc59 bytes) 2024-04-24T15:32:42.709 INFO:teuthology.orchestra.run.smithi096.stdout:1562 trunc from 0xa00604a to 0x75002d1 2024-04-24T15:32:42.716 INFO:teuthology.orchestra.run.smithi096.stdout:1564 punch from 0x553f84a to 0x554ba8b, (0xc241 bytes) 2024-04-24T15:32:42.716 INFO:teuthology.orchestra.run.smithi096.stdout:1565 writesame 0x31f1fb7 thru 0x31f5dc7 data_size 0x1(0x3e11 bytes) 2024-04-24T15:32:42.731 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:42.731 INFO:teuthology.orchestra.run.smithi096.stdout:1567 read 0x6a750f4 thru 0x6a810bc (0xbfc9 bytes) 2024-04-24T15:32:42.732 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:42.732 INFO:teuthology.orchestra.run.smithi096.stdout:1571 read 0x46faba0 thru 0x46fbfe2 (0x1443 bytes) 2024-04-24T15:32:42.738 INFO:teuthology.orchestra.run.smithi096.stdout:1572 write 0x805c136 thru 0x8065509 (0x93d4 bytes) 2024-04-24T15:32:42.741 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:42.741 INFO:teuthology.orchestra.run.smithi096.stdout:1574 write 0x2ee8f3 thru 0x2ef0d4 (0x7e2 bytes) 2024-04-24T15:32:42.742 INFO:teuthology.orchestra.run.smithi096.stdout:1575 write 0x45aa653 thru 0x45b33c9 (0x8d77 bytes) 2024-04-24T15:32:42.744 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:42.744 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:42.744 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:42.744 INFO:teuthology.orchestra.run.smithi096.stdout:1579 read 0x7d916b1 thru 0x7d9dc22 (0xc572 bytes) 2024-04-24T15:32:42.749 INFO:teuthology.orchestra.run.smithi096.stdout:1580 trunc from 0x806550a to 0xc141c7e 2024-04-24T15:32:42.754 INFO:teuthology.orchestra.run.smithi096.stdout:1581 write 0x6c00460 thru 0x6c010c7 (0xc68 bytes) 2024-04-24T15:32:42.759 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:42.759 INFO:teuthology.orchestra.run.smithi096.stdout:1583 write 0x981a7ab thru 0x98208f7 (0x614d bytes) 2024-04-24T15:32:42.760 INFO:teuthology.orchestra.run.smithi096.stdout:1584 write 0x46ff370 thru 0x4704c37 (0x58c8 bytes) 2024-04-24T15:32:42.762 INFO:teuthology.orchestra.run.smithi096.stdout:1585 read 0x2ac980 thru 0x2aef51 (0x25d2 bytes) 2024-04-24T15:32:42.762 INFO:teuthology.orchestra.run.smithi096.stdout:1586 write 0xc10bf68 thru 0xc10ed3e (0x2dd7 bytes) 2024-04-24T15:32:42.767 INFO:teuthology.orchestra.run.smithi096.stdout:1587 write 0x3565cd1 thru 0x3574957 (0xec87 bytes) 2024-04-24T15:32:42.769 INFO:teuthology.orchestra.run.smithi096.stdout:1588 clone 7 order 22 su 2097152 sc 13 2024-04-24T15:32:42.796 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:32:42.798+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:32:42.822 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:32:42.822 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:32:42.822 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-04-24T15:32:42.822 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-04-24T15:32:42.822 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:32:42.832 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-24T15:32:42.897 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:32:42.899+0000 7f23abcf6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 44021) UID: 0 2024-04-24T15:32:42.996 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:32:42.996 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-04-24T15:32:42.996 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-04-24T15:32:42.996 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-04-24T15:32:42.996 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.996 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14043 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2383872~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:42.996 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892347+0000", 2024-04-24T15:32:42.997 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.1037864469999992, 2024-04-24T15:32:42.997 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038537279000000001, 2024-04-24T15:32:42.997 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:42.997 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:42.997 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:42.997 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:42.997 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:42.997 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14043 2024-04-24T15:32:42.997 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.997 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:42.997 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.997 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:42.997 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892347+0000", 2024-04-24T15:32:42.997 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:42.998 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.998 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.998 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:42.998 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892347+0000", 2024-04-24T15:32:42.998 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:42.998 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.998 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.998 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:42.998 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892347+0000", 2024-04-24T15:32:42.998 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:42.998 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.998 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.998 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:42.999 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892349+0000", 2024-04-24T15:32:42.999 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3400000000000001e-06 2024-04-24T15:32:42.999 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.999 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:42.999 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892357+0000", 2024-04-24T15:32:42.999 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1540000000000002e-06 2024-04-24T15:32:42.999 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.999 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:42.999 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892360+0000", 2024-04-24T15:32:42.999 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5829999999999998e-06 2024-04-24T15:32:42.999 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:42.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:42.999 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:42.999 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930795+0000", 2024-04-24T15:32:43.000 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038434590999999997 2024-04-24T15:32:43.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.000 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.000 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:43.000 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930836+0000", 2024-04-24T15:32:43.000 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0827000000000001e-05 2024-04-24T15:32:43.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.000 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.000 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:43.000 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930885+0000", 2024-04-24T15:32:43.000 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9029000000000001e-05 2024-04-24T15:32:43.000 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:43.000 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:43.000 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:43.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.000 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.001 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14044 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2424832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:43.001 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892372+0000", 2024-04-24T15:32:43.001 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.1037618029999994, 2024-04-24T15:32:43.001 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038624890000000002, 2024-04-24T15:32:43.001 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:43.001 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:43.001 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:43.001 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:43.001 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:43.001 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14044 2024-04-24T15:32:43.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.001 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:43.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.001 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:43.001 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892372+0000", 2024-04-24T15:32:43.002 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:43.002 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.002 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.002 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:43.002 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892372+0000", 2024-04-24T15:32:43.002 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:43.002 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.002 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.002 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:43.002 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892372+0000", 2024-04-24T15:32:43.002 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:43.002 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.002 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.002 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:43.002 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892374+0000", 2024-04-24T15:32:43.003 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4899999999999999e-06 2024-04-24T15:32:43.003 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.003 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.003 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:43.003 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892380+0000", 2024-04-24T15:32:43.003 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.631e-06 2024-04-24T15:32:43.003 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.003 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.003 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:43.003 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892383+0000", 2024-04-24T15:32:43.003 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7400000000000002e-06 2024-04-24T15:32:43.003 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.003 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.003 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:43.003 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930905+0000", 2024-04-24T15:32:43.004 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038521396999999999 2024-04-24T15:32:43.004 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.004 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.004 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:43.004 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930949+0000", 2024-04-24T15:32:43.004 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4153999999999998e-05 2024-04-24T15:32:43.004 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.004 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.004 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:43.004 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930997+0000", 2024-04-24T15:32:43.004 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7998999999999999e-05 2024-04-24T15:32:43.004 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:43.004 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:43.004 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:43.004 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.005 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.005 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14045 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2465792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:43.005 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892387+0000", 2024-04-24T15:32:43.005 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.1037465359999992, 2024-04-24T15:32:43.005 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038720933999999999, 2024-04-24T15:32:43.005 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:43.005 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:43.005 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:43.005 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:43.005 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:43.005 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14045 2024-04-24T15:32:43.005 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.005 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:43.005 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.006 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:43.006 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892387+0000", 2024-04-24T15:32:43.006 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:43.006 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.006 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.006 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:43.006 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892387+0000", 2024-04-24T15:32:43.006 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:43.006 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.006 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.006 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:43.006 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892387+0000", 2024-04-24T15:32:43.006 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:43.006 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.007 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.007 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:43.007 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892392+0000", 2024-04-24T15:32:43.007 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0039999999999999e-06 2024-04-24T15:32:43.007 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.007 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.007 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:43.007 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892398+0000", 2024-04-24T15:32:43.007 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0900000000000001e-06 2024-04-24T15:32:43.007 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.007 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.007 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:43.007 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892402+0000", 2024-04-24T15:32:43.007 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.399e-06 2024-04-24T15:32:43.007 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.007 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.007 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:43.008 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931018+0000", 2024-04-24T15:32:43.008 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038616551999999998 2024-04-24T15:32:43.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.008 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.008 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:43.008 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931059+0000", 2024-04-24T15:32:43.008 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1205000000000003e-05 2024-04-24T15:32:43.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.008 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.008 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:43.008 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931108+0000", 2024-04-24T15:32:43.008 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8996000000000003e-05 2024-04-24T15:32:43.008 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:43.008 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:43.008 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:43.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.009 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14046 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2506752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:43.009 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892406+0000", 2024-04-24T15:32:43.009 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.1037279459999993, 2024-04-24T15:32:43.009 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038814507999999998, 2024-04-24T15:32:43.009 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:43.009 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:43.009 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:43.009 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:43.009 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:43.009 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14046 2024-04-24T15:32:43.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.009 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:43.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.010 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:43.010 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892406+0000", 2024-04-24T15:32:43.010 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:43.010 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.010 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.010 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:43.010 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892406+0000", 2024-04-24T15:32:43.010 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:43.010 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.010 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.010 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:43.010 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892406+0000", 2024-04-24T15:32:43.011 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:43.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.011 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.011 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:43.011 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892413+0000", 2024-04-24T15:32:43.011 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7449999999999995e-06 2024-04-24T15:32:43.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.011 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.011 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:43.011 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892419+0000", 2024-04-24T15:32:43.011 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5790000000000003e-06 2024-04-24T15:32:43.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.011 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.011 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:43.012 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892423+0000", 2024-04-24T15:32:43.012 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8199999999999998e-06 2024-04-24T15:32:43.012 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.012 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.012 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:43.012 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931128+0000", 2024-04-24T15:32:43.012 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038705502000000003 2024-04-24T15:32:43.012 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.012 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.012 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:43.012 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931172+0000", 2024-04-24T15:32:43.012 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4066999999999997e-05 2024-04-24T15:32:43.012 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.012 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.012 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:43.013 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931220+0000", 2024-04-24T15:32:43.013 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8167999999999997e-05 2024-04-24T15:32:43.013 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:43.013 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:43.013 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:43.013 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.013 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.013 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14047 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2547712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:43.013 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892427+0000", 2024-04-24T15:32:43.013 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.1037067169999997, 2024-04-24T15:32:43.013 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038904506999999998, 2024-04-24T15:32:43.013 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:43.013 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:43.014 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:43.014 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:43.014 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:43.014 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14047 2024-04-24T15:32:43.014 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.014 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:43.014 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.014 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:43.014 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892427+0000", 2024-04-24T15:32:43.014 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:43.014 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.014 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.014 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:43.015 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892427+0000", 2024-04-24T15:32:43.015 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:43.015 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.015 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.015 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:43.015 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892427+0000", 2024-04-24T15:32:43.015 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:43.015 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.015 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.015 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:43.015 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892429+0000", 2024-04-24T15:32:43.015 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3099999999999999e-06 2024-04-24T15:32:43.015 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.015 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.016 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:43.016 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892435+0000", 2024-04-24T15:32:43.016 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5520000000000004e-06 2024-04-24T15:32:43.016 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.016 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.016 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:43.016 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892438+0000", 2024-04-24T15:32:43.016 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4240000000000002e-06 2024-04-24T15:32:43.016 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.016 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.016 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:43.016 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931242+0000", 2024-04-24T15:32:43.016 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038804162000000003 2024-04-24T15:32:43.016 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.016 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.016 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:43.017 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931284+0000", 2024-04-24T15:32:43.017 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1414e-05 2024-04-24T15:32:43.017 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.017 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.017 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:43.017 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931332+0000", 2024-04-24T15:32:43.017 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8024e-05 2024-04-24T15:32:43.017 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:43.017 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:43.017 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:43.017 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.017 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.017 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14048 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2588672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:43.017 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892442+0000", 2024-04-24T15:32:43.017 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.1036917539999997, 2024-04-24T15:32:43.018 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039003132000000003, 2024-04-24T15:32:43.018 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:43.018 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:43.018 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:43.018 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:43.018 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:43.018 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14048 2024-04-24T15:32:43.018 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.018 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:43.018 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.018 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:43.018 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892442+0000", 2024-04-24T15:32:43.018 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:43.018 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.018 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.019 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:43.019 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892442+0000", 2024-04-24T15:32:43.019 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:43.019 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.019 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.019 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:43.019 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892442+0000", 2024-04-24T15:32:43.019 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:43.019 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.019 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.019 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:43.019 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892444+0000", 2024-04-24T15:32:43.019 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1169999999999998e-06 2024-04-24T15:32:43.019 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.020 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.020 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:43.020 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892449+0000", 2024-04-24T15:32:43.020 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1599999999999997e-06 2024-04-24T15:32:43.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.020 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.020 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:43.020 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892453+0000", 2024-04-24T15:32:43.020 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5930000000000002e-06 2024-04-24T15:32:43.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.020 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.020 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:43.020 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931352+0000", 2024-04-24T15:32:43.020 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038899361 2024-04-24T15:32:43.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.020 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.021 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:43.021 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931398+0000", 2024-04-24T15:32:43.021 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6315e-05 2024-04-24T15:32:43.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.021 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.021 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:43.021 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931445+0000", 2024-04-24T15:32:43.021 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6950000000000003e-05 2024-04-24T15:32:43.021 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:43.021 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:43.021 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:43.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.021 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.021 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14049 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2629632~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:43.021 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892458+0000", 2024-04-24T15:32:43.021 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.1036763339999993, 2024-04-24T15:32:43.022 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039103322000000003, 2024-04-24T15:32:43.022 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:43.022 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:43.022 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:43.022 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:43.022 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:43.022 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14049 2024-04-24T15:32:43.022 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.022 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:43.022 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.022 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:43.022 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892458+0000", 2024-04-24T15:32:43.022 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:43.022 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.022 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.023 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:43.023 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892458+0000", 2024-04-24T15:32:43.023 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:43.023 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.023 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.023 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:43.023 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892457+0000", 2024-04-24T15:32:43.023 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:43.023 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.023 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.023 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:43.023 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892459+0000", 2024-04-24T15:32:43.023 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.305e-06 2024-04-24T15:32:43.023 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.023 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.023 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:43.024 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892465+0000", 2024-04-24T15:32:43.024 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4360000000000001e-06 2024-04-24T15:32:43.024 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.024 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.024 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:43.024 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892474+0000", 2024-04-24T15:32:43.024 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7639999999999994e-06 2024-04-24T15:32:43.024 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.024 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.024 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:43.024 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931466+0000", 2024-04-24T15:32:43.024 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038992644 2024-04-24T15:32:43.024 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.024 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.025 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:43.025 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931512+0000", 2024-04-24T15:32:43.025 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5200000000000001e-05 2024-04-24T15:32:43.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.025 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.025 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:43.025 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931561+0000", 2024-04-24T15:32:43.025 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9357999999999997e-05 2024-04-24T15:32:43.025 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:43.025 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:43.025 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:43.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.025 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.025 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14050 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2670592~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:43.025 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892478+0000", 2024-04-24T15:32:43.026 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.1036557469999995, 2024-04-24T15:32:43.026 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039195102000000002, 2024-04-24T15:32:43.026 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:43.026 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:43.026 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:43.026 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:43.026 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:43.026 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14050 2024-04-24T15:32:43.026 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.026 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:43.026 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.026 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:43.026 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892478+0000", 2024-04-24T15:32:43.026 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:43.026 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.027 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.027 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:43.027 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892478+0000", 2024-04-24T15:32:43.027 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:43.027 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.027 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.027 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:43.027 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892478+0000", 2024-04-24T15:32:43.027 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:43.027 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.027 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.027 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:43.027 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892480+0000", 2024-04-24T15:32:43.027 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3300000000000001e-06 2024-04-24T15:32:43.027 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.027 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.028 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:43.028 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892486+0000", 2024-04-24T15:32:43.028 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9200000000000001e-06 2024-04-24T15:32:43.028 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.028 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.028 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:43.028 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892494+0000", 2024-04-24T15:32:43.028 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5159999999999997e-06 2024-04-24T15:32:43.028 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.028 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.028 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:43.028 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931581+0000", 2024-04-24T15:32:43.028 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039087894999999998 2024-04-24T15:32:43.028 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.028 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.029 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:43.029 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931626+0000", 2024-04-24T15:32:43.029 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4943e-05 2024-04-24T15:32:43.029 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.029 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.029 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:43.029 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931673+0000", 2024-04-24T15:32:43.029 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6906000000000001e-05 2024-04-24T15:32:43.029 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:43.029 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:43.029 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:43.029 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.029 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.029 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14051 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2711552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:43.029 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892507+0000", 2024-04-24T15:32:43.030 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.1036270479999999, 2024-04-24T15:32:43.030 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039274393999999997, 2024-04-24T15:32:43.030 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:43.030 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:43.030 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:43.030 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:43.030 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:43.030 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14051 2024-04-24T15:32:43.030 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.030 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:43.030 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.030 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:43.030 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892507+0000", 2024-04-24T15:32:43.030 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:43.030 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.031 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.031 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:43.031 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892507+0000", 2024-04-24T15:32:43.031 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:43.031 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.031 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.031 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:43.031 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892506+0000", 2024-04-24T15:32:43.031 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:43.031 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.031 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.031 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:43.031 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892509+0000", 2024-04-24T15:32:43.031 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8760000000000001e-06 2024-04-24T15:32:43.031 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.031 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.032 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:43.032 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892515+0000", 2024-04-24T15:32:43.032 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1840000000000004e-06 2024-04-24T15:32:43.032 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.032 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.032 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:43.032 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892519+0000", 2024-04-24T15:32:43.032 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7570000000000002e-06 2024-04-24T15:32:43.032 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.032 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.032 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:43.032 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931694+0000", 2024-04-24T15:32:43.032 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039174960000000002 2024-04-24T15:32:43.032 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.032 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.032 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:43.033 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931734+0000", 2024-04-24T15:32:43.033 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0284000000000003e-05 2024-04-24T15:32:43.033 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.033 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.033 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:43.033 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931781+0000", 2024-04-24T15:32:43.033 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6813000000000001e-05 2024-04-24T15:32:43.033 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:43.033 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:43.033 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:43.033 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.033 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.033 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14052 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2752512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:43.033 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892524+0000", 2024-04-24T15:32:43.034 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.1036099010000004, 2024-04-24T15:32:43.034 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039371122000000001, 2024-04-24T15:32:43.034 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:43.034 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:43.034 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:43.034 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:43.034 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:43.034 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14052 2024-04-24T15:32:43.034 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.034 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:43.034 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.034 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:43.034 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892524+0000", 2024-04-24T15:32:43.035 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:43.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.035 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.035 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:43.035 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892524+0000", 2024-04-24T15:32:43.035 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:43.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.035 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.035 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:43.035 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892524+0000", 2024-04-24T15:32:43.035 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:43.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.035 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.036 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:43.036 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892526+0000", 2024-04-24T15:32:43.036 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3630000000000001e-06 2024-04-24T15:32:43.036 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.036 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.036 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:43.036 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892538+0000", 2024-04-24T15:32:43.036 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1677e-05 2024-04-24T15:32:43.036 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.036 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.036 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:43.036 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892541+0000", 2024-04-24T15:32:43.036 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.822e-06 2024-04-24T15:32:43.036 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.037 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:43.037 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931802+0000", 2024-04-24T15:32:43.037 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039260911000000003 2024-04-24T15:32:43.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.037 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:43.037 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931847+0000", 2024-04-24T15:32:43.037 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4450999999999998e-05 2024-04-24T15:32:43.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.037 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:43.037 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931895+0000", 2024-04-24T15:32:43.037 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8257999999999998e-05 2024-04-24T15:32:43.038 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:43.038 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:43.038 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:43.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.038 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.038 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14053 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2793472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:43.038 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892546+0000", 2024-04-24T15:32:43.038 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.1035879229999992, 2024-04-24T15:32:43.038 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039457712999999998, 2024-04-24T15:32:43.038 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:43.038 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:43.038 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:43.038 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:43.038 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:43.039 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14053 2024-04-24T15:32:43.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.039 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:43.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:43.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892546+0000", 2024-04-24T15:32:43.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:43.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:43.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892546+0000", 2024-04-24T15:32:43.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:43.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:43.040 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892546+0000", 2024-04-24T15:32:43.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:43.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.040 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:43.040 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892548+0000", 2024-04-24T15:32:43.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3120000000000001e-06 2024-04-24T15:32:43.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.040 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:43.040 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892553+0000", 2024-04-24T15:32:43.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2320000000000001e-06 2024-04-24T15:32:43.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.040 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:43.040 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892559+0000", 2024-04-24T15:32:43.041 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5480000000000001e-06 2024-04-24T15:32:43.041 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.041 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:43.041 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931916+0000", 2024-04-24T15:32:43.041 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039357180999999998 2024-04-24T15:32:43.041 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.041 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:43.041 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931956+0000", 2024-04-24T15:32:43.041 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0565000000000002e-05 2024-04-24T15:32:43.041 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.041 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:43.041 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932004+0000", 2024-04-24T15:32:43.041 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7224000000000001e-05 2024-04-24T15:32:43.042 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:43.042 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:43.042 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:43.042 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.042 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.042 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14054 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2834432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:43.042 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892563+0000", 2024-04-24T15:32:43.042 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.1035712709999999, 2024-04-24T15:32:43.042 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039552406999999998, 2024-04-24T15:32:43.042 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:43.042 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:43.042 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:43.042 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:43.042 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:43.043 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14054 2024-04-24T15:32:43.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.043 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:43.043 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.043 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:43.043 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892563+0000", 2024-04-24T15:32:43.043 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:43.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.043 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.043 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:43.043 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892563+0000", 2024-04-24T15:32:43.043 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:43.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.043 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.043 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:43.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892562+0000", 2024-04-24T15:32:43.044 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:43.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:43.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892564+0000", 2024-04-24T15:32:43.044 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2029999999999999e-06 2024-04-24T15:32:43.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:43.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892570+0000", 2024-04-24T15:32:43.044 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1460000000000004e-06 2024-04-24T15:32:43.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:43.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892573+0000", 2024-04-24T15:32:43.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2030000000000001e-06 2024-04-24T15:32:43.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.045 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.045 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:43.045 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932024+0000", 2024-04-24T15:32:43.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039451696000000001 2024-04-24T15:32:43.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.045 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.045 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:43.045 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932068+0000", 2024-04-24T15:32:43.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3970000000000001e-05 2024-04-24T15:32:43.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.045 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.045 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:43.046 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932115+0000", 2024-04-24T15:32:43.046 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6569000000000001e-05 2024-04-24T15:32:43.046 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:43.046 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:43.046 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:43.046 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.046 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.046 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14055 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2875392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:43.046 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892591+0000", 2024-04-24T15:32:43.046 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.103543298, 2024-04-24T15:32:43.046 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039634309, 2024-04-24T15:32:43.046 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:43.046 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:43.046 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:43.046 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:43.047 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:43.047 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14055 2024-04-24T15:32:43.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.047 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:43.047 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.047 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:43.047 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892591+0000", 2024-04-24T15:32:43.047 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:43.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.047 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.047 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:43.047 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892591+0000", 2024-04-24T15:32:43.047 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:43.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.047 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.047 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:43.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892590+0000", 2024-04-24T15:32:43.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:43.048 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:43.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892592+0000", 2024-04-24T15:32:43.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7519999999999999e-06 2024-04-24T15:32:43.048 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:43.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892596+0000", 2024-04-24T15:32:43.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.608e-06 2024-04-24T15:32:43.048 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:43.049 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892598+0000", 2024-04-24T15:32:43.049 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.531e-06 2024-04-24T15:32:43.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.049 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:43.049 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932136+0000", 2024-04-24T15:32:43.049 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039537668999999998 2024-04-24T15:32:43.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.049 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:43.049 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932177+0000", 2024-04-24T15:32:43.049 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0880000000000002e-05 2024-04-24T15:32:43.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.049 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:43.050 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932225+0000", 2024-04-24T15:32:43.050 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8223999999999998e-05 2024-04-24T15:32:43.050 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:43.050 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:43.050 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:43.050 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.050 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.050 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14056 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2916352~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:43.050 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892600+0000", 2024-04-24T15:32:43.050 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.1035334639999999, 2024-04-24T15:32:43.050 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039736447000000001, 2024-04-24T15:32:43.050 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:43.050 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:43.050 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:43.050 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:43.051 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:43.051 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14056 2024-04-24T15:32:43.051 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.051 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:43.051 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.051 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:43.051 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892600+0000", 2024-04-24T15:32:43.051 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:43.051 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.051 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.051 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:43.051 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892600+0000", 2024-04-24T15:32:43.051 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:43.051 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.051 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.051 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:43.052 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892600+0000", 2024-04-24T15:32:43.052 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967296 2024-04-24T15:32:43.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.052 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:43.052 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892602+0000", 2024-04-24T15:32:43.052 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.576e-06 2024-04-24T15:32:43.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.052 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:43.052 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892605+0000", 2024-04-24T15:32:43.052 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.574e-06 2024-04-24T15:32:43.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.052 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:43.052 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892608+0000", 2024-04-24T15:32:43.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3800000000000001e-06 2024-04-24T15:32:43.053 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.053 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:43.053 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932245+0000", 2024-04-24T15:32:43.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039637378000000001 2024-04-24T15:32:43.053 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.053 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:43.053 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932290+0000", 2024-04-24T15:32:43.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4734999999999998e-05 2024-04-24T15:32:43.053 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.053 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:43.053 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932337+0000", 2024-04-24T15:32:43.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7033000000000002e-05 2024-04-24T15:32:43.054 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:43.054 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:43.054 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:43.054 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.054 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.054 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14062 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 3162112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:43.054 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.894556+0000", 2024-04-24T15:32:43.054 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.1015778940000001, 2024-04-24T15:32:43.054 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.03845316, 2024-04-24T15:32:43.054 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:43.054 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:43.054 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:43.055 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:43.055 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:43.055 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14062 2024-04-24T15:32:43.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.055 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:43.055 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.055 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:43.055 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894556+0000", 2024-04-24T15:32:43.055 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:43.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.055 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.055 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:43.055 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894556+0000", 2024-04-24T15:32:43.055 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:43.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.055 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.056 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:43.056 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894556+0000", 2024-04-24T15:32:43.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967296 2024-04-24T15:32:43.056 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.056 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.056 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:43.056 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894557+0000", 2024-04-24T15:32:43.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3230000000000001e-06 2024-04-24T15:32:43.056 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.056 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.056 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:43.056 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894560+0000", 2024-04-24T15:32:43.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7719999999999999e-06 2024-04-24T15:32:43.056 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.056 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.056 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:43.057 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894562+0000", 2024-04-24T15:32:43.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9479999999999998e-06 2024-04-24T15:32:43.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:43.057 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932917+0000", 2024-04-24T15:32:43.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038355490999999999 2024-04-24T15:32:43.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:43.057 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932961+0000", 2024-04-24T15:32:43.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3302e-05 2024-04-24T15:32:43.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:43.058 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933009+0000", 2024-04-24T15:32:43.058 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8535000000000002e-05 2024-04-24T15:32:43.058 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:43.058 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:43.058 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:43.058 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.058 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.058 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14063 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 3203072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:43.058 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.894564+0000", 2024-04-24T15:32:43.058 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.1015698730000008, 2024-04-24T15:32:43.058 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038554995000000002, 2024-04-24T15:32:43.058 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:43.058 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:43.058 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:43.058 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:43.058 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:43.059 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14063 2024-04-24T15:32:43.059 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.059 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:43.059 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.059 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:43.059 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894564+0000", 2024-04-24T15:32:43.059 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:43.059 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.059 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.059 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:43.059 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894564+0000", 2024-04-24T15:32:43.059 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:43.059 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.059 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.060 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:43.060 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894564+0000", 2024-04-24T15:32:43.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967296 2024-04-24T15:32:43.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.060 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.060 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:43.060 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894565+0000", 2024-04-24T15:32:43.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2640000000000001e-06 2024-04-24T15:32:43.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.060 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.060 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:43.060 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894569+0000", 2024-04-24T15:32:43.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1710000000000004e-06 2024-04-24T15:32:43.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.060 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.061 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:43.061 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894573+0000", 2024-04-24T15:32:43.061 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4410000000000002e-06 2024-04-24T15:32:43.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.061 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:43.061 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933030+0000", 2024-04-24T15:32:43.061 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038457709999999999 2024-04-24T15:32:43.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.061 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:43.061 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933072+0000", 2024-04-24T15:32:43.061 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.193e-05 2024-04-24T15:32:43.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.062 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:43.062 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933119+0000", 2024-04-24T15:32:43.062 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6715000000000003e-05 2024-04-24T15:32:43.062 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:43.062 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:43.062 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:43.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.062 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14064 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 3244032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:43.062 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.894591+0000", 2024-04-24T15:32:43.062 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.1015432310000008, 2024-04-24T15:32:43.062 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038639806999999998, 2024-04-24T15:32:43.062 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:43.063 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:43.063 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:43.063 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:43.063 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:43.063 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14064 2024-04-24T15:32:43.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.063 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:43.063 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.063 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:43.063 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894591+0000", 2024-04-24T15:32:43.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:43.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.063 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.063 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:43.063 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894591+0000", 2024-04-24T15:32:43.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:43.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:43.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894590+0000", 2024-04-24T15:32:43.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:43.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:43.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894593+0000", 2024-04-24T15:32:43.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4719999999999998e-06 2024-04-24T15:32:43.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:43.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894598+0000", 2024-04-24T15:32:43.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.57e-06 2024-04-24T15:32:43.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:43.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894602+0000", 2024-04-24T15:32:43.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.614e-06 2024-04-24T15:32:43.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:43.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933139+0000", 2024-04-24T15:32:43.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038536929999999997 2024-04-24T15:32:43.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:43.066 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933183+0000", 2024-04-24T15:32:43.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4217e-05 2024-04-24T15:32:43.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.066 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:43.066 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933230+0000", 2024-04-24T15:32:43.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7434e-05 2024-04-24T15:32:43.066 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:43.066 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:43.066 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:43.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.066 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14065 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 3284992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:43.066 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.894606+0000", 2024-04-24T15:32:43.066 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.101527849, 2024-04-24T15:32:43.067 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038734126000000001, 2024-04-24T15:32:43.067 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:43.067 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:43.067 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:43.067 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:43.067 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:43.067 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14065 2024-04-24T15:32:43.067 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.067 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:43.067 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.067 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:43.067 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894606+0000", 2024-04-24T15:32:43.067 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:43.067 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.068 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:43.068 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894606+0000", 2024-04-24T15:32:43.068 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:43.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.068 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:43.068 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894606+0000", 2024-04-24T15:32:43.068 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:43.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.068 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:43.068 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894608+0000", 2024-04-24T15:32:43.068 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0880000000000002e-06 2024-04-24T15:32:43.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.069 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:43.069 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894614+0000", 2024-04-24T15:32:43.069 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7830000000000004e-06 2024-04-24T15:32:43.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.069 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:43.069 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894621+0000", 2024-04-24T15:32:43.069 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5809999999999999e-06 2024-04-24T15:32:43.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.069 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:43.069 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933251+0000", 2024-04-24T15:32:43.070 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038629995 2024-04-24T15:32:43.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.070 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.070 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:43.070 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933292+0000", 2024-04-24T15:32:43.070 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0380000000000003e-05 2024-04-24T15:32:43.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.070 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.070 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:43.070 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933340+0000", 2024-04-24T15:32:43.070 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8662999999999998e-05 2024-04-24T15:32:43.070 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:43.071 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:43.071 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:43.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.071 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.071 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14066 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 3325952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:43.071 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.894625+0000", 2024-04-24T15:32:43.071 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.1015084099999992, 2024-04-24T15:32:43.071 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038825539999999999, 2024-04-24T15:32:43.071 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:43.071 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:43.071 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:43.071 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:43.071 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:43.071 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14066 2024-04-24T15:32:43.072 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.072 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:43.072 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.072 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:43.072 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894625+0000", 2024-04-24T15:32:43.072 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:43.072 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.072 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.072 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:43.072 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894625+0000", 2024-04-24T15:32:43.072 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:43.072 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.072 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.072 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:43.072 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894625+0000", 2024-04-24T15:32:43.073 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:43.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.073 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.073 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:43.073 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894627+0000", 2024-04-24T15:32:43.073 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.092e-06 2024-04-24T15:32:43.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.073 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.073 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:43.073 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894635+0000", 2024-04-24T15:32:43.073 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6660000000000008e-06 2024-04-24T15:32:43.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.073 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.073 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:43.073 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894639+0000", 2024-04-24T15:32:43.074 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.799e-06 2024-04-24T15:32:43.074 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.074 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.074 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:43.074 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933360+0000", 2024-04-24T15:32:43.074 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038721569999999997 2024-04-24T15:32:43.074 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.074 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.074 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:43.074 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933403+0000", 2024-04-24T15:32:43.074 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2605999999999998e-05 2024-04-24T15:32:43.074 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.074 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.075 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:43.075 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933451+0000", 2024-04-24T15:32:43.075 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8149999999999998e-05 2024-04-24T15:32:43.075 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:43.075 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:43.075 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:43.075 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.075 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.075 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14067 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 3366912~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:43.075 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.894644+0000", 2024-04-24T15:32:43.075 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.1014903520000008, 2024-04-24T15:32:43.075 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038926068000000001, 2024-04-24T15:32:43.075 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:43.075 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:43.075 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:43.076 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:43.076 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:43.076 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14067 2024-04-24T15:32:43.076 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.076 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:43.076 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.076 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:43.076 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894644+0000", 2024-04-24T15:32:43.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:43.076 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.076 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.076 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:43.076 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894644+0000", 2024-04-24T15:32:43.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:43.076 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.076 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.077 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:43.077 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894643+0000", 2024-04-24T15:32:43.077 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:43.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.077 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:43.077 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894646+0000", 2024-04-24T15:32:43.077 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3379999999999999e-06 2024-04-24T15:32:43.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.077 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:43.077 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894651+0000", 2024-04-24T15:32:43.077 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.485e-06 2024-04-24T15:32:43.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.078 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:43.078 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894655+0000", 2024-04-24T15:32:43.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.709e-06 2024-04-24T15:32:43.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.078 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.078 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:43.078 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933472+0000", 2024-04-24T15:32:43.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038816992000000002 2024-04-24T15:32:43.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.078 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.078 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:43.078 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933521+0000", 2024-04-24T15:32:43.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8841999999999998e-05 2024-04-24T15:32:43.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:43.078 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:43.079 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:43.079 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933570+0000", 2024-04-24T15:32:43.079 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9063000000000001e-05 2024-04-24T15:32:43.079 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:43.079 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:43.079 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:43.079 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:43.079 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:43.079 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:32:43.080 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-24T15:32:43.081 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:32:43.081+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:32:43.181 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:32:43.182+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:32:43.216 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-24T15:32:43.221 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:32:43.221 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:32:43.221 INFO:teuthology.orchestra.run.smithi151.stdout: "num_ops": 0 2024-04-24T15:32:43.221 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:32:43.232 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-24T15:32:43.265 INFO:teuthology.orchestra.run.smithi096.stdout:flattening image image_client.0-clone6 2024-04-24T15:32:43.282 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:32:43.282+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:32:43.383 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:32:43.383+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:32:43.397 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:32:43.398 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:32:43.398 INFO:teuthology.orchestra.run.smithi151.stdout: "complaint_time": 30, 2024-04-24T15:32:43.398 INFO:teuthology.orchestra.run.smithi151.stdout: "num_blocked_ops": 0 2024-04-24T15:32:43.398 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:32:43.408 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-24T15:32:43.411 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #5, image image_client.0-clone5 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent6 2024-04-24T15:32:43.437 INFO:teuthology.orchestra.run.smithi096.stdout:1589 write 0xc7bdc20 thru 0xc7c6fa1 (0x9382 bytes) 2024-04-24T15:32:43.440 INFO:teuthology.orchestra.run.smithi096.stdout:1591 writesame 0x2ab2a6d thru 0x2ab4e86 data_size 0x1(0x241a bytes) 2024-04-24T15:32:43.454 INFO:teuthology.orchestra.run.smithi096.stdout:1592 writesame 0xe31888f thru 0xe31f7d0 data_size 0x65(0x6f42 bytes) 2024-04-24T15:32:43.454 INFO:teuthology.orchestra.run.smithi096.stdout:1593 writesame 0x12c894a thru 0x12cf625 data_size 0x2(0x6cdc bytes) 2024-04-24T15:32:43.463 INFO:teuthology.orchestra.run.smithi096.stdout:1594 read 0x84c0e93 thru 0x84cca19 (0xbb87 bytes) 2024-04-24T15:32:43.464 INFO:teuthology.orchestra.run.smithi096.stdout:1596 trunc from 0xe31f7d1 to 0xdda6006 2024-04-24T15:32:43.469 INFO:teuthology.orchestra.run.smithi096.stdout:1597 punch from 0x41d002f to 0x41d6c85, (0x6c56 bytes) 2024-04-24T15:32:43.476 INFO:teuthology.orchestra.run.smithi096.stdout:1598 trunc from 0xdda6006 to 0x7f882c8 2024-04-24T15:32:43.484 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:32:43.485+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:32:43.499 INFO:teuthology.orchestra.run.smithi096.stdout:1599 writesame 0x374c63f thru 0x374ea3a data_size 0x1c(0x23fc bytes) 2024-04-24T15:32:43.502 INFO:teuthology.orchestra.run.smithi096.stdout:1602 punch from 0x254053c to 0x2542510, (0x1fd4 bytes) 2024-04-24T15:32:43.502 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:43.504 INFO:teuthology.orchestra.run.smithi096.stdout:1604 trunc from 0x7f882c8 to 0x984ae6e 2024-04-24T15:32:43.508 INFO:teuthology.orchestra.run.smithi096.stdout:1607 write 0x2589c78 thru 0x258f7db (0x5b64 bytes) 2024-04-24T15:32:43.522 INFO:teuthology.orchestra.run.smithi096.stdout:1608 trunc from 0x984ae6e to 0x1e9791a 2024-04-24T15:32:43.546 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:32:43.546 INFO:teuthology.orchestra.run.smithi151.stdout: "size": 20, 2024-04-24T15:32:43.546 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 600, 2024-04-24T15:32:43.546 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [ 2024-04-24T15:32:43.546 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.546 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:2507 4.e 4:729aaa1b:::rbd_data.114a1f60cb66.000000000000029a:head [set-alloc-hint object_size 262144 write_size 262144,write 233472~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-24T15:32:43.546 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:23.072715+0000", 2024-04-24T15:32:43.546 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 20.472418376, 2024-04-24T15:32:43.546 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0065961370000000002, 2024-04-24T15:32:43.547 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:43.547 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:43.547 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:43.547 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:43.547 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:43.547 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 2507 2024-04-24T15:32:43.547 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.547 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:43.547 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.547 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:43.547 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.072715+0000", 2024-04-24T15:32:43.547 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:43.547 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.547 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.548 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:43.548 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.072715+0000", 2024-04-24T15:32:43.548 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:43.548 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.548 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.548 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:43.548 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.072713+0000", 2024-04-24T15:32:43.548 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999981 2024-04-24T15:32:43.548 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.548 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.548 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:43.548 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.072727+0000", 2024-04-24T15:32:43.548 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3549e-05 2024-04-24T15:32:43.548 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.549 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.549 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:43.549 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.072739+0000", 2024-04-24T15:32:43.549 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.1973999999999999e-05 2024-04-24T15:32:43.549 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.549 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.549 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:43.549 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.072746+0000", 2024-04-24T15:32:43.549 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2849999999999997e-06 2024-04-24T15:32:43.549 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.549 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.549 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:43.549 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.072798+0000", 2024-04-24T15:32:43.549 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.2289e-05 2024-04-24T15:32:43.550 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.550 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.550 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op must wait for map", 2024-04-24T15:32:43.550 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.072806+0000", 2024-04-24T15:32:43.550 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.8469999999999996e-06 2024-04-24T15:32:43.550 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.550 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.550 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:43.550 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.074503+0000", 2024-04-24T15:32:43.550 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0016967029999999999 2024-04-24T15:32:43.550 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.550 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.550 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:43.550 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.074593+0000", 2024-04-24T15:32:43.551 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.0313000000000001e-05 2024-04-24T15:32:43.551 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.551 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.551 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 6", 2024-04-24T15:32:43.551 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.074732+0000", 2024-04-24T15:32:43.551 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000138376 2024-04-24T15:32:43.551 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.551 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.551 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:43.551 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.078538+0000", 2024-04-24T15:32:43.551 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0038068749999999999 2024-04-24T15:32:43.551 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.551 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.551 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:43.552 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.079232+0000", 2024-04-24T15:32:43.552 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00069326900000000005 2024-04-24T15:32:43.552 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.552 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.552 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:43.552 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.079273+0000", 2024-04-24T15:32:43.552 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.1236000000000003e-05 2024-04-24T15:32:43.552 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.552 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.552 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:43.552 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.079311+0000", 2024-04-24T15:32:43.552 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8497000000000001e-05 2024-04-24T15:32:43.552 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:43.553 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:43.553 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:43.553 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.553 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.553 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_repop(client.4426.0:2563 4.1d e34/27 4:b841be81:::rbd_data.114a1f60cb66.00000000000001d7:head v 34'17, mlcod=32'15)", 2024-04-24T15:32:43.553 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.108404+0000", 2024-04-24T15:32:43.553 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 19.436729843999998, 2024-04-24T15:32:43.553 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0073539469999999996, 2024-04-24T15:32:43.553 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:43.553 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:43.553 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:43.553 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.553 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:43.553 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.108404+0000", 2024-04-24T15:32:43.553 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:43.554 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.554 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.554 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:43.554 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.108404+0000", 2024-04-24T15:32:43.554 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:43.554 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.554 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.554 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:43.554 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.108403+0000", 2024-04-24T15:32:43.554 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:43.554 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.554 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.555 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:43.555 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.108420+0000", 2024-04-24T15:32:43.555 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.6980999999999999e-05 2024-04-24T15:32:43.555 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.555 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.555 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:43.555 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.108428+0000", 2024-04-24T15:32:43.555 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0560000000000005e-06 2024-04-24T15:32:43.555 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.555 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.555 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:43.555 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.108432+0000", 2024-04-24T15:32:43.555 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.4060000000000001e-06 2024-04-24T15:32:43.555 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.556 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.556 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:43.556 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.108462+0000", 2024-04-24T15:32:43.556 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.008e-05 2024-04-24T15:32:43.556 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.556 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.556 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:43.556 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.108475+0000", 2024-04-24T15:32:43.556 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3018e-05 2024-04-24T15:32:43.556 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.556 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.556 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:43.556 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.115726+0000", 2024-04-24T15:32:43.556 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0072510040000000001 2024-04-24T15:32:43.556 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.557 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.557 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:43.557 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.115758+0000", 2024-04-24T15:32:43.557 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.2092999999999999e-05 2024-04-24T15:32:43.557 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:43.557 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:43.557 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:43.557 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.557 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.557 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:2620 4.18 4:1e938922:::rbd_data.114a1f60cb66.0000000000000028:head [stat,write 163840~6144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:43.557 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.151073+0000", 2024-04-24T15:32:43.557 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 19.394061131000001, 2024-04-24T15:32:43.557 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0063989210000000001, 2024-04-24T15:32:43.557 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:43.557 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:43.558 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:43.558 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:43.558 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:43.558 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 2620 2024-04-24T15:32:43.558 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.558 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:43.558 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.558 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:43.558 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.151073+0000", 2024-04-24T15:32:43.558 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:43.558 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.558 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.558 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:43.559 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.151073+0000", 2024-04-24T15:32:43.559 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:43.559 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.559 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.559 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:43.559 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.151072+0000", 2024-04-24T15:32:43.559 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:43.559 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.559 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.559 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:43.559 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.151083+0000", 2024-04-24T15:32:43.559 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0801999999999999e-05 2024-04-24T15:32:43.559 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.559 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.559 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:43.560 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.151086+0000", 2024-04-24T15:32:43.560 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.3900000000000002e-06 2024-04-24T15:32:43.560 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.560 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.560 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:43.560 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.151089+0000", 2024-04-24T15:32:43.560 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.294e-06 2024-04-24T15:32:43.560 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.560 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.560 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:43.560 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.152198+0000", 2024-04-24T15:32:43.560 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.001109407 2024-04-24T15:32:43.560 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.560 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.561 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:43.561 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.152273+0000", 2024-04-24T15:32:43.561 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.4419999999999996e-05 2024-04-24T15:32:43.561 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.561 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.561 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 6", 2024-04-24T15:32:43.561 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.152396+0000", 2024-04-24T15:32:43.561 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00012322400000000001 2024-04-24T15:32:43.561 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.561 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.561 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:43.561 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.157242+0000", 2024-04-24T15:32:43.561 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0048462369999999998 2024-04-24T15:32:43.562 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.562 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.562 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:43.562 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.157414+0000", 2024-04-24T15:32:43.562 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000172411 2024-04-24T15:32:43.562 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.562 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.562 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:43.562 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.157451+0000", 2024-04-24T15:32:43.562 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.6269000000000001e-05 2024-04-24T15:32:43.562 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.562 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.562 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:43.563 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.157471+0000", 2024-04-24T15:32:43.563 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.0741e-05 2024-04-24T15:32:43.563 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:43.563 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:43.563 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:43.563 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.563 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.563 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:2623 4.18 4:1b4f38fa:::rbd_data.114a1f60cb66.000000000000002b:head [stat,write 161792~6144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:43.563 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.151108+0000", 2024-04-24T15:32:43.563 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 19.394025869, 2024-04-24T15:32:43.563 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0064461170000000003, 2024-04-24T15:32:43.563 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:43.563 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:43.563 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:43.564 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:43.564 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:43.564 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 2623 2024-04-24T15:32:43.564 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.564 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:43.564 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.564 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:43.564 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.151108+0000", 2024-04-24T15:32:43.564 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:43.564 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.564 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.564 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:43.564 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.151108+0000", 2024-04-24T15:32:43.564 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:43.565 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.565 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.565 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:43.565 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.151108+0000", 2024-04-24T15:32:43.565 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:43.565 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.565 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.565 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:43.565 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.151117+0000", 2024-04-24T15:32:43.565 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.8560000000000002e-06 2024-04-24T15:32:43.565 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.565 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.565 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:43.565 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.151122+0000", 2024-04-24T15:32:43.566 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5390000000000001e-06 2024-04-24T15:32:43.566 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.566 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.566 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:43.566 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.151124+0000", 2024-04-24T15:32:43.566 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.2800000000000002e-06 2024-04-24T15:32:43.566 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.566 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.566 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:43.566 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.152676+0000", 2024-04-24T15:32:43.566 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.001552124 2024-04-24T15:32:43.566 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.566 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.567 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:43.567 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.152759+0000", 2024-04-24T15:32:43.567 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.2688000000000005e-05 2024-04-24T15:32:43.567 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.567 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.567 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 6", 2024-04-24T15:32:43.567 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.152867+0000", 2024-04-24T15:32:43.567 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00010779400000000001 2024-04-24T15:32:43.567 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.567 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.567 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:43.567 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.157259+0000", 2024-04-24T15:32:43.567 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0043918230000000004 2024-04-24T15:32:43.567 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.567 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.568 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:43.568 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.157498+0000", 2024-04-24T15:32:43.568 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000239829 2024-04-24T15:32:43.568 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.568 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.568 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:43.568 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.157525+0000", 2024-04-24T15:32:43.568 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.6868000000000001e-05 2024-04-24T15:32:43.568 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.568 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.568 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:43.568 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.157554+0000", 2024-04-24T15:32:43.568 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8602000000000002e-05 2024-04-24T15:32:43.568 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:43.568 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:43.569 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:43.569 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.569 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.569 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:3618 4.18 4:1f7d02ce:::rbd_data.114a1f60cb66.0000000000000161:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:43.569 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.581260+0000", 2024-04-24T15:32:43.569 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 18.963873511999999, 2024-04-24T15:32:43.569 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0068184120000000003, 2024-04-24T15:32:43.569 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:43.569 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:43.569 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:43.569 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:43.569 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:43.569 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 3618 2024-04-24T15:32:43.569 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.569 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:43.570 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.570 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:43.570 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.581260+0000", 2024-04-24T15:32:43.570 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:43.570 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.570 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.570 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:43.570 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.581260+0000", 2024-04-24T15:32:43.570 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:43.570 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.570 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.570 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:43.570 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.581259+0000", 2024-04-24T15:32:43.571 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999999 2024-04-24T15:32:43.571 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.571 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.571 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:43.571 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.581265+0000", 2024-04-24T15:32:43.571 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.003e-06 2024-04-24T15:32:43.571 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.571 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.571 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:43.571 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.581278+0000", 2024-04-24T15:32:43.571 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.2483000000000001e-05 2024-04-24T15:32:43.571 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.571 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.571 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:43.571 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.581284+0000", 2024-04-24T15:32:43.572 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.7399999999999998e-06 2024-04-24T15:32:43.572 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.572 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.572 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:43.572 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.581783+0000", 2024-04-24T15:32:43.572 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00049892199999999995 2024-04-24T15:32:43.572 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.572 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.572 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:43.572 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.581902+0000", 2024-04-24T15:32:43.572 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000118272 2024-04-24T15:32:43.572 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.572 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.572 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 6", 2024-04-24T15:32:43.572 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.583068+0000", 2024-04-24T15:32:43.573 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0011665759999999999 2024-04-24T15:32:43.573 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.573 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.573 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:43.573 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.587081+0000", 2024-04-24T15:32:43.573 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.004012781 2024-04-24T15:32:43.573 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.573 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.573 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:43.573 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.587997+0000", 2024-04-24T15:32:43.573 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00091637700000000001 2024-04-24T15:32:43.573 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.573 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.573 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:43.573 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.588041+0000", 2024-04-24T15:32:43.574 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3619999999999999e-05 2024-04-24T15:32:43.574 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.574 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.574 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:43.574 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.588079+0000", 2024-04-24T15:32:43.574 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.7654000000000002e-05 2024-04-24T15:32:43.574 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:43.574 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:43.574 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:43.574 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.574 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.575 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:3767 4.9 4:908f8f18:::rbd_data.114a1f60cb66.000000000000001b:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:43.575 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.699557+0000", 2024-04-24T15:32:43.575 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 18.845576786999999, 2024-04-24T15:32:43.575 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0088231639999999997, 2024-04-24T15:32:43.575 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:43.575 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:43.575 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:43.575 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:43.575 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:43.575 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 3767 2024-04-24T15:32:43.575 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.575 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:43.575 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.575 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:43.576 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699557+0000", 2024-04-24T15:32:43.576 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:43.576 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.576 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.576 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:43.576 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699557+0000", 2024-04-24T15:32:43.576 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:43.576 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.576 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.576 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:43.576 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699556+0000", 2024-04-24T15:32:43.576 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999999 2024-04-24T15:32:43.576 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.576 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.577 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:43.577 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699561+0000", 2024-04-24T15:32:43.577 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.0440000000000003e-06 2024-04-24T15:32:43.577 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.577 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.577 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:43.577 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699571+0000", 2024-04-24T15:32:43.577 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0237999999999999e-05 2024-04-24T15:32:43.577 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.577 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.577 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:43.577 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699582+0000", 2024-04-24T15:32:43.577 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.1009000000000001e-05 2024-04-24T15:32:43.577 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.577 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.578 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:43.578 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699607+0000", 2024-04-24T15:32:43.578 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.5024999999999998e-05 2024-04-24T15:32:43.578 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.578 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.578 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:43.578 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699671+0000", 2024-04-24T15:32:43.578 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.3219999999999994e-05 2024-04-24T15:32:43.578 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.578 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.578 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:43.578 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.700952+0000", 2024-04-24T15:32:43.578 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.001281952 2024-04-24T15:32:43.578 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.578 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.579 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:43.579 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.705912+0000", 2024-04-24T15:32:43.579 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0049599889999999997 2024-04-24T15:32:43.579 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.579 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.579 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:43.579 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.708300+0000", 2024-04-24T15:32:43.579 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.002387934 2024-04-24T15:32:43.579 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.579 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.579 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:43.579 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.708345+0000", 2024-04-24T15:32:43.579 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4175000000000003e-05 2024-04-24T15:32:43.579 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.579 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.580 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:43.580 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.708380+0000", 2024-04-24T15:32:43.580 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5460999999999998e-05 2024-04-24T15:32:43.580 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:43.580 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:43.580 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:43.580 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.580 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.580 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:3770 4.13 4:cb3c841f:::rbd_data.114a1f60cb66.000000000000001e:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:43.580 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.699597+0000", 2024-04-24T15:32:43.580 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 18.845536323000001, 2024-04-24T15:32:43.580 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0083605450000000005, 2024-04-24T15:32:43.580 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:43.580 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:43.581 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:43.581 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:43.581 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:43.581 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 3770 2024-04-24T15:32:43.581 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.581 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:43.581 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.581 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:43.581 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699597+0000", 2024-04-24T15:32:43.581 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:43.581 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.581 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.581 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:43.581 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699597+0000", 2024-04-24T15:32:43.582 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:43.582 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.582 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.582 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:43.582 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699597+0000", 2024-04-24T15:32:43.582 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:43.582 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.582 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.582 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:43.582 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699600+0000", 2024-04-24T15:32:43.582 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.371e-06 2024-04-24T15:32:43.582 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.582 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.582 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:43.582 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699611+0000", 2024-04-24T15:32:43.583 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0552e-05 2024-04-24T15:32:43.583 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.583 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.583 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:43.583 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699615+0000", 2024-04-24T15:32:43.583 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5689999999999998e-06 2024-04-24T15:32:43.583 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.583 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.583 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:43.583 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699692+0000", 2024-04-24T15:32:43.583 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.6664999999999999e-05 2024-04-24T15:32:43.583 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.583 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.583 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:43.584 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699740+0000", 2024-04-24T15:32:43.584 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8032999999999999e-05 2024-04-24T15:32:43.584 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.584 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.584 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 6", 2024-04-24T15:32:43.584 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.700940+0000", 2024-04-24T15:32:43.584 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.001199978 2024-04-24T15:32:43.584 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.584 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.584 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:43.584 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.706397+0000", 2024-04-24T15:32:43.584 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0054569459999999998 2024-04-24T15:32:43.584 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.584 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.585 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:43.585 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.707870+0000", 2024-04-24T15:32:43.585 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.001473294 2024-04-24T15:32:43.585 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.585 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.585 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:43.585 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.707920+0000", 2024-04-24T15:32:43.585 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.994e-05 2024-04-24T15:32:43.585 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.585 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.585 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:43.585 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.707958+0000", 2024-04-24T15:32:43.585 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.7761e-05 2024-04-24T15:32:43.585 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:43.586 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:43.586 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:43.586 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.586 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.586 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:3773 4.e 4:7173e872:::rbd_data.114a1f60cb66.0000000000000021:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:43.586 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.699632+0000", 2024-04-24T15:32:43.586 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 18.845501760000001, 2024-04-24T15:32:43.586 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0068017260000000001, 2024-04-24T15:32:43.586 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:43.586 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:43.586 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:43.586 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:43.586 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:43.587 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 3773 2024-04-24T15:32:43.587 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.587 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:43.587 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.587 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:43.587 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699632+0000", 2024-04-24T15:32:43.587 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:43.587 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.587 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.587 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:43.587 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699632+0000", 2024-04-24T15:32:43.587 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:43.587 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.587 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.588 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:43.588 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699631+0000", 2024-04-24T15:32:43.588 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:43.588 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.588 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.588 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:43.588 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699639+0000", 2024-04-24T15:32:43.588 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7319999999999998e-06 2024-04-24T15:32:43.588 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.588 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.588 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:43.588 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699646+0000", 2024-04-24T15:32:43.588 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1330000000000001e-06 2024-04-24T15:32:43.588 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.589 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.589 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:43.589 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699651+0000", 2024-04-24T15:32:43.589 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.9200000000000003e-06 2024-04-24T15:32:43.589 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.589 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.589 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:43.589 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699689+0000", 2024-04-24T15:32:43.589 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.7979999999999999e-05 2024-04-24T15:32:43.589 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.589 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.589 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:43.589 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699759+0000", 2024-04-24T15:32:43.589 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.9468000000000006e-05 2024-04-24T15:32:43.590 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.590 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.590 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 6", 2024-04-24T15:32:43.590 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.701164+0000", 2024-04-24T15:32:43.590 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0014056629999999999 2024-04-24T15:32:43.590 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.590 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.590 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:43.590 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.705975+0000", 2024-04-24T15:32:43.590 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0048109340000000002 2024-04-24T15:32:43.590 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.590 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.590 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:43.591 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.706359+0000", 2024-04-24T15:32:43.591 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00038331699999999999 2024-04-24T15:32:43.591 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.591 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.591 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:43.591 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.706407+0000", 2024-04-24T15:32:43.591 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8544999999999997e-05 2024-04-24T15:32:43.591 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.591 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.591 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:43.591 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.706434+0000", 2024-04-24T15:32:43.591 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.6553000000000001e-05 2024-04-24T15:32:43.591 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:43.591 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:43.592 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:43.592 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.592 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.592 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:3794 4.18 4:1efc862d:::rbd_data.114a1f60cb66.0000000000000024:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:43.592 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.723446+0000", 2024-04-24T15:32:43.592 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 18.821687240999999, 2024-04-24T15:32:43.592 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.007572957, 2024-04-24T15:32:43.592 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:43.592 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:43.592 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:43.592 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:43.592 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:43.592 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 3794 2024-04-24T15:32:43.592 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.592 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:43.593 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.593 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:43.593 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723446+0000", 2024-04-24T15:32:43.593 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:43.593 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.593 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.593 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:43.593 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723446+0000", 2024-04-24T15:32:43.593 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:43.593 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.593 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.593 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:43.593 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723446+0000", 2024-04-24T15:32:43.593 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999999 2024-04-24T15:32:43.593 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.594 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.594 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:43.594 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723450+0000", 2024-04-24T15:32:43.594 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.1729999999999997e-06 2024-04-24T15:32:43.594 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.594 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.594 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:43.594 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723458+0000", 2024-04-24T15:32:43.594 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.653e-06 2024-04-24T15:32:43.594 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.594 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.594 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:43.595 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723465+0000", 2024-04-24T15:32:43.595 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.4450000000000001e-06 2024-04-24T15:32:43.595 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.595 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.595 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:43.595 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723501+0000", 2024-04-24T15:32:43.595 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5726000000000003e-05 2024-04-24T15:32:43.595 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.595 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.595 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:43.595 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723547+0000", 2024-04-24T15:32:43.595 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6479e-05 2024-04-24T15:32:43.595 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.596 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.596 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 6", 2024-04-24T15:32:43.596 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.724384+0000", 2024-04-24T15:32:43.596 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00083723800000000002 2024-04-24T15:32:43.596 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.596 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.596 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:43.596 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.730925+0000", 2024-04-24T15:32:43.596 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0065409179999999997 2024-04-24T15:32:43.596 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.596 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.596 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:43.596 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.730969+0000", 2024-04-24T15:32:43.596 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3869999999999998e-05 2024-04-24T15:32:43.597 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.597 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.597 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:43.597 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.730997+0000", 2024-04-24T15:32:43.597 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7783000000000001e-05 2024-04-24T15:32:43.597 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.597 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.597 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:43.597 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.731019+0000", 2024-04-24T15:32:43.597 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.2427999999999999e-05 2024-04-24T15:32:43.597 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:43.597 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:43.598 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:43.598 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.598 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.598 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:3797 4.18 4:1d8bdb81:::rbd_data.114a1f60cb66.0000000000000027:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:43.598 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.723479+0000", 2024-04-24T15:32:43.598 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 18.821654231, 2024-04-24T15:32:43.598 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.013378915, 2024-04-24T15:32:43.598 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:43.598 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:43.598 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:43.598 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:43.598 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:43.598 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 3797 2024-04-24T15:32:43.598 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.599 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:43.599 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.599 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:43.599 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723479+0000", 2024-04-24T15:32:43.599 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:43.599 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.599 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.599 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:43.599 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723479+0000", 2024-04-24T15:32:43.599 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:43.599 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.599 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.599 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:43.599 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723479+0000", 2024-04-24T15:32:43.600 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:43.600 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.600 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.600 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:43.600 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723482+0000", 2024-04-24T15:32:43.600 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5410000000000001e-06 2024-04-24T15:32:43.600 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.600 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.600 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:43.600 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723492+0000", 2024-04-24T15:32:43.600 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.6879999999999994e-06 2024-04-24T15:32:43.600 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.600 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.600 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:43.601 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723496+0000", 2024-04-24T15:32:43.601 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.34e-06 2024-04-24T15:32:43.601 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.601 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.601 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:43.601 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.728008+0000", 2024-04-24T15:32:43.601 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0045115049999999999 2024-04-24T15:32:43.601 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.601 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.601 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:43.601 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.728058+0000", 2024-04-24T15:32:43.601 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.0201000000000003e-05 2024-04-24T15:32:43.601 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.601 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.602 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 6", 2024-04-24T15:32:43.602 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.728499+0000", 2024-04-24T15:32:43.602 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00044129299999999999 2024-04-24T15:32:43.602 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.602 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.602 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:43.602 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.730962+0000", 2024-04-24T15:32:43.602 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0024628430000000002 2024-04-24T15:32:43.602 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.602 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.602 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:43.602 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.736810+0000", 2024-04-24T15:32:43.602 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0058474490000000002 2024-04-24T15:32:43.603 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.603 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.603 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:43.603 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.736837+0000", 2024-04-24T15:32:43.603 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7251999999999999e-05 2024-04-24T15:32:43.603 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.603 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.603 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:43.603 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.736858+0000", 2024-04-24T15:32:43.603 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.1387999999999999e-05 2024-04-24T15:32:43.603 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:43.603 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:43.603 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:43.603 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.603 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.604 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:3798 4.18 4:1e938922:::rbd_data.114a1f60cb66.0000000000000028:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:43.604 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.723506+0000", 2024-04-24T15:32:43.604 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 18.821627817, 2024-04-24T15:32:43.604 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.013398188, 2024-04-24T15:32:43.604 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:43.604 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:43.604 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:43.604 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:43.604 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:43.604 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 3798 2024-04-24T15:32:43.604 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.604 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:43.605 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.605 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:43.605 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723506+0000", 2024-04-24T15:32:43.605 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:43.605 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.605 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.605 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:43.605 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723506+0000", 2024-04-24T15:32:43.605 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:43.605 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.605 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.605 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:43.605 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723505+0000", 2024-04-24T15:32:43.605 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:43.605 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.606 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.606 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:43.606 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723509+0000", 2024-04-24T15:32:43.606 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.2349999999999999e-06 2024-04-24T15:32:43.606 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.606 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.606 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:43.606 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723515+0000", 2024-04-24T15:32:43.606 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.3239999999999999e-06 2024-04-24T15:32:43.606 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.606 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.606 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:43.606 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723519+0000", 2024-04-24T15:32:43.606 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8580000000000002e-06 2024-04-24T15:32:43.606 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.607 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.607 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:43.607 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.731045+0000", 2024-04-24T15:32:43.607 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0075259309999999996 2024-04-24T15:32:43.607 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.607 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.607 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:43.607 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.731090+0000", 2024-04-24T15:32:43.607 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4805999999999998e-05 2024-04-24T15:32:43.607 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.607 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.607 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 6", 2024-04-24T15:32:43.607 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.731571+0000", 2024-04-24T15:32:43.607 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00048136400000000002 2024-04-24T15:32:43.607 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.607 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.608 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:43.608 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.736784+0000", 2024-04-24T15:32:43.608 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0052135269999999999 2024-04-24T15:32:43.608 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.608 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.608 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:43.608 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.736865+0000", 2024-04-24T15:32:43.608 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0141000000000004e-05 2024-04-24T15:32:43.608 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.608 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.608 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:43.608 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.736887+0000", 2024-04-24T15:32:43.608 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.2194000000000001e-05 2024-04-24T15:32:43.608 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.608 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.609 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:43.609 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.736904+0000", 2024-04-24T15:32:43.609 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.7302000000000001e-05 2024-04-24T15:32:43.609 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:43.609 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:43.609 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:43.609 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.609 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.609 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:3801 4.18 4:1b4f38fa:::rbd_data.114a1f60cb66.000000000000002b:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:43.609 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.723629+0000", 2024-04-24T15:32:43.609 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 18.821504673, 2024-04-24T15:32:43.609 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.013474534, 2024-04-24T15:32:43.609 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:43.609 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:43.609 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:43.610 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:43.610 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:43.610 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 3801 2024-04-24T15:32:43.610 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.610 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:43.610 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.610 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:43.610 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723629+0000", 2024-04-24T15:32:43.610 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:43.610 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.610 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.611 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:43.611 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723629+0000", 2024-04-24T15:32:43.611 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:43.611 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.611 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.611 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:43.611 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723628+0000", 2024-04-24T15:32:43.611 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:43.611 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.611 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.611 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:43.611 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723632+0000", 2024-04-24T15:32:43.611 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.4690000000000002e-06 2024-04-24T15:32:43.612 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.612 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.612 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:43.612 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723640+0000", 2024-04-24T15:32:43.612 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3990000000000004e-06 2024-04-24T15:32:43.612 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.612 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.612 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:43.612 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723645+0000", 2024-04-24T15:32:43.612 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3699999999999997e-06 2024-04-24T15:32:43.612 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.612 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.612 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:43.612 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.733996+0000", 2024-04-24T15:32:43.613 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.010351111 2024-04-24T15:32:43.613 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.613 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.613 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:43.613 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.734045+0000", 2024-04-24T15:32:43.613 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.9166000000000003e-05 2024-04-24T15:32:43.613 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.613 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.613 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 6", 2024-04-24T15:32:43.613 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.734461+0000", 2024-04-24T15:32:43.613 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00041584 2024-04-24T15:32:43.613 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.613 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.613 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:43.614 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.736806+0000", 2024-04-24T15:32:43.614 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.002345232 2024-04-24T15:32:43.614 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.614 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.614 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:43.614 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.737065+0000", 2024-04-24T15:32:43.614 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00025913100000000001 2024-04-24T15:32:43.614 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.614 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.614 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:43.614 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.737086+0000", 2024-04-24T15:32:43.614 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.1180000000000001e-05 2024-04-24T15:32:43.614 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.615 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.615 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:43.615 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.737104+0000", 2024-04-24T15:32:43.615 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.7238999999999999e-05 2024-04-24T15:32:43.615 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:43.615 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:43.615 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:43.615 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.615 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.615 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:3896 4.1b 4:dd750b71:::rbd_data.114a1f60cb66.0000000000000058:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:43.615 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.797438+0000", 2024-04-24T15:32:43.615 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 18.747695448000002, 2024-04-24T15:32:43.615 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0070008420000000002, 2024-04-24T15:32:43.615 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:43.616 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:43.616 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:43.616 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:43.616 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:43.616 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 3896 2024-04-24T15:32:43.616 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.616 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:43.616 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.616 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:43.616 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797438+0000", 2024-04-24T15:32:43.616 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:43.616 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.616 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.616 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:43.617 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797438+0000", 2024-04-24T15:32:43.617 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:43.617 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.617 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.617 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:43.617 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797438+0000", 2024-04-24T15:32:43.617 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:43.617 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.617 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.617 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:43.617 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797441+0000", 2024-04-24T15:32:43.617 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8779999999999998e-06 2024-04-24T15:32:43.617 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.617 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.618 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:43.618 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797447+0000", 2024-04-24T15:32:43.618 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.6989999999999997e-06 2024-04-24T15:32:43.618 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.618 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.618 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:43.618 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797451+0000", 2024-04-24T15:32:43.618 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9689999999999996e-06 2024-04-24T15:32:43.618 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.618 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.618 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:43.618 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797915+0000", 2024-04-24T15:32:43.618 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00046327400000000001 2024-04-24T15:32:43.619 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.619 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.619 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:43.619 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797993+0000", 2024-04-24T15:32:43.619 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.8584000000000005e-05 2024-04-24T15:32:43.619 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.619 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.619 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:43.619 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.798090+0000", 2024-04-24T15:32:43.619 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.6638000000000005e-05 2024-04-24T15:32:43.619 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.619 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.619 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:43.620 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.799021+0000", 2024-04-24T15:32:43.620 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00093112299999999996 2024-04-24T15:32:43.620 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.620 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.620 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:43.620 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.804372+0000", 2024-04-24T15:32:43.620 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0053506789999999997 2024-04-24T15:32:43.620 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.620 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.620 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:43.620 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.804405+0000", 2024-04-24T15:32:43.620 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.3015999999999997e-05 2024-04-24T15:32:43.620 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.620 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.620 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:43.621 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.804439+0000", 2024-04-24T15:32:43.621 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.4431000000000003e-05 2024-04-24T15:32:43.621 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:43.621 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:43.621 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:43.621 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.621 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.621 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:3900 4.1a 4:5b94e8b4:::rbd_data.114a1f60cb66.000000000000006e:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:43.621 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.797456+0000", 2024-04-24T15:32:43.621 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 18.747677748000001, 2024-04-24T15:32:43.621 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.006973524, 2024-04-24T15:32:43.621 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:43.621 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:43.621 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:43.621 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:43.621 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:43.622 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 3900 2024-04-24T15:32:43.622 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.622 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:43.622 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.622 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:43.622 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797456+0000", 2024-04-24T15:32:43.622 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:43.622 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.622 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.622 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:43.622 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797456+0000", 2024-04-24T15:32:43.622 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:43.622 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.622 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.623 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:43.623 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797456+0000", 2024-04-24T15:32:43.623 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:43.623 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.623 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.623 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:43.623 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797458+0000", 2024-04-24T15:32:43.623 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.3209999999999999e-06 2024-04-24T15:32:43.623 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.623 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.623 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:43.623 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797464+0000", 2024-04-24T15:32:43.623 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.7579999999999998e-06 2024-04-24T15:32:43.623 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.623 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.624 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:43.624 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797467+0000", 2024-04-24T15:32:43.624 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.585e-06 2024-04-24T15:32:43.624 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.624 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.624 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:43.624 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797504+0000", 2024-04-24T15:32:43.624 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.7120999999999999e-05 2024-04-24T15:32:43.624 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.624 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.624 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:43.624 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797578+0000", 2024-04-24T15:32:43.624 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.3693000000000001e-05 2024-04-24T15:32:43.624 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.624 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.624 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:43.625 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797675+0000", 2024-04-24T15:32:43.625 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.7454000000000006e-05 2024-04-24T15:32:43.625 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.625 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.625 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:43.625 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.799033+0000", 2024-04-24T15:32:43.625 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.001357104 2024-04-24T15:32:43.625 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.625 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.625 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:43.625 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.804357+0000", 2024-04-24T15:32:43.625 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0053247490000000001 2024-04-24T15:32:43.625 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.625 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.625 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:43.626 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.804397+0000", 2024-04-24T15:32:43.626 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9573000000000002e-05 2024-04-24T15:32:43.626 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.626 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.626 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:43.626 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.804429+0000", 2024-04-24T15:32:43.626 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.2529e-05 2024-04-24T15:32:43.626 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:43.626 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:43.626 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:43.626 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.626 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.626 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:3923 4.9 4:94be6b6c:::rbd_data.114a1f60cb66.0000000000000097:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:43.626 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.806292+0000", 2024-04-24T15:32:43.627 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 18.738841885999999, 2024-04-24T15:32:43.627 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0062496729999999999, 2024-04-24T15:32:43.627 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:43.627 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:43.627 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:43.627 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:43.627 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:43.627 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 3923 2024-04-24T15:32:43.627 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.627 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:43.627 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.627 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:43.627 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.806292+0000", 2024-04-24T15:32:43.627 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:43.628 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.628 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.628 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:43.628 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.806292+0000", 2024-04-24T15:32:43.628 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:43.628 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.628 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.628 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:43.628 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.806291+0000", 2024-04-24T15:32:43.628 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999999 2024-04-24T15:32:43.628 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.628 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.629 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:43.629 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.806296+0000", 2024-04-24T15:32:43.629 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5730000000000001e-06 2024-04-24T15:32:43.629 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.629 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.629 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:43.629 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.806306+0000", 2024-04-24T15:32:43.629 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0409e-05 2024-04-24T15:32:43.629 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.629 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.629 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:43.629 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.806311+0000", 2024-04-24T15:32:43.629 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.7129999999999996e-06 2024-04-24T15:32:43.629 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.630 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.630 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:43.630 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.806350+0000", 2024-04-24T15:32:43.630 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9351999999999999e-05 2024-04-24T15:32:43.630 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.630 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.630 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:43.630 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.806431+0000", 2024-04-24T15:32:43.630 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.1050000000000005e-05 2024-04-24T15:32:43.630 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.630 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.630 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:43.630 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.806539+0000", 2024-04-24T15:32:43.630 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000107723 2024-04-24T15:32:43.630 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.631 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.631 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:43.631 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.809607+0000", 2024-04-24T15:32:43.631 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.003068468 2024-04-24T15:32:43.631 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.631 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.631 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:43.631 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.812493+0000", 2024-04-24T15:32:43.631 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0028862689999999999 2024-04-24T15:32:43.631 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.631 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.631 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:43.631 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.812521+0000", 2024-04-24T15:32:43.631 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7226e-05 2024-04-24T15:32:43.631 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.631 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.632 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:43.632 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.812541+0000", 2024-04-24T15:32:43.632 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.0729000000000001e-05 2024-04-24T15:32:43.632 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:43.632 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:43.632 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:43.632 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.632 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.632 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:3975 4.1b 4:d9227084:::rbd_data.114a1f60cb66.000000000000015f:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:43.632 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.827427+0000", 2024-04-24T15:32:43.632 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 18.717706442000001, 2024-04-24T15:32:43.632 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0061324140000000001, 2024-04-24T15:32:43.632 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:43.632 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:43.633 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:43.633 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:43.633 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:43.633 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 3975 2024-04-24T15:32:43.633 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.633 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:43.633 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.633 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:43.633 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.827427+0000", 2024-04-24T15:32:43.633 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:43.633 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.633 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.633 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:43.633 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.827427+0000", 2024-04-24T15:32:43.633 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:43.633 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.634 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.634 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:43.634 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.827427+0000", 2024-04-24T15:32:43.634 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:43.634 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.634 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.634 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:43.634 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.827431+0000", 2024-04-24T15:32:43.634 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2509999999999996e-06 2024-04-24T15:32:43.634 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.634 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.634 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:43.634 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.827440+0000", 2024-04-24T15:32:43.635 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.816e-06 2024-04-24T15:32:43.635 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.635 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.635 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:43.635 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.827445+0000", 2024-04-24T15:32:43.635 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.2059999999999998e-06 2024-04-24T15:32:43.635 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.635 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.635 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:43.635 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.827872+0000", 2024-04-24T15:32:43.636 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00042740299999999997 2024-04-24T15:32:43.636 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.636 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.636 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:43.636 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.827941+0000", 2024-04-24T15:32:43.636 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.8652000000000006e-05 2024-04-24T15:32:43.636 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.636 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.636 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:43.636 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.828048+0000", 2024-04-24T15:32:43.636 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000106941 2024-04-24T15:32:43.636 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.636 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.637 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:43.637 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.831351+0000", 2024-04-24T15:32:43.637 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0033031010000000001 2024-04-24T15:32:43.637 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.637 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.637 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:43.637 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.833495+0000", 2024-04-24T15:32:43.637 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0021436379999999998 2024-04-24T15:32:43.637 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.637 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.637 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:43.637 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.833526+0000", 2024-04-24T15:32:43.637 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.1436999999999997e-05 2024-04-24T15:32:43.637 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.638 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.638 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:43.638 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.833560+0000", 2024-04-24T15:32:43.638 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.3560999999999999e-05 2024-04-24T15:32:43.638 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:43.638 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:43.638 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:43.638 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.638 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.638 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:4010 4.1 4:813d513a:::rbd_data.114a1f60cb66.00000000000001c1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:43.638 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.845178+0000", 2024-04-24T15:32:43.638 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 18.699956094000001, 2024-04-24T15:32:43.638 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.007702108, 2024-04-24T15:32:43.639 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:43.639 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:43.639 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:43.639 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:43.639 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:43.639 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 4010 2024-04-24T15:32:43.639 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.639 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:43.639 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.639 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:43.639 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.845178+0000", 2024-04-24T15:32:43.639 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:43.639 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.639 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.639 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:43.639 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.845178+0000", 2024-04-24T15:32:43.640 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:43.640 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.640 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.640 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:43.640 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.845177+0000", 2024-04-24T15:32:43.640 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:43.640 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.640 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.640 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:43.640 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.845180+0000", 2024-04-24T15:32:43.640 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8119999999999999e-06 2024-04-24T15:32:43.640 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.640 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.640 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:43.640 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.845186+0000", 2024-04-24T15:32:43.641 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.9649999999999996e-06 2024-04-24T15:32:43.641 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.641 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.641 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:43.641 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.845189+0000", 2024-04-24T15:32:43.641 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.6040000000000001e-06 2024-04-24T15:32:43.641 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.641 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.641 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:43.641 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.845220+0000", 2024-04-24T15:32:43.641 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.0207000000000001e-05 2024-04-24T15:32:43.641 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.642 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.642 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:43.642 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.845262+0000", 2024-04-24T15:32:43.642 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2716000000000002e-05 2024-04-24T15:32:43.642 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.642 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.642 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 5", 2024-04-24T15:32:43.642 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.845326+0000", 2024-04-24T15:32:43.642 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.3490000000000004e-05 2024-04-24T15:32:43.642 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.642 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.642 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:43.643 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.849620+0000", 2024-04-24T15:32:43.643 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0042944769999999997 2024-04-24T15:32:43.643 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.643 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.643 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:43.643 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.852802+0000", 2024-04-24T15:32:43.643 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.003181798 2024-04-24T15:32:43.643 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.643 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.643 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:43.643 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.852846+0000", 2024-04-24T15:32:43.643 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3606000000000002e-05 2024-04-24T15:32:43.643 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.643 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.643 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:43.644 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.852880+0000", 2024-04-24T15:32:43.644 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.3908999999999997e-05 2024-04-24T15:32:43.644 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:43.644 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:43.644 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:43.644 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.644 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.644 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:4256 4.13 4:c8361c81:::rbd_data.114a1f60cb66.0000000000000316:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:43.644 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:25.054726+0000", 2024-04-24T15:32:43.644 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 18.490407650000002, 2024-04-24T15:32:43.644 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0079439720000000005, 2024-04-24T15:32:43.644 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:43.644 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:43.644 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:43.644 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:43.644 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:43.645 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 4256 2024-04-24T15:32:43.645 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.645 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:43.645 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.645 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:43.645 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.054726+0000", 2024-04-24T15:32:43.645 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:43.645 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.645 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.645 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:43.645 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.054726+0000", 2024-04-24T15:32:43.645 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:43.645 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.645 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.646 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:43.646 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.054725+0000", 2024-04-24T15:32:43.646 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999999 2024-04-24T15:32:43.646 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.646 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.646 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:43.646 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.054730+0000", 2024-04-24T15:32:43.646 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.4729999999999999e-06 2024-04-24T15:32:43.646 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.646 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.646 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:43.646 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.054742+0000", 2024-04-24T15:32:43.647 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.1489e-05 2024-04-24T15:32:43.647 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.647 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.647 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:43.647 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.054748+0000", 2024-04-24T15:32:43.647 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.879e-06 2024-04-24T15:32:43.647 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.647 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.647 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:43.647 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.054791+0000", 2024-04-24T15:32:43.647 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2673999999999998e-05 2024-04-24T15:32:43.647 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.647 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.647 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:43.647 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.054872+0000", 2024-04-24T15:32:43.648 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.1254999999999994e-05 2024-04-24T15:32:43.648 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.648 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.648 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 6", 2024-04-24T15:32:43.648 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.054978+0000", 2024-04-24T15:32:43.648 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00010624 2024-04-24T15:32:43.648 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.648 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.648 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:43.648 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.055893+0000", 2024-04-24T15:32:43.648 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00091453300000000003 2024-04-24T15:32:43.648 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.648 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.648 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:43.649 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.062620+0000", 2024-04-24T15:32:43.649 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0067271839999999998 2024-04-24T15:32:43.649 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.649 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.649 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:43.649 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.062645+0000", 2024-04-24T15:32:43.649 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.5491e-05 2024-04-24T15:32:43.649 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.649 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.649 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:43.649 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.062670+0000", 2024-04-24T15:32:43.649 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.474e-05 2024-04-24T15:32:43.649 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:43.649 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:43.649 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:43.650 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.650 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.650 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:7398 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-04-24T15:32:43.650 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:27.771937+0000", 2024-04-24T15:32:43.650 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 15.773196299, 2024-04-24T15:32:43.650 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.015783228, 2024-04-24T15:32:43.650 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:43.650 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:43.650 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:43.650 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:43.650 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:43.650 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 7398 2024-04-24T15:32:43.650 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.650 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:43.650 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.651 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:43.651 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:27.771937+0000", 2024-04-24T15:32:43.651 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:43.651 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.651 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.651 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:43.651 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:27.771937+0000", 2024-04-24T15:32:43.651 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:43.651 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.651 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.651 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:43.651 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:27.771936+0000", 2024-04-24T15:32:43.651 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999986 2024-04-24T15:32:43.651 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.652 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.652 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:43.652 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:27.771944+0000", 2024-04-24T15:32:43.652 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.8960000000000003e-06 2024-04-24T15:32:43.652 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.652 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.652 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:43.652 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:27.771955+0000", 2024-04-24T15:32:43.652 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0951000000000001e-05 2024-04-24T15:32:43.652 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.652 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.652 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:43.652 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:27.771959+0000", 2024-04-24T15:32:43.652 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.69e-06 2024-04-24T15:32:43.652 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.653 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.653 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:43.653 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:27.771998+0000", 2024-04-24T15:32:43.653 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8705000000000003e-05 2024-04-24T15:32:43.653 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.653 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.653 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:43.653 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:27.772054+0000", 2024-04-24T15:32:43.653 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.5420000000000001e-05 2024-04-24T15:32:43.653 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.653 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.653 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:43.653 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:27.774882+0000", 2024-04-24T15:32:43.653 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.002828344 2024-04-24T15:32:43.653 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.653 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.654 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:43.654 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:27.787163+0000", 2024-04-24T15:32:43.654 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.012281556000000001 2024-04-24T15:32:43.654 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.654 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.654 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:43.654 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:27.787651+0000", 2024-04-24T15:32:43.654 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00048708600000000002 2024-04-24T15:32:43.654 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.654 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.654 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:43.654 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:27.787691+0000", 2024-04-24T15:32:43.654 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0531999999999998e-05 2024-04-24T15:32:43.654 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.654 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.655 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:43.655 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:27.787721+0000", 2024-04-24T15:32:43.655 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.9518000000000001e-05 2024-04-24T15:32:43.655 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:43.655 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:43.655 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:43.655 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.655 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.655 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:7825 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-04-24T15:32:43.655 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:28.311523+0000", 2024-04-24T15:32:43.655 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 15.233610240000001, 2024-04-24T15:32:43.655 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.019381513, 2024-04-24T15:32:43.655 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:43.655 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:43.656 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:43.656 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:43.656 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:43.656 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 7825 2024-04-24T15:32:43.656 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.656 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:43.656 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.656 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:43.656 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:28.311523+0000", 2024-04-24T15:32:43.656 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:43.656 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.656 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.656 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:43.656 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:28.311523+0000", 2024-04-24T15:32:43.656 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:43.656 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.657 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.657 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:43.657 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:28.311523+0000", 2024-04-24T15:32:43.657 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:43.657 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.657 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.657 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:43.657 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:28.311527+0000", 2024-04-24T15:32:43.657 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6290000000000001e-06 2024-04-24T15:32:43.657 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.657 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.657 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:43.657 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:28.311538+0000", 2024-04-24T15:32:43.657 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0239e-05 2024-04-24T15:32:43.657 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.658 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.658 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:43.658 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:28.311542+0000", 2024-04-24T15:32:43.658 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3059999999999999e-06 2024-04-24T15:32:43.658 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.658 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.658 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:43.658 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:28.311621+0000", 2024-04-24T15:32:43.658 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.8671999999999995e-05 2024-04-24T15:32:43.658 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.658 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.658 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:43.658 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:28.311667+0000", 2024-04-24T15:32:43.658 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6842999999999999e-05 2024-04-24T15:32:43.658 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.658 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.659 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:43.659 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:28.315077+0000", 2024-04-24T15:32:43.659 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0034100889999999998 2024-04-24T15:32:43.659 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.659 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.659 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:43.659 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:28.330557+0000", 2024-04-24T15:32:43.659 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.015479100000000001 2024-04-24T15:32:43.659 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.659 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.659 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:43.659 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:28.330843+0000", 2024-04-24T15:32:43.659 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000286526 2024-04-24T15:32:43.659 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.659 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.660 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:43.660 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:28.330877+0000", 2024-04-24T15:32:43.660 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.4119999999999999e-05 2024-04-24T15:32:43.660 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:43.660 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:43.660 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:43.660 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:28.330905+0000", 2024-04-24T15:32:43.660 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7688999999999999e-05 2024-04-24T15:32:43.660 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:43.660 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:43.660 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:43.660 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:43.660 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:43.660 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:32:43.662 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-24T15:32:43.665 INFO:teuthology.orchestra.run.smithi096.stdout:1609 writesame 0xb073c0e thru 0xb080cfa data_size 0x13(0xd0ed bytes) 2024-04-24T15:32:43.665 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:43.665 INFO:teuthology.orchestra.run.smithi096.stdout:1611 read 0x772cda4 thru 0x77335fa (0x6857 bytes) 2024-04-24T15:32:43.665 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:43.665 INFO:teuthology.orchestra.run.smithi096.stdout:1615 read 0x2a9ede0 thru 0x2aa08e8 (0x1b09 bytes) 2024-04-24T15:32:43.665 INFO:teuthology.orchestra.run.smithi096.stdout:1616 punch from 0x37f14fb to 0x37f62fb, (0x4e00 bytes) 2024-04-24T15:32:43.665 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:43.665 INFO:teuthology.orchestra.run.smithi096.stdout:1618 punch from 0x6202cc6 to 0x620d412, (0xa74c bytes) 2024-04-24T15:32:43.665 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:43.665 INFO:teuthology.orchestra.run.smithi096.stdout:1620 write 0x95cbaba thru 0x95d506c (0x95b3 bytes) 2024-04-24T15:32:43.665 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:43.665 INFO:teuthology.orchestra.run.smithi096.stdout:1622 punch from 0x7872359 to 0x78808e4, (0xe58b bytes) 2024-04-24T15:32:43.666 INFO:teuthology.orchestra.run.smithi096.stdout:1623 write 0x7c6e245 thru 0x7c73494 (0x5250 bytes) 2024-04-24T15:32:43.666 INFO:teuthology.orchestra.run.smithi096.stdout:1624 write 0x52bb662 thru 0x52c3a64 (0x8403 bytes) 2024-04-24T15:32:43.666 INFO:teuthology.orchestra.run.smithi096.stdout:1625 writesame 0x3ffe8fd thru 0x40036aa data_size 0x2(0x4dae bytes) 2024-04-24T15:32:43.666 INFO:teuthology.orchestra.run.smithi096.stdout:1629 write 0x923aeca thru 0x924a24b (0xf382 bytes) 2024-04-24T15:32:43.666 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:43.666 INFO:teuthology.orchestra.run.smithi096.stdout:1631 write 0x5690404 thru 0x569b721 (0xb31e bytes) 2024-04-24T15:32:43.666 INFO:teuthology.orchestra.run.smithi096.stdout:1632 write 0x8d25aa2 thru 0x8d33803 (0xdd62 bytes) 2024-04-24T15:32:43.666 INFO:teuthology.orchestra.run.smithi096.stdout:1633 write 0xb3f9d63 thru 0xb3fc9cd (0x2c6b bytes) 2024-04-24T15:32:43.666 INFO:teuthology.orchestra.run.smithi096.stdout:1634 writesame 0x75862eb thru 0x7587a39 data_size 0x27(0x174f bytes) 2024-04-24T15:32:43.666 INFO:teuthology.orchestra.run.smithi096.stdout:1635 write 0x1f12292 thru 0x1f17e61 (0x5bd0 bytes) 2024-04-24T15:32:43.666 INFO:teuthology.orchestra.run.smithi096.stdout:1636 read 0x80e9d3a thru 0x80efcb5 (0x5f7c bytes) 2024-04-24T15:32:43.666 INFO:teuthology.orchestra.run.smithi096.stdout:1638 read 0x901ebb9 thru 0x902c12c (0xd574 bytes) 2024-04-24T15:32:43.666 INFO:teuthology.orchestra.run.smithi096.stdout:1639 read 0x7b0a85e thru 0x7b0ae18 (0x5bb bytes) 2024-04-24T15:32:43.666 INFO:teuthology.orchestra.run.smithi096.stdout:1640 write 0x785cab9 thru 0x7860e67 (0x43af bytes) 2024-04-24T15:32:43.666 INFO:teuthology.orchestra.run.smithi096.stdout:1642 write 0x5881886 thru 0x588b793 (0x9f0e bytes) 2024-04-24T15:32:43.667 INFO:teuthology.orchestra.run.smithi096.stdout:1644 write 0x5ae78d3 thru 0x5ae8295 (0x9c3 bytes) 2024-04-24T15:32:43.667 INFO:teuthology.orchestra.run.smithi096.stdout:1645 writesame 0x8a5e962 thru 0x8a5f203 data_size 0x11(0x8a2 bytes) 2024-04-24T15:32:43.667 INFO:teuthology.orchestra.run.smithi096.stdout:1647 writesame 0x8aeed18 thru 0x8af0d9e data_size 0x1(0x2087 bytes) 2024-04-24T15:32:43.667 INFO:teuthology.orchestra.run.smithi096.stdout:1648 read 0x3b4ad59 thru 0x3b57281 (0xc529 bytes) 2024-04-24T15:32:43.667 INFO:teuthology.orchestra.run.smithi096.stdout:1649 read 0x61c7923 thru 0x61c84c4 (0xba2 bytes) 2024-04-24T15:32:43.667 INFO:teuthology.orchestra.run.smithi096.stdout:1650 writesame 0x9ddc9d9 thru 0x9dec9a5 data_size 0x7(0xffcd bytes) 2024-04-24T15:32:43.667 INFO:teuthology.orchestra.run.smithi096.stdout:1651 writesame 0x1532025 thru 0x153b54d data_size 0x23(0x9529 bytes) 2024-04-24T15:32:43.667 INFO:teuthology.orchestra.run.smithi096.stdout:1652 write 0x294e9fc thru 0x295da63 (0xf068 bytes) 2024-04-24T15:32:43.667 INFO:teuthology.orchestra.run.smithi096.stdout:1653 trunc from 0xb3fc9ce to 0x2940620 2024-04-24T15:32:43.667 INFO:teuthology.orchestra.run.smithi096.stdout:1654 writesame 0x53adf8 thru 0x54a370 data_size 0x1(0xf579 bytes) 2024-04-24T15:32:43.667 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:32:43.662+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:32:43.681 INFO:teuthology.orchestra.run.smithi096.stdout:1656 write 0x487a69 thru 0x48ed18 (0x72b0 bytes) 2024-04-24T15:32:43.682 INFO:teuthology.orchestra.run.smithi096.stdout:1659 writesame 0x17e1211 thru 0x17e3f5f data_size 0x1(0x2d4f bytes) 2024-04-24T15:32:43.690 INFO:teuthology.orchestra.run.smithi096.stdout:1660 trunc from 0x2940620 to 0x43e3910 2024-04-24T15:32:43.701 INFO:teuthology.orchestra.run.smithi096.stdout:1661 writesame 0x5e87abb thru 0x5e88c33 data_size 0x15(0x1179 bytes) 2024-04-24T15:32:43.701 INFO:teuthology.orchestra.run.smithi096.stdout:1662 writesame 0x72a504 thru 0x72c8de data_size 0x43(0x23db bytes) 2024-04-24T15:32:43.702 INFO:teuthology.orchestra.run.smithi096.stdout:1663 punch from 0x329a372 to 0x329b598, (0x1226 bytes) 2024-04-24T15:32:43.702 INFO:teuthology.orchestra.run.smithi096.stdout:1664 write 0x25a18e7 thru 0x25aa1a2 (0x88bc bytes) 2024-04-24T15:32:43.704 INFO:teuthology.orchestra.run.smithi096.stdout:1665 writesame 0x376815 thru 0x378ae0 data_size 0x22(0x22cc bytes) 2024-04-24T15:32:43.741 INFO:teuthology.orchestra.run.smithi096.stdout:1666 writesame 0xbe0a56f thru 0xbe0f2ef data_size 0x1(0x4d81 bytes) 2024-04-24T15:32:43.751 INFO:teuthology.orchestra.run.smithi096.stdout:1667 trunc from 0xbe0f2f0 to 0x3b333d7 2024-04-24T15:32:43.761 INFO:teuthology.orchestra.run.smithi096.stdout:1668 write 0x40888ce thru 0x408a9e5 (0x2118 bytes) 2024-04-24T15:32:43.762 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:32:43.762+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:32:43.763 INFO:teuthology.orchestra.run.smithi096.stdout:1670 read 0x11b76f5 thru 0x11bd27f (0x5b8b bytes) 2024-04-24T15:32:43.764 INFO:teuthology.orchestra.run.smithi096.stdout:1671 read 0x5d7fce thru 0x5e777e (0xf7b1 bytes) 2024-04-24T15:32:43.781 INFO:teuthology.orchestra.run.smithi096.stdout:1676 writesame 0xd9b3acc thru 0xd9c0bb1 data_size 0x9(0xd0e6 bytes) 2024-04-24T15:32:43.793 INFO:teuthology.orchestra.run.smithi096.stdout:1678 trunc from 0xd9c0bb2 to 0x33ad315 2024-04-24T15:32:43.819 INFO:teuthology.orchestra.run.smithi096.stdout:1680 write 0xbd9eead thru 0xbdadd29 (0xee7d bytes) 2024-04-24T15:32:43.822 INFO:teuthology.orchestra.run.smithi096.stdout:1681 write 0x63e5f3d thru 0x63e8595 (0x2659 bytes) 2024-04-24T15:32:43.825 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:43.825 INFO:teuthology.orchestra.run.smithi096.stdout:1683 read 0x9b5f592 thru 0x9b6f1a8 (0xfc17 bytes) 2024-04-24T15:32:43.825 INFO:teuthology.orchestra.run.smithi096.stdout:1684 read 0x2527aae thru 0x25353fa (0xd94d bytes) 2024-04-24T15:32:43.830 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:32:43.830 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:32:43.830 INFO:teuthology.orchestra.run.smithi151.stdout: "num_ops": 0 2024-04-24T15:32:43.830 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:32:43.831 INFO:teuthology.orchestra.run.smithi096.stdout:1685 write 0xd5c1a10 thru 0xd5cd5ca (0xbbbb bytes) 2024-04-24T15:32:43.834 INFO:teuthology.orchestra.run.smithi096.stdout:1687 read 0x778078 thru 0x78720e (0xf197 bytes) 2024-04-24T15:32:43.834 INFO:teuthology.orchestra.run.smithi096.stdout:1688 write 0x7c9da2a thru 0x7caae6e (0xd445 bytes) 2024-04-24T15:32:43.837 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-24T15:32:43.838 INFO:teuthology.orchestra.run.smithi096.stdout:1689 write 0xe3c574 thru 0xe40794 (0x4221 bytes) 2024-04-24T15:32:43.840 INFO:teuthology.orchestra.run.smithi096.stdout:1691 punch from 0x77927d to 0x77ca3f, (0x37c2 bytes) 2024-04-24T15:32:43.840 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:43.840 INFO:teuthology.orchestra.run.smithi096.stdout:1693 read 0x451e88b thru 0x451ef1e (0x694 bytes) 2024-04-24T15:32:43.840 INFO:teuthology.orchestra.run.smithi096.stdout:1694 write 0xc639fab thru 0xc63f902 (0x5958 bytes) 2024-04-24T15:32:43.841 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-24T15:32:43.842 INFO:teuthology.orchestra.run.smithi096.stdout:1695 write 0x90ab0e1 thru 0x90b064a (0x556a bytes) 2024-04-24T15:32:43.844 INFO:teuthology.orchestra.run.smithi096.stdout:1697 punch from 0x8227af4 to 0x8234967, (0xce73 bytes) 2024-04-24T15:32:43.845 INFO:teuthology.orchestra.run.smithi096.stdout:1699 trunc from 0xd5cd5cb to 0xc8dde59 2024-04-24T15:32:43.870 INFO:teuthology.orchestra.run.smithi096.stdout:1700 trunc from 0xc8dde59 to 0xe89652 2024-04-24T15:32:43.884 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:32:43.885+0000 7f23abcf6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 44021) UID: 0 2024-04-24T15:32:43.896 INFO:teuthology.orchestra.run.smithi096.stdout:1701 punch from 0x8fdc2f to 0x90c5d9, (0xe9aa bytes) 2024-04-24T15:32:43.900 INFO:teuthology.orchestra.run.smithi096.stdout:1702 trunc from 0xe89652 to 0x40f9426 2024-04-24T15:32:43.906 INFO:teuthology.orchestra.run.smithi096.stdout:1703 read 0x278caaf thru 0x279257d (0x5acf bytes) 2024-04-24T15:32:43.906 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:43.906 INFO:teuthology.orchestra.run.smithi096.stdout:1709 read 0x1eca9fe thru 0x1ed6304 (0xb907 bytes) 2024-04-24T15:32:43.907 INFO:teuthology.orchestra.run.smithi096.stdout:1710 punch from 0x36b19f6 to 0x36b518a, (0x3794 bytes) 2024-04-24T15:32:43.907 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:43.907 INFO:teuthology.orchestra.run.smithi096.stdout:1712 read 0x296e54a thru 0x297c209 (0xdcc0 bytes) 2024-04-24T15:32:43.910 INFO:teuthology.orchestra.run.smithi096.stdout:1715 trunc from 0x40f9426 to 0x6cb1c8b 2024-04-24T15:32:43.916 INFO:teuthology.orchestra.run.smithi096.stdout:1717 read 0x1e047fc thru 0x1e0a04b (0x5850 bytes) 2024-04-24T15:32:43.917 INFO:teuthology.orchestra.run.smithi096.stdout:1718 punch from 0xf8049c to 0xf872d0, (0x6e34 bytes) 2024-04-24T15:32:43.917 INFO:teuthology.orchestra.run.smithi096.stdout:1720 read 0x5a4690a thru 0x5a4f018 (0x870f bytes) 2024-04-24T15:32:43.928 INFO:teuthology.orchestra.run.smithi096.stdout:1723 write 0xb9c240f thru 0xb9cec47 (0xc839 bytes) 2024-04-24T15:32:43.932 INFO:teuthology.orchestra.run.smithi096.stdout:1724 write 0x9788fa5 thru 0x9792944 (0x99a0 bytes) 2024-04-24T15:32:43.935 INFO:teuthology.orchestra.run.smithi096.stdout:1726 read 0x4e26c1d thru 0x4e272b5 (0x699 bytes) 2024-04-24T15:32:43.939 INFO:teuthology.orchestra.run.smithi096.stdout:1727 trunc from 0xb9cec48 to 0x8a6f231 2024-04-24T15:32:43.956 INFO:teuthology.orchestra.run.smithi096.stdout:1729 trunc from 0x8a6f231 to 0xb6e12d8 2024-04-24T15:32:43.963 INFO:teuthology.orchestra.run.smithi096.stdout:1731 punch from 0xb3060fb to 0xb306e41, (0xd46 bytes) 2024-04-24T15:32:43.963 INFO:teuthology.orchestra.run.smithi096.stdout:1733 read 0x6cbc3e2 thru 0x6cca850 (0xe46f bytes) 2024-04-24T15:32:43.963 INFO:teuthology.orchestra.run.smithi096.stdout:1734 punch from 0xb1d91b5 to 0xb1daa72, (0x18bd bytes) 2024-04-24T15:32:43.963 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:43.964 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:32:43.964+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:32:43.969 INFO:teuthology.orchestra.run.smithi096.stdout:1737 write 0xcffdf4a thru 0xd0061e4 (0x829b bytes) 2024-04-24T15:32:43.972 INFO:teuthology.orchestra.run.smithi096.stdout:1739 write 0x732c5ab thru 0x733792f (0xb385 bytes) 2024-04-24T15:32:43.975 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:32:43.975 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:32:43.975 INFO:teuthology.orchestra.run.smithi151.stdout: "complaint_time": 30, 2024-04-24T15:32:43.975 INFO:teuthology.orchestra.run.smithi151.stdout: "num_blocked_ops": 0 2024-04-24T15:32:43.975 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:32:43.975 INFO:teuthology.orchestra.run.smithi096.stdout:1743 read 0x12fe57c thru 0x1301e4e (0x38d3 bytes) 2024-04-24T15:32:43.984 INFO:teuthology.orchestra.run.smithi096.stdout:1748 trunc from 0xd0061e5 to 0x5e9da71 2024-04-24T15:32:43.986 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-24T15:32:43.992 INFO:teuthology.orchestra.run.smithi096.stdout:1750 write 0x105c3d thru 0x10731f (0x16e3 bytes) 2024-04-24T15:32:43.994 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:43.994 INFO:teuthology.orchestra.run.smithi096.stdout:1752 read 0x58b76c1 thru 0x58b7c89 (0x5c9 bytes) 2024-04-24T15:32:43.997 INFO:teuthology.orchestra.run.smithi096.stdout:1754 trunc from 0x5e9da71 to 0x32fb95e 2024-04-24T15:32:44.005 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-04-24T15:32:44.011 INFO:teuthology.orchestra.run.smithi096.stdout:1756 trunc from 0x32fb95e to 0xb1b642e 2024-04-24T15:32:44.021 INFO:teuthology.orchestra.run.smithi096.stdout:1757 trunc from 0xb1b642e to 0xe84f169 2024-04-24T15:32:44.025 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-04-24T15:32:44.025 INFO:teuthology.orchestra.run.smithi096.stdout:1759 write 0xb6b57c6 thru 0xb6bb32a (0x5b65 bytes) 2024-04-24T15:32:44.028 INFO:teuthology.orchestra.run.smithi096.stdout:1760 write 0xba25e17 thru 0xba33f4e (0xe138 bytes) 2024-04-24T15:32:44.031 INFO:teuthology.orchestra.run.smithi096.stdout:1761 punch from 0xfe5e3e to 0xfef0b5, (0x9277 bytes) 2024-04-24T15:32:44.031 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:44.031 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:44.031 INFO:teuthology.orchestra.run.smithi096.stdout:1765 write 0xd36317f thru 0xd36a474 (0x72f6 bytes) 2024-04-24T15:32:44.033 INFO:teuthology.orchestra.run.smithi096.stdout:1766 write 0x88909f3 thru 0x8890f18 (0x526 bytes) 2024-04-24T15:32:44.037 INFO:teuthology.orchestra.run.smithi096.stdout:1767 writesame 0x21a6787 thru 0x21a8033 data_size 0x1(0x18ad bytes) 2024-04-24T15:32:44.039 INFO:teuthology.orchestra.run.smithi096.stdout:1769 clone 8 order 22 su 32768 sc 15 2024-04-24T15:32:44.044 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-04-24T15:32:44.062 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-04-24T15:32:44.065 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:32:44.065+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:32:44.082 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-04-24T15:32:44.100 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-04-24T15:32:44.118 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-04-24T15:32:44.136 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-04-24T15:32:44.138 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T15:32:44.138 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T15:32:44.138 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T15:32:44.138 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T15:32:44.138 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T15:32:44.138 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T15:32:44.138 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T15:32:44.138 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T15:32:44.152 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:44.152 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:44.162 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-24T15:32:44.189 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:32:44.189+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:32:44.214 INFO:teuthology.orchestra.run.smithi096.stderr:nodeep-scrub is set 2024-04-24T15:32:44.263 INFO:teuthology.orchestra.run.smithi096.stdout:truncating image image_client.0-clone7 from 0xe84f169 (overlap 0xe89652) to 0x0 2024-04-24T15:32:44.267 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:32:44.267+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:32:44.292 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:44.292 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:44.302 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-24T15:32:44.307 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #6, image image_client.0-clone6 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent7 2024-04-24T15:32:44.369 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:32:44.369+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:32:44.427 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:44.427 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:44.437 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-24T15:32:44.470 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:32:44.470+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:32:44.524 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:44.524 INFO:teuthology.orchestra.run.smithi096.stdout:1772 write 0x81ec552 thru 0x81f250c (0x5fbb bytes) 2024-04-24T15:32:44.532 INFO:teuthology.orchestra.run.smithi096.stdout:1773 read 0x1f426f4 thru 0x1f451f7 (0x2b04 bytes) 2024-04-24T15:32:44.532 INFO:teuthology.orchestra.run.smithi096.stdout:1774 write 0xb1e9b9 thru 0xb2c34d (0xd995 bytes) 2024-04-24T15:32:44.549 INFO:teuthology.orchestra.run.smithi096.stdout:1776 read 0x13ef1c thru 0x145870 (0x6955 bytes) 2024-04-24T15:32:44.550 INFO:teuthology.orchestra.run.smithi096.stdout:1778 write 0x9ee2335 thru 0x9ee7de4 (0x5ab0 bytes) 2024-04-24T15:32:44.558 INFO:teuthology.orchestra.run.smithi096.stdout:1779 trunc from 0xe84f169 to 0xbeda034 2024-04-24T15:32:44.571 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:32:44.571+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:32:44.575 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:44.575 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:44.581 INFO:teuthology.orchestra.run.smithi096.stdout:1780 clone 9 order 18 su 16384 sc 3 2024-04-24T15:32:44.586 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-24T15:32:44.672 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:32:44.672+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:32:44.709 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:44.709 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:44.719 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-24T15:32:44.772 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:32:44.772+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:32:44.842 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:44.842 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:44.852 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-24T15:32:44.895 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:32:44.896+0000 7f23abcf6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 44021) UID: 0 2024-04-24T15:32:44.974 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:32:44.974+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:32:44.974 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:44.975 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:44.984 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-24T15:32:45.074 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:32:45.074+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:32:45.109 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:45.109 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:45.119 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-24T15:32:45.204 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:32:45.204+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:32:45.276 INFO:teuthology.orchestra.run.smithi096.stdout:truncating image image_client.0-clone8 from 0xbeda034 (overlap 0xbeda034) to 0x0 2024-04-24T15:32:45.276 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:32:45.276+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:32:45.302 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:45.302 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:45.321 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-24T15:32:45.398 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #7, image image_client.0-clone7 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent8 2024-04-24T15:32:45.399 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:32:45.399+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:32:45.399 INFO:teuthology.orchestra.run.smithi096.stdout:1782 write 0x9aa0350 thru 0x9aa53e7 (0x5098 bytes) 2024-04-24T15:32:45.407 INFO:teuthology.orchestra.run.smithi096.stdout:1784 punch from 0x7412aa1 to 0x7417d1d, (0x527c bytes) 2024-04-24T15:32:45.407 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:45.407 INFO:teuthology.orchestra.run.smithi096.stdout:1786 punch from 0x42e4075 to 0x42ee1a9, (0xa134 bytes) 2024-04-24T15:32:45.407 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:45.407 INFO:teuthology.orchestra.run.smithi096.stdout:1788 write 0x72c2564 thru 0x72c56cb (0x3168 bytes) 2024-04-24T15:32:45.412 INFO:teuthology.orchestra.run.smithi096.stdout:1789 read 0xa37452d thru 0xa3745ad (0x81 bytes) 2024-04-24T15:32:45.412 INFO:teuthology.orchestra.run.smithi096.stdout:1790 punch from 0x4e8d2bb to 0x4e9b7ea, (0xe52f bytes) 2024-04-24T15:32:45.412 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:45.412 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:45.412 INFO:teuthology.orchestra.run.smithi096.stdout:1793 write 0x5f75d82 thru 0x5f7d830 (0x7aaf bytes) 2024-04-24T15:32:45.422 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:45.422 INFO:teuthology.orchestra.run.smithi096.stdout:1798 read 0xb595614 thru 0xb59ba05 (0x63f2 bytes) 2024-04-24T15:32:45.434 INFO:teuthology.orchestra.run.smithi096.stdout:1800 trunc from 0xbeda034 to 0x1f773aa 2024-04-24T15:32:45.460 INFO:teuthology.orchestra.run.smithi096.stdout:1801 trunc from 0x1f773aa to 0x42a1392 2024-04-24T15:32:45.466 INFO:teuthology.orchestra.run.smithi096.stdout:1802 read 0x3565a8e thru 0x3571214 (0xb787 bytes) 2024-04-24T15:32:45.466 INFO:teuthology.orchestra.run.smithi096.stdout:1804 writesame 0x37e5591 thru 0x37f394e data_size 0x7b(0xe3be bytes) 2024-04-24T15:32:45.477 INFO:teuthology.orchestra.run.smithi096.stdout:1805 write 0x2e51186 thru 0x2e5b524 (0xa39f bytes) 2024-04-24T15:32:45.478 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:32:45.479+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:32:45.480 INFO:teuthology.orchestra.run.smithi096.stdout:1806 read 0x252c8a5 thru 0x25334c7 (0x6c23 bytes) 2024-04-24T15:32:45.480 INFO:teuthology.orchestra.run.smithi096.stdout:1808 write 0x41ed994 thru 0x41f9e5b (0xc4c8 bytes) 2024-04-24T15:32:45.489 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:45.498 INFO:teuthology.orchestra.run.smithi096.stdout:1811 trunc from 0x42a1392 to 0xc0b2fa7 2024-04-24T15:32:45.502 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:45.502 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:45.505 INFO:teuthology.orchestra.run.smithi096.stdout:1814 trunc from 0xc0b2fa7 to 0xa47b40d 2024-04-24T15:32:45.510 INFO:teuthology.orchestra.run.smithi096.stdout:1815 write 0x7a28548 thru 0x7a31a46 (0x94ff bytes) 2024-04-24T15:32:45.515 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-24T15:32:45.520 INFO:teuthology.orchestra.run.smithi096.stdout:1817 read 0x77d8367 thru 0x77e45a0 (0xc23a bytes) 2024-04-24T15:32:45.527 INFO:teuthology.orchestra.run.smithi096.stdout:1819 write 0xbc9bcb4 thru 0xbc9dfed (0x233a bytes) 2024-04-24T15:32:45.532 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:45.539 INFO:teuthology.orchestra.run.smithi096.stdout:1822 write 0xd4b1bd3 thru 0xd4b8d93 (0x71c1 bytes) 2024-04-24T15:32:45.549 INFO:teuthology.orchestra.run.smithi096.stdout:1825 trunc from 0xd4b8d94 to 0x8a8217a 2024-04-24T15:32:45.563 INFO:teuthology.orchestra.run.smithi096.stdout:1826 read 0x3405e2b thru 0x340b863 (0x5a39 bytes) 2024-04-24T15:32:45.563 INFO:teuthology.orchestra.run.smithi096.stdout:1827 read 0x3047223 thru 0x30555e8 (0xe3c6 bytes) 2024-04-24T15:32:45.569 INFO:teuthology.orchestra.run.smithi096.stdout:1828 trunc from 0x8a8217a to 0x399394b 2024-04-24T15:32:45.579 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:32:45.579+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:32:45.594 INFO:teuthology.orchestra.run.smithi096.stdout:1829 writesame 0xd793d4f thru 0xd7976a5 data_size 0x15(0x3957 bytes) 2024-04-24T15:32:45.594 INFO:teuthology.orchestra.run.smithi096.stdout:1831 writesame 0x78e9390 thru 0x78f0e38 data_size 0x9(0x7aa9 bytes) 2024-04-24T15:32:45.598 INFO:teuthology.orchestra.run.smithi096.stdout:1833 writesame 0x2f92835 thru 0x2f9b7cc data_size 0x14(0x8f98 bytes) 2024-04-24T15:32:45.602 INFO:teuthology.orchestra.run.smithi096.stdout:1834 read 0x30dffdc thru 0x30e5dae (0x5dd3 bytes) 2024-04-24T15:32:45.615 INFO:teuthology.orchestra.run.smithi096.stdout:1835 trunc from 0xd7976a6 to 0x23cf479 2024-04-24T15:32:45.631 INFO:teuthology.orchestra.run.smithi096.stdout:1836 writesame 0x278d03f thru 0x27929fa data_size 0x2(0x59bc bytes) 2024-04-24T15:32:45.631 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:45.637 INFO:teuthology.orchestra.run.smithi096.stdout:1839 trunc from 0x27929fb to 0x836d3c3 2024-04-24T15:32:45.642 INFO:teuthology.orchestra.run.smithi096.stdout:1840 read 0x724260a thru 0x7242fcf (0x9c6 bytes) 2024-04-24T15:32:45.642 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:45.647 INFO:teuthology.orchestra.run.smithi096.stdout:1843 trunc from 0x836d3c3 to 0xc6c2c86 2024-04-24T15:32:45.651 INFO:teuthology.orchestra.run.smithi096.stdout:1845 punch from 0x5fccd9d to 0x5fd4eeb, (0x814e bytes) 2024-04-24T15:32:45.653 INFO:teuthology.orchestra.run.smithi096.stdout:1847 trunc from 0xc6c2c86 to 0xad4d7b7 2024-04-24T15:32:45.658 INFO:teuthology.orchestra.run.smithi096.stdout:1848 read 0x8b2e4e thru 0x8bf2ad (0xc460 bytes) 2024-04-24T15:32:45.658 INFO:teuthology.orchestra.run.smithi096.stdout:1851 read 0x171c28d thru 0x1729683 (0xd3f7 bytes) 2024-04-24T15:32:45.659 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:45.659 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:45.659 INFO:teuthology.orchestra.run.smithi096.stdout:1854 read 0x8da86a3 thru 0x8db62cc (0xdc2a bytes) 2024-04-24T15:32:45.659 INFO:teuthology.orchestra.run.smithi096.stdout:1855 read 0x2f7578f thru 0x2f830c2 (0xd934 bytes) 2024-04-24T15:32:45.659 INFO:teuthology.orchestra.run.smithi096.stdout:1856 read 0x66613fb thru 0x6662f27 (0x1b2d bytes) 2024-04-24T15:32:45.662 INFO:teuthology.orchestra.run.smithi096.stdout:1858 trunc from 0xad4d7b7 to 0xd679f73 2024-04-24T15:32:45.666 INFO:teuthology.orchestra.run.smithi096.stdout:1860 read 0xc21831f thru 0xc22801e (0xfd00 bytes) 2024-04-24T15:32:45.666 INFO:teuthology.orchestra.run.smithi096.stdout:1861 write 0xbb80b34 thru 0xbb86c77 (0x6144 bytes) 2024-04-24T15:32:45.669 INFO:teuthology.orchestra.run.smithi096.stdout:1863 write 0x7550031 thru 0x755a624 (0xa5f4 bytes) 2024-04-24T15:32:45.674 INFO:teuthology.orchestra.run.smithi096.stdout:1864 punch from 0x422a300 to 0x422b1c4, (0xec4 bytes) 2024-04-24T15:32:45.674 INFO:teuthology.orchestra.run.smithi096.stdout:1865 read 0x7db183a thru 0x7db6498 (0x4c5f bytes) 2024-04-24T15:32:45.674 INFO:teuthology.orchestra.run.smithi096.stdout:1866 read 0x29e0d6e thru 0x29e6fed (0x6280 bytes) 2024-04-24T15:32:45.674 INFO:teuthology.orchestra.run.smithi096.stdout:1867 writesame 0xc8b9dbf thru 0xc8bfdc0 data_size 0x1(0x6002 bytes) 2024-04-24T15:32:45.676 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:45.676 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:45.682 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:32:45.682+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:32:45.686 INFO:teuthology.orchestra.run.smithi096.stdout:1868 write 0xbcfda7e thru 0xbd0b836 (0xddb9 bytes) 2024-04-24T15:32:45.688 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-24T15:32:45.695 INFO:teuthology.orchestra.run.smithi096.stdout:1869 punch from 0x4e3d12f to 0x4e3e2f5, (0x11c6 bytes) 2024-04-24T15:32:45.695 INFO:teuthology.orchestra.run.smithi096.stdout:1871 read 0x91d74ef thru 0x91e2edf (0xb9f1 bytes) 2024-04-24T15:32:45.702 INFO:teuthology.orchestra.run.smithi096.stdout:1872 writesame 0xda216e9 thru 0xda2766e data_size 0x1(0x5f86 bytes) 2024-04-24T15:32:45.702 INFO:teuthology.orchestra.run.smithi096.stdout:1873 read 0xa419841 thru 0xa41ee74 (0x5634 bytes) 2024-04-24T15:32:45.717 INFO:teuthology.orchestra.run.smithi096.stdout:1875 trunc from 0xda2766f to 0xfcae5f 2024-04-24T15:32:45.741 INFO:teuthology.orchestra.run.smithi096.stdout:1876 read 0x985e8b thru 0x98e487 (0x85fd bytes) 2024-04-24T15:32:45.742 INFO:teuthology.orchestra.run.smithi096.stdout:1877 punch from 0x9d9d24 to 0x9da5d7, (0x8b3 bytes) 2024-04-24T15:32:45.760 INFO:teuthology.orchestra.run.smithi096.stdout:1878 trunc from 0xfcae5f to 0xe4d3376 2024-04-24T15:32:45.766 INFO:teuthology.orchestra.run.smithi096.stdout:1882 read 0xd8ebc thru 0xe2f05 (0xa04a bytes) 2024-04-24T15:32:45.766 INFO:teuthology.orchestra.run.smithi096.stdout:1883 writesame 0xb5d91cb thru 0xb5e067e data_size 0x61(0x74b4 bytes) 2024-04-24T15:32:45.776 INFO:teuthology.orchestra.run.smithi096.stdout:1885 write 0x3e49318 thru 0x3e54bc3 (0xb8ac bytes) 2024-04-24T15:32:45.780 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:45.780 INFO:teuthology.orchestra.run.smithi096.stdout:1890 punch from 0x6b06499 to 0x6b15e5d, (0xf9c4 bytes) 2024-04-24T15:32:45.783 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:32:45.783+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:32:45.793 INFO:teuthology.orchestra.run.smithi096.stdout:1892 trunc from 0xe4d3376 to 0x363b085 2024-04-24T15:32:45.802 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:45.802 INFO:teuthology.orchestra.run.smithi096.stdout:1895 read 0xb529f6 thru 0xb5a78d (0x7d98 bytes) 2024-04-24T15:32:45.811 INFO:teuthology.orchestra.run.smithi096.stdout:1896 write 0x5e9f424 thru 0x5ea8e74 (0x9a51 bytes) 2024-04-24T15:32:45.825 INFO:teuthology.orchestra.run.smithi096.stdout:1897 writesame 0x8d85b8b thru 0x8d91e0c data_size 0x56(0xc282 bytes) 2024-04-24T15:32:45.825 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:45.832 INFO:teuthology.orchestra.run.smithi096.stdout:1901 write 0xb4a3977 thru 0xb4a7a73 (0x40fd bytes) 2024-04-24T15:32:45.843 INFO:teuthology.orchestra.run.smithi096.stdout:1908 write 0xd185b69 thru 0xd1954a3 (0xf93b bytes) 2024-04-24T15:32:45.848 INFO:teuthology.orchestra.run.smithi096.stdout:1909 read 0x9a9ecb2 thru 0x9aaceec (0xe23b bytes) 2024-04-24T15:32:45.849 INFO:teuthology.orchestra.run.smithi096.stdout:1911 trunc from 0xd1954a4 to 0xc68595e 2024-04-24T15:32:45.853 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:45.853 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:45.857 INFO:teuthology.orchestra.run.smithi096.stdout:1912 write 0x73c718e thru 0x73cbe44 (0x4cb7 bytes) 2024-04-24T15:32:45.861 INFO:teuthology.orchestra.run.smithi096.stdout:1913 writesame 0x3176534 thru 0x317bdac data_size 0x1d(0x5879 bytes) 2024-04-24T15:32:45.864 INFO:teuthology.orchestra.run.smithi096.stdout:1914 write 0x7b3f983 thru 0x7b4733f (0x79bd bytes) 2024-04-24T15:32:45.865 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-24T15:32:45.869 INFO:teuthology.orchestra.run.smithi096.stdout:1915 read 0x18505dc thru 0x1854642 (0x4067 bytes) 2024-04-24T15:32:45.869 INFO:teuthology.orchestra.run.smithi096.stdout:1917 read 0x5872eb0 thru 0x587caa8 (0x9bf9 bytes) 2024-04-24T15:32:45.869 INFO:teuthology.orchestra.run.smithi096.stdout:1918 punch from 0x217a9d4 to 0x2182130, (0x775c bytes) 2024-04-24T15:32:45.869 INFO:teuthology.orchestra.run.smithi096.stdout:1919 read 0xacca789 thru 0xaccbdc2 (0x163a bytes) 2024-04-24T15:32:45.869 INFO:teuthology.orchestra.run.smithi096.stdout:1920 read 0x230f582 thru 0x2310c88 (0x1707 bytes) 2024-04-24T15:32:45.869 INFO:teuthology.orchestra.run.smithi096.stdout:1921 write 0x9ec3fd5 thru 0x9ec9903 (0x592f bytes) 2024-04-24T15:32:45.884 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:32:45.885+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:32:45.889 INFO:teuthology.orchestra.run.smithi096.stdout:1922 trunc from 0xc68595e to 0x1088d37 2024-04-24T15:32:45.910 INFO:teuthology.orchestra.run.smithi096.stdout:1924 writesame 0x490f1b8 thru 0x4918f1d data_size 0x1(0x9d66 bytes) 2024-04-24T15:32:45.917 INFO:teuthology.orchestra.run.smithi096.stdout:1927 write 0x6adc9a1 thru 0x6aeb24b (0xe8ab bytes) 2024-04-24T15:32:45.925 INFO:teuthology.orchestra.run.smithi096.stdout:1928 trunc from 0x6aeb24c to 0x91e3ce7 2024-04-24T15:32:45.935 INFO:teuthology.orchestra.run.smithi096.stdout:1933 write 0x9f025ce thru 0x9f11627 (0xf05a bytes) 2024-04-24T15:32:45.939 INFO:teuthology.orchestra.run.smithi096.stdout:1935 trunc from 0x9f11628 to 0xa761d72 2024-04-24T15:32:45.944 INFO:teuthology.orchestra.run.smithi096.stdout:1936 punch from 0x48f7f8e to 0x48f978e, (0x1800 bytes) 2024-04-24T15:32:45.944 INFO:teuthology.orchestra.run.smithi096.stdout:1937 writesame 0x3273b0d thru 0x327deb0 data_size 0x6(0xa3a4 bytes) 2024-04-24T15:32:45.951 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:45.951 INFO:teuthology.orchestra.run.smithi096.stdout:1940 read 0x8591f5a thru 0x859ac9a (0x8d41 bytes) 2024-04-24T15:32:45.952 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:45.952 INFO:teuthology.orchestra.run.smithi096.stdout:1943 write 0x419f459 thru 0x41ab359 (0xbf01 bytes) 2024-04-24T15:32:45.960 INFO:teuthology.orchestra.run.smithi096.stdout:1944 writesame 0x4a67424 thru 0x4a699de data_size 0x1(0x25bb bytes) 2024-04-24T15:32:45.967 INFO:teuthology.orchestra.run.smithi096.stdout:1947 write 0x66c5fe6 thru 0x66c75f8 (0x1613 bytes) 2024-04-24T15:32:45.969 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:45.969 INFO:teuthology.orchestra.run.smithi096.stdout:1949 write 0x436f603 thru 0x437e610 (0xf00e bytes) 2024-04-24T15:32:45.978 INFO:teuthology.orchestra.run.smithi096.stdout:1954 read 0x8562dae thru 0x8572593 (0xf7e6 bytes) 2024-04-24T15:32:45.978 INFO:teuthology.orchestra.run.smithi096.stdout:1955 write 0x278aff7 thru 0x279163d (0x6647 bytes) 2024-04-24T15:32:45.984 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:32:45.985+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:32:45.995 INFO:teuthology.orchestra.run.smithi096.stdout:1956 write 0xcc5e378 thru 0xcc6cce3 (0xe96c bytes) 2024-04-24T15:32:46.012 INFO:teuthology.orchestra.run.smithi096.stdout:1957 trunc from 0xcc6cce4 to 0x2865099 2024-04-24T15:32:46.022 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:46.022 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:46.035 INFO:teuthology.orchestra.run.smithi096.stdout:1958 trunc from 0x2865099 to 0xc62fe58 2024-04-24T15:32:46.040 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-24T15:32:46.042 INFO:teuthology.orchestra.run.smithi096.stdout:1961 writesame 0x14f52b9 thru 0x14ff38a data_size 0x2e(0xa0d2 bytes) 2024-04-24T15:32:46.048 INFO:teuthology.orchestra.run.smithi096.stdout:1962 trunc from 0xc62fe58 to 0xd1e6751 2024-04-24T15:32:46.053 INFO:teuthology.orchestra.run.smithi096.stdout:1963 punch from 0xa96a7d6 to 0xa971672, (0x6e9c bytes) 2024-04-24T15:32:46.053 INFO:teuthology.orchestra.run.smithi096.stdout:1965 write 0x9199d2c thru 0x919b921 (0x1bf6 bytes) 2024-04-24T15:32:46.056 INFO:teuthology.orchestra.run.smithi096.stdout:1967 read 0x7862d15 thru 0x78725bb (0xf8a7 bytes) 2024-04-24T15:32:46.056 INFO:teuthology.orchestra.run.smithi096.stdout:1968 write 0x2198e4d thru 0x21a16a8 (0x885c bytes) 2024-04-24T15:32:46.060 INFO:teuthology.orchestra.run.smithi096.stdout:1969 write 0x391f083 thru 0x39210e2 (0x2060 bytes) 2024-04-24T15:32:46.068 INFO:teuthology.orchestra.run.smithi096.stdout:1971 write 0xd48415e thru 0xd493e05 (0xfca8 bytes) 2024-04-24T15:32:46.073 INFO:teuthology.orchestra.run.smithi096.stdout:1972 read 0x5e8811c thru 0x5e8a1ed (0x20d2 bytes) 2024-04-24T15:32:46.073 INFO:teuthology.orchestra.run.smithi096.stdout:1973 writesame 0x2ecbe2b thru 0x2ed70ab data_size 0x1(0xb281 bytes) 2024-04-24T15:32:46.086 INFO:teuthology.orchestra.run.smithi096.stdout:1974 write 0x5b77194 thru 0x5b7d544 (0x63b1 bytes) 2024-04-24T15:32:46.086 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:32:46.086+0000 7f23abcf6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 44021) UID: 0 2024-04-24T15:32:46.097 INFO:teuthology.orchestra.run.smithi096.stdout:1975 trunc from 0xd493e06 to 0x77a2962 2024-04-24T15:32:46.107 INFO:teuthology.orchestra.run.smithi096.stdout:1976 read 0x7224a1c thru 0x7229cc7 (0x52ac bytes) 2024-04-24T15:32:46.107 INFO:teuthology.orchestra.run.smithi096.stdout:1980 punch from 0x499e7fc to 0x49aae1e, (0xc622 bytes) 2024-04-24T15:32:46.113 INFO:teuthology.orchestra.run.smithi096.stdout:1981 trunc from 0x77a2962 to 0x2d4a075 2024-04-24T15:32:46.141 INFO:teuthology.orchestra.run.smithi096.stdout:1982 write 0xe6c8bd1 thru 0xe6cb0c1 (0x24f1 bytes) 2024-04-24T15:32:46.143 INFO:teuthology.orchestra.run.smithi096.stdout:1985 write 0xa8c59fa thru 0xa8d04b0 (0xaab7 bytes) 2024-04-24T15:32:46.148 INFO:teuthology.orchestra.run.smithi096.stdout:truncating to largest ever: 0xee00a52 2024-04-24T15:32:46.148 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-24T15:32:46.149 INFO:teuthology.orchestra.run.smithi096.stdout:1986 trunc from 0xe6cb0c2 to 0xee00a52 2024-04-24T15:32:46.153 INFO:teuthology.orchestra.run.smithi096.stdout:1988 writesame 0x85e43d7 thru 0x85e58fb data_size 0x1(0x1525 bytes) 2024-04-24T15:32:46.169 INFO:teuthology.orchestra.run.smithi096.stdout:1989 trunc from 0xee00a52 to 0x540a194 2024-04-24T15:32:46.187 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:32:46.187+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:32:46.190 INFO:teuthology.orchestra.run.smithi096.stdout:1990 write 0xa3d4c3f thru 0xa3da308 (0x56ca bytes) 2024-04-24T15:32:46.193 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:46.193 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:46.197 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:46.197 INFO:teuthology.orchestra.run.smithi096.stdout:1992 punch from 0x528f85b to 0x5299c60, (0xa405 bytes) 2024-04-24T15:32:46.206 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-24T15:32:46.207 INFO:teuthology.orchestra.run.smithi096.stdout:1993 write 0xdb44b28 thru 0xdb5445b (0xf934 bytes) 2024-04-24T15:32:46.215 INFO:teuthology.orchestra.run.smithi096.stdout:1997 write 0x825498c thru 0x8257322 (0x2997 bytes) 2024-04-24T15:32:46.217 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 6, 7] out_osds: [1, 4, 5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-04-24T15:32:46.217 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-24T15:32:46.217 INFO:tasks.thrashosds.thrasher:Removing osd 7, in_osds are: [0, 2, 3, 6, 7] 2024-04-24T15:32:46.217 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 7 2024-04-24T15:32:46.218 INFO:teuthology.orchestra.run.smithi096.stdout:1998 write 0x2b2cf2b thru 0x2b3c045 (0xf11b bytes) 2024-04-24T15:32:46.227 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-24T15:32:46.229 INFO:teuthology.orchestra.run.smithi096.stdout:2000 writesame 0x61c19d3 thru 0x61c28b5 data_size 0x1(0xee3 bytes) 2024-04-24T15:32:46.233 INFO:teuthology.orchestra.run.smithi096.stdout:2002 punch from 0x646ecf8 to 0x6476023, (0x732b bytes) 2024-04-24T15:32:46.233 INFO:teuthology.orchestra.run.smithi096.stdout:2003 writesame 0x61a9197 thru 0x61ad872 data_size 0xa(0x46dc bytes) 2024-04-24T15:32:46.237 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:46.237 INFO:teuthology.orchestra.run.smithi096.stdout:2005 writesame 0x7690298 thru 0x769491d data_size 0x66(0x4686 bytes) 2024-04-24T15:32:46.243 INFO:teuthology.orchestra.run.smithi096.stdout:2006 read 0x2a28e7f thru 0x2a338fa (0xaa7c bytes) 2024-04-24T15:32:46.243 INFO:teuthology.orchestra.run.smithi096.stdout:2008 write 0x80937c0 thru 0x809adaf (0x75f0 bytes) 2024-04-24T15:32:46.252 INFO:teuthology.orchestra.run.smithi096.stdout:2009 writesame 0x4a57b3c thru 0x4a5994c data_size 0x1(0x1e11 bytes) 2024-04-24T15:32:46.262 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:46.262 INFO:teuthology.orchestra.run.smithi096.stdout:2011 write 0xd973cc0 thru 0xd980edf (0xd220 bytes) 2024-04-24T15:32:46.284 INFO:teuthology.orchestra.run.smithi096.stdout:2012 write 0xdeba250 thru 0xdec9e4f (0xfc00 bytes) 2024-04-24T15:32:46.293 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:32:46.292+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:32:46.293 INFO:teuthology.orchestra.run.smithi096.stdout:2015 write 0xb9547cf thru 0xb96153d (0xcd6f bytes) 2024-04-24T15:32:46.311 INFO:teuthology.orchestra.run.smithi096.stdout:2016 trunc from 0xdec9e50 to 0x68a7fbc 2024-04-24T15:32:46.338 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-04-24T15:32:46.347 INFO:teuthology.orchestra.run.smithi096.stdout:2017 writesame 0x896a130 thru 0x897293a data_size 0x39(0x880b bytes) 2024-04-24T15:32:46.357 INFO:teuthology.orchestra.run.smithi096.stdout:2018 writesame 0xd189735 thru 0xd191a41 data_size 0x35(0x830d bytes) 2024-04-24T15:32:46.357 INFO:teuthology.orchestra.run.smithi096.stdout:2019 trunc from 0xd191a42 to 0xd2b1aa3 2024-04-24T15:32:46.361 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-04-24T15:32:46.362 INFO:teuthology.orchestra.run.smithi096.stdout:2021 writesame 0xc317c0 thru 0xc32aa0 data_size 0x9(0x12e1 bytes) 2024-04-24T15:32:46.376 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:46.376 INFO:teuthology.orchestra.run.smithi096.stdout:2023 trunc from 0xd2b1aa3 to 0xc777d8c 2024-04-24T15:32:46.383 INFO:teuthology.orchestra.run.smithi096.stdout:2024 read 0x1086a96 thru 0x1089aed (0x3058 bytes) 2024-04-24T15:32:46.383 INFO:teuthology.orchestra.run.smithi096.stdout:2025 punch from 0xafff7a5 to 0xb00a2b0, (0xab0b bytes) 2024-04-24T15:32:46.383 INFO:teuthology.orchestra.run.smithi096.stdout:2027 punch from 0x52a4cf8 to 0x52afb1e, (0xae26 bytes) 2024-04-24T15:32:46.385 INFO:teuthology.orchestra.run.smithi096.stdout:2028 trunc from 0xc777d8c to 0xe1b5a16 2024-04-24T15:32:46.389 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:32:46.389+0000 7f23abcf6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 44021) UID: 0 2024-04-24T15:32:46.389 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:46.389 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:46.396 INFO:teuthology.orchestra.run.smithi096.stdout:2029 write 0xd1adcd7 thru 0xd1b9a6c (0xbd96 bytes) 2024-04-24T15:32:46.397 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-04-24T15:32:46.405 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-24T15:32:46.405 INFO:teuthology.orchestra.run.smithi096.stdout:2030 writesame 0xcfe145e thru 0xcfed62b data_size 0x3(0xc1ce bytes) 2024-04-24T15:32:46.423 INFO:teuthology.orchestra.run.smithi096.stdout:2031 writesame 0x5c0d350 thru 0x5c0e2b9 data_size 0x1(0xf6a bytes) 2024-04-24T15:32:46.428 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-04-24T15:32:46.431 INFO:teuthology.orchestra.run.smithi096.stdout:2032 read 0x4404801 thru 0x4413e14 (0xf614 bytes) 2024-04-24T15:32:46.431 INFO:teuthology.orchestra.run.smithi096.stdout:2034 writesame 0xd7d5ec8 thru 0xd7e4490 data_size 0x41(0xe5c9 bytes) 2024-04-24T15:32:46.438 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:46.438 INFO:teuthology.orchestra.run.smithi096.stdout:2036 writesame 0x9570b9b thru 0x957e5cf data_size 0x1(0xda35 bytes) 2024-04-24T15:32:46.456 INFO:teuthology.orchestra.run.smithi096.stdout:2037 write 0x5f96000 thru 0x5fa5356 (0xf357 bytes) 2024-04-24T15:32:46.457 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-04-24T15:32:46.461 INFO:teuthology.orchestra.run.smithi096.stdout:2038 trunc from 0xe1b5a16 to 0xeae16c2 2024-04-24T15:32:46.466 INFO:teuthology.orchestra.run.smithi096.stdout:2039 read 0x6213de0 thru 0x6218256 (0x4477 bytes) 2024-04-24T15:32:46.467 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:46.467 INFO:teuthology.orchestra.run.smithi096.stdout:2041 write 0x3e4d567 thru 0x3e4e723 (0x11bd bytes) 2024-04-24T15:32:46.471 INFO:teuthology.orchestra.run.smithi096.stdout:2042 writesame 0xde132cb thru 0xde18df6 data_size 0x1e(0x5b2c bytes) 2024-04-24T15:32:46.481 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-04-24T15:32:46.481 INFO:teuthology.orchestra.run.smithi096.stdout:2043 punch from 0x507c8c0 to 0x5088b48, (0xc288 bytes) 2024-04-24T15:32:46.481 INFO:teuthology.orchestra.run.smithi096.stdout:2044 writesame 0xc7b59f3 thru 0xc7c27db data_size 0x3(0xcde9 bytes) 2024-04-24T15:32:46.489 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:32:46.490+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:32:46.493 INFO:teuthology.orchestra.run.smithi096.stdout:2045 writesame 0xc413c01 thru 0xc416bfc data_size 0x25(0x2ffc bytes) 2024-04-24T15:32:46.499 INFO:teuthology.orchestra.run.smithi096.stdout:2047 write 0xca3dada thru 0xca4c8f2 (0xee19 bytes) 2024-04-24T15:32:46.506 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-04-24T15:32:46.508 INFO:teuthology.orchestra.run.smithi096.stdout:2048 trunc from 0xeae16c2 to 0xc32f782 2024-04-24T15:32:46.530 INFO:teuthology.orchestra.run.smithi096.stdout:2049 trunc from 0xc32f782 to 0xab7cc0e 2024-04-24T15:32:46.537 INFO:teuthology.orchestra.run.smithi096.stdout:2050 read 0x3b9491b thru 0x3ba257b (0xdc61 bytes) 2024-04-24T15:32:46.537 INFO:teuthology.orchestra.run.smithi096.stdout:2051 read 0x348198a thru 0x3482329 (0x9a0 bytes) 2024-04-24T15:32:46.541 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-04-24T15:32:46.541 INFO:teuthology.orchestra.run.smithi096.stdout:2052 trunc from 0xab7cc0e to 0xdce2207 2024-04-24T15:32:46.543 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T15:32:46.543 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T15:32:46.543 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T15:32:46.543 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T15:32:46.543 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T15:32:46.543 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T15:32:46.544 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T15:32:46.544 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T15:32:46.549 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:46.549 INFO:teuthology.orchestra.run.smithi096.stdout:2054 read 0xcafe353 thru 0xcb03683 (0x5331 bytes) 2024-04-24T15:32:46.549 INFO:teuthology.orchestra.run.smithi096.stdout:2055 punch from 0x123186d to 0x123265f, (0xdf2 bytes) 2024-04-24T15:32:46.555 INFO:teuthology.orchestra.run.smithi096.stdout:2056 trunc from 0xdce2207 to 0x9d4b2ac 2024-04-24T15:32:46.562 INFO:teuthology.orchestra.run.smithi096.stdout:2057 writesame 0x86441b0 thru 0x8653569 data_size 0x3(0xf3ba bytes) 2024-04-24T15:32:46.572 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:32:46.573 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-04-24T15:32:46.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-04-24T15:32:46.573 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-04-24T15:32:46.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.573 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4426.0:2434 4.d e32/26 4:b7840674:::rbd_data.114a1f60cb66.00000000000001a7:head v 32'12, mlcod=32'10)", 2024-04-24T15:32:46.573 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:23.024172+0000", 2024-04-24T15:32:46.573 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.547889049999998, 2024-04-24T15:32:46.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057919520000000004, 2024-04-24T15:32:46.573 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:46.573 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:46.573 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:46.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.573 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:46.574 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.024172+0000", 2024-04-24T15:32:46.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:46.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.574 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:46.574 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.024172+0000", 2024-04-24T15:32:46.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:46.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.574 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:46.574 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.024171+0000", 2024-04-24T15:32:46.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:46.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:46.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.024178+0000", 2024-04-24T15:32:46.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2420000000000003e-06 2024-04-24T15:32:46.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:46.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.024191+0000", 2024-04-24T15:32:46.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2444e-05 2024-04-24T15:32:46.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:46.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.024197+0000", 2024-04-24T15:32:46.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2249999999999997e-06 2024-04-24T15:32:46.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:46.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.024251+0000", 2024-04-24T15:32:46.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4221000000000001e-05 2024-04-24T15:32:46.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:46.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.024270+0000", 2024-04-24T15:32:46.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9151999999999999e-05 2024-04-24T15:32:46.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:46.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.029925+0000", 2024-04-24T15:32:46.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0056546160000000003 2024-04-24T15:32:46.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:46.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.029964+0000", 2024-04-24T15:32:46.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8855e-05 2024-04-24T15:32:46.577 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:46.577 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:46.577 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:46.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.577 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4426.0:2510 4.12 e33/27 4:4fee0517:::rbd_data.114a1f60cb66.000000000000029d:head v 33'19, mlcod=32'17)", 2024-04-24T15:32:46.577 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:23.078201+0000", 2024-04-24T15:32:46.577 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.493859551, 2024-04-24T15:32:46.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0060097689999999999, 2024-04-24T15:32:46.578 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:46.578 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:46.578 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:46.578 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.578 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:46.578 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.078201+0000", 2024-04-24T15:32:46.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:46.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.578 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.578 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:46.578 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.078201+0000", 2024-04-24T15:32:46.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:46.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.578 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.579 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:46.579 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.078201+0000", 2024-04-24T15:32:46.579 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:46.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.579 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:46.579 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.078213+0000", 2024-04-24T15:32:46.579 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2261e-05 2024-04-24T15:32:46.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.579 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:46.579 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.078223+0000", 2024-04-24T15:32:46.579 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4709999999999996e-06 2024-04-24T15:32:46.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:46.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.078230+0000", 2024-04-24T15:32:46.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8979999999999999e-06 2024-04-24T15:32:46.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:46.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.079842+0000", 2024-04-24T15:32:46.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001612048 2024-04-24T15:32:46.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:46.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.079870+0000", 2024-04-24T15:32:46.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8487e-05 2024-04-24T15:32:46.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:46.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.084169+0000", 2024-04-24T15:32:46.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0042991490000000004 2024-04-24T15:32:46.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:46.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.084211+0000", 2024-04-24T15:32:46.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1792000000000002e-05 2024-04-24T15:32:46.581 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:46.581 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:46.581 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:46.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.582 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:2522 4.6 4:60ceedf0:::rbd_data.114a1f60cb66.000000000000011d:head [set-alloc-hint object_size 262144 write_size 262144,write 110592~2048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-24T15:32:46.582 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:23.088240+0000", 2024-04-24T15:32:46.582 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.483820637000001, 2024-04-24T15:32:46.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.988232797, 2024-04-24T15:32:46.582 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:46.582 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:46.582 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:46.582 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:46.582 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:46.582 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2522 2024-04-24T15:32:46.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.582 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:46.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:46.583 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088240+0000", 2024-04-24T15:32:46.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:46.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:46.583 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088240+0000", 2024-04-24T15:32:46.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:46.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:46.583 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088238+0000", 2024-04-24T15:32:46.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999981 2024-04-24T15:32:46.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:46.584 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088254+0000", 2024-04-24T15:32:46.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5562000000000001e-05 2024-04-24T15:32:46.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:46.584 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088275+0000", 2024-04-24T15:32:46.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1256000000000001e-05 2024-04-24T15:32:46.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:46.584 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088283+0000", 2024-04-24T15:32:46.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3219999999999993e-06 2024-04-24T15:32:46.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:46.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088351+0000", 2024-04-24T15:32:46.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7595000000000006e-05 2024-04-24T15:32:46.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-04-24T15:32:46.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088362+0000", 2024-04-24T15:32:46.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1764e-05 2024-04-24T15:32:46.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:46.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.073748+0000", 2024-04-24T15:32:46.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98538541899999998 2024-04-24T15:32:46.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:46.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.073925+0000", 2024-04-24T15:32:46.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017747600000000001 2024-04-24T15:32:46.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:46.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.074092+0000", 2024-04-24T15:32:46.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000166622 2024-04-24T15:32:46.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:46.587 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.074774+0000", 2024-04-24T15:32:46.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00068225000000000002 2024-04-24T15:32:46.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.587 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:46.587 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.076412+0000", 2024-04-24T15:32:46.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001638189 2024-04-24T15:32:46.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.587 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:46.587 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.076446+0000", 2024-04-24T15:32:46.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3587000000000002e-05 2024-04-24T15:32:46.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:46.588 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.076473+0000", 2024-04-24T15:32:46.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6886999999999999e-05 2024-04-24T15:32:46.588 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:46.588 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:46.588 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:46.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.588 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:2755 4.1c 4:3c8baac7:::rbd_data.114a1f60cb66.00000000000000a8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:46.588 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.252831+0000", 2024-04-24T15:32:46.588 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 22.319229823000001, 2024-04-24T15:32:46.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057169500000000002, 2024-04-24T15:32:46.588 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:46.588 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:46.589 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:46.589 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:46.589 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:46.589 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2755 2024-04-24T15:32:46.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.589 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:46.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.589 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:46.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.252831+0000", 2024-04-24T15:32:46.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:46.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.589 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:46.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.252831+0000", 2024-04-24T15:32:46.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:46.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.590 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:46.590 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.252830+0000", 2024-04-24T15:32:46.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:46.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.590 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:46.590 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.252835+0000", 2024-04-24T15:32:46.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3739999999999997e-06 2024-04-24T15:32:46.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.591 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:46.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.252846+0000", 2024-04-24T15:32:46.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.039e-05 2024-04-24T15:32:46.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.591 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:46.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.252850+0000", 2024-04-24T15:32:46.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4100000000000001e-06 2024-04-24T15:32:46.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.591 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:46.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.252885+0000", 2024-04-24T15:32:46.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5213999999999999e-05 2024-04-24T15:32:46.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:46.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.252957+0000", 2024-04-24T15:32:46.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2285000000000003e-05 2024-04-24T15:32:46.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:46.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.253053+0000", 2024-04-24T15:32:46.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5841999999999995e-05 2024-04-24T15:32:46.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:46.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.255905+0000", 2024-04-24T15:32:46.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028514539999999998 2024-04-24T15:32:46.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:46.593 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.258474+0000", 2024-04-24T15:32:46.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0025695679999999999 2024-04-24T15:32:46.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:46.593 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.258518+0000", 2024-04-24T15:32:46.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3242999999999999e-05 2024-04-24T15:32:46.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:46.593 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.258548+0000", 2024-04-24T15:32:46.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0318999999999999e-05 2024-04-24T15:32:46.593 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:46.594 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:46.594 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:46.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.594 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.594 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4426.0:2761 4.19 e34/26 4:9e865ec5:::rbd_data.114a1f60cb66.00000000000000c0:head v 34'14, mlcod=34'12)", 2024-04-24T15:32:46.594 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.255783+0000", 2024-04-24T15:32:46.594 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 22.316278226000001, 2024-04-24T15:32:46.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057401149999999996, 2024-04-24T15:32:46.594 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:46.594 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:46.594 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:46.594 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.594 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:46.595 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.255783+0000", 2024-04-24T15:32:46.595 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:46.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.595 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:46.595 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.255783+0000", 2024-04-24T15:32:46.595 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:46.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.595 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:46.595 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.255782+0000", 2024-04-24T15:32:46.595 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:46.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.595 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:46.596 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.255794+0000", 2024-04-24T15:32:46.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2429000000000001e-05 2024-04-24T15:32:46.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:46.596 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.255802+0000", 2024-04-24T15:32:46.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2139999999999999e-06 2024-04-24T15:32:46.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:46.596 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.255807+0000", 2024-04-24T15:32:46.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6500000000000001e-06 2024-04-24T15:32:46.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:46.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.255846+0000", 2024-04-24T15:32:46.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8497000000000001e-05 2024-04-24T15:32:46.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:46.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.255867+0000", 2024-04-24T15:32:46.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0733999999999998e-05 2024-04-24T15:32:46.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:46.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.261440+0000", 2024-04-24T15:32:46.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0055733099999999997 2024-04-24T15:32:46.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.598 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:46.598 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.261523+0000", 2024-04-24T15:32:46.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2764999999999998e-05 2024-04-24T15:32:46.598 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:46.598 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:46.598 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:46.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.598 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:3772 4.4 4:24a7cfbc:::rbd_data.114a1f60cb66.0000000000000020:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:46.598 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.699611+0000", 2024-04-24T15:32:46.598 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 21.872449806999999, 2024-04-24T15:32:46.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0060871340000000001, 2024-04-24T15:32:46.598 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:46.599 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:46.599 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:46.599 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:46.599 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:46.599 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3772 2024-04-24T15:32:46.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.599 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:46.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:46.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.699611+0000", 2024-04-24T15:32:46.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:46.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:46.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.699611+0000", 2024-04-24T15:32:46.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:46.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:46.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.699609+0000", 2024-04-24T15:32:46.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999981 2024-04-24T15:32:46.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:46.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.699618+0000", 2024-04-24T15:32:46.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3859999999999996e-06 2024-04-24T15:32:46.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:46.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.699637+0000", 2024-04-24T15:32:46.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8688000000000001e-05 2024-04-24T15:32:46.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:46.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.699647+0000", 2024-04-24T15:32:46.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.029e-05 2024-04-24T15:32:46.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:46.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.699692+0000", 2024-04-24T15:32:46.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4400000000000002e-05 2024-04-24T15:32:46.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:46.602 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.699751+0000", 2024-04-24T15:32:46.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8953999999999998e-05 2024-04-24T15:32:46.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:46.602 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.700249+0000", 2024-04-24T15:32:46.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00049869900000000002 2024-04-24T15:32:46.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:46.602 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.705560+0000", 2024-04-24T15:32:46.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0053110550000000003 2024-04-24T15:32:46.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:46.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.705602+0000", 2024-04-24T15:32:46.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1689e-05 2024-04-24T15:32:46.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:46.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.705652+0000", 2024-04-24T15:32:46.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9740000000000001e-05 2024-04-24T15:32:46.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:46.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.705698+0000", 2024-04-24T15:32:46.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6329000000000003e-05 2024-04-24T15:32:46.603 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:46.604 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:46.604 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:46.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.604 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:3774 4.4 4:25b094fe:::rbd_data.114a1f60cb66.0000000000000022:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:46.604 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.699669+0000", 2024-04-24T15:32:46.604 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 21.872391675999999, 2024-04-24T15:32:46.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.006296678, 2024-04-24T15:32:46.604 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:46.604 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:46.604 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:46.604 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:46.604 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:46.604 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3774 2024-04-24T15:32:46.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.605 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:46.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:46.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.699669+0000", 2024-04-24T15:32:46.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:46.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:46.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.699669+0000", 2024-04-24T15:32:46.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:46.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:46.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.699669+0000", 2024-04-24T15:32:46.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:46.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:46.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.699672+0000", 2024-04-24T15:32:46.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2830000000000002e-06 2024-04-24T15:32:46.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:46.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.699677+0000", 2024-04-24T15:32:46.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7530000000000001e-06 2024-04-24T15:32:46.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:46.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.699680+0000", 2024-04-24T15:32:46.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0130000000000002e-06 2024-04-24T15:32:46.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:46.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.702736+0000", 2024-04-24T15:32:46.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0030567379999999998 2024-04-24T15:32:46.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:46.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.702797+0000", 2024-04-24T15:32:46.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0563000000000003e-05 2024-04-24T15:32:46.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:46.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.703297+0000", 2024-04-24T15:32:46.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00049955300000000002 2024-04-24T15:32:46.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:46.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.705595+0000", 2024-04-24T15:32:46.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002298076 2024-04-24T15:32:46.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:46.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.705870+0000", 2024-04-24T15:32:46.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000275285 2024-04-24T15:32:46.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:46.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.705923+0000", 2024-04-24T15:32:46.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3229999999999997e-05 2024-04-24T15:32:46.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:46.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.705966+0000", 2024-04-24T15:32:46.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2636999999999999e-05 2024-04-24T15:32:46.609 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:46.609 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:46.609 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:46.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.609 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:3795 4.1c 4:3a2027c5:::rbd_data.114a1f60cb66.0000000000000025:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:46.610 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.723461+0000", 2024-04-24T15:32:46.610 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 21.848599819, 2024-04-24T15:32:46.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0066452719999999998, 2024-04-24T15:32:46.610 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:46.610 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:46.610 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:46.610 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:46.610 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:46.610 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3795 2024-04-24T15:32:46.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.610 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:46.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:46.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.723461+0000", 2024-04-24T15:32:46.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:46.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:46.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.723461+0000", 2024-04-24T15:32:46.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:46.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:46.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.723459+0000", 2024-04-24T15:32:46.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999986 2024-04-24T15:32:46.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:46.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.723470+0000", 2024-04-24T15:32:46.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0097e-05 2024-04-24T15:32:46.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:46.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.723484+0000", 2024-04-24T15:32:46.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4338000000000001e-05 2024-04-24T15:32:46.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:46.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.723496+0000", 2024-04-24T15:32:46.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2125e-05 2024-04-24T15:32:46.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:46.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.723547+0000", 2024-04-24T15:32:46.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0909000000000003e-05 2024-04-24T15:32:46.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:46.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.723639+0000", 2024-04-24T15:32:46.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1844999999999999e-05 2024-04-24T15:32:46.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:46.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.724463+0000", 2024-04-24T15:32:46.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00082389099999999999 2024-04-24T15:32:46.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.614 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:46.614 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.728863+0000", 2024-04-24T15:32:46.614 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0044007610000000004 2024-04-24T15:32:46.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.614 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:46.614 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.730017+0000", 2024-04-24T15:32:46.614 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0011534410000000001 2024-04-24T15:32:46.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.614 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:46.614 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.730067+0000", 2024-04-24T15:32:46.614 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0430000000000003e-05 2024-04-24T15:32:46.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:46.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.730106+0000", 2024-04-24T15:32:46.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8902000000000001e-05 2024-04-24T15:32:46.615 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:46.615 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:46.615 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:46.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.615 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:3976 4.1c 4:396ab59e:::rbd_data.114a1f60cb66.0000000000000160:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:46.615 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.827920+0000", 2024-04-24T15:32:46.615 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 21.744140938000001, 2024-04-24T15:32:46.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057086940000000003, 2024-04-24T15:32:46.616 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:46.616 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:46.616 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:46.616 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:46.616 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:46.616 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3976 2024-04-24T15:32:46.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.616 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:46.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:46.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.827920+0000", 2024-04-24T15:32:46.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:46.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:46.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.827920+0000", 2024-04-24T15:32:46.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:46.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:46.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.827919+0000", 2024-04-24T15:32:46.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:46.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:46.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.827924+0000", 2024-04-24T15:32:46.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4270000000000001e-06 2024-04-24T15:32:46.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.618 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:46.618 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.827934+0000", 2024-04-24T15:32:46.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0606e-05 2024-04-24T15:32:46.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.618 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:46.618 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.827939+0000", 2024-04-24T15:32:46.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.792e-06 2024-04-24T15:32:46.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.618 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:46.618 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.827967+0000", 2024-04-24T15:32:46.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7919000000000002e-05 2024-04-24T15:32:46.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:46.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.828011+0000", 2024-04-24T15:32:46.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3896000000000001e-05 2024-04-24T15:32:46.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:46.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.828070+0000", 2024-04-24T15:32:46.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9348999999999997e-05 2024-04-24T15:32:46.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:46.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.830513+0000", 2024-04-24T15:32:46.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024429669999999999 2024-04-24T15:32:46.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:46.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.833567+0000", 2024-04-24T15:32:46.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0030542590000000001 2024-04-24T15:32:46.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:46.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.833608+0000", 2024-04-24T15:32:46.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1148e-05 2024-04-24T15:32:46.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:46.621 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.833629+0000", 2024-04-24T15:32:46.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0066999999999999e-05 2024-04-24T15:32:46.621 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:46.621 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:46.621 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:46.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.621 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:3987 4.1c 4:3a30a8f2:::rbd_data.114a1f60cb66.0000000000000174:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:46.621 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.833638+0000", 2024-04-24T15:32:46.621 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 21.738422658000001, 2024-04-24T15:32:46.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057825430000000002, 2024-04-24T15:32:46.621 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:46.621 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:46.621 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:46.622 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:46.622 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:46.622 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3987 2024-04-24T15:32:46.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.622 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:46.622 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.622 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:46.622 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.833638+0000", 2024-04-24T15:32:46.622 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:46.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.622 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.622 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:46.622 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.833638+0000", 2024-04-24T15:32:46.623 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:46.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.623 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:46.623 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.833637+0000", 2024-04-24T15:32:46.623 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:46.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.623 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:46.623 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.833646+0000", 2024-04-24T15:32:46.623 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0180000000000007e-06 2024-04-24T15:32:46.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.623 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:46.623 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.833659+0000", 2024-04-24T15:32:46.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.313e-05 2024-04-24T15:32:46.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.624 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:46.624 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.833664+0000", 2024-04-24T15:32:46.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.006e-06 2024-04-24T15:32:46.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.624 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:46.624 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.833697+0000", 2024-04-24T15:32:46.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2978000000000002e-05 2024-04-24T15:32:46.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.624 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:46.625 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.833756+0000", 2024-04-24T15:32:46.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8811000000000003e-05 2024-04-24T15:32:46.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.625 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:46.625 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.833823+0000", 2024-04-24T15:32:46.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7225999999999996e-05 2024-04-24T15:32:46.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.625 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:46.625 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.838767+0000", 2024-04-24T15:32:46.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0049435809999999998 2024-04-24T15:32:46.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.625 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:46.626 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.839341+0000", 2024-04-24T15:32:46.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00057366800000000005 2024-04-24T15:32:46.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.626 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:46.626 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.839385+0000", 2024-04-24T15:32:46.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4597e-05 2024-04-24T15:32:46.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.626 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:46.626 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.839421+0000", 2024-04-24T15:32:46.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5516999999999999e-05 2024-04-24T15:32:46.626 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:46.627 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:46.627 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:46.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.627 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:4005 4.6 4:64968d7a:::rbd_data.114a1f60cb66.00000000000001bc:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:46.627 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.841940+0000", 2024-04-24T15:32:46.627 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 21.730120399, 2024-04-24T15:32:46.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.005717481, 2024-04-24T15:32:46.627 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:46.627 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:46.627 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:46.627 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:46.627 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:46.627 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4005 2024-04-24T15:32:46.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.628 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:46.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:46.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.841940+0000", 2024-04-24T15:32:46.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:46.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:46.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.841940+0000", 2024-04-24T15:32:46.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:46.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:46.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.841940+0000", 2024-04-24T15:32:46.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:46.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:46.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.841944+0000", 2024-04-24T15:32:46.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.904e-06 2024-04-24T15:32:46.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:46.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.841954+0000", 2024-04-24T15:32:46.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7499999999999998e-06 2024-04-24T15:32:46.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:46.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.841960+0000", 2024-04-24T15:32:46.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4399999999999996e-06 2024-04-24T15:32:46.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:46.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.841991+0000", 2024-04-24T15:32:46.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1615000000000001e-05 2024-04-24T15:32:46.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:46.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.842067+0000", 2024-04-24T15:32:46.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6002999999999997e-05 2024-04-24T15:32:46.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:46.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.842168+0000", 2024-04-24T15:32:46.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000100474 2024-04-24T15:32:46.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:46.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.847470+0000", 2024-04-24T15:32:46.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0053021830000000002 2024-04-24T15:32:46.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:46.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.847577+0000", 2024-04-24T15:32:46.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000107425 2024-04-24T15:32:46.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:46.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.847622+0000", 2024-04-24T15:32:46.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5015999999999997e-05 2024-04-24T15:32:46.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:46.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.847658+0000", 2024-04-24T15:32:46.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5509999999999997e-05 2024-04-24T15:32:46.632 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:46.632 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:46.632 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:46.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.633 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:4119 4.6 4:6588fd86:::rbd_data.114a1f60cb66.0000000000000179:head [set-alloc-hint object_size 262144 write_size 262144,write 151552~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:46.633 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.935980+0000", 2024-04-24T15:32:46.633 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 21.636080530000001, 2024-04-24T15:32:46.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.006739391, 2024-04-24T15:32:46.633 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:46.633 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:46.633 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:46.633 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:46.633 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:46.633 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4119 2024-04-24T15:32:46.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.633 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:46.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.633 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:46.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.935980+0000", 2024-04-24T15:32:46.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:46.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:46.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.935980+0000", 2024-04-24T15:32:46.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:46.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:46.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.935980+0000", 2024-04-24T15:32:46.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:46.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:46.635 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.935987+0000", 2024-04-24T15:32:46.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1590000000000004e-06 2024-04-24T15:32:46.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:46.635 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.935996+0000", 2024-04-24T15:32:46.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8389999999999996e-06 2024-04-24T15:32:46.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:46.635 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.936001+0000", 2024-04-24T15:32:46.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6890000000000001e-06 2024-04-24T15:32:46.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:46.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.936456+0000", 2024-04-24T15:32:46.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00045432600000000001 2024-04-24T15:32:46.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:46.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.936530+0000", 2024-04-24T15:32:46.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4550999999999998e-05 2024-04-24T15:32:46.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:46.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.936663+0000", 2024-04-24T15:32:46.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000133225 2024-04-24T15:32:46.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.637 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:46.637 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.938066+0000", 2024-04-24T15:32:46.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0014031969999999999 2024-04-24T15:32:46.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.637 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:46.637 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.942636+0000", 2024-04-24T15:32:46.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0045691270000000001 2024-04-24T15:32:46.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.637 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:46.637 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.942684+0000", 2024-04-24T15:32:46.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8123e-05 2024-04-24T15:32:46.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.638 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:46.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.942720+0000", 2024-04-24T15:32:46.638 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5880000000000002e-05 2024-04-24T15:32:46.638 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:46.638 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:46.638 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:46.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.638 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:9497 4.1c 4:3e8473f6:::rbd_header.114a1f60cb66:head [watch reconnect cookie 140672528495600 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:46.638 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.312983+0000", 2024-04-24T15:32:46.638 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.259077720000001, 2024-04-24T15:32:46.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0070173939999999997, 2024-04-24T15:32:46.639 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:46.639 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:46.639 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:46.639 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:46.639 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:46.639 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 9497 2024-04-24T15:32:46.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.639 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:46.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.639 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:46.639 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.312983+0000", 2024-04-24T15:32:46.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:46.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:46.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.312983+0000", 2024-04-24T15:32:46.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:46.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:46.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.312983+0000", 2024-04-24T15:32:46.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:46.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:46.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.312989+0000", 2024-04-24T15:32:46.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3090000000000001e-06 2024-04-24T15:32:46.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.641 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:46.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.312995+0000", 2024-04-24T15:32:46.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.477e-06 2024-04-24T15:32:46.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.641 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:46.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.312999+0000", 2024-04-24T15:32:46.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8489999999999999e-06 2024-04-24T15:32:46.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.641 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:46.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.313327+0000", 2024-04-24T15:32:46.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00032772399999999999 2024-04-24T15:32:46.642 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.642 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.642 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:46.642 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.313391+0000", 2024-04-24T15:32:46.642 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4388999999999996e-05 2024-04-24T15:32:46.642 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.642 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.642 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:46.642 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.320000+0000", 2024-04-24T15:32:46.642 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0066090660000000002 2024-04-24T15:32:46.642 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:46.642 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:46.642 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:46.642 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.643 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:19861 4.d 4:b2ab6102:::rbd_data.114ae15b9a20.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:46.643 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:34.404476+0000", 2024-04-24T15:32:46.643 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 12.167584905, 2024-04-24T15:32:46.643 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0072684150000000003, 2024-04-24T15:32:46.643 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:46.643 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:46.643 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:46.643 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:46.643 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:46.643 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 19861 2024-04-24T15:32:46.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.643 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:46.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:46.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:34.404476+0000", 2024-04-24T15:32:46.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:46.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:46.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:34.404476+0000", 2024-04-24T15:32:46.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:46.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:46.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:34.404473+0000", 2024-04-24T15:32:46.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999971 2024-04-24T15:32:46.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:46.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:34.405518+0000", 2024-04-24T15:32:46.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0010449490000000001 2024-04-24T15:32:46.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:46.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:34.405557+0000", 2024-04-24T15:32:46.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8427999999999999e-05 2024-04-24T15:32:46.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:46.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:34.405569+0000", 2024-04-24T15:32:46.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2129e-05 2024-04-24T15:32:46.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.646 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:46.646 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:34.405625+0000", 2024-04-24T15:32:46.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5980999999999998e-05 2024-04-24T15:32:46.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.646 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:46.646 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:34.405788+0000", 2024-04-24T15:32:46.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016350499999999999 2024-04-24T15:32:46.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.646 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 6", 2024-04-24T15:32:46.646 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:34.406065+0000", 2024-04-24T15:32:46.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000276631 2024-04-24T15:32:46.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.647 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:46.647 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:34.409212+0000", 2024-04-24T15:32:46.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003147066 2024-04-24T15:32:46.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.647 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:46.647 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:34.411631+0000", 2024-04-24T15:32:46.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024194960000000001 2024-04-24T15:32:46.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.647 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:46.647 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:34.411693+0000", 2024-04-24T15:32:46.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1383999999999994e-05 2024-04-24T15:32:46.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.648 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:46.648 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:34.411744+0000", 2024-04-24T15:32:46.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1474000000000002e-05 2024-04-24T15:32:46.648 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:46.648 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:46.648 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:46.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.648 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:24347 4.6 4:628d3169:::rbd_data.114ae15b9a20.000000000000004b:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-04-24T15:32:46.648 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:37.251701+0000", 2024-04-24T15:32:46.649 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.3203592989999997, 2024-04-24T15:32:46.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0087394719999999999, 2024-04-24T15:32:46.649 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:46.649 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:46.649 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:46.649 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:46.649 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:46.649 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 24347 2024-04-24T15:32:46.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.649 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:46.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:46.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.251701+0000", 2024-04-24T15:32:46.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:46.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.650 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:46.650 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.251701+0000", 2024-04-24T15:32:46.650 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:46.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.650 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:46.650 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.251700+0000", 2024-04-24T15:32:46.650 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999986 2024-04-24T15:32:46.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.650 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:46.650 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.251708+0000", 2024-04-24T15:32:46.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.49e-06 2024-04-24T15:32:46.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.651 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:46.651 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.251724+0000", 2024-04-24T15:32:46.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5803000000000001e-05 2024-04-24T15:32:46.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.651 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:46.651 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.251732+0000", 2024-04-24T15:32:46.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8460000000000008e-06 2024-04-24T15:32:46.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.651 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:46.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.251775+0000", 2024-04-24T15:32:46.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3102000000000002e-05 2024-04-24T15:32:46.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:46.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.251847+0000", 2024-04-24T15:32:46.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1482000000000004e-05 2024-04-24T15:32:46.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:46.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.252681+0000", 2024-04-24T15:32:46.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00083451599999999999 2024-04-24T15:32:46.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:46.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.260266+0000", 2024-04-24T15:32:46.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0075852599999999999 2024-04-24T15:32:46.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:46.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.260345+0000", 2024-04-24T15:32:46.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8824000000000003e-05 2024-04-24T15:32:46.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:46.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.260397+0000", 2024-04-24T15:32:46.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1573999999999998e-05 2024-04-24T15:32:46.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.654 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:46.654 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.260441+0000", 2024-04-24T15:32:46.654 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4195e-05 2024-04-24T15:32:46.654 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:46.654 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:46.654 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:46.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.654 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:24351 4.6 4:61b5c231:::rbd_data.114ae15b9a20.000000000000004f:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-04-24T15:32:46.654 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:37.254276+0000", 2024-04-24T15:32:46.654 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.3177852219999995, 2024-04-24T15:32:46.654 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0064981930000000002, 2024-04-24T15:32:46.655 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:46.655 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:46.655 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:46.655 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:46.655 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:46.655 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 24351 2024-04-24T15:32:46.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.655 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:46.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.655 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:46.655 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.254276+0000", 2024-04-24T15:32:46.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:46.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.655 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:46.656 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.254276+0000", 2024-04-24T15:32:46.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:46.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.656 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:46.656 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.254274+0000", 2024-04-24T15:32:46.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999981 2024-04-24T15:32:46.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.656 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:46.656 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.254283+0000", 2024-04-24T15:32:46.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.346e-06 2024-04-24T15:32:46.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:46.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.254298+0000", 2024-04-24T15:32:46.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5318000000000001e-05 2024-04-24T15:32:46.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:46.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.254306+0000", 2024-04-24T15:32:46.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.622e-06 2024-04-24T15:32:46.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:46.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.255950+0000", 2024-04-24T15:32:46.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001644473 2024-04-24T15:32:46.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:46.658 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.256055+0000", 2024-04-24T15:32:46.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104273 2024-04-24T15:32:46.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:46.658 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.256982+0000", 2024-04-24T15:32:46.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00092704699999999996 2024-04-24T15:32:46.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:46.658 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.260520+0000", 2024-04-24T15:32:46.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035386979999999998 2024-04-24T15:32:46.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.659 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:46.659 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.260692+0000", 2024-04-24T15:32:46.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017125300000000001 2024-04-24T15:32:46.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.659 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:46.659 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.260738+0000", 2024-04-24T15:32:46.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6820999999999998e-05 2024-04-24T15:32:46.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.659 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:46.659 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.260774+0000", 2024-04-24T15:32:46.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5237999999999997e-05 2024-04-24T15:32:46.660 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:46.660 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:46.660 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:46.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.660 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:24393 4.4 4:2764ffd7:::rbd_data.114ae15b9a20.0000000000000031:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-04-24T15:32:46.660 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:37.296111+0000", 2024-04-24T15:32:46.660 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.2759500890000002, 2024-04-24T15:32:46.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0058595019999999999, 2024-04-24T15:32:46.660 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:46.660 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:46.660 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:46.661 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:46.661 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:46.661 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 24393 2024-04-24T15:32:46.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.661 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:46.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:46.661 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.296111+0000", 2024-04-24T15:32:46.661 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:46.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:46.661 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.296111+0000", 2024-04-24T15:32:46.661 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:46.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.662 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:46.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.296110+0000", 2024-04-24T15:32:46.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:46.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.662 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:46.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.296114+0000", 2024-04-24T15:32:46.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4240000000000003e-06 2024-04-24T15:32:46.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.662 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:46.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.296125+0000", 2024-04-24T15:32:46.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0869999999999999e-05 2024-04-24T15:32:46.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.663 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:46.663 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.296130+0000", 2024-04-24T15:32:46.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7859999999999996e-06 2024-04-24T15:32:46.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.663 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:46.663 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.296174+0000", 2024-04-24T15:32:46.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3492999999999998e-05 2024-04-24T15:32:46.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.664 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:46.664 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.296258+0000", 2024-04-24T15:32:46.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3981999999999997e-05 2024-04-24T15:32:46.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.664 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:46.664 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.296383+0000", 2024-04-24T15:32:46.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000125143 2024-04-24T15:32:46.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.664 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:46.664 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.298229+0000", 2024-04-24T15:32:46.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018462260000000001 2024-04-24T15:32:46.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:46.665 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.301889+0000", 2024-04-24T15:32:46.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003660122 2024-04-24T15:32:46.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:46.665 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.301936+0000", 2024-04-24T15:32:46.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7284999999999999e-05 2024-04-24T15:32:46.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:46.665 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.301970+0000", 2024-04-24T15:32:46.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3892e-05 2024-04-24T15:32:46.666 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:46.666 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:46.666 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:46.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.666 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4426.0:25918 4.12 e50/27 4:4ab5cdb0:::rbd_data.114aaa0086f7.0000000000000002:head v 50'171, mlcod=48'169)", 2024-04-24T15:32:46.666 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:42.687598+0000", 2024-04-24T15:32:46.666 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.8844623870000001, 2024-04-24T15:32:46.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0066213660000000001, 2024-04-24T15:32:46.666 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:46.666 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:46.666 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:46.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.667 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:46.667 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:42.687598+0000", 2024-04-24T15:32:46.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:46.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.667 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:46.667 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:42.687598+0000", 2024-04-24T15:32:46.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:46.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.667 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:46.667 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:42.687598+0000", 2024-04-24T15:32:46.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:46.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.668 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:46.668 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:42.691504+0000", 2024-04-24T15:32:46.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0039066459999999997 2024-04-24T15:32:46.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.668 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:46.668 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:42.691524+0000", 2024-04-24T15:32:46.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9415999999999998e-05 2024-04-24T15:32:46.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.668 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:46.668 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:42.691534+0000", 2024-04-24T15:32:46.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9119999999999993e-06 2024-04-24T15:32:46.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:46.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:42.691573+0000", 2024-04-24T15:32:46.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9242000000000001e-05 2024-04-24T15:32:46.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:46.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:42.691593+0000", 2024-04-24T15:32:46.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0380000000000001e-05 2024-04-24T15:32:46.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:46.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:42.694173+0000", 2024-04-24T15:32:46.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002579895 2024-04-24T15:32:46.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:46.670 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:42.694220+0000", 2024-04-24T15:32:46.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6424000000000001e-05 2024-04-24T15:32:46.670 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:46.670 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:46.670 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:46.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.670 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4426.0:26638 4.12 e51/27 4:4e5034f1:::rbd_data.114a8e7208a5.0000000000000000:head v 51'173, mlcod=50'171)", 2024-04-24T15:32:46.670 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:43.886615+0000", 2024-04-24T15:32:46.670 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 2.6854461970000001, 2024-04-24T15:32:46.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0061157260000000001, 2024-04-24T15:32:46.670 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:46.671 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:46.671 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:46.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.671 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:46.671 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:43.886615+0000", 2024-04-24T15:32:46.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:46.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.671 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:46.671 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:43.886615+0000", 2024-04-24T15:32:46.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:46.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.671 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:46.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:43.886614+0000", 2024-04-24T15:32:46.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:32:46.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.672 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:46.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:43.890633+0000", 2024-04-24T15:32:46.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0040196110000000002 2024-04-24T15:32:46.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.672 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:46.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:43.890649+0000", 2024-04-24T15:32:46.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6147999999999999e-05 2024-04-24T15:32:46.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.673 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:46.673 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:43.890656+0000", 2024-04-24T15:32:46.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1959999999999996e-06 2024-04-24T15:32:46.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.673 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:46.673 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:43.890693+0000", 2024-04-24T15:32:46.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7651000000000003e-05 2024-04-24T15:32:46.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.673 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:46.673 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:43.890711+0000", 2024-04-24T15:32:46.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7592e-05 2024-04-24T15:32:46.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.674 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:46.674 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:43.892696+0000", 2024-04-24T15:32:46.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0019849870000000001 2024-04-24T15:32:46.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.674 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:46.674 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:43.892730+0000", 2024-04-24T15:32:46.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4385999999999999e-05 2024-04-24T15:32:46.674 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:46.674 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:46.674 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:46.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.675 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:29863 4.2 4:45d67f64:::rbd_data.114a2e5eebc.0000000000000255:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-24T15:32:46.675 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:46.442532+0000", 2024-04-24T15:32:46.675 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.12952892799999999, 2024-04-24T15:32:46.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014574231999999999, 2024-04-24T15:32:46.675 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:46.675 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:46.675 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:46.675 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:46.675 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:46.675 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 29863 2024-04-24T15:32:46.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.675 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:46.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:46.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.442532+0000", 2024-04-24T15:32:46.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:46.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:46.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.442532+0000", 2024-04-24T15:32:46.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:46.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:46.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.442530+0000", 2024-04-24T15:32:46.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999976 2024-04-24T15:32:46.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.677 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:46.677 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.442540+0000", 2024-04-24T15:32:46.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0284e-05 2024-04-24T15:32:46.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.677 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:46.677 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.442560+0000", 2024-04-24T15:32:46.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0097000000000001e-05 2024-04-24T15:32:46.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.677 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:46.677 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.442569+0000", 2024-04-24T15:32:46.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2e-06 2024-04-24T15:32:46.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:46.678 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.442641+0000", 2024-04-24T15:32:46.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1514999999999995e-05 2024-04-24T15:32:46.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:46.678 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.442787+0000", 2024-04-24T15:32:46.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014640599999999999 2024-04-24T15:32:46.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-24T15:32:46.678 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.445547+0000", 2024-04-24T15:32:46.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0027599109999999999 2024-04-24T15:32:46.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.679 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:46.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.456969+0000", 2024-04-24T15:32:46.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011422406 2024-04-24T15:32:46.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.679 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:32:46.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.456997+0000", 2024-04-24T15:32:46.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7132999999999999e-05 2024-04-24T15:32:46.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.679 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:32:46.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.457050+0000", 2024-04-24T15:32:46.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3093000000000001e-05 2024-04-24T15:32:46.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:46.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:46.680 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:46.680 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.457106+0000", 2024-04-24T15:32:46.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6467e-05 2024-04-24T15:32:46.680 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:46.680 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:46.680 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:46.680 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:46.680 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:46.680 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:32:46.681 INFO:teuthology.orchestra.run.smithi096.stdout:2058 punch from 0x5d3403a to 0x5d3ca7a, (0x8a40 bytes) 2024-04-24T15:32:46.682 INFO:teuthology.orchestra.run.smithi096.stdout:2060 read 0xb32672 thru 0xb3995c (0x72eb bytes) 2024-04-24T15:32:46.682 INFO:teuthology.orchestra.run.smithi096.stdout:2061 trunc from 0x9d4b2ac to 0x43b4c70 2024-04-24T15:32:46.683 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-24T15:32:46.684 INFO:teuthology.orchestra.run.smithi096.stdout:2065 writesame 0xf9170f thru 0xf9ae7d data_size 0x1(0x976f bytes) 2024-04-24T15:32:46.685 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:46.685 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:46.685 INFO:teuthology.orchestra.run.smithi096.stdout:2068 read 0x1f6d00f thru 0x1f77c9d (0xac8f bytes) 2024-04-24T15:32:46.685 INFO:teuthology.orchestra.run.smithi096.stdout:2069 read 0x36a485 thru 0x376e58 (0xc9d4 bytes) 2024-04-24T15:32:46.685 INFO:teuthology.orchestra.run.smithi096.stdout:2070 write 0xe1eadef thru 0xe1f9f64 (0xf176 bytes) 2024-04-24T15:32:46.685 INFO:teuthology.orchestra.run.smithi096.stdout:2072 writesame 0xc68c022 thru 0xc694eed data_size 0xd(0x8ecc bytes) 2024-04-24T15:32:46.685 INFO:teuthology.orchestra.run.smithi096.stdout:2073 punch from 0xcbb30a0 to 0xcbb721c, (0x417c bytes) 2024-04-24T15:32:46.685 INFO:teuthology.orchestra.run.smithi096.stdout:2074 read 0xc5c6b21 thru 0xc5d0312 (0x97f2 bytes) 2024-04-24T15:32:46.685 INFO:teuthology.orchestra.run.smithi096.stdout:2075 write 0x3d70e9d thru 0x3d7a297 (0x93fb bytes) 2024-04-24T15:32:46.685 INFO:teuthology.orchestra.run.smithi096.stdout:2076 writesame 0xe34c516 thru 0xe352a29 data_size 0x2(0x6514 bytes) 2024-04-24T15:32:46.685 INFO:teuthology.orchestra.run.smithi096.stdout:2078 write 0xa41c154 thru 0xa41d0c2 (0xf6f bytes) 2024-04-24T15:32:46.685 INFO:teuthology.orchestra.run.smithi096.stdout:2079 read 0x11f444a thru 0x11f88ba (0x4471 bytes) 2024-04-24T15:32:46.685 INFO:teuthology.orchestra.run.smithi096.stdout:2080 read 0x52b6f6b thru 0x52c08d0 (0x9966 bytes) 2024-04-24T15:32:46.686 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:46.686 INFO:teuthology.orchestra.run.smithi096.stdout:2082 read 0x20c2c4 thru 0x2188a3 (0xc5e0 bytes) 2024-04-24T15:32:46.686 INFO:teuthology.orchestra.run.smithi096.stdout:2083 trunc from 0xe352a2a to 0xb7da6f6 2024-04-24T15:32:46.686 INFO:teuthology.orchestra.run.smithi096.stdout:2084 punch from 0x5baf5ed to 0x5bb2511, (0x2f24 bytes) 2024-04-24T15:32:46.686 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:46.686 INFO:teuthology.orchestra.run.smithi096.stdout:2087 writesame 0x541fb4d thru 0x542f9e0 data_size 0x6(0xfe94 bytes) 2024-04-24T15:32:46.686 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:32:46.682+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:32:46.686 INFO:teuthology.orchestra.run.smithi096.stdout:2088 trunc from 0xb7da6f6 to 0x5b9a445 2024-04-24T15:32:46.690 INFO:teuthology.orchestra.run.smithi096.stdout:2089 write 0x20017ab thru 0x200e82c (0xd082 bytes) 2024-04-24T15:32:46.701 INFO:tasks.ceph.mon.a.smithi096.stderr:2024-04-24T15:32:46.701+0000 7fa88ddc8700 -1 mon.a@0(leader).osd e53 definitely_dead 0 2024-04-24T15:32:46.704 INFO:teuthology.orchestra.run.smithi096.stdout:2090 trunc from 0x5b9a445 to 0x836a0c 2024-04-24T15:32:46.730 INFO:teuthology.orchestra.run.smithi096.stdout:2092 clone 10 order 18 su 8192 sc 8 2024-04-24T15:32:46.782 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:32:46.782+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:32:46.821 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:32:46.821 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:32:46.822 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-04-24T15:32:46.822 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:32:46.831 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-24T15:32:46.882 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:32:46.883+0000 7f23abcf6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 44021) UID: 0 2024-04-24T15:32:46.984 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:32:46.985+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:32:46.994 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:32:46.995 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:32:46.995 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-04-24T15:32:46.995 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-04-24T15:32:46.995 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:32:47.004 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-24T15:32:47.085 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:32:47.085+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:32:47.129 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:32:47.130 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-04-24T15:32:47.130 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-04-24T15:32:47.130 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-04-24T15:32:47.130 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.130 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14043 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2383872~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:47.130 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892347+0000", 2024-04-24T15:32:47.130 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.237211715999999, 2024-04-24T15:32:47.130 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038537279000000001, 2024-04-24T15:32:47.130 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:47.130 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:47.130 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:47.130 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:47.130 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:47.131 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14043 2024-04-24T15:32:47.131 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.131 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:47.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.131 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:47.131 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892347+0000", 2024-04-24T15:32:47.131 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:47.131 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.131 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:47.131 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892347+0000", 2024-04-24T15:32:47.131 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:47.131 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.132 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:47.132 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892347+0000", 2024-04-24T15:32:47.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:47.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.132 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.132 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:47.132 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892349+0000", 2024-04-24T15:32:47.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3400000000000001e-06 2024-04-24T15:32:47.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.132 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.132 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:47.132 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892357+0000", 2024-04-24T15:32:47.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1540000000000002e-06 2024-04-24T15:32:47.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:47.133 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892360+0000", 2024-04-24T15:32:47.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5829999999999998e-06 2024-04-24T15:32:47.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:47.133 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930795+0000", 2024-04-24T15:32:47.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038434590999999997 2024-04-24T15:32:47.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:47.133 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930836+0000", 2024-04-24T15:32:47.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0827000000000001e-05 2024-04-24T15:32:47.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.134 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.134 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:47.134 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930885+0000", 2024-04-24T15:32:47.134 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9029000000000001e-05 2024-04-24T15:32:47.134 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:47.134 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:47.134 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:47.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.134 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.135 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14044 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2424832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:47.135 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892372+0000", 2024-04-24T15:32:47.135 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.237187071999999, 2024-04-24T15:32:47.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038624890000000002, 2024-04-24T15:32:47.135 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:47.135 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:47.135 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:47.135 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:47.135 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:47.135 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14044 2024-04-24T15:32:47.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.135 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:47.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:47.136 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892372+0000", 2024-04-24T15:32:47.136 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:47.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.136 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.136 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:47.136 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892372+0000", 2024-04-24T15:32:47.136 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:47.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.136 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.136 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:47.136 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892372+0000", 2024-04-24T15:32:47.136 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:47.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.136 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.137 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:47.137 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892374+0000", 2024-04-24T15:32:47.137 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4899999999999999e-06 2024-04-24T15:32:47.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.137 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.137 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:47.137 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892380+0000", 2024-04-24T15:32:47.137 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.631e-06 2024-04-24T15:32:47.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.137 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.137 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:47.137 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892383+0000", 2024-04-24T15:32:47.137 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7400000000000002e-06 2024-04-24T15:32:47.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.138 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.138 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:47.138 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930905+0000", 2024-04-24T15:32:47.138 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038521396999999999 2024-04-24T15:32:47.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.138 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.138 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:47.138 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930949+0000", 2024-04-24T15:32:47.138 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4153999999999998e-05 2024-04-24T15:32:47.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.138 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.138 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:47.138 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930997+0000", 2024-04-24T15:32:47.138 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7998999999999999e-05 2024-04-24T15:32:47.139 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:47.139 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:47.139 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:47.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.139 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.139 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14045 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2465792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:47.139 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892387+0000", 2024-04-24T15:32:47.139 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.237171804999999, 2024-04-24T15:32:47.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038720933999999999, 2024-04-24T15:32:47.139 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:47.139 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:47.139 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:47.139 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:47.139 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:47.139 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14045 2024-04-24T15:32:47.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.140 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:47.140 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.140 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:47.140 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892387+0000", 2024-04-24T15:32:47.140 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:47.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.140 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.140 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:47.140 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892387+0000", 2024-04-24T15:32:47.140 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:47.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.140 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.140 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:47.141 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892387+0000", 2024-04-24T15:32:47.141 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:47.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.141 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:47.141 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892392+0000", 2024-04-24T15:32:47.141 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0039999999999999e-06 2024-04-24T15:32:47.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.141 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:47.141 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892398+0000", 2024-04-24T15:32:47.141 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0900000000000001e-06 2024-04-24T15:32:47.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.142 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:47.142 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892402+0000", 2024-04-24T15:32:47.142 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.399e-06 2024-04-24T15:32:47.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.142 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.142 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:47.142 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931018+0000", 2024-04-24T15:32:47.142 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038616551999999998 2024-04-24T15:32:47.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.142 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.142 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:47.142 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931059+0000", 2024-04-24T15:32:47.142 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1205000000000003e-05 2024-04-24T15:32:47.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.143 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.143 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:47.143 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931108+0000", 2024-04-24T15:32:47.143 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8996000000000003e-05 2024-04-24T15:32:47.143 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:47.143 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:47.143 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:47.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.143 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.143 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14046 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2506752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:47.143 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892406+0000", 2024-04-24T15:32:47.143 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.237153214999999, 2024-04-24T15:32:47.143 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038814507999999998, 2024-04-24T15:32:47.143 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:47.144 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:47.144 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:47.144 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:47.144 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:47.144 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14046 2024-04-24T15:32:47.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.144 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:47.144 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.144 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:47.144 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892406+0000", 2024-04-24T15:32:47.144 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:47.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.144 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.144 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:47.145 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892406+0000", 2024-04-24T15:32:47.145 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:47.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.145 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.145 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:47.145 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892406+0000", 2024-04-24T15:32:47.145 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:47.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.145 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.145 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:47.145 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892413+0000", 2024-04-24T15:32:47.145 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7449999999999995e-06 2024-04-24T15:32:47.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.145 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.146 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:47.146 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892419+0000", 2024-04-24T15:32:47.146 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5790000000000003e-06 2024-04-24T15:32:47.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.146 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.146 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:47.146 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892423+0000", 2024-04-24T15:32:47.146 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8199999999999998e-06 2024-04-24T15:32:47.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.146 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.146 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:47.146 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931128+0000", 2024-04-24T15:32:47.146 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038705502000000003 2024-04-24T15:32:47.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.147 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:47.147 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931172+0000", 2024-04-24T15:32:47.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4066999999999997e-05 2024-04-24T15:32:47.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.147 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:47.147 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931220+0000", 2024-04-24T15:32:47.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8167999999999997e-05 2024-04-24T15:32:47.147 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:47.147 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:47.147 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:47.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.148 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14047 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2547712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:47.148 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892427+0000", 2024-04-24T15:32:47.148 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.237131986, 2024-04-24T15:32:47.148 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038904506999999998, 2024-04-24T15:32:47.148 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:47.148 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:47.148 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:47.148 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:47.148 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:47.148 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14047 2024-04-24T15:32:47.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.148 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:47.148 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.149 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:47.149 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892427+0000", 2024-04-24T15:32:47.149 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:47.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.149 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.149 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:47.149 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892427+0000", 2024-04-24T15:32:47.149 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:47.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.149 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.149 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:47.149 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892427+0000", 2024-04-24T15:32:47.149 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:47.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.149 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.150 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:47.150 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892429+0000", 2024-04-24T15:32:47.150 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3099999999999999e-06 2024-04-24T15:32:47.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.150 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.150 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:47.150 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892435+0000", 2024-04-24T15:32:47.150 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5520000000000004e-06 2024-04-24T15:32:47.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.150 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.150 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:47.150 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892438+0000", 2024-04-24T15:32:47.150 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4240000000000002e-06 2024-04-24T15:32:47.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.151 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:47.151 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931242+0000", 2024-04-24T15:32:47.151 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038804162000000003 2024-04-24T15:32:47.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.151 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:47.151 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931284+0000", 2024-04-24T15:32:47.151 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1414e-05 2024-04-24T15:32:47.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.151 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:47.151 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931332+0000", 2024-04-24T15:32:47.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8024e-05 2024-04-24T15:32:47.152 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:47.152 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:47.152 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:47.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.152 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14048 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2588672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:47.152 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892442+0000", 2024-04-24T15:32:47.152 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.237117023, 2024-04-24T15:32:47.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039003132000000003, 2024-04-24T15:32:47.152 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:47.152 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:47.152 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:47.152 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:47.152 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:47.153 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14048 2024-04-24T15:32:47.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.153 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:47.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:47.153 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892442+0000", 2024-04-24T15:32:47.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:47.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:47.153 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892442+0000", 2024-04-24T15:32:47.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:47.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.154 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:47.154 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892442+0000", 2024-04-24T15:32:47.154 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:47.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.154 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.154 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:47.154 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892444+0000", 2024-04-24T15:32:47.154 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1169999999999998e-06 2024-04-24T15:32:47.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.154 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.154 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:47.154 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892449+0000", 2024-04-24T15:32:47.154 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1599999999999997e-06 2024-04-24T15:32:47.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.155 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:47.155 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892453+0000", 2024-04-24T15:32:47.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5930000000000002e-06 2024-04-24T15:32:47.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.155 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:47.155 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931352+0000", 2024-04-24T15:32:47.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038899361 2024-04-24T15:32:47.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.155 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:47.155 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931398+0000", 2024-04-24T15:32:47.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6315e-05 2024-04-24T15:32:47.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.156 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:47.156 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931445+0000", 2024-04-24T15:32:47.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6950000000000003e-05 2024-04-24T15:32:47.156 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:47.156 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:47.156 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:47.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.156 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14049 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2629632~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:47.156 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892458+0000", 2024-04-24T15:32:47.156 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.237101602999999, 2024-04-24T15:32:47.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039103322000000003, 2024-04-24T15:32:47.157 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:47.157 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:47.157 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:47.157 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:47.157 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:47.157 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14049 2024-04-24T15:32:47.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.157 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:47.157 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.157 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:47.157 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892458+0000", 2024-04-24T15:32:47.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:47.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.157 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.158 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:47.158 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892458+0000", 2024-04-24T15:32:47.158 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:47.158 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.158 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.158 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:47.158 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892457+0000", 2024-04-24T15:32:47.158 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:47.158 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.158 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.158 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:47.158 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892459+0000", 2024-04-24T15:32:47.158 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.305e-06 2024-04-24T15:32:47.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.159 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.159 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:47.159 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892465+0000", 2024-04-24T15:32:47.159 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4360000000000001e-06 2024-04-24T15:32:47.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.159 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.159 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:47.159 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892474+0000", 2024-04-24T15:32:47.159 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7639999999999994e-06 2024-04-24T15:32:47.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.159 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.159 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:47.159 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931466+0000", 2024-04-24T15:32:47.159 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038992644 2024-04-24T15:32:47.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:47.160 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931512+0000", 2024-04-24T15:32:47.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5200000000000001e-05 2024-04-24T15:32:47.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:47.160 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931561+0000", 2024-04-24T15:32:47.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9357999999999997e-05 2024-04-24T15:32:47.160 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:47.160 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:47.160 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:47.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.161 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14050 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2670592~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:47.161 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892478+0000", 2024-04-24T15:32:47.161 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.237081015999999, 2024-04-24T15:32:47.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039195102000000002, 2024-04-24T15:32:47.161 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:47.161 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:47.161 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:47.161 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:47.161 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:47.161 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14050 2024-04-24T15:32:47.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.161 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:47.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:47.162 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892478+0000", 2024-04-24T15:32:47.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:47.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:47.162 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892478+0000", 2024-04-24T15:32:47.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:47.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:47.162 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892478+0000", 2024-04-24T15:32:47.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:47.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.163 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.163 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:47.163 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892480+0000", 2024-04-24T15:32:47.163 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3300000000000001e-06 2024-04-24T15:32:47.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.163 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.163 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:47.163 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892486+0000", 2024-04-24T15:32:47.163 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9200000000000001e-06 2024-04-24T15:32:47.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.163 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.163 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:47.163 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892494+0000", 2024-04-24T15:32:47.163 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5159999999999997e-06 2024-04-24T15:32:47.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.164 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:47.164 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931581+0000", 2024-04-24T15:32:47.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039087894999999998 2024-04-24T15:32:47.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.164 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:47.164 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931626+0000", 2024-04-24T15:32:47.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4943e-05 2024-04-24T15:32:47.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.164 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:47.164 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931673+0000", 2024-04-24T15:32:47.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6906000000000001e-05 2024-04-24T15:32:47.165 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:47.165 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:47.165 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:47.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.165 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14051 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2711552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:47.165 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892507+0000", 2024-04-24T15:32:47.165 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.237052317, 2024-04-24T15:32:47.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039274393999999997, 2024-04-24T15:32:47.165 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:47.165 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:47.165 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:47.165 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:47.165 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:47.165 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14051 2024-04-24T15:32:47.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.166 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:47.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.166 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:47.166 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892507+0000", 2024-04-24T15:32:47.166 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:47.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.166 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:47.166 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892507+0000", 2024-04-24T15:32:47.166 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:47.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.167 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:47.167 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892506+0000", 2024-04-24T15:32:47.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:47.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.167 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:47.167 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892509+0000", 2024-04-24T15:32:47.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8760000000000001e-06 2024-04-24T15:32:47.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.167 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:47.167 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892515+0000", 2024-04-24T15:32:47.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1840000000000004e-06 2024-04-24T15:32:47.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.168 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:47.168 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892519+0000", 2024-04-24T15:32:47.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7570000000000002e-06 2024-04-24T15:32:47.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.168 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:47.168 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931694+0000", 2024-04-24T15:32:47.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039174960000000002 2024-04-24T15:32:47.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.168 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:47.168 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931734+0000", 2024-04-24T15:32:47.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0284000000000003e-05 2024-04-24T15:32:47.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.169 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.169 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:47.169 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931781+0000", 2024-04-24T15:32:47.169 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6813000000000001e-05 2024-04-24T15:32:47.169 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:47.169 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:47.169 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:47.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.169 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.169 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14052 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2752512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:47.169 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892524+0000", 2024-04-24T15:32:47.169 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.23703517, 2024-04-24T15:32:47.169 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039371122000000001, 2024-04-24T15:32:47.169 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:47.169 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:47.170 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:47.170 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:47.170 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:47.170 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14052 2024-04-24T15:32:47.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.170 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:47.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.170 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:47.170 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892524+0000", 2024-04-24T15:32:47.170 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:47.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.170 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:47.170 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892524+0000", 2024-04-24T15:32:47.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:47.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:47.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892524+0000", 2024-04-24T15:32:47.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:47.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:47.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892526+0000", 2024-04-24T15:32:47.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3630000000000001e-06 2024-04-24T15:32:47.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:47.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892538+0000", 2024-04-24T15:32:47.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1677e-05 2024-04-24T15:32:47.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:47.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892541+0000", 2024-04-24T15:32:47.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.822e-06 2024-04-24T15:32:47.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:47.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931802+0000", 2024-04-24T15:32:47.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039260911000000003 2024-04-24T15:32:47.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.173 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:47.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931847+0000", 2024-04-24T15:32:47.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4450999999999998e-05 2024-04-24T15:32:47.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.173 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:47.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931895+0000", 2024-04-24T15:32:47.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8257999999999998e-05 2024-04-24T15:32:47.173 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:47.173 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:47.173 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:47.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.173 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14053 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2793472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:47.174 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892546+0000", 2024-04-24T15:32:47.174 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.237013191999999, 2024-04-24T15:32:47.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039457712999999998, 2024-04-24T15:32:47.174 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:47.174 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:47.174 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:47.174 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:47.174 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:47.174 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14053 2024-04-24T15:32:47.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.174 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:47.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.174 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:47.174 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892546+0000", 2024-04-24T15:32:47.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:47.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:47.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892546+0000", 2024-04-24T15:32:47.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:47.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:47.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892546+0000", 2024-04-24T15:32:47.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:47.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:47.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892548+0000", 2024-04-24T15:32:47.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3120000000000001e-06 2024-04-24T15:32:47.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:47.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892553+0000", 2024-04-24T15:32:47.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2320000000000001e-06 2024-04-24T15:32:47.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:47.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892559+0000", 2024-04-24T15:32:47.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5480000000000001e-06 2024-04-24T15:32:47.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:47.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931916+0000", 2024-04-24T15:32:47.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039357180999999998 2024-04-24T15:32:47.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:47.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931956+0000", 2024-04-24T15:32:47.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0565000000000002e-05 2024-04-24T15:32:47.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:47.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932004+0000", 2024-04-24T15:32:47.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7224000000000001e-05 2024-04-24T15:32:47.177 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:47.178 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:47.178 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:47.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.178 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14054 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2834432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:47.178 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892563+0000", 2024-04-24T15:32:47.178 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.23699654, 2024-04-24T15:32:47.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039552406999999998, 2024-04-24T15:32:47.178 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:47.178 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:47.178 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:47.178 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:47.178 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:47.178 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14054 2024-04-24T15:32:47.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.179 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:47.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.179 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:47.179 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892563+0000", 2024-04-24T15:32:47.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:47.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.179 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:47.179 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892563+0000", 2024-04-24T15:32:47.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:47.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.179 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:47.179 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892562+0000", 2024-04-24T15:32:47.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:47.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:47.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892564+0000", 2024-04-24T15:32:47.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2029999999999999e-06 2024-04-24T15:32:47.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:47.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892570+0000", 2024-04-24T15:32:47.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1460000000000004e-06 2024-04-24T15:32:47.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:47.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892573+0000", 2024-04-24T15:32:47.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2030000000000001e-06 2024-04-24T15:32:47.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:47.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932024+0000", 2024-04-24T15:32:47.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039451696000000001 2024-04-24T15:32:47.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:47.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932068+0000", 2024-04-24T15:32:47.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3970000000000001e-05 2024-04-24T15:32:47.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:47.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932115+0000", 2024-04-24T15:32:47.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6569000000000001e-05 2024-04-24T15:32:47.182 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:47.182 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:47.182 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:47.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.182 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14055 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2875392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:47.182 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892591+0000", 2024-04-24T15:32:47.182 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.236968567, 2024-04-24T15:32:47.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039634309, 2024-04-24T15:32:47.182 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:47.182 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:47.182 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:47.183 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:47.183 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:47.183 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14055 2024-04-24T15:32:47.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.183 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:47.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:47.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892591+0000", 2024-04-24T15:32:47.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:47.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:47.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892591+0000", 2024-04-24T15:32:47.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:47.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.184 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:47.184 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892590+0000", 2024-04-24T15:32:47.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:47.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.184 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:47.184 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892592+0000", 2024-04-24T15:32:47.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7519999999999999e-06 2024-04-24T15:32:47.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.184 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:47.184 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892596+0000", 2024-04-24T15:32:47.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.608e-06 2024-04-24T15:32:47.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.185 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:47.185 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892598+0000", 2024-04-24T15:32:47.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.531e-06 2024-04-24T15:32:47.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.185 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:47.185 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932136+0000", 2024-04-24T15:32:47.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039537668999999998 2024-04-24T15:32:47.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.185 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:47.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932177+0000", 2024-04-24T15:32:47.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0880000000000002e-05 2024-04-24T15:32:47.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:47.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932225+0000", 2024-04-24T15:32:47.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8223999999999998e-05 2024-04-24T15:32:47.186 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:47.186 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:47.186 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:47.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.186 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14056 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2916352~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:47.186 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892600+0000", 2024-04-24T15:32:47.187 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.236958733, 2024-04-24T15:32:47.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039736447000000001, 2024-04-24T15:32:47.187 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:47.187 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:47.187 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:47.187 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:47.187 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:47.187 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14056 2024-04-24T15:32:47.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.187 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:47.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:47.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892600+0000", 2024-04-24T15:32:47.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:47.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:47.188 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892600+0000", 2024-04-24T15:32:47.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:47.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:47.188 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892600+0000", 2024-04-24T15:32:47.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967296 2024-04-24T15:32:47.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:47.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892602+0000", 2024-04-24T15:32:47.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.576e-06 2024-04-24T15:32:47.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:47.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892605+0000", 2024-04-24T15:32:47.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.574e-06 2024-04-24T15:32:47.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:47.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892608+0000", 2024-04-24T15:32:47.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3800000000000001e-06 2024-04-24T15:32:47.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:47.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932245+0000", 2024-04-24T15:32:47.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039637378000000001 2024-04-24T15:32:47.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:47.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932290+0000", 2024-04-24T15:32:47.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4734999999999998e-05 2024-04-24T15:32:47.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:47.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932337+0000", 2024-04-24T15:32:47.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7033000000000002e-05 2024-04-24T15:32:47.191 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:47.191 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:47.191 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:47.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.191 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14062 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 3162112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:47.191 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.894556+0000", 2024-04-24T15:32:47.191 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.235003163, 2024-04-24T15:32:47.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.03845316, 2024-04-24T15:32:47.191 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:47.192 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:47.192 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:47.192 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:47.192 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:47.192 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14062 2024-04-24T15:32:47.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.192 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:47.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:47.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894556+0000", 2024-04-24T15:32:47.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:47.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:47.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894556+0000", 2024-04-24T15:32:47.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:47.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:47.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894556+0000", 2024-04-24T15:32:47.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967296 2024-04-24T15:32:47.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:47.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894557+0000", 2024-04-24T15:32:47.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3230000000000001e-06 2024-04-24T15:32:47.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:47.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894560+0000", 2024-04-24T15:32:47.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7719999999999999e-06 2024-04-24T15:32:47.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:47.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894562+0000", 2024-04-24T15:32:47.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9479999999999998e-06 2024-04-24T15:32:47.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:47.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932917+0000", 2024-04-24T15:32:47.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038355490999999999 2024-04-24T15:32:47.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:47.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932961+0000", 2024-04-24T15:32:47.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3302e-05 2024-04-24T15:32:47.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:47.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933009+0000", 2024-04-24T15:32:47.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8535000000000002e-05 2024-04-24T15:32:47.195 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:47.195 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:47.195 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:47.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.196 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14063 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 3203072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:47.196 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.894564+0000", 2024-04-24T15:32:47.196 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.234995142000001, 2024-04-24T15:32:47.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038554995000000002, 2024-04-24T15:32:47.196 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:47.196 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:47.196 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:47.196 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:47.196 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:47.196 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14063 2024-04-24T15:32:47.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.196 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:47.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:47.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894564+0000", 2024-04-24T15:32:47.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:47.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:47.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894564+0000", 2024-04-24T15:32:47.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:47.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:47.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894564+0000", 2024-04-24T15:32:47.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967296 2024-04-24T15:32:47.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.198 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:47.198 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894565+0000", 2024-04-24T15:32:47.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2640000000000001e-06 2024-04-24T15:32:47.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.198 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:47.198 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894569+0000", 2024-04-24T15:32:47.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1710000000000004e-06 2024-04-24T15:32:47.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.198 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:47.198 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894573+0000", 2024-04-24T15:32:47.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4410000000000002e-06 2024-04-24T15:32:47.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:47.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933030+0000", 2024-04-24T15:32:47.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038457709999999999 2024-04-24T15:32:47.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:47.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933072+0000", 2024-04-24T15:32:47.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.193e-05 2024-04-24T15:32:47.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:47.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933119+0000", 2024-04-24T15:32:47.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6715000000000003e-05 2024-04-24T15:32:47.200 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:47.200 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:47.200 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:47.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.200 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14064 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 3244032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:47.200 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.894591+0000", 2024-04-24T15:32:47.200 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.234968500000001, 2024-04-24T15:32:47.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038639806999999998, 2024-04-24T15:32:47.200 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:47.200 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:47.201 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:47.201 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:47.201 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:47.201 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14064 2024-04-24T15:32:47.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.201 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:47.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:47.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894591+0000", 2024-04-24T15:32:47.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:47.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:47.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894591+0000", 2024-04-24T15:32:47.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:47.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:47.202 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894590+0000", 2024-04-24T15:32:47.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:47.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:47.202 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894593+0000", 2024-04-24T15:32:47.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4719999999999998e-06 2024-04-24T15:32:47.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:47.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894598+0000", 2024-04-24T15:32:47.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.57e-06 2024-04-24T15:32:47.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:47.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894602+0000", 2024-04-24T15:32:47.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.614e-06 2024-04-24T15:32:47.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:47.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933139+0000", 2024-04-24T15:32:47.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038536929999999997 2024-04-24T15:32:47.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:47.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933183+0000", 2024-04-24T15:32:47.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4217e-05 2024-04-24T15:32:47.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:47.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933230+0000", 2024-04-24T15:32:47.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7434e-05 2024-04-24T15:32:47.204 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:47.204 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:47.204 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:47.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.205 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14065 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 3284992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:47.205 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.894606+0000", 2024-04-24T15:32:47.205 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.234953118, 2024-04-24T15:32:47.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038734126000000001, 2024-04-24T15:32:47.205 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:47.205 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:47.205 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:47.205 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:47.205 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:47.205 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14065 2024-04-24T15:32:47.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.205 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:47.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.206 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:47.206 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894606+0000", 2024-04-24T15:32:47.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:47.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.206 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.206 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:47.206 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894606+0000", 2024-04-24T15:32:47.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:47.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.206 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.206 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:47.206 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894606+0000", 2024-04-24T15:32:47.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:47.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.207 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:47.207 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894608+0000", 2024-04-24T15:32:47.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0880000000000002e-06 2024-04-24T15:32:47.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.207 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:47.207 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894614+0000", 2024-04-24T15:32:47.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7830000000000004e-06 2024-04-24T15:32:47.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.207 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:47.207 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894621+0000", 2024-04-24T15:32:47.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5809999999999999e-06 2024-04-24T15:32:47.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:47.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933251+0000", 2024-04-24T15:32:47.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038629995 2024-04-24T15:32:47.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:47.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933292+0000", 2024-04-24T15:32:47.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0380000000000003e-05 2024-04-24T15:32:47.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:47.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933340+0000", 2024-04-24T15:32:47.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8662999999999998e-05 2024-04-24T15:32:47.209 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:47.209 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:47.209 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:47.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.209 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14066 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 3325952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:47.209 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.894625+0000", 2024-04-24T15:32:47.209 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.234933678999999, 2024-04-24T15:32:47.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038825539999999999, 2024-04-24T15:32:47.209 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:47.209 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:47.209 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:47.209 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:47.210 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:47.210 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14066 2024-04-24T15:32:47.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.210 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:47.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.210 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:47.210 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894625+0000", 2024-04-24T15:32:47.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:47.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.210 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:47.210 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894625+0000", 2024-04-24T15:32:47.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:47.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.211 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:47.211 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894625+0000", 2024-04-24T15:32:47.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:47.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.211 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:47.211 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894627+0000", 2024-04-24T15:32:47.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.092e-06 2024-04-24T15:32:47.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.211 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:47.211 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894635+0000", 2024-04-24T15:32:47.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6660000000000008e-06 2024-04-24T15:32:47.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:47.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894639+0000", 2024-04-24T15:32:47.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.799e-06 2024-04-24T15:32:47.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:47.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933360+0000", 2024-04-24T15:32:47.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038721569999999997 2024-04-24T15:32:47.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:47.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933403+0000", 2024-04-24T15:32:47.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2605999999999998e-05 2024-04-24T15:32:47.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:47.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933451+0000", 2024-04-24T15:32:47.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8149999999999998e-05 2024-04-24T15:32:47.213 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:47.213 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:47.213 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:47.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.213 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14067 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 3366912~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:47.213 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.894644+0000", 2024-04-24T15:32:47.213 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.234915621000001, 2024-04-24T15:32:47.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038926068000000001, 2024-04-24T15:32:47.214 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:47.214 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:47.214 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:47.214 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:47.214 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:47.214 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14067 2024-04-24T15:32:47.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.214 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:47.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:47.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894644+0000", 2024-04-24T15:32:47.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:47.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:47.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894644+0000", 2024-04-24T15:32:47.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:47.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:47.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894643+0000", 2024-04-24T15:32:47.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:47.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:47.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894646+0000", 2024-04-24T15:32:47.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3379999999999999e-06 2024-04-24T15:32:47.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:47.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894651+0000", 2024-04-24T15:32:47.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.485e-06 2024-04-24T15:32:47.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:47.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894655+0000", 2024-04-24T15:32:47.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.709e-06 2024-04-24T15:32:47.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:47.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933472+0000", 2024-04-24T15:32:47.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038816992000000002 2024-04-24T15:32:47.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:47.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933521+0000", 2024-04-24T15:32:47.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8841999999999998e-05 2024-04-24T15:32:47.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:47.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:47.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:47.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933570+0000", 2024-04-24T15:32:47.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9063000000000001e-05 2024-04-24T15:32:47.217 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:47.217 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:47.217 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:47.217 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:47.217 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:47.218 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:32:47.219 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-24T15:32:47.220 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:32:47.219+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:32:47.236 INFO:teuthology.orchestra.run.smithi096.stderr:marked out osd.7. 2024-04-24T15:32:47.240 INFO:teuthology.orchestra.run.smithi096.stderr:noscrub is unset 2024-04-24T15:32:47.320 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:32:47.320+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:32:47.351 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:32:47.351 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:32:47.351 INFO:teuthology.orchestra.run.smithi151.stdout: "num_ops": 0 2024-04-24T15:32:47.351 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:32:47.360 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-24T15:32:47.445 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:32:47.446+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:32:47.522 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:32:47.522+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:32:47.523 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:32:47.523 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:32:47.523 INFO:teuthology.orchestra.run.smithi151.stdout: "complaint_time": 30, 2024-04-24T15:32:47.523 INFO:teuthology.orchestra.run.smithi151.stdout: "num_blocked_ops": 0 2024-04-24T15:32:47.523 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:32:47.534 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-24T15:32:47.623 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:32:47.623+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:32:47.704 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:32:47.704 INFO:teuthology.orchestra.run.smithi151.stdout: "size": 20, 2024-04-24T15:32:47.704 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 600, 2024-04-24T15:32:47.704 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [ 2024-04-24T15:32:47.705 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.705 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:2507 4.e 4:729aaa1b:::rbd_data.114a1f60cb66.000000000000029a:head [set-alloc-hint object_size 262144 write_size 262144,write 233472~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-24T15:32:47.705 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:23.072715+0000", 2024-04-24T15:32:47.705 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 24.630949052999998, 2024-04-24T15:32:47.705 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0065961370000000002, 2024-04-24T15:32:47.705 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:47.705 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:47.705 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:47.705 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:47.705 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:47.705 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 2507 2024-04-24T15:32:47.705 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.705 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:47.706 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.706 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:47.706 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.072715+0000", 2024-04-24T15:32:47.706 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:47.706 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.706 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.706 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:47.706 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.072715+0000", 2024-04-24T15:32:47.706 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:47.706 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.706 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.706 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:47.706 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.072713+0000", 2024-04-24T15:32:47.706 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999981 2024-04-24T15:32:47.707 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.707 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.707 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:47.707 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.072727+0000", 2024-04-24T15:32:47.707 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3549e-05 2024-04-24T15:32:47.707 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.707 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.707 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:47.707 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.072739+0000", 2024-04-24T15:32:47.707 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.1973999999999999e-05 2024-04-24T15:32:47.707 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.707 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.707 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:47.707 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.072746+0000", 2024-04-24T15:32:47.707 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2849999999999997e-06 2024-04-24T15:32:47.708 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.708 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.708 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:47.708 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.072798+0000", 2024-04-24T15:32:47.708 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.2289e-05 2024-04-24T15:32:47.708 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.708 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.708 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op must wait for map", 2024-04-24T15:32:47.708 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.072806+0000", 2024-04-24T15:32:47.708 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.8469999999999996e-06 2024-04-24T15:32:47.708 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.708 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.708 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:47.708 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.074503+0000", 2024-04-24T15:32:47.709 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0016967029999999999 2024-04-24T15:32:47.709 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.709 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.709 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:47.709 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.074593+0000", 2024-04-24T15:32:47.709 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.0313000000000001e-05 2024-04-24T15:32:47.709 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.709 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.709 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 6", 2024-04-24T15:32:47.709 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.074732+0000", 2024-04-24T15:32:47.709 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000138376 2024-04-24T15:32:47.709 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.709 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.709 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:47.710 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.078538+0000", 2024-04-24T15:32:47.710 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0038068749999999999 2024-04-24T15:32:47.710 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.710 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.710 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:47.710 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.079232+0000", 2024-04-24T15:32:47.710 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00069326900000000005 2024-04-24T15:32:47.710 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.710 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.710 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:47.710 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.079273+0000", 2024-04-24T15:32:47.710 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.1236000000000003e-05 2024-04-24T15:32:47.710 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.711 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.711 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:47.711 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.079311+0000", 2024-04-24T15:32:47.711 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8497000000000001e-05 2024-04-24T15:32:47.711 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:47.711 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:47.711 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:47.711 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.711 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.711 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_repop(client.4426.0:2563 4.1d e34/27 4:b841be81:::rbd_data.114a1f60cb66.00000000000001d7:head v 34'17, mlcod=32'15)", 2024-04-24T15:32:47.711 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.108404+0000", 2024-04-24T15:32:47.711 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 23.595260521, 2024-04-24T15:32:47.711 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0073539469999999996, 2024-04-24T15:32:47.711 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:47.711 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:47.712 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:47.712 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.712 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:47.712 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.108404+0000", 2024-04-24T15:32:47.712 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:47.712 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.712 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.712 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:47.712 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.108404+0000", 2024-04-24T15:32:47.712 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:47.712 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.712 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.712 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:47.712 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.108403+0000", 2024-04-24T15:32:47.713 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:47.713 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.713 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.713 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:47.713 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.108420+0000", 2024-04-24T15:32:47.713 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.6980999999999999e-05 2024-04-24T15:32:47.713 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.713 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.713 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:47.713 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.108428+0000", 2024-04-24T15:32:47.713 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0560000000000005e-06 2024-04-24T15:32:47.713 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.713 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.713 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:47.713 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.108432+0000", 2024-04-24T15:32:47.714 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.4060000000000001e-06 2024-04-24T15:32:47.714 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.714 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.714 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:47.714 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.108462+0000", 2024-04-24T15:32:47.714 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.008e-05 2024-04-24T15:32:47.714 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.714 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.714 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:47.714 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.108475+0000", 2024-04-24T15:32:47.714 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3018e-05 2024-04-24T15:32:47.714 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.714 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.715 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:47.715 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.115726+0000", 2024-04-24T15:32:47.715 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0072510040000000001 2024-04-24T15:32:47.715 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.715 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.715 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:47.715 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.115758+0000", 2024-04-24T15:32:47.715 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.2092999999999999e-05 2024-04-24T15:32:47.715 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:47.715 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:47.715 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:47.715 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.715 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.715 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:2620 4.18 4:1e938922:::rbd_data.114a1f60cb66.0000000000000028:head [stat,write 163840~6144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:47.715 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.151073+0000", 2024-04-24T15:32:47.716 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 23.552591807999999, 2024-04-24T15:32:47.716 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0063989210000000001, 2024-04-24T15:32:47.716 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:47.716 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:47.716 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:47.716 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:47.716 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:47.716 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 2620 2024-04-24T15:32:47.716 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.716 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:47.716 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.716 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:47.716 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.151073+0000", 2024-04-24T15:32:47.716 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:47.717 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.717 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.717 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:47.717 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.151073+0000", 2024-04-24T15:32:47.717 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:47.717 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.717 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.717 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:47.717 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.151072+0000", 2024-04-24T15:32:47.717 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:47.717 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.717 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.717 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:47.717 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.151083+0000", 2024-04-24T15:32:47.717 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0801999999999999e-05 2024-04-24T15:32:47.717 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.718 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.718 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:47.718 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.151086+0000", 2024-04-24T15:32:47.718 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.3900000000000002e-06 2024-04-24T15:32:47.718 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.718 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.718 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:47.718 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.151089+0000", 2024-04-24T15:32:47.718 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.294e-06 2024-04-24T15:32:47.718 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.718 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.718 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:47.718 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.152198+0000", 2024-04-24T15:32:47.718 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.001109407 2024-04-24T15:32:47.719 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.719 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.719 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:47.719 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.152273+0000", 2024-04-24T15:32:47.719 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.4419999999999996e-05 2024-04-24T15:32:47.719 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.719 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.719 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 6", 2024-04-24T15:32:47.719 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.152396+0000", 2024-04-24T15:32:47.719 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00012322400000000001 2024-04-24T15:32:47.719 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.719 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.719 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:47.719 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.157242+0000", 2024-04-24T15:32:47.720 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0048462369999999998 2024-04-24T15:32:47.720 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.720 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.720 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:47.720 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.157414+0000", 2024-04-24T15:32:47.720 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000172411 2024-04-24T15:32:47.720 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.720 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.720 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:47.720 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.157451+0000", 2024-04-24T15:32:47.720 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.6269000000000001e-05 2024-04-24T15:32:47.720 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.720 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.720 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:47.721 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.157471+0000", 2024-04-24T15:32:47.721 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.0741e-05 2024-04-24T15:32:47.721 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:47.721 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:47.721 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:47.721 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.721 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.721 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:2623 4.18 4:1b4f38fa:::rbd_data.114a1f60cb66.000000000000002b:head [stat,write 161792~6144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:47.721 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.151108+0000", 2024-04-24T15:32:47.721 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 23.552556546000002, 2024-04-24T15:32:47.721 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0064461170000000003, 2024-04-24T15:32:47.721 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:47.721 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:47.721 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:47.722 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:47.722 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:47.722 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 2623 2024-04-24T15:32:47.722 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.722 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:47.722 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.722 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:47.722 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.151108+0000", 2024-04-24T15:32:47.722 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:47.722 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.722 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.722 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:47.722 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.151108+0000", 2024-04-24T15:32:47.723 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:47.723 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.723 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.723 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:47.723 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.151108+0000", 2024-04-24T15:32:47.723 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:47.723 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.723 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.723 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:47.723 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.151117+0000", 2024-04-24T15:32:47.723 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.8560000000000002e-06 2024-04-24T15:32:47.723 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.723 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.723 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:47.723 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.151122+0000", 2024-04-24T15:32:47.724 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5390000000000001e-06 2024-04-24T15:32:47.724 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.724 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.724 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:47.724 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.151124+0000", 2024-04-24T15:32:47.724 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.2800000000000002e-06 2024-04-24T15:32:47.724 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.724 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.724 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:47.724 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.152676+0000", 2024-04-24T15:32:47.724 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.001552124 2024-04-24T15:32:47.724 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.724 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.724 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:47.725 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.152759+0000", 2024-04-24T15:32:47.725 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.2688000000000005e-05 2024-04-24T15:32:47.725 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.725 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.725 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 6", 2024-04-24T15:32:47.725 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.152867+0000", 2024-04-24T15:32:47.725 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00010779400000000001 2024-04-24T15:32:47.725 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.725 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.725 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:47.725 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.157259+0000", 2024-04-24T15:32:47.725 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0043918230000000004 2024-04-24T15:32:47.725 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.725 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.725 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:47.726 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.157498+0000", 2024-04-24T15:32:47.726 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000239829 2024-04-24T15:32:47.726 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.726 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.726 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:47.726 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.157525+0000", 2024-04-24T15:32:47.726 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.6868000000000001e-05 2024-04-24T15:32:47.726 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.726 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.726 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:47.726 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.157554+0000", 2024-04-24T15:32:47.726 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8602000000000002e-05 2024-04-24T15:32:47.726 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:47.727 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:47.727 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:47.727 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.727 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.727 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:3618 4.18 4:1f7d02ce:::rbd_data.114a1f60cb66.0000000000000161:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:47.727 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.581260+0000", 2024-04-24T15:32:47.727 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 23.122404189000001, 2024-04-24T15:32:47.727 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0068184120000000003, 2024-04-24T15:32:47.727 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:47.727 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:47.727 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:47.727 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:47.727 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:47.727 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 3618 2024-04-24T15:32:47.727 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.728 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:47.728 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.728 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:47.728 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.581260+0000", 2024-04-24T15:32:47.728 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:47.728 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.728 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.728 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:47.728 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.581260+0000", 2024-04-24T15:32:47.728 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:47.728 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.728 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.728 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:47.728 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.581259+0000", 2024-04-24T15:32:47.728 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999999 2024-04-24T15:32:47.729 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.729 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.729 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:47.729 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.581265+0000", 2024-04-24T15:32:47.729 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.003e-06 2024-04-24T15:32:47.729 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.729 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.729 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:47.729 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.581278+0000", 2024-04-24T15:32:47.729 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.2483000000000001e-05 2024-04-24T15:32:47.729 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.729 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.729 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:47.730 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.581284+0000", 2024-04-24T15:32:47.730 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.7399999999999998e-06 2024-04-24T15:32:47.730 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.730 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.730 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:47.730 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.581783+0000", 2024-04-24T15:32:47.730 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00049892199999999995 2024-04-24T15:32:47.730 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.730 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.730 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:47.731 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.581902+0000", 2024-04-24T15:32:47.731 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000118272 2024-04-24T15:32:47.731 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.731 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.731 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 6", 2024-04-24T15:32:47.731 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.583068+0000", 2024-04-24T15:32:47.731 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0011665759999999999 2024-04-24T15:32:47.731 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.731 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.731 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:47.731 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.587081+0000", 2024-04-24T15:32:47.731 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.004012781 2024-04-24T15:32:47.731 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.731 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.732 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:47.732 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.587997+0000", 2024-04-24T15:32:47.732 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00091637700000000001 2024-04-24T15:32:47.732 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.732 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.732 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:47.732 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.588041+0000", 2024-04-24T15:32:47.732 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3619999999999999e-05 2024-04-24T15:32:47.732 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.732 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.732 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:47.732 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.588079+0000", 2024-04-24T15:32:47.732 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.7654000000000002e-05 2024-04-24T15:32:47.732 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:47.733 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:47.733 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:47.733 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.733 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.733 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:3767 4.9 4:908f8f18:::rbd_data.114a1f60cb66.000000000000001b:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:47.733 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.699557+0000", 2024-04-24T15:32:47.733 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 23.004107464000001, 2024-04-24T15:32:47.733 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0088231639999999997, 2024-04-24T15:32:47.733 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:47.733 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:47.733 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:47.733 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:47.733 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:47.733 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 3767 2024-04-24T15:32:47.734 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.734 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:47.734 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.734 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:47.734 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699557+0000", 2024-04-24T15:32:47.734 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:47.734 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.734 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.734 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:47.734 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699557+0000", 2024-04-24T15:32:47.734 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:47.734 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.734 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.734 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:47.735 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699556+0000", 2024-04-24T15:32:47.735 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999999 2024-04-24T15:32:47.735 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.735 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.735 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:47.735 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699561+0000", 2024-04-24T15:32:47.735 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.0440000000000003e-06 2024-04-24T15:32:47.735 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.735 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.735 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:47.735 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699571+0000", 2024-04-24T15:32:47.735 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0237999999999999e-05 2024-04-24T15:32:47.735 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.735 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.736 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:47.736 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699582+0000", 2024-04-24T15:32:47.736 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.1009000000000001e-05 2024-04-24T15:32:47.736 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.736 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.736 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:47.736 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699607+0000", 2024-04-24T15:32:47.736 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.5024999999999998e-05 2024-04-24T15:32:47.736 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.736 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.736 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:47.736 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699671+0000", 2024-04-24T15:32:47.736 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.3219999999999994e-05 2024-04-24T15:32:47.736 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.736 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.737 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:47.737 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.700952+0000", 2024-04-24T15:32:47.737 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.001281952 2024-04-24T15:32:47.737 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.737 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.737 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:47.737 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.705912+0000", 2024-04-24T15:32:47.737 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0049599889999999997 2024-04-24T15:32:47.738 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.738 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.738 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:47.738 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.708300+0000", 2024-04-24T15:32:47.738 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.002387934 2024-04-24T15:32:47.738 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.738 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.738 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:47.738 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.708345+0000", 2024-04-24T15:32:47.738 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4175000000000003e-05 2024-04-24T15:32:47.738 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.738 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.738 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:47.739 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.708380+0000", 2024-04-24T15:32:47.739 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5460999999999998e-05 2024-04-24T15:32:47.739 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:47.739 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:47.739 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:47.739 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.739 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.739 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:3770 4.13 4:cb3c841f:::rbd_data.114a1f60cb66.000000000000001e:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:47.739 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.699597+0000", 2024-04-24T15:32:47.739 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 23.004066999999999, 2024-04-24T15:32:47.739 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0083605450000000005, 2024-04-24T15:32:47.739 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:47.739 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:47.740 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:47.740 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:47.740 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:47.740 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 3770 2024-04-24T15:32:47.740 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.740 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:47.740 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.740 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:47.740 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699597+0000", 2024-04-24T15:32:47.740 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:47.740 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.740 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.740 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:47.740 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699597+0000", 2024-04-24T15:32:47.741 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:47.741 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.741 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.741 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:47.741 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699597+0000", 2024-04-24T15:32:47.741 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:47.741 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.741 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.741 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:47.741 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699600+0000", 2024-04-24T15:32:47.741 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.371e-06 2024-04-24T15:32:47.741 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.741 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.741 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:47.741 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699611+0000", 2024-04-24T15:32:47.742 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0552e-05 2024-04-24T15:32:47.742 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.742 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.742 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:47.742 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699615+0000", 2024-04-24T15:32:47.742 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5689999999999998e-06 2024-04-24T15:32:47.742 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.742 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.742 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:47.742 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699692+0000", 2024-04-24T15:32:47.742 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.6664999999999999e-05 2024-04-24T15:32:47.742 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.742 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.743 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:47.743 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699740+0000", 2024-04-24T15:32:47.743 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8032999999999999e-05 2024-04-24T15:32:47.743 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.743 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.743 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 6", 2024-04-24T15:32:47.743 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.700940+0000", 2024-04-24T15:32:47.743 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.001199978 2024-04-24T15:32:47.743 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.743 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.743 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:47.743 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.706397+0000", 2024-04-24T15:32:47.743 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0054569459999999998 2024-04-24T15:32:47.743 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.743 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.744 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:47.744 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.707870+0000", 2024-04-24T15:32:47.744 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.001473294 2024-04-24T15:32:47.744 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.744 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.744 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:47.744 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.707920+0000", 2024-04-24T15:32:47.744 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.994e-05 2024-04-24T15:32:47.744 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.744 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.744 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:47.744 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.707958+0000", 2024-04-24T15:32:47.744 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.7761e-05 2024-04-24T15:32:47.744 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:47.745 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:47.745 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:47.745 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.745 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.745 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:3773 4.e 4:7173e872:::rbd_data.114a1f60cb66.0000000000000021:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:47.745 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.699632+0000", 2024-04-24T15:32:47.745 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 23.004032436999999, 2024-04-24T15:32:47.745 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0068017260000000001, 2024-04-24T15:32:47.745 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:47.745 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:47.745 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:47.745 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:47.745 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:47.746 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 3773 2024-04-24T15:32:47.746 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.746 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:47.746 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.746 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:47.746 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699632+0000", 2024-04-24T15:32:47.746 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:47.746 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.746 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.746 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:47.746 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699632+0000", 2024-04-24T15:32:47.746 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:47.746 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.746 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.747 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:47.747 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699631+0000", 2024-04-24T15:32:47.747 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:47.747 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.747 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.747 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:47.747 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699639+0000", 2024-04-24T15:32:47.747 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7319999999999998e-06 2024-04-24T15:32:47.747 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.747 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.747 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:47.747 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699646+0000", 2024-04-24T15:32:47.747 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1330000000000001e-06 2024-04-24T15:32:47.748 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.748 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.748 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:47.748 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699651+0000", 2024-04-24T15:32:47.748 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.9200000000000003e-06 2024-04-24T15:32:47.748 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.748 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.748 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:47.748 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699689+0000", 2024-04-24T15:32:47.748 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.7979999999999999e-05 2024-04-24T15:32:47.748 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.748 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.748 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:47.748 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699759+0000", 2024-04-24T15:32:47.748 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.9468000000000006e-05 2024-04-24T15:32:47.749 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.749 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.749 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 6", 2024-04-24T15:32:47.749 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.701164+0000", 2024-04-24T15:32:47.749 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0014056629999999999 2024-04-24T15:32:47.749 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.749 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.749 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:47.749 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.705975+0000", 2024-04-24T15:32:47.749 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0048109340000000002 2024-04-24T15:32:47.749 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.749 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.749 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:47.749 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.706359+0000", 2024-04-24T15:32:47.750 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00038331699999999999 2024-04-24T15:32:47.750 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.750 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.750 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:47.750 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.706407+0000", 2024-04-24T15:32:47.750 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8544999999999997e-05 2024-04-24T15:32:47.750 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.750 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.750 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:47.750 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.706434+0000", 2024-04-24T15:32:47.750 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.6553000000000001e-05 2024-04-24T15:32:47.750 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:47.750 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:47.751 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:47.751 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.751 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.751 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:3794 4.18 4:1efc862d:::rbd_data.114a1f60cb66.0000000000000024:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:47.751 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.723446+0000", 2024-04-24T15:32:47.751 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 22.980217918000001, 2024-04-24T15:32:47.751 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.007572957, 2024-04-24T15:32:47.751 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:47.751 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:47.751 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:47.751 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:47.751 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:47.751 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 3794 2024-04-24T15:32:47.751 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.752 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:47.752 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.752 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:47.752 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723446+0000", 2024-04-24T15:32:47.752 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:47.752 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.752 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.752 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:47.752 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723446+0000", 2024-04-24T15:32:47.752 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:47.752 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.752 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.752 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:47.752 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723446+0000", 2024-04-24T15:32:47.753 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999999 2024-04-24T15:32:47.753 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.753 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.753 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:47.753 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723450+0000", 2024-04-24T15:32:47.753 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.1729999999999997e-06 2024-04-24T15:32:47.753 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.753 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.753 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:47.753 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723458+0000", 2024-04-24T15:32:47.753 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.653e-06 2024-04-24T15:32:47.753 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.753 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.753 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:47.754 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723465+0000", 2024-04-24T15:32:47.754 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.4450000000000001e-06 2024-04-24T15:32:47.754 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.754 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.754 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:47.754 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723501+0000", 2024-04-24T15:32:47.754 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5726000000000003e-05 2024-04-24T15:32:47.754 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.754 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.754 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:47.754 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723547+0000", 2024-04-24T15:32:47.754 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6479e-05 2024-04-24T15:32:47.754 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.754 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.755 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 6", 2024-04-24T15:32:47.755 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.724384+0000", 2024-04-24T15:32:47.755 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00083723800000000002 2024-04-24T15:32:47.755 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.755 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.755 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:47.755 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.730925+0000", 2024-04-24T15:32:47.755 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0065409179999999997 2024-04-24T15:32:47.755 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.755 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.755 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:47.755 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.730969+0000", 2024-04-24T15:32:47.755 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3869999999999998e-05 2024-04-24T15:32:47.755 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.756 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.756 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:47.756 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.730997+0000", 2024-04-24T15:32:47.756 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7783000000000001e-05 2024-04-24T15:32:47.756 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.756 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.756 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:47.756 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.731019+0000", 2024-04-24T15:32:47.756 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.2427999999999999e-05 2024-04-24T15:32:47.756 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:47.756 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:47.756 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:47.756 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.756 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.756 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:3797 4.18 4:1d8bdb81:::rbd_data.114a1f60cb66.0000000000000027:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:47.757 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.723479+0000", 2024-04-24T15:32:47.757 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 22.980184907999998, 2024-04-24T15:32:47.757 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.013378915, 2024-04-24T15:32:47.757 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:47.757 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:47.757 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:47.757 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:47.757 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:47.757 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 3797 2024-04-24T15:32:47.757 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.757 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:47.757 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.757 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:47.757 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723479+0000", 2024-04-24T15:32:47.757 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:47.758 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.758 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.758 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:47.758 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723479+0000", 2024-04-24T15:32:47.758 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:47.758 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.758 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.758 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:47.758 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723479+0000", 2024-04-24T15:32:47.758 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:47.758 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.758 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.758 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:47.758 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723482+0000", 2024-04-24T15:32:47.759 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5410000000000001e-06 2024-04-24T15:32:47.759 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.759 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.759 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:47.759 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723492+0000", 2024-04-24T15:32:47.759 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.6879999999999994e-06 2024-04-24T15:32:47.759 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.759 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.759 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:47.759 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723496+0000", 2024-04-24T15:32:47.759 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.34e-06 2024-04-24T15:32:47.759 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.759 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.759 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:47.760 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.728008+0000", 2024-04-24T15:32:47.760 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0045115049999999999 2024-04-24T15:32:47.760 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.760 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.760 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:47.760 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.728058+0000", 2024-04-24T15:32:47.760 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.0201000000000003e-05 2024-04-24T15:32:47.760 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.760 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.760 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 6", 2024-04-24T15:32:47.760 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.728499+0000", 2024-04-24T15:32:47.760 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00044129299999999999 2024-04-24T15:32:47.760 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.760 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.761 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:47.761 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.730962+0000", 2024-04-24T15:32:47.761 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0024628430000000002 2024-04-24T15:32:47.761 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.761 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.761 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:47.761 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.736810+0000", 2024-04-24T15:32:47.761 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0058474490000000002 2024-04-24T15:32:47.761 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.761 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.761 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:47.761 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.736837+0000", 2024-04-24T15:32:47.761 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7251999999999999e-05 2024-04-24T15:32:47.761 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.762 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.762 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:47.762 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.736858+0000", 2024-04-24T15:32:47.762 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.1387999999999999e-05 2024-04-24T15:32:47.762 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:47.762 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:47.762 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:47.762 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.762 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.762 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:3798 4.18 4:1e938922:::rbd_data.114a1f60cb66.0000000000000028:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:47.762 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.723506+0000", 2024-04-24T15:32:47.762 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 22.980158494000001, 2024-04-24T15:32:47.762 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.013398188, 2024-04-24T15:32:47.762 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:47.763 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:47.763 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:47.763 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:47.763 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:47.763 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 3798 2024-04-24T15:32:47.763 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.763 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:47.763 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.763 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:47.763 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723506+0000", 2024-04-24T15:32:47.763 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:47.763 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.763 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.763 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:47.764 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723506+0000", 2024-04-24T15:32:47.764 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:47.764 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.764 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.764 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:47.764 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723505+0000", 2024-04-24T15:32:47.764 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:47.764 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.764 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.764 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:47.764 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723509+0000", 2024-04-24T15:32:47.764 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.2349999999999999e-06 2024-04-24T15:32:47.764 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.764 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.765 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:47.765 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723515+0000", 2024-04-24T15:32:47.765 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.3239999999999999e-06 2024-04-24T15:32:47.765 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.765 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.765 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:47.765 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723519+0000", 2024-04-24T15:32:47.765 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8580000000000002e-06 2024-04-24T15:32:47.765 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.765 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.765 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:47.765 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.731045+0000", 2024-04-24T15:32:47.765 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0075259309999999996 2024-04-24T15:32:47.765 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.765 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.766 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:47.766 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.731090+0000", 2024-04-24T15:32:47.766 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4805999999999998e-05 2024-04-24T15:32:47.766 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.766 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.766 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 6", 2024-04-24T15:32:47.766 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.731571+0000", 2024-04-24T15:32:47.766 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00048136400000000002 2024-04-24T15:32:47.766 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.766 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.766 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:47.766 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.736784+0000", 2024-04-24T15:32:47.766 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0052135269999999999 2024-04-24T15:32:47.767 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.767 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.767 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:47.767 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.736865+0000", 2024-04-24T15:32:47.767 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0141000000000004e-05 2024-04-24T15:32:47.767 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.767 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.767 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:47.767 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.736887+0000", 2024-04-24T15:32:47.767 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.2194000000000001e-05 2024-04-24T15:32:47.767 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.767 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.767 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:47.767 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.736904+0000", 2024-04-24T15:32:47.767 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.7302000000000001e-05 2024-04-24T15:32:47.768 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:47.768 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:47.768 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:47.768 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.768 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.768 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:3801 4.18 4:1b4f38fa:::rbd_data.114a1f60cb66.000000000000002b:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:47.768 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.723629+0000", 2024-04-24T15:32:47.768 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 22.980035350000001, 2024-04-24T15:32:47.768 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.013474534, 2024-04-24T15:32:47.768 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:47.768 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:47.768 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:47.768 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:47.768 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:47.768 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 3801 2024-04-24T15:32:47.769 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.769 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:47.769 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.769 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:47.769 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723629+0000", 2024-04-24T15:32:47.769 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:47.769 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.769 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.769 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:47.769 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723629+0000", 2024-04-24T15:32:47.769 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:47.769 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.769 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.769 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:47.770 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723628+0000", 2024-04-24T15:32:47.770 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:47.770 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.770 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.770 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:47.770 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723632+0000", 2024-04-24T15:32:47.770 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.4690000000000002e-06 2024-04-24T15:32:47.770 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.770 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.770 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:47.770 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723640+0000", 2024-04-24T15:32:47.770 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3990000000000004e-06 2024-04-24T15:32:47.770 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.771 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.771 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:47.771 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723645+0000", 2024-04-24T15:32:47.771 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3699999999999997e-06 2024-04-24T15:32:47.771 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.771 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.771 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:47.771 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.733996+0000", 2024-04-24T15:32:47.771 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.010351111 2024-04-24T15:32:47.771 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.771 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.771 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:47.771 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.734045+0000", 2024-04-24T15:32:47.771 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.9166000000000003e-05 2024-04-24T15:32:47.772 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.772 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.772 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 6", 2024-04-24T15:32:47.772 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.734461+0000", 2024-04-24T15:32:47.772 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00041584 2024-04-24T15:32:47.772 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.772 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.772 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:47.772 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.736806+0000", 2024-04-24T15:32:47.772 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.002345232 2024-04-24T15:32:47.772 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.772 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.772 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:47.772 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.737065+0000", 2024-04-24T15:32:47.773 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00025913100000000001 2024-04-24T15:32:47.773 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.773 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.773 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:47.773 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.737086+0000", 2024-04-24T15:32:47.773 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.1180000000000001e-05 2024-04-24T15:32:47.773 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.773 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.773 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:47.773 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.737104+0000", 2024-04-24T15:32:47.773 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.7238999999999999e-05 2024-04-24T15:32:47.773 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:47.773 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:47.774 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:47.774 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.774 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.774 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:3896 4.1b 4:dd750b71:::rbd_data.114a1f60cb66.0000000000000058:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:47.774 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.797438+0000", 2024-04-24T15:32:47.774 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 22.906226125, 2024-04-24T15:32:47.774 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0070008420000000002, 2024-04-24T15:32:47.774 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:47.774 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:47.774 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:47.774 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:47.774 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:47.774 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 3896 2024-04-24T15:32:47.775 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.775 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:47.775 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.775 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:47.775 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797438+0000", 2024-04-24T15:32:47.775 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:47.775 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.775 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.775 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:47.775 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797438+0000", 2024-04-24T15:32:47.775 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:47.775 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.775 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.775 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:47.776 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797438+0000", 2024-04-24T15:32:47.776 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:47.776 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.776 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.776 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:47.776 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797441+0000", 2024-04-24T15:32:47.776 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8779999999999998e-06 2024-04-24T15:32:47.776 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.776 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.776 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:47.776 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797447+0000", 2024-04-24T15:32:47.776 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.6989999999999997e-06 2024-04-24T15:32:47.776 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.776 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.777 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:47.777 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797451+0000", 2024-04-24T15:32:47.777 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9689999999999996e-06 2024-04-24T15:32:47.777 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.777 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.777 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:47.777 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797915+0000", 2024-04-24T15:32:47.777 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00046327400000000001 2024-04-24T15:32:47.777 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.777 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.777 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:47.777 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797993+0000", 2024-04-24T15:32:47.777 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.8584000000000005e-05 2024-04-24T15:32:47.777 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.778 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.778 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:47.778 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.798090+0000", 2024-04-24T15:32:47.778 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.6638000000000005e-05 2024-04-24T15:32:47.778 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.778 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.778 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:47.778 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.799021+0000", 2024-04-24T15:32:47.778 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00093112299999999996 2024-04-24T15:32:47.778 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.778 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.778 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:47.778 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.804372+0000", 2024-04-24T15:32:47.778 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0053506789999999997 2024-04-24T15:32:47.779 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.779 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.779 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:47.779 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.804405+0000", 2024-04-24T15:32:47.779 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.3015999999999997e-05 2024-04-24T15:32:47.779 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.779 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.779 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:47.779 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.804439+0000", 2024-04-24T15:32:47.779 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.4431000000000003e-05 2024-04-24T15:32:47.779 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:47.779 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:47.779 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:47.779 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.779 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.780 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:3900 4.1a 4:5b94e8b4:::rbd_data.114a1f60cb66.000000000000006e:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:47.780 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.797456+0000", 2024-04-24T15:32:47.780 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 22.906208424999999, 2024-04-24T15:32:47.780 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.006973524, 2024-04-24T15:32:47.780 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:47.780 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:47.780 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:47.780 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:47.780 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:47.780 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 3900 2024-04-24T15:32:47.780 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.780 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:47.780 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.780 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:47.781 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797456+0000", 2024-04-24T15:32:47.781 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:47.781 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.781 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.781 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:47.781 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797456+0000", 2024-04-24T15:32:47.781 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:47.781 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.781 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.781 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:47.781 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797456+0000", 2024-04-24T15:32:47.781 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:47.781 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.781 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.782 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:47.782 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797458+0000", 2024-04-24T15:32:47.782 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.3209999999999999e-06 2024-04-24T15:32:47.782 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.782 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.782 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:47.782 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797464+0000", 2024-04-24T15:32:47.782 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.7579999999999998e-06 2024-04-24T15:32:47.782 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.782 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.782 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:47.782 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797467+0000", 2024-04-24T15:32:47.782 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.585e-06 2024-04-24T15:32:47.782 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.783 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.783 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:47.783 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797504+0000", 2024-04-24T15:32:47.783 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.7120999999999999e-05 2024-04-24T15:32:47.783 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.783 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.783 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:47.783 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797578+0000", 2024-04-24T15:32:47.783 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.3693000000000001e-05 2024-04-24T15:32:47.783 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.783 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.783 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:47.783 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797675+0000", 2024-04-24T15:32:47.783 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.7454000000000006e-05 2024-04-24T15:32:47.783 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.784 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.784 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:47.784 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.799033+0000", 2024-04-24T15:32:47.784 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.001357104 2024-04-24T15:32:47.784 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.784 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.784 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:47.784 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.804357+0000", 2024-04-24T15:32:47.784 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0053247490000000001 2024-04-24T15:32:47.784 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.784 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.784 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:47.784 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.804397+0000", 2024-04-24T15:32:47.784 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9573000000000002e-05 2024-04-24T15:32:47.785 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.785 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.785 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:47.785 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.804429+0000", 2024-04-24T15:32:47.785 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.2529e-05 2024-04-24T15:32:47.785 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:47.785 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:47.785 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:47.785 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.785 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.785 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:3923 4.9 4:94be6b6c:::rbd_data.114a1f60cb66.0000000000000097:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:47.785 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.806292+0000", 2024-04-24T15:32:47.785 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 22.897372563000001, 2024-04-24T15:32:47.785 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0062496729999999999, 2024-04-24T15:32:47.785 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:47.786 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:47.786 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:47.786 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:47.786 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:47.786 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 3923 2024-04-24T15:32:47.786 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.786 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:47.786 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.786 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:47.786 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.806292+0000", 2024-04-24T15:32:47.786 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:47.786 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.786 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.787 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:47.787 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.806292+0000", 2024-04-24T15:32:47.787 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:47.787 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.787 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.787 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:47.787 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.806291+0000", 2024-04-24T15:32:47.787 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999999 2024-04-24T15:32:47.787 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.787 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.787 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:47.787 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.806296+0000", 2024-04-24T15:32:47.787 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5730000000000001e-06 2024-04-24T15:32:47.787 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.788 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.788 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:47.788 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.806306+0000", 2024-04-24T15:32:47.788 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0409e-05 2024-04-24T15:32:47.788 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.788 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.788 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:47.788 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.806311+0000", 2024-04-24T15:32:47.788 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.7129999999999996e-06 2024-04-24T15:32:47.788 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.788 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.788 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:47.788 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.806350+0000", 2024-04-24T15:32:47.788 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9351999999999999e-05 2024-04-24T15:32:47.789 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.789 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.789 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:47.789 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.806431+0000", 2024-04-24T15:32:47.789 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.1050000000000005e-05 2024-04-24T15:32:47.789 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.789 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.789 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:47.789 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.806539+0000", 2024-04-24T15:32:47.789 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000107723 2024-04-24T15:32:47.789 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.789 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.789 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:47.789 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.809607+0000", 2024-04-24T15:32:47.790 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.003068468 2024-04-24T15:32:47.790 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.790 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.790 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:47.790 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.812493+0000", 2024-04-24T15:32:47.790 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0028862689999999999 2024-04-24T15:32:47.790 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.790 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.790 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:47.790 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.812521+0000", 2024-04-24T15:32:47.790 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7226e-05 2024-04-24T15:32:47.790 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.790 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.790 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:47.791 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.812541+0000", 2024-04-24T15:32:47.791 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.0729000000000001e-05 2024-04-24T15:32:47.791 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:47.791 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:47.791 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:47.791 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.791 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.791 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:3975 4.1b 4:d9227084:::rbd_data.114a1f60cb66.000000000000015f:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:47.791 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.827427+0000", 2024-04-24T15:32:47.791 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 22.876237118999999, 2024-04-24T15:32:47.791 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0061324140000000001, 2024-04-24T15:32:47.791 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:47.791 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:47.791 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:47.792 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:47.792 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:47.792 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 3975 2024-04-24T15:32:47.792 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.792 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:47.792 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.792 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:47.792 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.827427+0000", 2024-04-24T15:32:47.792 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:47.792 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.792 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.793 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:47.793 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.827427+0000", 2024-04-24T15:32:47.793 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:47.793 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.793 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.793 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:47.793 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.827427+0000", 2024-04-24T15:32:47.793 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:47.793 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.793 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.793 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:47.793 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.827431+0000", 2024-04-24T15:32:47.793 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2509999999999996e-06 2024-04-24T15:32:47.794 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.794 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.794 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:47.794 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.827440+0000", 2024-04-24T15:32:47.794 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.816e-06 2024-04-24T15:32:47.794 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.794 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.794 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:47.794 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.827445+0000", 2024-04-24T15:32:47.794 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.2059999999999998e-06 2024-04-24T15:32:47.794 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.794 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.794 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:47.794 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.827872+0000", 2024-04-24T15:32:47.794 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00042740299999999997 2024-04-24T15:32:47.795 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.795 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.795 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:47.795 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.827941+0000", 2024-04-24T15:32:47.795 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.8652000000000006e-05 2024-04-24T15:32:47.795 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.795 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.795 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:47.795 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.828048+0000", 2024-04-24T15:32:47.795 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000106941 2024-04-24T15:32:47.795 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.795 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.795 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:47.795 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.831351+0000", 2024-04-24T15:32:47.796 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0033031010000000001 2024-04-24T15:32:47.796 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.796 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.796 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:47.796 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.833495+0000", 2024-04-24T15:32:47.796 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0021436379999999998 2024-04-24T15:32:47.796 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.796 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.796 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:47.796 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.833526+0000", 2024-04-24T15:32:47.796 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.1436999999999997e-05 2024-04-24T15:32:47.796 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.796 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.796 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:47.797 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.833560+0000", 2024-04-24T15:32:47.797 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.3560999999999999e-05 2024-04-24T15:32:47.797 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:47.797 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:47.797 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:47.797 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.797 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.797 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:4010 4.1 4:813d513a:::rbd_data.114a1f60cb66.00000000000001c1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:47.797 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.845178+0000", 2024-04-24T15:32:47.797 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 22.858486770999999, 2024-04-24T15:32:47.797 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.007702108, 2024-04-24T15:32:47.797 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:47.797 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:47.797 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:47.797 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:47.798 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:47.798 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 4010 2024-04-24T15:32:47.798 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.798 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:47.798 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.798 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:47.798 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.845178+0000", 2024-04-24T15:32:47.798 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:47.798 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.798 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.798 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:47.798 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.845178+0000", 2024-04-24T15:32:47.798 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:47.798 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.799 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.799 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:47.799 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.845177+0000", 2024-04-24T15:32:47.799 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:47.799 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.799 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.799 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:47.799 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.845180+0000", 2024-04-24T15:32:47.799 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8119999999999999e-06 2024-04-24T15:32:47.799 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.799 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.799 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:47.799 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.845186+0000", 2024-04-24T15:32:47.799 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.9649999999999996e-06 2024-04-24T15:32:47.799 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.800 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.800 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:47.800 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.845189+0000", 2024-04-24T15:32:47.800 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.6040000000000001e-06 2024-04-24T15:32:47.800 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.800 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.800 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:47.800 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.845220+0000", 2024-04-24T15:32:47.800 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.0207000000000001e-05 2024-04-24T15:32:47.800 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.800 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.800 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:47.800 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.845262+0000", 2024-04-24T15:32:47.801 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2716000000000002e-05 2024-04-24T15:32:47.801 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.801 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.801 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 5", 2024-04-24T15:32:47.801 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.845326+0000", 2024-04-24T15:32:47.801 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.3490000000000004e-05 2024-04-24T15:32:47.801 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.801 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.801 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:47.801 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.849620+0000", 2024-04-24T15:32:47.801 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0042944769999999997 2024-04-24T15:32:47.801 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.801 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.801 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:47.801 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.852802+0000", 2024-04-24T15:32:47.802 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.003181798 2024-04-24T15:32:47.802 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.802 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.802 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:47.802 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.852846+0000", 2024-04-24T15:32:47.802 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3606000000000002e-05 2024-04-24T15:32:47.802 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.802 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.802 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:47.802 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.852880+0000", 2024-04-24T15:32:47.802 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.3908999999999997e-05 2024-04-24T15:32:47.802 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:47.802 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:47.803 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:47.803 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.803 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.803 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:4256 4.13 4:c8361c81:::rbd_data.114a1f60cb66.0000000000000316:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:47.803 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:25.054726+0000", 2024-04-24T15:32:47.803 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 22.648938327, 2024-04-24T15:32:47.803 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0079439720000000005, 2024-04-24T15:32:47.803 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:47.803 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:47.803 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:47.803 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:47.803 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:47.803 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 4256 2024-04-24T15:32:47.803 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.804 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:47.804 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.804 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:47.804 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.054726+0000", 2024-04-24T15:32:47.804 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:47.804 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.804 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.804 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:47.804 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.054726+0000", 2024-04-24T15:32:47.804 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:47.804 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.804 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.804 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:47.804 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.054725+0000", 2024-04-24T15:32:47.805 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999999 2024-04-24T15:32:47.805 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.805 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.805 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:47.805 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.054730+0000", 2024-04-24T15:32:47.805 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.4729999999999999e-06 2024-04-24T15:32:47.805 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.805 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.805 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:47.805 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.054742+0000", 2024-04-24T15:32:47.805 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.1489e-05 2024-04-24T15:32:47.805 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.805 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.805 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:47.805 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.054748+0000", 2024-04-24T15:32:47.806 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.879e-06 2024-04-24T15:32:47.806 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.806 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.806 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:47.806 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.054791+0000", 2024-04-24T15:32:47.806 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2673999999999998e-05 2024-04-24T15:32:47.806 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.806 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.806 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:47.806 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.054872+0000", 2024-04-24T15:32:47.806 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.1254999999999994e-05 2024-04-24T15:32:47.806 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.806 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.807 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 6", 2024-04-24T15:32:47.807 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.054978+0000", 2024-04-24T15:32:47.807 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00010624 2024-04-24T15:32:47.807 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.807 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.807 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:47.807 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.055893+0000", 2024-04-24T15:32:47.807 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00091453300000000003 2024-04-24T15:32:47.807 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.807 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.807 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:47.807 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.062620+0000", 2024-04-24T15:32:47.807 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0067271839999999998 2024-04-24T15:32:47.807 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.807 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.808 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:47.808 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.062645+0000", 2024-04-24T15:32:47.808 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.5491e-05 2024-04-24T15:32:47.808 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.808 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.808 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:47.808 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.062670+0000", 2024-04-24T15:32:47.808 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.474e-05 2024-04-24T15:32:47.808 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:47.808 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:47.808 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:47.808 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.808 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.808 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:7398 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-04-24T15:32:47.808 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:27.771937+0000", 2024-04-24T15:32:47.809 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 19.931726976, 2024-04-24T15:32:47.809 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.015783228, 2024-04-24T15:32:47.809 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:47.809 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:47.809 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:47.809 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:47.809 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:47.809 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 7398 2024-04-24T15:32:47.809 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.809 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:47.809 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.809 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:47.809 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:27.771937+0000", 2024-04-24T15:32:47.809 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:47.809 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.810 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.810 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:47.810 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:27.771937+0000", 2024-04-24T15:32:47.810 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:47.810 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.810 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.810 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:47.810 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:27.771936+0000", 2024-04-24T15:32:47.810 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999986 2024-04-24T15:32:47.810 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.810 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.810 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:47.810 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:27.771944+0000", 2024-04-24T15:32:47.810 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.8960000000000003e-06 2024-04-24T15:32:47.811 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.811 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.811 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:47.811 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:27.771955+0000", 2024-04-24T15:32:47.811 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0951000000000001e-05 2024-04-24T15:32:47.811 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.811 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.811 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:47.811 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:27.771959+0000", 2024-04-24T15:32:47.811 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.69e-06 2024-04-24T15:32:47.811 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.811 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.811 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:47.811 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:27.771998+0000", 2024-04-24T15:32:47.811 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8705000000000003e-05 2024-04-24T15:32:47.811 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.812 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.812 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:47.812 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:27.772054+0000", 2024-04-24T15:32:47.812 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.5420000000000001e-05 2024-04-24T15:32:47.812 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.812 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.812 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:47.812 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:27.774882+0000", 2024-04-24T15:32:47.812 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.002828344 2024-04-24T15:32:47.812 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.812 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.812 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:47.812 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:27.787163+0000", 2024-04-24T15:32:47.812 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.012281556000000001 2024-04-24T15:32:47.812 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.813 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.813 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:47.813 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:27.787651+0000", 2024-04-24T15:32:47.813 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00048708600000000002 2024-04-24T15:32:47.813 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.813 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.813 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:47.813 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:27.787691+0000", 2024-04-24T15:32:47.813 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0531999999999998e-05 2024-04-24T15:32:47.813 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.813 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.813 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:47.813 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:27.787721+0000", 2024-04-24T15:32:47.814 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.9518000000000001e-05 2024-04-24T15:32:47.814 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:47.814 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:47.814 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:47.814 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.814 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.814 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:7825 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-04-24T15:32:47.814 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:28.311523+0000", 2024-04-24T15:32:47.814 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 19.392140916999999, 2024-04-24T15:32:47.814 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.019381513, 2024-04-24T15:32:47.814 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:47.814 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:47.814 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:47.814 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:47.815 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:47.815 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 7825 2024-04-24T15:32:47.815 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.815 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:47.815 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.815 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:47.815 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:28.311523+0000", 2024-04-24T15:32:47.815 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:47.815 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.815 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.815 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:47.815 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:28.311523+0000", 2024-04-24T15:32:47.815 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:47.815 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.816 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.816 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:47.816 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:28.311523+0000", 2024-04-24T15:32:47.816 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:47.816 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.816 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.816 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:47.816 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:28.311527+0000", 2024-04-24T15:32:47.816 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6290000000000001e-06 2024-04-24T15:32:47.816 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.816 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.816 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:47.816 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:28.311538+0000", 2024-04-24T15:32:47.816 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0239e-05 2024-04-24T15:32:47.817 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.817 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.817 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:47.817 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:28.311542+0000", 2024-04-24T15:32:47.817 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3059999999999999e-06 2024-04-24T15:32:47.817 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.817 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.817 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:47.817 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:28.311621+0000", 2024-04-24T15:32:47.817 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.8671999999999995e-05 2024-04-24T15:32:47.817 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.817 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.817 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:47.817 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:28.311667+0000", 2024-04-24T15:32:47.818 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6842999999999999e-05 2024-04-24T15:32:47.818 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.818 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.818 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:47.818 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:28.315077+0000", 2024-04-24T15:32:47.818 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0034100889999999998 2024-04-24T15:32:47.818 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.818 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.818 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:47.818 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:28.330557+0000", 2024-04-24T15:32:47.818 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.015479100000000001 2024-04-24T15:32:47.818 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.818 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.818 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:47.818 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:28.330843+0000", 2024-04-24T15:32:47.819 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000286526 2024-04-24T15:32:47.819 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.819 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.819 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:47.819 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:28.330877+0000", 2024-04-24T15:32:47.819 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.4119999999999999e-05 2024-04-24T15:32:47.819 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:47.819 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:47.819 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:47.819 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:28.330905+0000", 2024-04-24T15:32:47.819 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7688999999999999e-05 2024-04-24T15:32:47.819 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:47.819 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:47.819 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:47.820 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:47.820 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:47.820 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:32:47.821 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-24T15:32:47.822 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:32:47.821+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:32:47.922 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:32:47.923+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:32:47.946 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:32:47.946 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:32:47.946 INFO:teuthology.orchestra.run.smithi151.stdout: "num_ops": 0 2024-04-24T15:32:47.946 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:32:47.954 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-24T15:32:48.023 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:32:48.024+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:32:48.077 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:32:48.077 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:32:48.077 INFO:teuthology.orchestra.run.smithi151.stdout: "complaint_time": 30, 2024-04-24T15:32:48.077 INFO:teuthology.orchestra.run.smithi151.stdout: "num_blocked_ops": 0 2024-04-24T15:32:48.077 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:32:48.087 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-24T15:32:48.124 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:32:48.124+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:32:48.211 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:32:48.212 INFO:teuthology.orchestra.run.smithi151.stdout: "size": 20, 2024-04-24T15:32:48.212 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 600, 2024-04-24T15:32:48.212 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [ 2024-04-24T15:32:48.212 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.212 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23613 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 14688256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:48.212 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.792522+0000", 2024-04-24T15:32:48.212 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 13.418926892, 2024-04-24T15:32:48.212 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040507356000000001, 2024-04-24T15:32:48.212 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:48.212 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:48.212 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:48.212 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:48.213 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:48.213 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23613 2024-04-24T15:32:48.213 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.213 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:48.213 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.213 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:48.213 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792522+0000", 2024-04-24T15:32:48.213 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.213 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.213 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.213 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:48.213 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792522+0000", 2024-04-24T15:32:48.213 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.213 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.214 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.214 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:48.214 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792522+0000", 2024-04-24T15:32:48.214 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:48.214 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.214 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.214 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:48.214 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792523+0000", 2024-04-24T15:32:48.214 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.576e-06 2024-04-24T15:32:48.214 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.214 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.214 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:48.214 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792526+0000", 2024-04-24T15:32:48.214 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7439999999999999e-06 2024-04-24T15:32:48.215 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.215 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.215 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:48.215 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792528+0000", 2024-04-24T15:32:48.215 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.0600000000000002e-06 2024-04-24T15:32:48.215 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.215 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.215 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:48.215 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.832864+0000", 2024-04-24T15:32:48.215 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040335742000000001 2024-04-24T15:32:48.215 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.215 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.215 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:48.215 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.832953+0000", 2024-04-24T15:32:48.216 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.9281000000000001e-05 2024-04-24T15:32:48.216 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.216 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.216 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:48.216 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.833029+0000", 2024-04-24T15:32:48.216 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.6259000000000003e-05 2024-04-24T15:32:48.216 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.216 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:48.216 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.216 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.216 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.216 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23614 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 14729216~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:48.216 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.792530+0000", 2024-04-24T15:32:48.216 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 13.418918628, 2024-04-24T15:32:48.216 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040633957999999998, 2024-04-24T15:32:48.216 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:48.217 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:48.217 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:48.217 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:48.217 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:48.217 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23614 2024-04-24T15:32:48.217 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.217 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:48.217 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.217 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:48.217 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792530+0000", 2024-04-24T15:32:48.217 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.217 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.217 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.217 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:48.217 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792530+0000", 2024-04-24T15:32:48.218 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.218 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.218 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.218 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:48.218 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792530+0000", 2024-04-24T15:32:48.218 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:32:48.218 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.218 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.218 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:48.218 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792531+0000", 2024-04-24T15:32:48.218 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.46e-06 2024-04-24T15:32:48.218 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.218 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.218 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:48.218 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792534+0000", 2024-04-24T15:32:48.219 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8959999999999999e-06 2024-04-24T15:32:48.219 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.219 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.219 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:48.219 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792536+0000", 2024-04-24T15:32:48.219 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.841e-06 2024-04-24T15:32:48.219 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.219 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.219 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:48.219 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.833060+0000", 2024-04-24T15:32:48.219 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040524125000000001 2024-04-24T15:32:48.219 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.219 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.219 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:48.220 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.833108+0000", 2024-04-24T15:32:48.220 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.7547999999999999e-05 2024-04-24T15:32:48.220 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.220 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.220 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:48.220 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.833164+0000", 2024-04-24T15:32:48.220 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6297e-05 2024-04-24T15:32:48.220 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.220 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:48.220 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.220 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.220 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.220 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23623 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15097856~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:48.220 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.793887+0000", 2024-04-24T15:32:48.220 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 13.417561666999999, 2024-04-24T15:32:48.221 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040432784999999999, 2024-04-24T15:32:48.221 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:48.221 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:48.221 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:48.221 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:48.221 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:48.221 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23623 2024-04-24T15:32:48.221 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.221 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:48.221 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.221 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:48.221 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793887+0000", 2024-04-24T15:32:48.221 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.221 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.221 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.221 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:48.222 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793887+0000", 2024-04-24T15:32:48.222 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.222 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.222 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.222 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:48.222 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793887+0000", 2024-04-24T15:32:48.222 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:32:48.222 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.222 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.222 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:48.222 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793888+0000", 2024-04-24T15:32:48.222 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3659999999999999e-06 2024-04-24T15:32:48.222 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.222 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.223 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:48.223 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793891+0000", 2024-04-24T15:32:48.223 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8389999999999998e-06 2024-04-24T15:32:48.223 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.223 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.223 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:48.223 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793893+0000", 2024-04-24T15:32:48.223 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.9549999999999999e-06 2024-04-24T15:32:48.223 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.223 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.223 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:48.223 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834239+0000", 2024-04-24T15:32:48.223 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040345487999999999 2024-04-24T15:32:48.223 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.224 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.224 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:48.224 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834280+0000", 2024-04-24T15:32:48.224 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.1878000000000001e-05 2024-04-24T15:32:48.224 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.224 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.224 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:48.224 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834320+0000", 2024-04-24T15:32:48.224 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9480000000000001e-05 2024-04-24T15:32:48.224 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.224 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:48.224 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.224 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.224 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.225 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23624 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15138816~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:48.225 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.794053+0000", 2024-04-24T15:32:48.225 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 13.417395719, 2024-04-24T15:32:48.225 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040370801999999997, 2024-04-24T15:32:48.225 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:48.225 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:48.225 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:48.225 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:48.225 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:48.225 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23624 2024-04-24T15:32:48.225 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.225 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:48.225 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.226 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:48.226 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794053+0000", 2024-04-24T15:32:48.226 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.226 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.226 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.226 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:48.226 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794053+0000", 2024-04-24T15:32:48.226 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.226 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.226 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.226 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:48.226 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793895+0000", 2024-04-24T15:32:48.226 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998422 2024-04-24T15:32:48.226 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.227 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.227 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:48.227 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794057+0000", 2024-04-24T15:32:48.227 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00016149400000000001 2024-04-24T15:32:48.227 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.227 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.227 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:48.227 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794065+0000", 2024-04-24T15:32:48.227 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.5259999999999993e-06 2024-04-24T15:32:48.227 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.227 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.227 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:48.227 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794071+0000", 2024-04-24T15:32:48.227 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6149999999999996e-06 2024-04-24T15:32:48.228 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.228 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.228 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:48.228 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834341+0000", 2024-04-24T15:32:48.228 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040270366000000002 2024-04-24T15:32:48.228 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.228 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.228 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:48.228 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834381+0000", 2024-04-24T15:32:48.228 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0102999999999999e-05 2024-04-24T15:32:48.228 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.228 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.228 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:48.228 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834424+0000", 2024-04-24T15:32:48.228 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2713000000000003e-05 2024-04-24T15:32:48.229 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.229 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:48.229 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.229 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.229 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.229 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23625 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15179776~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:48.229 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.794160+0000", 2024-04-24T15:32:48.229 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 13.417288836999999, 2024-04-24T15:32:48.229 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040413004000000002, 2024-04-24T15:32:48.229 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:48.229 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:48.229 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:48.229 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:48.229 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:48.229 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23625 2024-04-24T15:32:48.230 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.230 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:48.230 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.230 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:48.230 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794160+0000", 2024-04-24T15:32:48.230 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.230 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.230 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.230 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:48.230 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794160+0000", 2024-04-24T15:32:48.230 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.230 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.230 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.230 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:48.230 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794086+0000", 2024-04-24T15:32:48.231 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999256 2024-04-24T15:32:48.231 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.231 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.231 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:48.231 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794163+0000", 2024-04-24T15:32:48.231 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7896000000000001e-05 2024-04-24T15:32:48.231 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.231 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.231 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:48.231 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794171+0000", 2024-04-24T15:32:48.231 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.9500000000000001e-06 2024-04-24T15:32:48.231 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.231 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.231 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:48.231 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794179+0000", 2024-04-24T15:32:48.232 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7970000000000001e-06 2024-04-24T15:32:48.232 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.232 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.232 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:48.232 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834451+0000", 2024-04-24T15:32:48.232 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040271492999999998 2024-04-24T15:32:48.232 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.232 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.232 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:48.232 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834519+0000", 2024-04-24T15:32:48.232 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.7984000000000005e-05 2024-04-24T15:32:48.233 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.233 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.233 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:48.233 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834573+0000", 2024-04-24T15:32:48.233 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.4339999999999998e-05 2024-04-24T15:32:48.233 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.233 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:48.233 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.233 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.234 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.234 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23629 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15343616~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:48.234 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.794748+0000", 2024-04-24T15:32:48.234 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 13.416701034999999, 2024-04-24T15:32:48.234 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040419718, 2024-04-24T15:32:48.234 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:48.234 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:48.234 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:48.234 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:48.234 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:48.234 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23629 2024-04-24T15:32:48.234 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.234 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:48.234 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.235 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:48.235 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794748+0000", 2024-04-24T15:32:48.235 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.235 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.235 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.235 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:48.235 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794748+0000", 2024-04-24T15:32:48.235 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.235 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.235 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.235 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:48.235 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794673+0000", 2024-04-24T15:32:48.235 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999251 2024-04-24T15:32:48.235 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.235 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.236 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:48.236 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794751+0000", 2024-04-24T15:32:48.236 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.8480999999999996e-05 2024-04-24T15:32:48.236 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.236 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.236 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:48.236 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794760+0000", 2024-04-24T15:32:48.236 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3429999999999996e-06 2024-04-24T15:32:48.236 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.236 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.236 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:48.236 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794764+0000", 2024-04-24T15:32:48.236 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5600000000000004e-06 2024-04-24T15:32:48.236 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.237 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.237 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:48.237 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835040+0000", 2024-04-24T15:32:48.237 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040275188000000003 2024-04-24T15:32:48.237 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.237 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.237 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:48.237 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835105+0000", 2024-04-24T15:32:48.237 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.5992000000000001e-05 2024-04-24T15:32:48.237 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.237 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.237 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:48.237 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835167+0000", 2024-04-24T15:32:48.237 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.1981000000000002e-05 2024-04-24T15:32:48.237 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.238 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:48.238 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.238 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.238 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.238 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23630 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15384576~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:48.238 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.794839+0000", 2024-04-24T15:32:48.238 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 13.416609965999999, 2024-04-24T15:32:48.238 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040523894999999997, 2024-04-24T15:32:48.238 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:48.238 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:48.238 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:48.238 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:48.238 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:48.238 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23630 2024-04-24T15:32:48.239 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.239 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:48.239 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.239 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:48.239 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794839+0000", 2024-04-24T15:32:48.239 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.239 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.239 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.239 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:48.239 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794839+0000", 2024-04-24T15:32:48.239 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.239 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.239 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.240 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:48.240 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794768+0000", 2024-04-24T15:32:48.240 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999294 2024-04-24T15:32:48.240 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.240 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.240 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:48.240 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794841+0000", 2024-04-24T15:32:48.240 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2793000000000006e-05 2024-04-24T15:32:48.240 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.240 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.240 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:48.240 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794846+0000", 2024-04-24T15:32:48.240 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.7230000000000004e-06 2024-04-24T15:32:48.240 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.241 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.241 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:48.241 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794848+0000", 2024-04-24T15:32:48.241 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7659999999999999e-06 2024-04-24T15:32:48.241 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.241 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.241 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:48.241 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835223+0000", 2024-04-24T15:32:48.241 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040374275000000001 2024-04-24T15:32:48.241 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.241 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.241 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:48.241 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835295+0000", 2024-04-24T15:32:48.241 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2633999999999996e-05 2024-04-24T15:32:48.241 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.242 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.242 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:48.242 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835363+0000", 2024-04-24T15:32:48.242 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.7355999999999997e-05 2024-04-24T15:32:48.242 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.242 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:48.242 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.242 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.242 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.242 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23631 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15425536~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:48.242 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.794956+0000", 2024-04-24T15:32:48.242 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 13.416493125000001, 2024-04-24T15:32:48.242 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040562383, 2024-04-24T15:32:48.243 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:48.243 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:48.243 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:48.243 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:48.243 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:48.243 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23631 2024-04-24T15:32:48.243 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.243 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:48.243 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.243 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:48.243 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794956+0000", 2024-04-24T15:32:48.243 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.243 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.243 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.243 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:48.244 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794956+0000", 2024-04-24T15:32:48.244 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.244 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.244 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.244 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:48.244 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794859+0000", 2024-04-24T15:32:48.244 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999032 2024-04-24T15:32:48.244 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.244 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.244 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:48.244 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794959+0000", 2024-04-24T15:32:48.244 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000100207 2024-04-24T15:32:48.244 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.244 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.245 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:48.245 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794967+0000", 2024-04-24T15:32:48.245 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7640000000000005e-06 2024-04-24T15:32:48.245 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.245 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.245 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:48.245 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794971+0000", 2024-04-24T15:32:48.245 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.335e-06 2024-04-24T15:32:48.245 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.245 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.245 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:48.245 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835388+0000", 2024-04-24T15:32:48.245 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040417340000000003 2024-04-24T15:32:48.245 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.246 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.246 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:48.246 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835459+0000", 2024-04-24T15:32:48.246 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.0118999999999997e-05 2024-04-24T15:32:48.246 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.246 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.246 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:48.246 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835518+0000", 2024-04-24T15:32:48.246 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.9438999999999998e-05 2024-04-24T15:32:48.246 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.246 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:48.246 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.246 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.247 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.247 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23632 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15466496~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:48.247 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795060+0000", 2024-04-24T15:32:48.247 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 13.416388415, 2024-04-24T15:32:48.247 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040667993, 2024-04-24T15:32:48.247 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:48.247 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:48.247 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:48.247 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:48.247 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:48.247 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23632 2024-04-24T15:32:48.247 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.247 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:48.247 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.247 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:48.248 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795060+0000", 2024-04-24T15:32:48.248 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.248 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.248 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.248 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:48.248 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795060+0000", 2024-04-24T15:32:48.248 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.248 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.248 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.248 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:48.248 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794976+0000", 2024-04-24T15:32:48.248 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999156 2024-04-24T15:32:48.248 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.248 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.249 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:48.249 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795064+0000", 2024-04-24T15:32:48.249 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.7709999999999999e-05 2024-04-24T15:32:48.249 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.249 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.249 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:48.249 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795072+0000", 2024-04-24T15:32:48.249 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3769999999999996e-06 2024-04-24T15:32:48.249 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.249 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.249 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:48.249 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795078+0000", 2024-04-24T15:32:48.249 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.5079999999999996e-06 2024-04-24T15:32:48.249 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.250 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.250 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:48.250 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835541+0000", 2024-04-24T15:32:48.250 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040463440000000003 2024-04-24T15:32:48.250 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.250 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.250 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:48.250 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835657+0000", 2024-04-24T15:32:48.250 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00011572499999999999 2024-04-24T15:32:48.250 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.250 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.250 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:48.250 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835728+0000", 2024-04-24T15:32:48.250 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1402000000000005e-05 2024-04-24T15:32:48.251 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.251 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:48.251 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.251 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.251 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.251 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23633 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15507456~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:48.251 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795134+0000", 2024-04-24T15:32:48.251 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 13.416314760000001, 2024-04-24T15:32:48.251 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040748591000000001, 2024-04-24T15:32:48.251 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:48.251 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:48.251 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:48.251 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:48.251 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:48.251 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23633 2024-04-24T15:32:48.252 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.252 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:48.252 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.252 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:48.252 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795134+0000", 2024-04-24T15:32:48.252 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.252 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.252 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.252 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:48.252 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795134+0000", 2024-04-24T15:32:48.252 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.252 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.252 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.252 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:48.253 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795083+0000", 2024-04-24T15:32:48.253 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999949 2024-04-24T15:32:48.253 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.253 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.253 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:48.253 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795138+0000", 2024-04-24T15:32:48.253 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.4472000000000003e-05 2024-04-24T15:32:48.253 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.253 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.253 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:48.253 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795145+0000", 2024-04-24T15:32:48.253 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2779999999999996e-06 2024-04-24T15:32:48.253 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.253 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.254 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:48.254 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795149+0000", 2024-04-24T15:32:48.254 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.568e-06 2024-04-24T15:32:48.254 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.254 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.254 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:48.254 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835757+0000", 2024-04-24T15:32:48.254 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040608894999999999 2024-04-24T15:32:48.254 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.254 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.254 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:48.254 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835829+0000", 2024-04-24T15:32:48.255 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1879999999999996e-05 2024-04-24T15:32:48.255 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.255 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.255 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:48.255 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835883+0000", 2024-04-24T15:32:48.255 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.3346000000000001e-05 2024-04-24T15:32:48.255 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.255 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:48.255 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.255 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.255 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.255 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23634 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15548416~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:48.255 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795271+0000", 2024-04-24T15:32:48.255 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 13.416177726000001, 2024-04-24T15:32:48.255 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040755845999999998, 2024-04-24T15:32:48.256 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:48.256 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:48.256 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:48.256 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:48.256 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:48.256 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23634 2024-04-24T15:32:48.256 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.256 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:48.256 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.256 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:48.256 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795271+0000", 2024-04-24T15:32:48.256 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.256 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.256 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.256 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:48.257 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795271+0000", 2024-04-24T15:32:48.257 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.257 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.257 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.257 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:48.257 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795152+0000", 2024-04-24T15:32:48.257 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998813 2024-04-24T15:32:48.257 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.257 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.257 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:48.257 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795276+0000", 2024-04-24T15:32:48.257 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00012401799999999999 2024-04-24T15:32:48.257 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.257 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.258 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:48.258 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795281+0000", 2024-04-24T15:32:48.258 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8860000000000003e-06 2024-04-24T15:32:48.258 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.258 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.258 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:48.258 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795284+0000", 2024-04-24T15:32:48.258 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.0450000000000001e-06 2024-04-24T15:32:48.258 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.258 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.258 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:48.258 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835905+0000", 2024-04-24T15:32:48.258 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040621035999999999 2024-04-24T15:32:48.258 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.258 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.259 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:48.259 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835971+0000", 2024-04-24T15:32:48.259 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.5314000000000005e-05 2024-04-24T15:32:48.259 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.259 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.259 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:48.259 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836027+0000", 2024-04-24T15:32:48.259 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6305999999999999e-05 2024-04-24T15:32:48.259 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.259 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:48.259 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.259 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.259 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.259 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23635 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15589376~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:48.259 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795382+0000", 2024-04-24T15:32:48.259 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 13.416066953, 2024-04-24T15:32:48.260 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040805363999999997, 2024-04-24T15:32:48.260 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:48.260 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:48.260 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:48.260 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:48.260 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:48.260 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23635 2024-04-24T15:32:48.260 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.260 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:48.260 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.260 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:48.260 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795382+0000", 2024-04-24T15:32:48.260 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.260 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.260 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.261 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:48.261 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795382+0000", 2024-04-24T15:32:48.261 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.261 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.261 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.261 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:48.261 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795288+0000", 2024-04-24T15:32:48.261 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999056 2024-04-24T15:32:48.261 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.261 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.261 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:48.261 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795385+0000", 2024-04-24T15:32:48.261 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.7288999999999996e-05 2024-04-24T15:32:48.261 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.261 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.262 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:48.262 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795392+0000", 2024-04-24T15:32:48.262 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.452e-06 2024-04-24T15:32:48.262 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.262 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.262 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:48.262 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795397+0000", 2024-04-24T15:32:48.262 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2860000000000001e-06 2024-04-24T15:32:48.262 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.262 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.262 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:48.262 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836054+0000", 2024-04-24T15:32:48.262 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040657581999999998 2024-04-24T15:32:48.262 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.262 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.263 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:48.263 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836124+0000", 2024-04-24T15:32:48.263 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.0020999999999999e-05 2024-04-24T15:32:48.263 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.263 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.263 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:48.263 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836187+0000", 2024-04-24T15:32:48.263 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.3009999999999995e-05 2024-04-24T15:32:48.263 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.263 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:48.263 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.263 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.263 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.263 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23636 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15630336~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:48.263 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795610+0000", 2024-04-24T15:32:48.264 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 13.415838595, 2024-04-24T15:32:48.264 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040730942999999999, 2024-04-24T15:32:48.264 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:48.264 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:48.264 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:48.264 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:48.264 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:48.264 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23636 2024-04-24T15:32:48.264 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.264 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:48.264 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.264 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:48.264 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795610+0000", 2024-04-24T15:32:48.264 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.264 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.264 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.265 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:48.265 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795610+0000", 2024-04-24T15:32:48.265 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.265 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.265 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.265 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:48.265 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795401+0000", 2024-04-24T15:32:48.265 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9997911 2024-04-24T15:32:48.265 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.265 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.265 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:48.265 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795614+0000", 2024-04-24T15:32:48.265 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00021228600000000001 2024-04-24T15:32:48.265 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.265 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.266 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:48.266 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795622+0000", 2024-04-24T15:32:48.266 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3920000000000003e-06 2024-04-24T15:32:48.266 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.266 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.266 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:48.266 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795627+0000", 2024-04-24T15:32:48.266 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8199999999999996e-06 2024-04-24T15:32:48.266 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.266 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.266 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:48.266 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836211+0000", 2024-04-24T15:32:48.266 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040583803000000002 2024-04-24T15:32:48.267 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.267 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.267 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:48.267 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836284+0000", 2024-04-24T15:32:48.267 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.3617999999999999e-05 2024-04-24T15:32:48.267 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.267 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.267 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:48.267 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836341+0000", 2024-04-24T15:32:48.267 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6959999999999997e-05 2024-04-24T15:32:48.267 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.267 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:48.267 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.268 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.268 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.268 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23637 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15671296~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:48.268 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795699+0000", 2024-04-24T15:32:48.268 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 13.415749428, 2024-04-24T15:32:48.268 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040775556999999997, 2024-04-24T15:32:48.268 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:48.268 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:48.268 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:48.268 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:48.268 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:48.268 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23637 2024-04-24T15:32:48.268 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.268 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:48.269 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.269 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:48.269 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795699+0000", 2024-04-24T15:32:48.269 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.269 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.269 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.269 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:48.269 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795699+0000", 2024-04-24T15:32:48.269 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.269 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.269 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.269 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:48.269 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795632+0000", 2024-04-24T15:32:48.270 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999323 2024-04-24T15:32:48.270 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.270 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.270 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:48.270 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795704+0000", 2024-04-24T15:32:48.270 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2428000000000005e-05 2024-04-24T15:32:48.270 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.270 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.270 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:48.270 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795715+0000", 2024-04-24T15:32:48.270 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.1532000000000001e-05 2024-04-24T15:32:48.270 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.270 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.270 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:48.271 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795721+0000", 2024-04-24T15:32:48.271 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.8200000000000002e-06 2024-04-24T15:32:48.271 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.271 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.271 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:48.271 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836364+0000", 2024-04-24T15:32:48.271 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040642997 2024-04-24T15:32:48.271 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.271 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.271 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:48.271 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836435+0000", 2024-04-24T15:32:48.271 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.0821000000000005e-05 2024-04-24T15:32:48.271 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.271 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.272 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:48.272 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836475+0000", 2024-04-24T15:32:48.272 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9796999999999999e-05 2024-04-24T15:32:48.272 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.272 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:48.272 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.272 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.272 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.272 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23638 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15712256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:48.272 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795932+0000", 2024-04-24T15:32:48.272 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 13.415516397999999, 2024-04-24T15:32:48.272 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040668682999999997, 2024-04-24T15:32:48.272 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:48.272 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:48.273 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:48.273 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:48.273 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:48.273 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23638 2024-04-24T15:32:48.273 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.273 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:48.273 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.273 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:48.273 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795932+0000", 2024-04-24T15:32:48.273 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.273 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.273 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.273 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:48.273 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795932+0000", 2024-04-24T15:32:48.273 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.274 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.274 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.274 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:48.274 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795728+0000", 2024-04-24T15:32:48.274 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9997959 2024-04-24T15:32:48.274 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.274 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.274 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:48.274 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795936+0000", 2024-04-24T15:32:48.274 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000208128 2024-04-24T15:32:48.274 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.274 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.274 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:48.275 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795945+0000", 2024-04-24T15:32:48.275 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.7110000000000001e-06 2024-04-24T15:32:48.275 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.275 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.275 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:48.275 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795957+0000", 2024-04-24T15:32:48.275 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.1800999999999999e-05 2024-04-24T15:32:48.275 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.275 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.275 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:48.275 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836505+0000", 2024-04-24T15:32:48.275 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040547622999999998 2024-04-24T15:32:48.275 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.275 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.276 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:48.276 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836561+0000", 2024-04-24T15:32:48.276 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6598999999999998e-05 2024-04-24T15:32:48.276 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.276 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.276 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:48.276 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836601+0000", 2024-04-24T15:32:48.276 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9907000000000003e-05 2024-04-24T15:32:48.276 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.276 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:48.276 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.276 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.276 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.276 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23639 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15753216~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:48.277 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.796071+0000", 2024-04-24T15:32:48.277 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 13.415378136999999, 2024-04-24T15:32:48.277 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040648199000000003, 2024-04-24T15:32:48.277 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:48.277 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:48.277 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:48.277 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:48.277 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:48.277 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23639 2024-04-24T15:32:48.277 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.277 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:48.277 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.277 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:48.277 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796071+0000", 2024-04-24T15:32:48.278 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.278 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.278 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.278 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:48.278 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796071+0000", 2024-04-24T15:32:48.278 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.278 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.278 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.278 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:48.278 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795967+0000", 2024-04-24T15:32:48.278 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998965 2024-04-24T15:32:48.278 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.278 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.278 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:48.278 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796074+0000", 2024-04-24T15:32:48.279 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000107013 2024-04-24T15:32:48.279 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.279 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.279 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:48.279 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796081+0000", 2024-04-24T15:32:48.279 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2590000000000003e-06 2024-04-24T15:32:48.279 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.279 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.279 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:48.279 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796085+0000", 2024-04-24T15:32:48.279 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5269999999999999e-06 2024-04-24T15:32:48.279 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.279 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.279 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:48.279 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836624+0000", 2024-04-24T15:32:48.280 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040539324000000002 2024-04-24T15:32:48.280 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.280 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.280 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:48.280 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836667+0000", 2024-04-24T15:32:48.280 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2562999999999999e-05 2024-04-24T15:32:48.280 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.280 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.280 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:48.280 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836719+0000", 2024-04-24T15:32:48.280 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.2015999999999998e-05 2024-04-24T15:32:48.280 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.280 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:48.280 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.280 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.281 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.281 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23640 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15794176~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:48.281 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.796492+0000", 2024-04-24T15:32:48.281 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 13.414956378999999, 2024-04-24T15:32:48.281 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040420000999999997, 2024-04-24T15:32:48.281 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:48.281 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:48.281 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:48.281 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:48.281 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:48.281 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23640 2024-04-24T15:32:48.281 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.281 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:48.281 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.282 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:48.282 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796492+0000", 2024-04-24T15:32:48.282 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.282 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.282 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.282 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:48.282 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796492+0000", 2024-04-24T15:32:48.282 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.282 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.282 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.282 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:48.282 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796088+0000", 2024-04-24T15:32:48.282 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9995961 2024-04-24T15:32:48.283 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.283 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.283 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:48.283 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796496+0000", 2024-04-24T15:32:48.283 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00040785700000000002 2024-04-24T15:32:48.283 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.283 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.283 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:48.283 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796503+0000", 2024-04-24T15:32:48.283 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1049999999999997e-06 2024-04-24T15:32:48.283 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.283 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.283 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:48.283 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796508+0000", 2024-04-24T15:32:48.283 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.155e-06 2024-04-24T15:32:48.284 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.284 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.284 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:48.284 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836756+0000", 2024-04-24T15:32:48.284 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040248850000000003 2024-04-24T15:32:48.284 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.284 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.284 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:48.284 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836841+0000", 2024-04-24T15:32:48.284 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.4735999999999996e-05 2024-04-24T15:32:48.284 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.284 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.284 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:48.284 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836912+0000", 2024-04-24T15:32:48.284 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1229999999999994e-05 2024-04-24T15:32:48.285 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.285 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:48.285 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.285 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.285 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.285 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23641 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15835136~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:48.285 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.796658+0000", 2024-04-24T15:32:48.285 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 13.414790303, 2024-04-24T15:32:48.285 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040420706000000001, 2024-04-24T15:32:48.285 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:48.285 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:48.285 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:48.285 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:48.285 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:48.285 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23641 2024-04-24T15:32:48.286 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.286 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:48.286 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.286 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:48.286 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796658+0000", 2024-04-24T15:32:48.286 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.286 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.286 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.286 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:48.286 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796658+0000", 2024-04-24T15:32:48.286 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.286 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.286 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.286 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:48.287 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796512+0000", 2024-04-24T15:32:48.287 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998531 2024-04-24T15:32:48.287 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.287 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.287 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:48.287 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796669+0000", 2024-04-24T15:32:48.287 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000157577 2024-04-24T15:32:48.287 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.287 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.287 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:48.287 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796677+0000", 2024-04-24T15:32:48.287 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7800000000000001e-06 2024-04-24T15:32:48.287 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.287 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.287 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:48.287 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796681+0000", 2024-04-24T15:32:48.288 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.721e-06 2024-04-24T15:32:48.288 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.288 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.288 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:48.288 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836941+0000", 2024-04-24T15:32:48.288 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040259907999999997 2024-04-24T15:32:48.288 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.288 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.288 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:48.288 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837017+0000", 2024-04-24T15:32:48.288 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.6071999999999999e-05 2024-04-24T15:32:48.288 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.288 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.288 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:48.289 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837079+0000", 2024-04-24T15:32:48.289 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.2358000000000002e-05 2024-04-24T15:32:48.289 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.289 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:48.289 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.289 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.289 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.289 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23645 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15998976~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:48.289 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.797360+0000", 2024-04-24T15:32:48.289 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 13.414088805, 2024-04-24T15:32:48.289 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040387645999999999, 2024-04-24T15:32:48.289 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:48.289 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:48.289 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:48.290 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:48.290 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:48.290 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23645 2024-04-24T15:32:48.290 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.290 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:48.290 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.290 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:48.290 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797360+0000", 2024-04-24T15:32:48.290 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.290 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.290 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.290 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:48.290 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797360+0000", 2024-04-24T15:32:48.291 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.291 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.291 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.291 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:48.291 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797288+0000", 2024-04-24T15:32:48.291 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999928 2024-04-24T15:32:48.291 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.291 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.291 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:48.291 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797364+0000", 2024-04-24T15:32:48.291 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.5347999999999998e-05 2024-04-24T15:32:48.291 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.291 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.291 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:48.292 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797372+0000", 2024-04-24T15:32:48.292 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.2409999999999995e-06 2024-04-24T15:32:48.292 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.292 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.292 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:48.292 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797376+0000", 2024-04-24T15:32:48.292 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.515e-06 2024-04-24T15:32:48.292 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.292 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.292 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:48.292 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837586+0000", 2024-04-24T15:32:48.292 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040210047999999998 2024-04-24T15:32:48.292 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.292 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.292 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:48.293 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837672+0000", 2024-04-24T15:32:48.293 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.5997999999999999e-05 2024-04-24T15:32:48.293 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.293 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.293 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:48.293 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837748+0000", 2024-04-24T15:32:48.293 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.5328999999999997e-05 2024-04-24T15:32:48.293 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.293 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:48.293 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.293 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.293 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.293 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23646 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 16039936~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:48.293 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.797498+0000", 2024-04-24T15:32:48.293 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 13.413950703999999, 2024-04-24T15:32:48.294 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040433139, 2024-04-24T15:32:48.294 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:48.294 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:48.294 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:48.294 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:48.294 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:48.294 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23646 2024-04-24T15:32:48.294 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.294 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:48.294 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.294 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:48.294 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797498+0000", 2024-04-24T15:32:48.294 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.294 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.294 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.295 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:48.295 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797498+0000", 2024-04-24T15:32:48.295 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.295 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.295 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.295 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:48.295 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797381+0000", 2024-04-24T15:32:48.295 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998832 2024-04-24T15:32:48.295 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.295 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.295 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:48.295 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797501+0000", 2024-04-24T15:32:48.295 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000119759 2024-04-24T15:32:48.295 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.295 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.296 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:48.296 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797510+0000", 2024-04-24T15:32:48.296 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.4770000000000003e-06 2024-04-24T15:32:48.296 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.296 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.296 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:48.296 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797515+0000", 2024-04-24T15:32:48.296 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8330000000000001e-06 2024-04-24T15:32:48.296 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.296 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.296 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:48.296 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837777+0000", 2024-04-24T15:32:48.296 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040262502999999998 2024-04-24T15:32:48.296 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.296 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.297 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:48.297 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837860+0000", 2024-04-24T15:32:48.297 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3002000000000003e-05 2024-04-24T15:32:48.297 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.297 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.297 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:48.297 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837931+0000", 2024-04-24T15:32:48.297 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1222000000000004e-05 2024-04-24T15:32:48.297 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.297 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:48.297 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.297 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.297 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:48.297 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:32:48.299 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-24T15:32:48.299 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:32:48.299+0000 7f23abcf6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 44021) UID: 0 2024-04-24T15:32:48.400 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:32:48.400+0000 7f23abcf6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 44021) UID: 0 2024-04-24T15:32:48.462 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:32:48.462 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:32:48.462 INFO:teuthology.orchestra.run.smithi151.stdout: "num_ops": 0 2024-04-24T15:32:48.463 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:32:48.472 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-24T15:32:48.501 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:32:48.501+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:32:48.557 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-24T15:32:48.604 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:32:48.604+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:32:48.636 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:32:48.636 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:32:48.636 INFO:teuthology.orchestra.run.smithi151.stdout: "complaint_time": 30, 2024-04-24T15:32:48.636 INFO:teuthology.orchestra.run.smithi151.stdout: "num_blocked_ops": 0 2024-04-24T15:32:48.636 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:32:48.649 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-24T15:32:48.702 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:32:48.702+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:32:48.734 INFO:teuthology.orchestra.run.smithi096.stdout:truncating image image_client.0-clone9 from 0x836a0c (overlap 0x836a0c) to 0x4ed93a 2024-04-24T15:32:48.739 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-04-24T15:32:48.773 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-04-24T15:32:48.798 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:32:48.798 INFO:teuthology.orchestra.run.smithi151.stdout: "size": 20, 2024-04-24T15:32:48.798 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 600, 2024-04-24T15:32:48.798 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [ 2024-04-24T15:32:48.798 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.798 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17317 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 11509760~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:48.798 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.224408+0000", 2024-04-24T15:32:48.798 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 14.573328044, 2024-04-24T15:32:48.798 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042604418999999998, 2024-04-24T15:32:48.798 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:48.798 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:48.799 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:48.799 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:48.799 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:48.799 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17317 2024-04-24T15:32:48.799 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.799 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:48.799 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.799 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:48.799 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224408+0000", 2024-04-24T15:32:48.799 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.799 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.799 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.799 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:48.799 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224408+0000", 2024-04-24T15:32:48.799 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.800 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.800 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.800 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:48.800 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224407+0000", 2024-04-24T15:32:48.800 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:48.800 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.800 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.800 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:48.800 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224409+0000", 2024-04-24T15:32:48.800 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.1059999999999998e-06 2024-04-24T15:32:48.800 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.800 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.801 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:48.801 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224415+0000", 2024-04-24T15:32:48.801 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6219999999999997e-06 2024-04-24T15:32:48.801 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.801 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.801 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:48.801 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224419+0000", 2024-04-24T15:32:48.801 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8340000000000001e-06 2024-04-24T15:32:48.801 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.801 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.801 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:48.801 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.266851+0000", 2024-04-24T15:32:48.802 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042431910000000003 2024-04-24T15:32:48.802 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.802 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.802 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:48.802 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.266931+0000", 2024-04-24T15:32:48.802 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0580000000000004e-05 2024-04-24T15:32:48.802 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.802 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.802 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:48.802 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267012+0000", 2024-04-24T15:32:48.802 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0698999999999994e-05 2024-04-24T15:32:48.802 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.802 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:48.803 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.803 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.803 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.803 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17318 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 11550720~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:48.803 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.224424+0000", 2024-04-24T15:32:48.803 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 14.573311675999999, 2024-04-24T15:32:48.803 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042771921999999997, 2024-04-24T15:32:48.803 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:48.803 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:48.803 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:48.803 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:48.803 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:48.803 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17318 2024-04-24T15:32:48.804 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.804 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:48.804 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.804 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:48.804 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224424+0000", 2024-04-24T15:32:48.804 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.804 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.804 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.804 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:48.804 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224424+0000", 2024-04-24T15:32:48.804 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.804 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.804 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.804 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:48.805 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224424+0000", 2024-04-24T15:32:48.805 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:48.805 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.805 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.805 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:48.805 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224434+0000", 2024-04-24T15:32:48.805 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.9699999999999994e-06 2024-04-24T15:32:48.805 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.805 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.805 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:48.805 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224443+0000", 2024-04-24T15:32:48.805 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.3689999999999996e-06 2024-04-24T15:32:48.805 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.805 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.805 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:48.806 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224447+0000", 2024-04-24T15:32:48.806 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.783e-06 2024-04-24T15:32:48.806 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.806 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.806 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:48.806 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267027+0000", 2024-04-24T15:32:48.806 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042579847999999997 2024-04-24T15:32:48.806 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.806 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.806 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:48.806 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267108+0000", 2024-04-24T15:32:48.806 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.1420000000000003e-05 2024-04-24T15:32:48.806 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.807 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.807 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:48.807 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267196+0000", 2024-04-24T15:32:48.807 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.7851999999999998e-05 2024-04-24T15:32:48.807 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.807 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:48.807 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.807 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.807 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.807 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17319 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 11591680~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:48.807 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.224453+0000", 2024-04-24T15:32:48.807 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 14.573282750000001, 2024-04-24T15:32:48.807 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042921783999999998, 2024-04-24T15:32:48.808 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:48.808 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:48.808 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:48.808 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:48.808 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:48.808 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17319 2024-04-24T15:32:48.808 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.808 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:48.808 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.808 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:48.808 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224453+0000", 2024-04-24T15:32:48.808 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.808 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.808 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.808 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:48.809 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224453+0000", 2024-04-24T15:32:48.809 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.809 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.809 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.809 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:48.809 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224452+0000", 2024-04-24T15:32:48.809 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:48.809 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.809 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.809 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:48.809 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224455+0000", 2024-04-24T15:32:48.809 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.289e-06 2024-04-24T15:32:48.809 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.810 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.810 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:48.810 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224461+0000", 2024-04-24T15:32:48.810 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.7300000000000002e-06 2024-04-24T15:32:48.810 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.810 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.810 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:48.810 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224464+0000", 2024-04-24T15:32:48.810 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.2890000000000002e-06 2024-04-24T15:32:48.810 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.810 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.810 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:48.810 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267218+0000", 2024-04-24T15:32:48.810 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042753696000000001 2024-04-24T15:32:48.811 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.811 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.811 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:48.811 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267295+0000", 2024-04-24T15:32:48.811 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7762999999999994e-05 2024-04-24T15:32:48.811 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.811 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.811 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:48.811 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267375+0000", 2024-04-24T15:32:48.811 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.9444999999999996e-05 2024-04-24T15:32:48.811 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.811 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:48.811 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.811 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.812 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.812 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17320 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 11632640~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:48.812 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.224468+0000", 2024-04-24T15:32:48.812 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 14.573267716, 2024-04-24T15:32:48.812 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043063615, 2024-04-24T15:32:48.812 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:48.812 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:48.812 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:48.812 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:48.812 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:48.812 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17320 2024-04-24T15:32:48.812 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.812 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:48.813 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.813 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:48.813 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224468+0000", 2024-04-24T15:32:48.813 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.813 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.813 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.813 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:48.813 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224468+0000", 2024-04-24T15:32:48.813 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.813 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.813 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.813 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:48.813 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224468+0000", 2024-04-24T15:32:48.813 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:48.814 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.814 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.814 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:48.814 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224470+0000", 2024-04-24T15:32:48.814 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.0600000000000002e-06 2024-04-24T15:32:48.814 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.814 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.814 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:48.814 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224475+0000", 2024-04-24T15:32:48.814 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.1059999999999999e-06 2024-04-24T15:32:48.814 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.814 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.814 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:48.815 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224478+0000", 2024-04-24T15:32:48.815 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.3100000000000001e-06 2024-04-24T15:32:48.815 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.815 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.815 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:48.815 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267386+0000", 2024-04-24T15:32:48.815 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042907529 2024-04-24T15:32:48.815 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.815 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.815 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:48.815 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267447+0000", 2024-04-24T15:32:48.815 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.1173000000000005e-05 2024-04-24T15:32:48.815 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.815 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.815 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:48.816 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267532+0000", 2024-04-24T15:32:48.816 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.4771999999999994e-05 2024-04-24T15:32:48.816 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.816 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:48.816 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.816 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.816 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.816 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17321 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 11673600~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:48.816 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.224499+0000", 2024-04-24T15:32:48.816 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 14.573236827000001, 2024-04-24T15:32:48.816 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043214447000000003, 2024-04-24T15:32:48.816 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:48.816 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:48.816 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:48.817 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:48.817 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:48.817 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17321 2024-04-24T15:32:48.817 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.817 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:48.817 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.817 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:48.817 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224499+0000", 2024-04-24T15:32:48.817 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.817 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.817 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.817 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:48.817 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224499+0000", 2024-04-24T15:32:48.817 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.818 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.818 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.818 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:48.818 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224499+0000", 2024-04-24T15:32:48.818 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:48.818 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.818 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.818 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:48.818 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224504+0000", 2024-04-24T15:32:48.818 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6869999999999999e-06 2024-04-24T15:32:48.818 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.818 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.818 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:48.819 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224509+0000", 2024-04-24T15:32:48.819 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2259999999999999e-06 2024-04-24T15:32:48.819 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.819 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.819 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:48.819 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224511+0000", 2024-04-24T15:32:48.819 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.418e-06 2024-04-24T15:32:48.819 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.819 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.819 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:48.819 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267552+0000", 2024-04-24T15:32:48.819 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043041455999999999 2024-04-24T15:32:48.819 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.820 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.820 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:48.820 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267629+0000", 2024-04-24T15:32:48.820 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7069000000000004e-05 2024-04-24T15:32:48.820 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.820 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.820 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:48.820 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267713+0000", 2024-04-24T15:32:48.820 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3843000000000003e-05 2024-04-24T15:32:48.820 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.820 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:48.820 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.820 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.820 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.821 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17322 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 11714560~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:48.821 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.224514+0000", 2024-04-24T15:32:48.821 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 14.573221588999999, 2024-04-24T15:32:48.821 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043339196000000003, 2024-04-24T15:32:48.821 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:48.821 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:48.821 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:48.821 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:48.821 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:48.821 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17322 2024-04-24T15:32:48.821 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.821 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:48.821 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.821 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:48.822 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224514+0000", 2024-04-24T15:32:48.822 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.822 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.822 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.822 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:48.822 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224514+0000", 2024-04-24T15:32:48.822 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.822 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.822 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.822 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:48.822 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224514+0000", 2024-04-24T15:32:48.822 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:48.822 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.823 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.823 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:48.823 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224515+0000", 2024-04-24T15:32:48.823 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.5570000000000001e-06 2024-04-24T15:32:48.823 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.823 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.823 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:48.823 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224518+0000", 2024-04-24T15:32:48.823 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.943e-06 2024-04-24T15:32:48.823 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.823 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.823 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:48.823 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224520+0000", 2024-04-24T15:32:48.823 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.9999999999999999e-06 2024-04-24T15:32:48.824 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.824 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.824 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:48.824 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267730+0000", 2024-04-24T15:32:48.824 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043209466000000002 2024-04-24T15:32:48.824 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.824 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.824 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:48.824 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267772+0000", 2024-04-24T15:32:48.824 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2453000000000002e-05 2024-04-24T15:32:48.824 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.824 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.824 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:48.824 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267853+0000", 2024-04-24T15:32:48.824 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.1094000000000007e-05 2024-04-24T15:32:48.825 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.825 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:48.825 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.825 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.825 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.825 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17323 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 11755520~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:48.825 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.224522+0000", 2024-04-24T15:32:48.825 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 14.573213249, 2024-04-24T15:32:48.825 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043506792000000002, 2024-04-24T15:32:48.825 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:48.825 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:48.825 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:48.825 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:48.825 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:48.825 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17323 2024-04-24T15:32:48.826 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.826 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:48.826 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.826 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:48.826 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224522+0000", 2024-04-24T15:32:48.826 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.826 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.826 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.826 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:48.826 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224522+0000", 2024-04-24T15:32:48.826 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.826 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.826 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.827 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:48.827 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224522+0000", 2024-04-24T15:32:48.827 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:32:48.827 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.827 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.827 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:48.827 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224524+0000", 2024-04-24T15:32:48.827 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3069999999999999e-06 2024-04-24T15:32:48.827 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.827 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.827 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:48.827 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224526+0000", 2024-04-24T15:32:48.827 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.773e-06 2024-04-24T15:32:48.827 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.827 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.828 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:48.828 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224529+0000", 2024-04-24T15:32:48.828 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.3e-06 2024-04-24T15:32:48.828 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.828 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.828 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:48.828 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267873+0000", 2024-04-24T15:32:48.828 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043344141000000003 2024-04-24T15:32:48.828 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.828 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.828 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:48.828 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267948+0000", 2024-04-24T15:32:48.828 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.4960000000000001e-05 2024-04-24T15:32:48.829 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.829 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.829 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:48.829 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.268029+0000", 2024-04-24T15:32:48.829 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.1520000000000006e-05 2024-04-24T15:32:48.829 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.829 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:48.829 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.829 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.829 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.829 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17328 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 11960320~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:48.829 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.226273+0000", 2024-04-24T15:32:48.829 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 14.571462264000001, 2024-04-24T15:32:48.829 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042524610999999997, 2024-04-24T15:32:48.830 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:48.830 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:48.830 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:48.830 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:48.830 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:48.830 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17328 2024-04-24T15:32:48.830 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.830 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:48.830 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.830 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:48.830 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226273+0000", 2024-04-24T15:32:48.830 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.830 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.831 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.831 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:48.831 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226273+0000", 2024-04-24T15:32:48.831 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.831 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.831 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.831 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:48.831 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226273+0000", 2024-04-24T15:32:48.831 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:32:48.831 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.831 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.831 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:48.831 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226275+0000", 2024-04-24T15:32:48.831 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.759e-06 2024-04-24T15:32:48.831 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.831 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.832 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:48.832 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226286+0000", 2024-04-24T15:32:48.832 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0841e-05 2024-04-24T15:32:48.832 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.832 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.832 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:48.832 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226290+0000", 2024-04-24T15:32:48.832 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.794e-06 2024-04-24T15:32:48.832 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.832 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.832 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:48.832 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.268693+0000", 2024-04-24T15:32:48.832 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042403519000000001 2024-04-24T15:32:48.832 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.832 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.832 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:48.833 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.268739+0000", 2024-04-24T15:32:48.833 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6149000000000001e-05 2024-04-24T15:32:48.833 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.833 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.833 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:48.833 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.268798+0000", 2024-04-24T15:32:48.833 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.8752999999999998e-05 2024-04-24T15:32:48.833 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.833 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:48.833 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.833 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.833 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.833 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17329 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 12001280~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:48.833 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.226295+0000", 2024-04-24T15:32:48.833 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 14.571441157000001, 2024-04-24T15:32:48.834 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042690771000000002, 2024-04-24T15:32:48.834 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:48.834 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:48.834 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:48.834 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:48.834 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:48.834 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17329 2024-04-24T15:32:48.834 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.834 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:48.834 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.834 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:48.834 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226295+0000", 2024-04-24T15:32:48.834 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.835 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.835 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.835 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:48.835 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226295+0000", 2024-04-24T15:32:48.835 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.835 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.835 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.835 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:48.835 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226294+0000", 2024-04-24T15:32:48.835 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:48.835 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.835 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.835 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:48.836 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226297+0000", 2024-04-24T15:32:48.836 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.0189999999999998e-06 2024-04-24T15:32:48.836 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.836 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.836 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:48.836 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226302+0000", 2024-04-24T15:32:48.836 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6260000000000003e-06 2024-04-24T15:32:48.836 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.836 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.836 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:48.836 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226304+0000", 2024-04-24T15:32:48.836 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.2230000000000001e-06 2024-04-24T15:32:48.836 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.837 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.837 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:48.837 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.268831+0000", 2024-04-24T15:32:48.837 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042527224000000002 2024-04-24T15:32:48.837 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.837 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.837 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:48.837 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.268904+0000", 2024-04-24T15:32:48.837 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.3168000000000001e-05 2024-04-24T15:32:48.837 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.837 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.837 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:48.837 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.268985+0000", 2024-04-24T15:32:48.837 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.1009999999999999e-05 2024-04-24T15:32:48.838 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.838 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:48.838 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.838 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.838 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.838 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17330 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 12042240~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:48.838 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.226307+0000", 2024-04-24T15:32:48.838 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 14.571429166, 2024-04-24T15:32:48.838 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042812009999999998, 2024-04-24T15:32:48.838 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:48.838 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:48.838 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:48.838 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:48.839 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:48.839 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17330 2024-04-24T15:32:48.839 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.839 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:48.839 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.839 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:48.839 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226307+0000", 2024-04-24T15:32:48.839 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.839 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.839 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.839 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:48.839 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226307+0000", 2024-04-24T15:32:48.839 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.839 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.840 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.840 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:48.840 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226306+0000", 2024-04-24T15:32:48.840 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:32:48.840 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.840 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.840 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:48.840 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226312+0000", 2024-04-24T15:32:48.840 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6180000000000003e-06 2024-04-24T15:32:48.840 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.840 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.840 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:48.840 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226316+0000", 2024-04-24T15:32:48.840 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2379999999999999e-06 2024-04-24T15:32:48.841 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.841 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.841 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:48.841 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226318+0000", 2024-04-24T15:32:48.841 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.3130000000000001e-06 2024-04-24T15:32:48.841 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.841 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.841 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:48.841 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269015+0000", 2024-04-24T15:32:48.841 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042696698999999998 2024-04-24T15:32:48.841 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.841 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.841 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:48.842 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269061+0000", 2024-04-24T15:32:48.842 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6193000000000003e-05 2024-04-24T15:32:48.842 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.842 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.842 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:48.842 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269119+0000", 2024-04-24T15:32:48.842 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.7169000000000001e-05 2024-04-24T15:32:48.842 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.842 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:48.842 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.842 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.842 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.842 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17331 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 12083200~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:48.843 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.226321+0000", 2024-04-24T15:32:48.843 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 14.571414734999999, 2024-04-24T15:32:48.843 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042976872999999999, 2024-04-24T15:32:48.843 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:48.843 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:48.843 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:48.843 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:48.843 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:48.843 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17331 2024-04-24T15:32:48.843 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.843 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:48.843 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.843 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:48.843 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226321+0000", 2024-04-24T15:32:48.843 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.843 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.844 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.844 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:48.844 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226321+0000", 2024-04-24T15:32:48.844 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.844 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.844 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.844 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:48.844 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226321+0000", 2024-04-24T15:32:48.844 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:32:48.844 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.844 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.844 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:48.844 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226322+0000", 2024-04-24T15:32:48.844 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.446e-06 2024-04-24T15:32:48.844 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.845 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.845 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:48.845 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226325+0000", 2024-04-24T15:32:48.845 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.004e-06 2024-04-24T15:32:48.845 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.845 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.845 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:48.845 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226327+0000", 2024-04-24T15:32:48.845 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.0990000000000001e-06 2024-04-24T15:32:48.845 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.845 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.845 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:48.845 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269137+0000", 2024-04-24T15:32:48.845 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042809381000000001 2024-04-24T15:32:48.845 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.845 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.846 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:48.846 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269217+0000", 2024-04-24T15:32:48.846 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.085e-05 2024-04-24T15:32:48.846 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.846 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.846 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:48.846 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269298+0000", 2024-04-24T15:32:48.846 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0317999999999999e-05 2024-04-24T15:32:48.846 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.846 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:48.846 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.846 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.846 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.846 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17332 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 12124160~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:48.847 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.226329+0000", 2024-04-24T15:32:48.847 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 14.571406247000001, 2024-04-24T15:32:48.847 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043084539999999998, 2024-04-24T15:32:48.847 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:48.847 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:48.847 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:48.847 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:48.847 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:48.847 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17332 2024-04-24T15:32:48.847 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.847 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:48.847 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.847 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:48.847 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226329+0000", 2024-04-24T15:32:48.847 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.847 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.848 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.848 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:48.848 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226329+0000", 2024-04-24T15:32:48.848 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.848 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.848 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.848 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:48.848 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226329+0000", 2024-04-24T15:32:48.848 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:32:48.848 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.848 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.848 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:48.848 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226331+0000", 2024-04-24T15:32:48.848 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.328e-06 2024-04-24T15:32:48.848 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.848 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.849 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:48.849 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226333+0000", 2024-04-24T15:32:48.849 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.9119999999999998e-06 2024-04-24T15:32:48.849 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.849 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.849 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:48.849 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226335+0000", 2024-04-24T15:32:48.849 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.9690000000000001e-06 2024-04-24T15:32:48.849 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.849 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.849 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:48.849 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269313+0000", 2024-04-24T15:32:48.849 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042977518999999999 2024-04-24T15:32:48.849 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.849 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.850 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:48.850 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269357+0000", 2024-04-24T15:32:48.850 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3748999999999997e-05 2024-04-24T15:32:48.850 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.850 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.850 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:48.850 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269414+0000", 2024-04-24T15:32:48.850 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.7265000000000001e-05 2024-04-24T15:32:48.850 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.850 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:48.850 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.850 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.850 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.850 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17333 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 12165120~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:48.850 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.226338+0000", 2024-04-24T15:32:48.851 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 14.571398110000001, 2024-04-24T15:32:48.851 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043264706999999999, 2024-04-24T15:32:48.851 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:48.851 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:48.851 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:48.851 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:48.851 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:48.851 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17333 2024-04-24T15:32:48.851 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.851 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:48.851 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.851 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:48.851 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226338+0000", 2024-04-24T15:32:48.851 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.851 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.852 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.852 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:48.852 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226338+0000", 2024-04-24T15:32:48.852 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.852 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.852 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.852 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:48.852 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226337+0000", 2024-04-24T15:32:48.852 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:32:48.852 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.852 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.852 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:48.852 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226339+0000", 2024-04-24T15:32:48.852 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.387e-06 2024-04-24T15:32:48.852 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.852 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.853 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:48.853 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226342+0000", 2024-04-24T15:32:48.853 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8059999999999999e-06 2024-04-24T15:32:48.853 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.853 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.853 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:48.853 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226344+0000", 2024-04-24T15:32:48.853 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.1390000000000002e-06 2024-04-24T15:32:48.853 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.853 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.853 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:48.853 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269445+0000", 2024-04-24T15:32:48.853 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043101457000000003 2024-04-24T15:32:48.853 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.854 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.854 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:48.854 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269520+0000", 2024-04-24T15:32:48.854 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.5278000000000007e-05 2024-04-24T15:32:48.854 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.854 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.854 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:48.854 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269602+0000", 2024-04-24T15:32:48.854 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.1847000000000004e-05 2024-04-24T15:32:48.854 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.854 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:48.854 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.854 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.854 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.855 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17334 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 12206080~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:48.855 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.226346+0000", 2024-04-24T15:32:48.855 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 14.571389694000001, 2024-04-24T15:32:48.855 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043468358999999998, 2024-04-24T15:32:48.855 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:48.855 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:48.855 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:48.855 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:48.855 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:48.855 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17334 2024-04-24T15:32:48.855 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.855 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:48.855 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.855 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:48.855 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226346+0000", 2024-04-24T15:32:48.855 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.856 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.856 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.856 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:48.856 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226346+0000", 2024-04-24T15:32:48.856 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.856 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.856 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.856 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:48.856 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226346+0000", 2024-04-24T15:32:48.856 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:32:48.856 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.856 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.856 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:48.856 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226347+0000", 2024-04-24T15:32:48.856 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.359e-06 2024-04-24T15:32:48.857 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.857 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.857 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:48.857 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226350+0000", 2024-04-24T15:32:48.857 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7800000000000001e-06 2024-04-24T15:32:48.857 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.857 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.857 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:48.857 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226352+0000", 2024-04-24T15:32:48.857 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.046e-06 2024-04-24T15:32:48.857 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.857 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.857 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:48.857 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269646+0000", 2024-04-24T15:32:48.858 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043294393 2024-04-24T15:32:48.858 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.858 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.858 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:48.858 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269732+0000", 2024-04-24T15:32:48.858 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.5716000000000004e-05 2024-04-24T15:32:48.858 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.858 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.858 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:48.858 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269814+0000", 2024-04-24T15:32:48.858 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.2270999999999999e-05 2024-04-24T15:32:48.858 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.858 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:48.858 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.859 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.859 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.859 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17342 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 12533760~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:48.859 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.228377+0000", 2024-04-24T15:32:48.859 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 14.5693588, 2024-04-24T15:32:48.859 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042573286000000002, 2024-04-24T15:32:48.859 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:48.859 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:48.859 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:48.859 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:48.859 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:48.859 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17342 2024-04-24T15:32:48.859 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.859 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:48.860 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.860 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:48.860 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228377+0000", 2024-04-24T15:32:48.860 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.860 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.860 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.860 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:48.860 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228377+0000", 2024-04-24T15:32:48.860 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.860 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.860 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.860 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:48.860 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228377+0000", 2024-04-24T15:32:48.860 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:48.860 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.861 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.861 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:48.861 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228389+0000", 2024-04-24T15:32:48.861 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.2262999999999999e-05 2024-04-24T15:32:48.861 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.861 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.861 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:48.861 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228398+0000", 2024-04-24T15:32:48.861 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.1079999999999999e-06 2024-04-24T15:32:48.861 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.861 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.861 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:48.861 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228402+0000", 2024-04-24T15:32:48.861 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8920000000000002e-06 2024-04-24T15:32:48.862 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.862 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.862 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:48.862 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.270842+0000", 2024-04-24T15:32:48.862 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042440193000000001 2024-04-24T15:32:48.862 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.862 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.862 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:48.862 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.270887+0000", 2024-04-24T15:32:48.862 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5359999999999999e-05 2024-04-24T15:32:48.862 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.862 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.862 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:48.862 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.270950+0000", 2024-04-24T15:32:48.863 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.2842999999999994e-05 2024-04-24T15:32:48.863 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.863 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:48.863 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.863 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.863 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.863 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17343 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 12574720~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:48.863 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.228407+0000", 2024-04-24T15:32:48.863 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 14.569328927000001, 2024-04-24T15:32:48.863 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042650325000000003, 2024-04-24T15:32:48.863 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:48.863 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:48.863 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:48.863 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:48.863 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:48.864 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17343 2024-04-24T15:32:48.864 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.864 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:48.864 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.864 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:48.864 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228407+0000", 2024-04-24T15:32:48.864 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.864 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.864 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.864 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:48.864 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228407+0000", 2024-04-24T15:32:48.864 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.864 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.864 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.864 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:48.865 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228406+0000", 2024-04-24T15:32:48.865 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:48.865 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.865 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.865 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:48.865 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228409+0000", 2024-04-24T15:32:48.865 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.3980000000000002e-06 2024-04-24T15:32:48.865 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.865 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.865 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:48.865 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228414+0000", 2024-04-24T15:32:48.865 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6590000000000004e-06 2024-04-24T15:32:48.865 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.865 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.866 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:48.866 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228418+0000", 2024-04-24T15:32:48.866 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.625e-06 2024-04-24T15:32:48.866 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.866 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.866 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:48.866 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.270966+0000", 2024-04-24T15:32:48.866 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042547709000000003 2024-04-24T15:32:48.866 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.866 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.866 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:48.866 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.271007+0000", 2024-04-24T15:32:48.866 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.1659000000000002e-05 2024-04-24T15:32:48.867 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.867 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.867 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:48.867 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.271057+0000", 2024-04-24T15:32:48.867 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.9679000000000003e-05 2024-04-24T15:32:48.867 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.867 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:48.867 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.867 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.867 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.867 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17344 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 12607488~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:48.867 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.228461+0000", 2024-04-24T15:32:48.867 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 14.569274939, 2024-04-24T15:32:48.867 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042713530999999999, 2024-04-24T15:32:48.868 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:48.868 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:48.868 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:48.868 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:48.868 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:48.868 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17344 2024-04-24T15:32:48.868 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.868 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:48.868 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.868 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:48.868 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228461+0000", 2024-04-24T15:32:48.868 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.868 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.868 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.869 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:48.869 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228461+0000", 2024-04-24T15:32:48.869 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.869 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.869 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.869 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:48.869 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228460+0000", 2024-04-24T15:32:48.869 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:48.869 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.869 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.869 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:48.869 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228464+0000", 2024-04-24T15:32:48.869 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.98e-06 2024-04-24T15:32:48.869 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.870 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.870 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:48.870 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228473+0000", 2024-04-24T15:32:48.870 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.0270000000000001e-06 2024-04-24T15:32:48.870 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.870 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.870 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:48.870 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228479+0000", 2024-04-24T15:32:48.870 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.5910000000000003e-06 2024-04-24T15:32:48.870 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.870 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.870 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:48.870 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.271081+0000", 2024-04-24T15:32:48.871 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042602025000000002 2024-04-24T15:32:48.871 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.871 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.871 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:48.871 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.271126+0000", 2024-04-24T15:32:48.871 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4904999999999998e-05 2024-04-24T15:32:48.871 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.871 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.871 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:48.871 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.271174+0000", 2024-04-24T15:32:48.871 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8603999999999997e-05 2024-04-24T15:32:48.871 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.871 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:48.871 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.872 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.872 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.872 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17345 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 12648448~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:48.872 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.228484+0000", 2024-04-24T15:32:48.872 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 14.569251402999999, 2024-04-24T15:32:48.872 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042811844000000002, 2024-04-24T15:32:48.872 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:48.872 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:48.872 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:48.872 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:48.872 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:48.872 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17345 2024-04-24T15:32:48.872 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.872 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:48.873 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.873 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:48.873 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228484+0000", 2024-04-24T15:32:48.873 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.873 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.873 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.873 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:48.873 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228484+0000", 2024-04-24T15:32:48.873 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.873 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.873 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.873 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:48.873 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228484+0000", 2024-04-24T15:32:48.874 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:48.874 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.874 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.874 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:48.874 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228487+0000", 2024-04-24T15:32:48.874 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.9950000000000001e-06 2024-04-24T15:32:48.874 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.874 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.874 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:48.874 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228493+0000", 2024-04-24T15:32:48.874 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.2870000000000001e-06 2024-04-24T15:32:48.874 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.874 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.875 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:48.875 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228497+0000", 2024-04-24T15:32:48.875 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.1219999999999997e-06 2024-04-24T15:32:48.875 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.875 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.875 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:48.875 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.271202+0000", 2024-04-24T15:32:48.875 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042704963999999998 2024-04-24T15:32:48.875 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.875 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.875 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:48.875 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.271250+0000", 2024-04-24T15:32:48.875 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8207000000000001e-05 2024-04-24T15:32:48.875 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.876 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.876 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:48.876 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.271296+0000", 2024-04-24T15:32:48.876 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5769999999999997e-05 2024-04-24T15:32:48.876 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.876 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:48.876 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.876 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.876 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.876 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17346 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 12689408~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:48.876 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.228502+0000", 2024-04-24T15:32:48.876 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 14.569233789, 2024-04-24T15:32:48.876 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042918317999999997, 2024-04-24T15:32:48.876 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:48.877 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:48.877 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:48.877 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:48.877 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:48.877 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17346 2024-04-24T15:32:48.877 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.877 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:48.877 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.877 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:48.877 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228502+0000", 2024-04-24T15:32:48.877 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.877 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.877 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.878 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:48.878 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228502+0000", 2024-04-24T15:32:48.878 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.878 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.878 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.878 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:48.878 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228502+0000", 2024-04-24T15:32:48.878 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:48.878 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.878 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.878 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:48.878 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228504+0000", 2024-04-24T15:32:48.878 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.497e-06 2024-04-24T15:32:48.879 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.879 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.879 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:48.879 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228510+0000", 2024-04-24T15:32:48.879 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.7429999999999999e-06 2024-04-24T15:32:48.879 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.879 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.879 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:48.879 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228514+0000", 2024-04-24T15:32:48.879 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0810000000000004e-06 2024-04-24T15:32:48.879 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.879 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.879 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:48.879 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.271330+0000", 2024-04-24T15:32:48.879 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042816616000000002 2024-04-24T15:32:48.880 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.880 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.880 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:48.880 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.271371+0000", 2024-04-24T15:32:48.880 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0222999999999998e-05 2024-04-24T15:32:48.880 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.880 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.880 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:48.880 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.271420+0000", 2024-04-24T15:32:48.880 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.9539000000000001e-05 2024-04-24T15:32:48.880 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.880 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:48.880 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.880 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.881 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.881 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17347 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 12730368~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:48.881 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.228518+0000", 2024-04-24T15:32:48.881 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 14.569217261, 2024-04-24T15:32:48.881 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043022429000000001, 2024-04-24T15:32:48.881 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:48.881 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:48.881 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:48.881 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:48.881 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:48.881 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17347 2024-04-24T15:32:48.881 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.881 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:48.881 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.882 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:48.882 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228518+0000", 2024-04-24T15:32:48.882 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.882 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.882 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.882 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:48.882 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228518+0000", 2024-04-24T15:32:48.882 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:48.882 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.882 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.882 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:48.882 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228518+0000", 2024-04-24T15:32:48.882 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:48.882 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.883 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.883 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:48.883 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228521+0000", 2024-04-24T15:32:48.883 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.4559999999999999e-06 2024-04-24T15:32:48.883 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.883 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.883 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:48.883 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228526+0000", 2024-04-24T15:32:48.883 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.7259999999999999e-06 2024-04-24T15:32:48.883 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.883 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.883 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:48.883 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228530+0000", 2024-04-24T15:32:48.884 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.658e-06 2024-04-24T15:32:48.884 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.884 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.884 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:48.884 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.271445+0000", 2024-04-24T15:32:48.884 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042915506999999999 2024-04-24T15:32:48.884 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.884 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.884 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:48.884 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.271490+0000", 2024-04-24T15:32:48.884 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4752000000000001e-05 2024-04-24T15:32:48.884 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:48.884 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:48.884 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:48.885 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.271541+0000", 2024-04-24T15:32:48.885 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.0699999999999999e-05 2024-04-24T15:32:48.885 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.885 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:48.885 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.885 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:48.885 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:48.885 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:32:48.886 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-04-24T15:32:48.886 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-04-24T15:32:48.887 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #8, image image_client.0-clone8 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent9 2024-04-24T15:32:48.887 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-04-24T15:32:48.888 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-24T15:32:48.888 INFO:teuthology.orchestra.run.smithi096.stdout:2093 writesame 0xaac4f5f thru 0xaacb3df data_size 0x1(0x6481 bytes) 2024-04-24T15:32:48.889 INFO:teuthology.orchestra.run.smithi096.stdout:2094 write 0xb7cadc1 thru 0xb7caee8 (0x128 bytes) 2024-04-24T15:32:48.889 INFO:teuthology.orchestra.run.smithi096.stdout:2095 trunc from 0xb7caee9 to 0xbb76f53 2024-04-24T15:32:48.889 INFO:teuthology.orchestra.run.smithi096.stdout:2097 read 0x313953 thru 0x322c47 (0xf2f5 bytes) 2024-04-24T15:32:48.889 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-04-24T15:32:48.889 INFO:teuthology.orchestra.run.smithi096.stdout:2098 trunc from 0xbb76f53 to 0x4a0831b 2024-04-24T15:32:48.890 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:32:48.886+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:32:48.909 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-04-24T15:32:48.913 INFO:teuthology.orchestra.run.smithi096.stdout:2099 writesame 0xee5873a thru 0xee676bc data_size 0x1(0xef83 bytes) 2024-04-24T15:32:48.913 INFO:teuthology.orchestra.run.smithi096.stdout:2100 punch from 0x48641f0 to 0x48732e3, (0xf0f3 bytes) 2024-04-24T15:32:48.913 INFO:teuthology.orchestra.run.smithi096.stdout:2101 read 0x20a94ae thru 0x20affbc (0x6b0f bytes) 2024-04-24T15:32:48.913 INFO:teuthology.orchestra.run.smithi096.stdout:2103 read 0x2109b9e thru 0x210a201 (0x664 bytes) 2024-04-24T15:32:48.914 INFO:teuthology.orchestra.run.smithi096.stdout:2104 read 0x6141885 thru 0x6145a7d (0x41f9 bytes) 2024-04-24T15:32:48.914 INFO:teuthology.orchestra.run.smithi096.stdout:2105 writesame 0x6972bb1 thru 0x6973a96 data_size 0x1(0xee6 bytes) 2024-04-24T15:32:48.916 INFO:teuthology.orchestra.run.smithi096.stdout:2106 write 0x1866fd0 thru 0x1867576 (0x5a7 bytes) 2024-04-24T15:32:48.922 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:48.922 INFO:teuthology.orchestra.run.smithi096.stdout:2108 writesame 0x33e53d3 thru 0x33ebeea data_size 0x5c(0x6b18 bytes) 2024-04-24T15:32:48.932 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-04-24T15:32:48.933 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T15:32:48.933 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T15:32:48.933 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T15:32:48.933 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T15:32:48.933 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T15:32:48.934 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T15:32:48.934 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T15:32:48.934 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T15:32:48.947 INFO:teuthology.orchestra.run.smithi096.stdout:2109 trunc from 0xee676bd to 0x382acd6 2024-04-24T15:32:48.957 INFO:teuthology.orchestra.run.smithi096.stdout:2111 writesame 0x2e67e88 thru 0x2e6c220 data_size 0x1(0x4399 bytes) 2024-04-24T15:32:48.982 INFO:teuthology.orchestra.run.smithi096.stdout:2113 write 0x99b0840 thru 0x99b3123 (0x28e4 bytes) 2024-04-24T15:32:48.987 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:32:48.987+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:32:48.987 INFO:teuthology.orchestra.run.smithi096.stdout:2114 read 0x6e534ed thru 0x6e54ecd (0x19e1 bytes) 2024-04-24T15:32:48.988 INFO:teuthology.orchestra.run.smithi096.stdout:2115 read 0x3df38cc thru 0x3df49f3 (0x1128 bytes) 2024-04-24T15:32:48.996 INFO:teuthology.orchestra.run.smithi096.stdout:2117 trunc from 0x99b3124 to 0x229b555 2024-04-24T15:32:49.017 INFO:teuthology.orchestra.run.smithi096.stdout:2118 writesame 0x330d6e3 thru 0x33159fd data_size 0x1(0x831b bytes) 2024-04-24T15:32:49.017 INFO:teuthology.orchestra.run.smithi096.stdout:2119 read 0x1b09e6 thru 0x1bcb25 (0xc140 bytes) 2024-04-24T15:32:49.019 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:49.019 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:49.023 INFO:teuthology.orchestra.run.smithi096.stdout:2122 trunc from 0x33159fe to 0x80eceeb 2024-04-24T15:32:49.029 INFO:teuthology.orchestra.run.smithi096.stdout:2123 write 0x172acc2 thru 0x1738e25 (0xe164 bytes) 2024-04-24T15:32:49.030 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-24T15:32:49.044 INFO:teuthology.orchestra.run.smithi096.stdout:2124 read 0xd6c68f thru 0xd799d3 (0xd345 bytes) 2024-04-24T15:32:49.051 INFO:teuthology.orchestra.run.smithi096.stdout:2125 trunc from 0x80eceeb to 0x1e7b540 2024-04-24T15:32:49.067 INFO:teuthology.orchestra.run.smithi096.stdout:2126 write 0x646c7f7 thru 0x646ca58 (0x262 bytes) 2024-04-24T15:32:49.070 INFO:teuthology.orchestra.run.smithi096.stdout:2127 punch from 0x38bf870 to 0x38c483d, (0x4fcd bytes) 2024-04-24T15:32:49.074 INFO:teuthology.orchestra.run.smithi096.stdout:2129 trunc from 0x646ca59 to 0x366aa12 2024-04-24T15:32:49.088 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:32:49.088+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:32:49.096 INFO:teuthology.orchestra.run.smithi096.stdout:2130 write 0x9fb0e66 thru 0x9fbf32e (0xe4c9 bytes) 2024-04-24T15:32:49.104 INFO:teuthology.orchestra.run.smithi096.stdout:2131 punch from 0x6e5ad86 to 0x6e649ce, (0x9c48 bytes) 2024-04-24T15:32:49.104 INFO:teuthology.orchestra.run.smithi096.stdout:2132 read 0x5ef7490 thru 0x5efc5f5 (0x5166 bytes) 2024-04-24T15:32:49.104 INFO:teuthology.orchestra.run.smithi096.stdout:2133 writesame 0x3070f42 thru 0x3080c7b data_size 0x1(0xfd3a bytes) 2024-04-24T15:32:49.127 INFO:teuthology.orchestra.run.smithi096.stdout:2135 write 0xbf4c574 thru 0xbf5be6d (0xf8fa bytes) 2024-04-24T15:32:49.142 INFO:teuthology.orchestra.run.smithi096.stdout:2137 punch from 0x7850515 to 0x7855f3d, (0x5a28 bytes) 2024-04-24T15:32:49.142 INFO:teuthology.orchestra.run.smithi096.stdout:2140 write 0x1e51222 thru 0x1e52246 (0x1025 bytes) 2024-04-24T15:32:49.146 INFO:teuthology.orchestra.run.smithi096.stdout:2141 writesame 0x875f273 thru 0x876882a data_size 0xc(0x95b8 bytes) 2024-04-24T15:32:49.159 INFO:teuthology.orchestra.run.smithi096.stdout:2142 read 0x4cf6531 thru 0x4d03d51 (0xd821 bytes) 2024-04-24T15:32:49.159 INFO:teuthology.orchestra.run.smithi096.stdout:2143 flatten 2024-04-24T15:32:49.165 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:49.165 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:49.175 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-24T15:32:49.219 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:32:49.219+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:32:49.239 INFO:teuthology.orchestra.run.smithi096.stdout:2144 write 0x2a2a1fb thru 0x2a2e62c (0x4432 bytes) 2024-04-24T15:32:49.249 INFO:teuthology.orchestra.run.smithi096.stdout:2146 read 0xb0fce3 thru 0xb119b3 (0x1cd1 bytes) 2024-04-24T15:32:49.249 INFO:teuthology.orchestra.run.smithi096.stdout:2147 punch from 0xc6b936 to 0xc727eb, (0x6eb5 bytes) 2024-04-24T15:32:49.249 INFO:teuthology.orchestra.run.smithi096.stdout:2148 punch from 0x7c767e0 to 0x7c7b103, (0x4923 bytes) 2024-04-24T15:32:49.249 INFO:teuthology.orchestra.run.smithi096.stdout:2149 read 0x8833977 thru 0x8834340 (0x9ca bytes) 2024-04-24T15:32:49.249 INFO:teuthology.orchestra.run.smithi096.stdout:2150 punch from 0x6a81d3e to 0x6a909bc, (0xec7e bytes) 2024-04-24T15:32:49.256 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-24T15:32:49.257 INFO:teuthology.orchestra.run.smithi096.stdout:2152 trunc from 0xbf5be6e to 0x4edee35 2024-04-24T15:32:49.273 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:49.273 INFO:teuthology.orchestra.run.smithi096.stdout:2156 read 0x28de787 thru 0x28ee706 (0xff80 bytes) 2024-04-24T15:32:49.273 INFO:teuthology.orchestra.run.smithi096.stdout:2157 write 0x1f7261f thru 0x1f79a03 (0x73e5 bytes) 2024-04-24T15:32:49.278 INFO:teuthology.orchestra.run.smithi096.stdout:2158 read 0x1d761a thru 0x1e387c (0xc263 bytes) 2024-04-24T15:32:49.279 INFO:teuthology.orchestra.run.smithi096.stdout:2161 punch from 0x1582ed4 to 0x15861d7, (0x3303 bytes) 2024-04-24T15:32:49.279 INFO:teuthology.orchestra.run.smithi096.stdout:2163 punch from 0x384189c to 0x3849040, (0x77a4 bytes) 2024-04-24T15:32:49.279 INFO:teuthology.orchestra.run.smithi096.stdout:2164 punch from 0xdc48ba to 0xdcd0a0, (0x87e6 bytes) 2024-04-24T15:32:49.286 INFO:teuthology.orchestra.run.smithi096.stdout:2165 writesame 0x5ef769c thru 0x5f01236 data_size 0x1f(0x9b9b bytes) 2024-04-24T15:32:49.286 INFO:teuthology.orchestra.run.smithi096.stdout:2166 write 0x109848b thru 0x109c82a (0x43a0 bytes) 2024-04-24T15:32:49.290 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:32:49.290+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:32:49.291 INFO:teuthology.orchestra.run.smithi096.stdout:2167 punch from 0xe31d76 to 0xe34878, (0x2b02 bytes) 2024-04-24T15:32:49.291 INFO:teuthology.orchestra.run.smithi096.stdout:2168 write 0x5798f2c thru 0x57a07a5 (0x787a bytes) 2024-04-24T15:32:49.307 INFO:teuthology.orchestra.run.smithi096.stdout:2171 writesame 0x9ca0851 thru 0x9cab698 data_size 0x9c(0xae48 bytes) 2024-04-24T15:32:49.309 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:49.310 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:49.318 INFO:teuthology.orchestra.run.smithi096.stdout:2172 write 0xd048374 thru 0xd0551c5 (0xce52 bytes) 2024-04-24T15:32:49.320 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-24T15:32:49.328 INFO:teuthology.orchestra.run.smithi096.stdout:2173 write 0xa4d8f4a thru 0xa4e81c2 (0xf279 bytes) 2024-04-24T15:32:49.341 INFO:teuthology.orchestra.run.smithi096.stdout:2174 trunc from 0xd0551c6 to 0xdcf222f 2024-04-24T15:32:49.347 INFO:teuthology.orchestra.run.smithi096.stdout:2175 writesame 0xc68bfd3 thru 0xc691e22 data_size 0x18(0x5e50 bytes) 2024-04-24T15:32:49.356 INFO:teuthology.orchestra.run.smithi096.stdout:2179 writesame 0x7a5d131 thru 0x7a5f23e data_size 0x1(0x210e bytes) 2024-04-24T15:32:49.368 INFO:teuthology.orchestra.run.smithi096.stdout:2180 write 0xe2160bf thru 0xe217fc3 (0x1f05 bytes) 2024-04-24T15:32:49.382 INFO:teuthology.orchestra.run.smithi096.stdout:2181 trunc from 0xe217fc4 to 0x64bc26e 2024-04-24T15:32:49.390 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:32:49.390+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:32:49.398 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:49.398 INFO:teuthology.orchestra.run.smithi096.stdout:2183 write 0x2f68183 thru 0x2f6b75a (0x35d8 bytes) 2024-04-24T15:32:49.412 INFO:teuthology.orchestra.run.smithi096.stdout:2184 read 0x5649f0a thru 0x56562eb (0xc3e2 bytes) 2024-04-24T15:32:49.418 INFO:teuthology.orchestra.run.smithi096.stdout:2186 trunc from 0x64bc26e to 0x295bb00 2024-04-24T15:32:49.438 INFO:teuthology.orchestra.run.smithi096.stdout:2188 writesame 0x2af6d5d thru 0x2affab3 data_size 0x7(0x8d57 bytes) 2024-04-24T15:32:49.443 INFO:teuthology.orchestra.run.smithi096.stdout:2190 trunc from 0x2affab4 to 0x618db1e 2024-04-24T15:32:49.449 INFO:teuthology.orchestra.run.smithi096.stdout:2191 write 0x42f3a24 thru 0x42f985a (0x5e37 bytes) 2024-04-24T15:32:49.457 INFO:teuthology.orchestra.run.smithi096.stdout:2193 punch from 0x29fe3a1 to 0x2a01380, (0x2fdf bytes) 2024-04-24T15:32:49.472 INFO:teuthology.orchestra.run.smithi096.stdout:2194 write 0xc835383 thru 0xc83cba8 (0x7826 bytes) 2024-04-24T15:32:49.478 INFO:teuthology.orchestra.run.smithi096.stdout:2195 punch from 0x11a9fe7 to 0x11aefd1, (0x4fea bytes) 2024-04-24T15:32:49.479 INFO:teuthology.orchestra.run.smithi096.stdout:2196 read 0x1e6e41f thru 0x1e75727 (0x7309 bytes) 2024-04-24T15:32:49.486 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:49.486 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:49.488 INFO:teuthology.orchestra.run.smithi096.stdout:2197 write 0xe53c80b thru 0xe547969 (0xb15f bytes) 2024-04-24T15:32:49.491 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:32:49.491+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:32:49.500 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-24T15:32:49.500 INFO:teuthology.orchestra.run.smithi096.stdout:2198 writesame 0xa457f44 thru 0xa4654f6 data_size 0x1(0xd5b3 bytes) 2024-04-24T15:32:49.517 INFO:teuthology.orchestra.run.smithi096.stdout:2199 punch from 0x18e6c9e to 0x18f6c53, (0xffb5 bytes) 2024-04-24T15:32:49.517 INFO:teuthology.orchestra.run.smithi096.stdout:2202 writesame 0x1311aeb thru 0x1319f3b data_size 0x7(0x8451 bytes) 2024-04-24T15:32:49.523 INFO:teuthology.orchestra.run.smithi096.stdout:2203 read 0x139df70 thru 0x139f24a (0x12db bytes) 2024-04-24T15:32:49.540 INFO:teuthology.orchestra.run.smithi096.stdout:2204 trunc from 0xe54796a to 0x20d544 2024-04-24T15:32:49.570 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:49.571 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:49.584 INFO:teuthology.orchestra.run.smithi096.stdout:2209 write 0x78da1d9 thru 0x78dd78b (0x35b3 bytes) 2024-04-24T15:32:49.591 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:32:49.592+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:32:49.597 INFO:teuthology.orchestra.run.smithi096.stdout:2211 write 0xb5770b7 thru 0xb57831e (0x1268 bytes) 2024-04-24T15:32:49.600 INFO:teuthology.orchestra.run.smithi096.stdout:2212 read 0x6f3f2e7 thru 0x6f4de8e (0xeba8 bytes) 2024-04-24T15:32:49.601 INFO:teuthology.orchestra.run.smithi096.stdout:2215 read 0x1f73b1f thru 0x1f74060 (0x542 bytes) 2024-04-24T15:32:49.601 INFO:teuthology.orchestra.run.smithi096.stdout:2217 write 0x8b7d7e8 thru 0x8b899d0 (0xc1e9 bytes) 2024-04-24T15:32:49.612 INFO:teuthology.orchestra.run.smithi096.stdout:2218 write 0x41f4957 thru 0x41fe569 (0x9c13 bytes) 2024-04-24T15:32:49.623 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:49.629 INFO:teuthology.orchestra.run.smithi096.stdout:2221 write 0xb5bdfa8 thru 0xb5c4c5e (0x6cb7 bytes) 2024-04-24T15:32:49.640 INFO:teuthology.orchestra.run.smithi096.stdout:2222 trunc from 0xb5c4c5f to 0xab42e90 2024-04-24T15:32:49.676 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:49.676 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:49.686 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-24T15:32:49.728 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:32:49.728+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:32:49.793 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:32:49.794+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:32:49.849 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:49.849 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:49.859 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-24T15:32:49.900 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:32:49.900+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:32:49.996 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:32:49.996+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:32:50.023 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:50.023 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:50.033 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-24T15:32:50.117 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:32:50.117+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:32:50.196 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:50.196 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:50.197 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:32:50.197+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:32:50.206 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-24T15:32:50.298 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:32:50.298+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:32:50.369 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:50.369 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:50.378 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-24T15:32:50.420 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:32:50.420+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:32:50.501 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:32:50.501+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:32:50.543 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:50.543 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:50.553 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-24T15:32:50.601 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:32:50.602+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:32:50.652 INFO:teuthology.orchestra.run.smithi096.stderr:nodeep-scrub is unset 2024-04-24T15:32:50.665 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:50.673 INFO:teuthology.orchestra.run.smithi096.stdout:2224 trunc from 0xab42e90 to 0x40a9f73 2024-04-24T15:32:50.689 INFO:teuthology.orchestra.run.smithi096.stdout:2225 punch from 0x1848a0a to 0x1851867, (0x8e5d bytes) 2024-04-24T15:32:50.689 INFO:teuthology.orchestra.run.smithi096.stdout:2226 punch from 0x99d635 to 0x9ab6fc, (0xe0c7 bytes) 2024-04-24T15:32:50.689 INFO:teuthology.orchestra.run.smithi096.stdout:2227 clone 11 order 24 su 4194304 sc 11 2024-04-24T15:32:50.702 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:32:50.702+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:32:50.729 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:50.729 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:50.739 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-24T15:32:50.823 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:32:50.823+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:32:50.903 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:50.904 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:50.904 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:32:50.904+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:32:50.914 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-24T15:32:50.955 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-24T15:32:51.004 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:32:51.004+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:32:51.084 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:51.084 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:51.094 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-24T15:32:51.097 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-04-24T15:32:51.105 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:32:51.105+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:32:51.118 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-04-24T15:32:51.137 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-04-24T15:32:51.156 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-04-24T15:32:51.175 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-04-24T15:32:51.193 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-04-24T15:32:51.206 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:32:51.206+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:32:51.212 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-04-24T15:32:51.224 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:32:51.224 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:32:51.224 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-04-24T15:32:51.224 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-04-24T15:32:51.224 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:32:51.231 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-04-24T15:32:51.233 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T15:32:51.233 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T15:32:51.233 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T15:32:51.233 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T15:32:51.233 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T15:32:51.233 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T15:32:51.233 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T15:32:51.233 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T15:32:51.234 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-24T15:32:51.328 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:32:51.328+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:32:51.407 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:32:51.407+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:32:51.408 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:32:51.408 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-04-24T15:32:51.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-04-24T15:32:51.408 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-04-24T15:32:51.408 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.408 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14043 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2383872~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:51.408 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892347+0000", 2024-04-24T15:32:51.408 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 17.516063047999999, 2024-04-24T15:32:51.409 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038537279000000001, 2024-04-24T15:32:51.409 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:51.409 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:51.409 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:51.409 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:51.409 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:51.409 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14043 2024-04-24T15:32:51.409 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.409 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:51.409 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.409 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:51.409 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892347+0000", 2024-04-24T15:32:51.409 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:51.409 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.409 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.409 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:51.410 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892347+0000", 2024-04-24T15:32:51.410 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:51.410 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.410 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.410 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:51.410 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892347+0000", 2024-04-24T15:32:51.410 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:51.410 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.410 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.410 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:51.410 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892349+0000", 2024-04-24T15:32:51.410 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3400000000000001e-06 2024-04-24T15:32:51.410 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.410 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.410 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:51.411 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892357+0000", 2024-04-24T15:32:51.411 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1540000000000002e-06 2024-04-24T15:32:51.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:51.411 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892360+0000", 2024-04-24T15:32:51.411 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5829999999999998e-06 2024-04-24T15:32:51.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:51.411 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930795+0000", 2024-04-24T15:32:51.411 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038434590999999997 2024-04-24T15:32:51.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:51.411 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930836+0000", 2024-04-24T15:32:51.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0827000000000001e-05 2024-04-24T15:32:51.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:51.412 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930885+0000", 2024-04-24T15:32:51.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9029000000000001e-05 2024-04-24T15:32:51.412 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:51.412 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:51.412 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:51.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.412 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14044 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2424832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:51.412 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892372+0000", 2024-04-24T15:32:51.412 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 17.516038404, 2024-04-24T15:32:51.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038624890000000002, 2024-04-24T15:32:51.413 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:51.413 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:51.413 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:51.413 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:51.413 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:51.413 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14044 2024-04-24T15:32:51.413 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.413 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:51.413 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.413 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:51.413 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892372+0000", 2024-04-24T15:32:51.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:51.413 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.414 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.414 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:51.414 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892372+0000", 2024-04-24T15:32:51.414 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:51.414 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.414 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.414 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:51.414 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892372+0000", 2024-04-24T15:32:51.414 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:51.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.415 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.415 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:51.415 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892374+0000", 2024-04-24T15:32:51.415 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4899999999999999e-06 2024-04-24T15:32:51.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.415 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.415 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:51.415 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892380+0000", 2024-04-24T15:32:51.415 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.631e-06 2024-04-24T15:32:51.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.415 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.416 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:51.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892383+0000", 2024-04-24T15:32:51.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7400000000000002e-06 2024-04-24T15:32:51.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.416 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.416 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:51.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930905+0000", 2024-04-24T15:32:51.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038521396999999999 2024-04-24T15:32:51.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.416 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.416 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:51.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930949+0000", 2024-04-24T15:32:51.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4153999999999998e-05 2024-04-24T15:32:51.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.417 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.417 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:51.417 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930997+0000", 2024-04-24T15:32:51.417 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7998999999999999e-05 2024-04-24T15:32:51.417 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:51.417 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:51.417 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:51.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.417 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.417 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14045 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2465792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:51.417 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892387+0000", 2024-04-24T15:32:51.417 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 17.516023137000001, 2024-04-24T15:32:51.417 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038720933999999999, 2024-04-24T15:32:51.417 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:51.418 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:51.418 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:51.418 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:51.418 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:51.418 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14045 2024-04-24T15:32:51.418 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.418 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:51.418 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.418 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:51.418 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892387+0000", 2024-04-24T15:32:51.418 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:51.418 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.418 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.418 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:51.419 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892387+0000", 2024-04-24T15:32:51.419 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:51.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.419 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.419 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:51.419 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892387+0000", 2024-04-24T15:32:51.419 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:51.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.419 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.419 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:51.419 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892392+0000", 2024-04-24T15:32:51.419 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0039999999999999e-06 2024-04-24T15:32:51.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.419 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.420 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:51.420 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892398+0000", 2024-04-24T15:32:51.420 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0900000000000001e-06 2024-04-24T15:32:51.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.420 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:51.420 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892402+0000", 2024-04-24T15:32:51.420 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.399e-06 2024-04-24T15:32:51.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.420 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:51.420 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931018+0000", 2024-04-24T15:32:51.420 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038616551999999998 2024-04-24T15:32:51.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.421 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:51.421 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931059+0000", 2024-04-24T15:32:51.421 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1205000000000003e-05 2024-04-24T15:32:51.421 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.421 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.421 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:51.421 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931108+0000", 2024-04-24T15:32:51.421 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8996000000000003e-05 2024-04-24T15:32:51.421 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:51.421 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:51.421 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:51.421 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.421 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.422 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14046 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2506752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:51.422 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892406+0000", 2024-04-24T15:32:51.422 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 17.516004547000001, 2024-04-24T15:32:51.422 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038814507999999998, 2024-04-24T15:32:51.422 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:51.422 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:51.422 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:51.422 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:51.422 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:51.422 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14046 2024-04-24T15:32:51.422 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.425 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:51.425 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.425 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:51.425 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892406+0000", 2024-04-24T15:32:51.426 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:51.426 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.426 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.426 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:51.426 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892406+0000", 2024-04-24T15:32:51.426 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:51.426 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.426 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.426 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:51.426 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892406+0000", 2024-04-24T15:32:51.426 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:51.426 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.426 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.426 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:51.427 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892413+0000", 2024-04-24T15:32:51.427 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7449999999999995e-06 2024-04-24T15:32:51.427 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.427 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.427 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:51.427 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892419+0000", 2024-04-24T15:32:51.427 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5790000000000003e-06 2024-04-24T15:32:51.427 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.427 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.427 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:51.427 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892423+0000", 2024-04-24T15:32:51.427 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8199999999999998e-06 2024-04-24T15:32:51.427 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.427 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.428 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:51.428 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931128+0000", 2024-04-24T15:32:51.428 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038705502000000003 2024-04-24T15:32:51.428 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.428 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.428 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:51.428 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931172+0000", 2024-04-24T15:32:51.428 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4066999999999997e-05 2024-04-24T15:32:51.428 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.428 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.428 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:51.428 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931220+0000", 2024-04-24T15:32:51.428 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8167999999999997e-05 2024-04-24T15:32:51.428 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:51.428 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:51.428 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:51.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.429 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.429 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14047 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2547712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:51.429 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892427+0000", 2024-04-24T15:32:51.429 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 17.515983318, 2024-04-24T15:32:51.429 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038904506999999998, 2024-04-24T15:32:51.429 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:51.429 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:51.429 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:51.429 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:51.429 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:51.429 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14047 2024-04-24T15:32:51.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.429 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:51.430 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.430 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:51.430 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892427+0000", 2024-04-24T15:32:51.430 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:51.430 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.430 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.430 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:51.430 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892427+0000", 2024-04-24T15:32:51.430 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:51.430 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.430 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.430 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:51.430 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892427+0000", 2024-04-24T15:32:51.430 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:51.430 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.431 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.431 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:51.431 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892429+0000", 2024-04-24T15:32:51.431 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3099999999999999e-06 2024-04-24T15:32:51.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.431 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.431 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:51.431 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892435+0000", 2024-04-24T15:32:51.431 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5520000000000004e-06 2024-04-24T15:32:51.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.431 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.431 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:51.431 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892438+0000", 2024-04-24T15:32:51.431 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4240000000000002e-06 2024-04-24T15:32:51.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.432 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.432 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:51.432 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931242+0000", 2024-04-24T15:32:51.432 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038804162000000003 2024-04-24T15:32:51.432 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.432 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.432 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:51.432 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931284+0000", 2024-04-24T15:32:51.432 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1414e-05 2024-04-24T15:32:51.432 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.432 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.432 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:51.432 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931332+0000", 2024-04-24T15:32:51.432 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8024e-05 2024-04-24T15:32:51.432 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:51.433 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:51.433 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:51.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.433 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.433 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14048 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2588672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:51.433 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892442+0000", 2024-04-24T15:32:51.433 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 17.515968354999998, 2024-04-24T15:32:51.433 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039003132000000003, 2024-04-24T15:32:51.433 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:51.433 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:51.433 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:51.433 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:51.433 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:51.433 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14048 2024-04-24T15:32:51.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.434 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:51.434 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.434 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:51.434 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892442+0000", 2024-04-24T15:32:51.434 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:51.434 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.434 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.434 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:51.434 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892442+0000", 2024-04-24T15:32:51.434 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:51.434 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.434 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.434 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:51.435 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892442+0000", 2024-04-24T15:32:51.435 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:51.435 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.435 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.435 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:51.435 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892444+0000", 2024-04-24T15:32:51.435 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1169999999999998e-06 2024-04-24T15:32:51.435 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.435 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.435 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:51.435 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892449+0000", 2024-04-24T15:32:51.435 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1599999999999997e-06 2024-04-24T15:32:51.435 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.435 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.435 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:51.435 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892453+0000", 2024-04-24T15:32:51.436 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5930000000000002e-06 2024-04-24T15:32:51.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.436 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:51.436 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931352+0000", 2024-04-24T15:32:51.436 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038899361 2024-04-24T15:32:51.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.436 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:51.436 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931398+0000", 2024-04-24T15:32:51.436 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6315e-05 2024-04-24T15:32:51.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.436 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:51.437 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931445+0000", 2024-04-24T15:32:51.437 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6950000000000003e-05 2024-04-24T15:32:51.437 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:51.437 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:51.437 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:51.437 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.437 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.437 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14049 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2629632~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:51.437 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892458+0000", 2024-04-24T15:32:51.437 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 17.515952935000001, 2024-04-24T15:32:51.437 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039103322000000003, 2024-04-24T15:32:51.437 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:51.437 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:51.437 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:51.437 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:51.437 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:51.438 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14049 2024-04-24T15:32:51.438 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.438 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:51.438 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.438 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:51.438 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892458+0000", 2024-04-24T15:32:51.438 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:51.438 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.438 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.438 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:51.438 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892458+0000", 2024-04-24T15:32:51.438 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:51.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.439 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:51.439 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892457+0000", 2024-04-24T15:32:51.439 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:51.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.439 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:51.439 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892459+0000", 2024-04-24T15:32:51.439 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.305e-06 2024-04-24T15:32:51.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.439 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:51.439 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892465+0000", 2024-04-24T15:32:51.439 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4360000000000001e-06 2024-04-24T15:32:51.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.440 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.440 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:51.440 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892474+0000", 2024-04-24T15:32:51.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7639999999999994e-06 2024-04-24T15:32:51.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.440 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.440 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:51.440 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931466+0000", 2024-04-24T15:32:51.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038992644 2024-04-24T15:32:51.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.440 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.440 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:51.440 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931512+0000", 2024-04-24T15:32:51.441 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5200000000000001e-05 2024-04-24T15:32:51.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.441 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:51.441 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931561+0000", 2024-04-24T15:32:51.441 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9357999999999997e-05 2024-04-24T15:32:51.441 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:51.441 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:51.441 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:51.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.441 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14050 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2670592~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:51.441 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892478+0000", 2024-04-24T15:32:51.442 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 17.515932348, 2024-04-24T15:32:51.442 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039195102000000002, 2024-04-24T15:32:51.442 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:51.442 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:51.442 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:51.442 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:51.442 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:51.442 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14050 2024-04-24T15:32:51.442 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.442 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:51.442 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.442 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:51.442 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892478+0000", 2024-04-24T15:32:51.443 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:51.443 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.443 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.443 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:51.443 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892478+0000", 2024-04-24T15:32:51.443 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:51.443 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.443 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.443 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:51.443 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892478+0000", 2024-04-24T15:32:51.443 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:51.443 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.443 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.443 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:51.443 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892480+0000", 2024-04-24T15:32:51.444 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3300000000000001e-06 2024-04-24T15:32:51.444 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.444 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.444 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:51.444 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892486+0000", 2024-04-24T15:32:51.444 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9200000000000001e-06 2024-04-24T15:32:51.444 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.444 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.444 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:51.444 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892494+0000", 2024-04-24T15:32:51.444 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5159999999999997e-06 2024-04-24T15:32:51.444 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.444 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.444 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:51.444 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931581+0000", 2024-04-24T15:32:51.445 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039087894999999998 2024-04-24T15:32:51.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.445 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.445 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:51.445 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931626+0000", 2024-04-24T15:32:51.445 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4943e-05 2024-04-24T15:32:51.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.445 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.445 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:51.445 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931673+0000", 2024-04-24T15:32:51.445 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6906000000000001e-05 2024-04-24T15:32:51.445 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:51.445 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:51.446 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:51.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.446 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.446 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14051 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2711552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:51.446 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892507+0000", 2024-04-24T15:32:51.446 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 17.515903648999998, 2024-04-24T15:32:51.446 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039274393999999997, 2024-04-24T15:32:51.446 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:51.446 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:51.446 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:51.446 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:51.446 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:51.446 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14051 2024-04-24T15:32:51.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.447 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:51.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.447 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:51.447 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892507+0000", 2024-04-24T15:32:51.447 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:51.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.447 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:51.447 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892507+0000", 2024-04-24T15:32:51.447 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:51.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.447 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:51.447 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892506+0000", 2024-04-24T15:32:51.447 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:51.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.448 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.448 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:51.448 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892509+0000", 2024-04-24T15:32:51.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8760000000000001e-06 2024-04-24T15:32:51.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.448 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.448 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:51.448 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892515+0000", 2024-04-24T15:32:51.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1840000000000004e-06 2024-04-24T15:32:51.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.448 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.448 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:51.448 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892519+0000", 2024-04-24T15:32:51.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7570000000000002e-06 2024-04-24T15:32:51.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.449 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:51.449 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931694+0000", 2024-04-24T15:32:51.449 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039174960000000002 2024-04-24T15:32:51.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.449 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:51.449 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931734+0000", 2024-04-24T15:32:51.449 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0284000000000003e-05 2024-04-24T15:32:51.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.449 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:51.449 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931781+0000", 2024-04-24T15:32:51.450 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6813000000000001e-05 2024-04-24T15:32:51.450 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:51.450 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:51.450 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:51.450 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.450 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.450 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14052 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2752512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:51.450 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892524+0000", 2024-04-24T15:32:51.450 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 17.515886502000001, 2024-04-24T15:32:51.450 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039371122000000001, 2024-04-24T15:32:51.450 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:51.450 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:51.450 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:51.451 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:51.451 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:51.451 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14052 2024-04-24T15:32:51.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.451 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:51.451 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.451 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:51.451 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892524+0000", 2024-04-24T15:32:51.451 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:51.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.451 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.451 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:51.451 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892524+0000", 2024-04-24T15:32:51.451 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:51.452 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.452 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.452 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:51.452 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892524+0000", 2024-04-24T15:32:51.452 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:51.452 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.452 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.452 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:51.452 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892526+0000", 2024-04-24T15:32:51.452 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3630000000000001e-06 2024-04-24T15:32:51.452 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.452 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.452 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:51.452 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892538+0000", 2024-04-24T15:32:51.452 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1677e-05 2024-04-24T15:32:51.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.453 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.453 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:51.453 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892541+0000", 2024-04-24T15:32:51.453 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.822e-06 2024-04-24T15:32:51.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.453 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.453 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:51.453 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931802+0000", 2024-04-24T15:32:51.453 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039260911000000003 2024-04-24T15:32:51.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.453 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.453 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:51.453 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931847+0000", 2024-04-24T15:32:51.453 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4450999999999998e-05 2024-04-24T15:32:51.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.454 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.454 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:51.454 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931895+0000", 2024-04-24T15:32:51.454 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8257999999999998e-05 2024-04-24T15:32:51.454 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:51.454 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:51.454 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:51.454 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.454 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.454 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14053 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2793472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:51.454 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892546+0000", 2024-04-24T15:32:51.454 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 17.515864524000001, 2024-04-24T15:32:51.454 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039457712999999998, 2024-04-24T15:32:51.454 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:51.454 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:51.455 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:51.455 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:51.455 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:51.455 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14053 2024-04-24T15:32:51.455 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.455 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:51.455 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.455 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:51.455 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892546+0000", 2024-04-24T15:32:51.455 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:51.455 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.455 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.455 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:51.455 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892546+0000", 2024-04-24T15:32:51.456 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:51.456 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.456 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.456 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:51.456 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892546+0000", 2024-04-24T15:32:51.456 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:51.456 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.456 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.456 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:51.456 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892548+0000", 2024-04-24T15:32:51.456 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3120000000000001e-06 2024-04-24T15:32:51.456 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.456 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.456 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:51.456 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892553+0000", 2024-04-24T15:32:51.457 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2320000000000001e-06 2024-04-24T15:32:51.457 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.457 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:51.457 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892559+0000", 2024-04-24T15:32:51.457 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5480000000000001e-06 2024-04-24T15:32:51.457 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.457 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:51.457 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931916+0000", 2024-04-24T15:32:51.457 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039357180999999998 2024-04-24T15:32:51.457 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.457 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:51.458 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931956+0000", 2024-04-24T15:32:51.458 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0565000000000002e-05 2024-04-24T15:32:51.458 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.458 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.458 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:51.458 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932004+0000", 2024-04-24T15:32:51.458 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7224000000000001e-05 2024-04-24T15:32:51.458 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:51.458 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:51.458 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:51.458 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.458 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.458 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14054 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2834432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:51.459 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892563+0000", 2024-04-24T15:32:51.459 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 17.515847871999998, 2024-04-24T15:32:51.459 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039552406999999998, 2024-04-24T15:32:51.459 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:51.459 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:51.459 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:51.459 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:51.459 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:51.459 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14054 2024-04-24T15:32:51.459 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.459 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:51.459 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.459 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:51.459 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892563+0000", 2024-04-24T15:32:51.459 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:51.459 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.460 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.460 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:51.460 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892563+0000", 2024-04-24T15:32:51.460 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:51.460 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.460 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.460 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:51.460 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892562+0000", 2024-04-24T15:32:51.460 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:51.460 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.460 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.460 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:51.460 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892564+0000", 2024-04-24T15:32:51.460 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2029999999999999e-06 2024-04-24T15:32:51.460 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.460 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.461 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:51.461 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892570+0000", 2024-04-24T15:32:51.461 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1460000000000004e-06 2024-04-24T15:32:51.461 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.461 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.461 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:51.461 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892573+0000", 2024-04-24T15:32:51.461 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2030000000000001e-06 2024-04-24T15:32:51.461 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.461 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.461 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:51.461 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932024+0000", 2024-04-24T15:32:51.461 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039451696000000001 2024-04-24T15:32:51.461 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.461 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.462 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:51.462 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932068+0000", 2024-04-24T15:32:51.462 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3970000000000001e-05 2024-04-24T15:32:51.462 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.462 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.462 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:51.462 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932115+0000", 2024-04-24T15:32:51.462 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6569000000000001e-05 2024-04-24T15:32:51.462 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:51.462 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:51.462 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:51.462 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.462 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.462 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14055 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2875392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:51.463 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892591+0000", 2024-04-24T15:32:51.463 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 17.515819899, 2024-04-24T15:32:51.463 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039634309, 2024-04-24T15:32:51.463 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:51.463 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:51.463 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:51.463 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:51.463 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:51.463 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14055 2024-04-24T15:32:51.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.463 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:51.463 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.463 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:51.464 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892591+0000", 2024-04-24T15:32:51.464 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:51.464 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.464 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.464 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:51.464 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892591+0000", 2024-04-24T15:32:51.464 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:51.464 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.464 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.464 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:51.464 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892590+0000", 2024-04-24T15:32:51.464 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:51.464 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.464 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.464 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:51.465 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892592+0000", 2024-04-24T15:32:51.465 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7519999999999999e-06 2024-04-24T15:32:51.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.465 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.465 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:51.465 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892596+0000", 2024-04-24T15:32:51.465 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.608e-06 2024-04-24T15:32:51.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.465 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.465 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:51.465 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892598+0000", 2024-04-24T15:32:51.465 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.531e-06 2024-04-24T15:32:51.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.465 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.465 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:51.466 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932136+0000", 2024-04-24T15:32:51.466 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039537668999999998 2024-04-24T15:32:51.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.466 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.466 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:51.466 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932177+0000", 2024-04-24T15:32:51.466 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0880000000000002e-05 2024-04-24T15:32:51.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.466 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.466 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:51.466 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932225+0000", 2024-04-24T15:32:51.466 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8223999999999998e-05 2024-04-24T15:32:51.467 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:51.467 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:51.467 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:51.467 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.467 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.467 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14056 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2916352~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:51.467 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892600+0000", 2024-04-24T15:32:51.467 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 17.515810065, 2024-04-24T15:32:51.467 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039736447000000001, 2024-04-24T15:32:51.467 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:51.467 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:51.467 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:51.467 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:51.467 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:51.468 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14056 2024-04-24T15:32:51.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.468 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:51.468 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.468 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:51.468 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892600+0000", 2024-04-24T15:32:51.468 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:51.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.468 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.468 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:51.468 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892600+0000", 2024-04-24T15:32:51.468 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:51.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.468 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.468 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:51.469 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892600+0000", 2024-04-24T15:32:51.469 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967296 2024-04-24T15:32:51.469 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.469 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.469 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:51.469 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892602+0000", 2024-04-24T15:32:51.469 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.576e-06 2024-04-24T15:32:51.469 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.469 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.469 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:51.469 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892605+0000", 2024-04-24T15:32:51.469 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.574e-06 2024-04-24T15:32:51.469 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.470 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.470 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:51.470 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892608+0000", 2024-04-24T15:32:51.470 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3800000000000001e-06 2024-04-24T15:32:51.470 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.470 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.470 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:51.470 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932245+0000", 2024-04-24T15:32:51.470 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039637378000000001 2024-04-24T15:32:51.470 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.471 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.471 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:51.471 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932290+0000", 2024-04-24T15:32:51.471 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4734999999999998e-05 2024-04-24T15:32:51.471 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.471 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.471 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:51.471 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932337+0000", 2024-04-24T15:32:51.471 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7033000000000002e-05 2024-04-24T15:32:51.471 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:51.471 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:51.471 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:51.471 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.471 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.471 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14063 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 3203072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:51.472 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.894564+0000", 2024-04-24T15:32:51.472 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 17.513846474000001, 2024-04-24T15:32:51.472 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038554995000000002, 2024-04-24T15:32:51.472 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:51.472 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:51.472 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:51.472 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:51.472 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:51.472 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14063 2024-04-24T15:32:51.472 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.472 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:51.472 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.472 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:51.472 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894564+0000", 2024-04-24T15:32:51.473 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:51.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.473 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.473 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:51.473 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894564+0000", 2024-04-24T15:32:51.473 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:51.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.473 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.473 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:51.473 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894564+0000", 2024-04-24T15:32:51.473 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967296 2024-04-24T15:32:51.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.473 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.473 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:51.474 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894565+0000", 2024-04-24T15:32:51.474 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2640000000000001e-06 2024-04-24T15:32:51.474 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.474 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.474 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:51.474 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894569+0000", 2024-04-24T15:32:51.474 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1710000000000004e-06 2024-04-24T15:32:51.474 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.474 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.474 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:51.474 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894573+0000", 2024-04-24T15:32:51.474 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4410000000000002e-06 2024-04-24T15:32:51.474 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.475 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.475 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:51.475 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933030+0000", 2024-04-24T15:32:51.475 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038457709999999999 2024-04-24T15:32:51.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.475 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.475 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:51.475 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933072+0000", 2024-04-24T15:32:51.475 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.193e-05 2024-04-24T15:32:51.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.475 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.475 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:51.475 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933119+0000", 2024-04-24T15:32:51.475 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6715000000000003e-05 2024-04-24T15:32:51.475 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:51.476 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:51.476 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:51.476 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.476 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.476 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14064 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 3244032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:51.476 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.894591+0000", 2024-04-24T15:32:51.476 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 17.513819831999999, 2024-04-24T15:32:51.476 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038639806999999998, 2024-04-24T15:32:51.476 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:51.476 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:51.476 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:51.476 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:51.476 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:51.476 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14064 2024-04-24T15:32:51.476 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.477 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:51.477 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.477 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:51.477 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894591+0000", 2024-04-24T15:32:51.477 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:51.477 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.477 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.477 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:51.477 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894591+0000", 2024-04-24T15:32:51.477 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:51.477 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.477 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.477 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:51.477 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894590+0000", 2024-04-24T15:32:51.478 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:51.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.478 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.478 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:51.478 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894593+0000", 2024-04-24T15:32:51.478 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4719999999999998e-06 2024-04-24T15:32:51.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.478 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.478 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:51.478 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894598+0000", 2024-04-24T15:32:51.478 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.57e-06 2024-04-24T15:32:51.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.478 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.479 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:51.479 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894602+0000", 2024-04-24T15:32:51.479 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.614e-06 2024-04-24T15:32:51.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.479 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.479 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:51.479 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933139+0000", 2024-04-24T15:32:51.479 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038536929999999997 2024-04-24T15:32:51.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.479 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.479 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:51.479 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933183+0000", 2024-04-24T15:32:51.479 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4217e-05 2024-04-24T15:32:51.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.480 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.480 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:51.480 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933230+0000", 2024-04-24T15:32:51.480 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7434e-05 2024-04-24T15:32:51.480 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:51.480 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:51.480 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:51.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.480 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.480 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14065 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 3284992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:51.480 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.894606+0000", 2024-04-24T15:32:51.480 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 17.513804449999999, 2024-04-24T15:32:51.480 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038734126000000001, 2024-04-24T15:32:51.481 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:51.481 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:51.481 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:51.481 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:51.481 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:51.481 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14065 2024-04-24T15:32:51.481 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.481 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:51.481 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.481 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:51.481 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894606+0000", 2024-04-24T15:32:51.481 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:51.481 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.481 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.482 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:51.482 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894606+0000", 2024-04-24T15:32:51.482 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:51.482 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.482 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.482 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:51.482 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894606+0000", 2024-04-24T15:32:51.482 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:51.482 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.482 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.482 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:51.482 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894608+0000", 2024-04-24T15:32:51.483 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0880000000000002e-06 2024-04-24T15:32:51.483 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.483 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.483 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:51.483 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894614+0000", 2024-04-24T15:32:51.483 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7830000000000004e-06 2024-04-24T15:32:51.483 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.483 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.483 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:51.483 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894621+0000", 2024-04-24T15:32:51.483 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5809999999999999e-06 2024-04-24T15:32:51.483 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.483 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.484 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:51.484 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933251+0000", 2024-04-24T15:32:51.484 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038629995 2024-04-24T15:32:51.484 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.484 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.484 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:51.484 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933292+0000", 2024-04-24T15:32:51.484 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0380000000000003e-05 2024-04-24T15:32:51.484 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.484 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.484 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:51.484 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933340+0000", 2024-04-24T15:32:51.484 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8662999999999998e-05 2024-04-24T15:32:51.484 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:51.484 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:51.485 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:51.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.485 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.485 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14066 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 3325952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:51.485 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.894625+0000", 2024-04-24T15:32:51.485 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 17.513785011, 2024-04-24T15:32:51.485 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038825539999999999, 2024-04-24T15:32:51.485 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:51.485 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:51.485 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:51.485 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:51.485 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:51.485 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14066 2024-04-24T15:32:51.486 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.486 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:51.486 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.486 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:51.486 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894625+0000", 2024-04-24T15:32:51.486 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:51.486 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.486 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.486 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:51.486 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894625+0000", 2024-04-24T15:32:51.486 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:51.486 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.487 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.487 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:51.487 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894625+0000", 2024-04-24T15:32:51.487 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:51.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.487 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.487 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:51.487 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894627+0000", 2024-04-24T15:32:51.487 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.092e-06 2024-04-24T15:32:51.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.487 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.487 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:51.487 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894635+0000", 2024-04-24T15:32:51.487 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6660000000000008e-06 2024-04-24T15:32:51.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.488 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.488 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:51.488 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894639+0000", 2024-04-24T15:32:51.488 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.799e-06 2024-04-24T15:32:51.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.488 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.488 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:51.488 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933360+0000", 2024-04-24T15:32:51.488 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038721569999999997 2024-04-24T15:32:51.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.488 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.488 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:51.488 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933403+0000", 2024-04-24T15:32:51.489 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2605999999999998e-05 2024-04-24T15:32:51.489 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.489 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.489 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:51.489 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933451+0000", 2024-04-24T15:32:51.489 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8149999999999998e-05 2024-04-24T15:32:51.489 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:51.489 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:51.489 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:51.489 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.489 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.489 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14067 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 3366912~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:51.489 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.894644+0000", 2024-04-24T15:32:51.489 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 17.513766953000001, 2024-04-24T15:32:51.490 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038926068000000001, 2024-04-24T15:32:51.490 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:51.490 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:51.490 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:51.490 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:51.490 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:51.490 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14067 2024-04-24T15:32:51.490 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.490 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:51.490 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.490 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:51.490 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894644+0000", 2024-04-24T15:32:51.490 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:51.491 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.491 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.491 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:51.491 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894644+0000", 2024-04-24T15:32:51.491 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:51.491 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.491 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.491 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:51.491 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894643+0000", 2024-04-24T15:32:51.491 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:51.491 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.491 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.491 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:51.491 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894646+0000", 2024-04-24T15:32:51.491 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3379999999999999e-06 2024-04-24T15:32:51.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.492 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.492 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:51.492 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894651+0000", 2024-04-24T15:32:51.492 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.485e-06 2024-04-24T15:32:51.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.492 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.492 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:51.492 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894655+0000", 2024-04-24T15:32:51.492 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.709e-06 2024-04-24T15:32:51.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.492 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.492 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:51.492 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933472+0000", 2024-04-24T15:32:51.492 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038816992000000002 2024-04-24T15:32:51.493 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.493 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.493 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:51.493 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933521+0000", 2024-04-24T15:32:51.493 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8841999999999998e-05 2024-04-24T15:32:51.493 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.493 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.493 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:51.493 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933570+0000", 2024-04-24T15:32:51.493 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9063000000000001e-05 2024-04-24T15:32:51.493 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:51.493 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:51.493 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:51.493 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.493 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.494 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:30240 4.0 4:059b194c:::rbd_object_map.114a2e5eebc:head [read 0~43] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-04-24T15:32:51.494 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:47.244473+0000", 2024-04-24T15:32:51.494 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.1639377670000002, 2024-04-24T15:32:51.494 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4146627299999999, 2024-04-24T15:32:51.494 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:32:51.494 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:32:51.494 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:32:51.494 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:32:51.494 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:51.494 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30240 2024-04-24T15:32:51.494 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.494 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:32:51.494 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.495 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:32:51.495 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.244473+0000", 2024-04-24T15:32:51.495 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:51.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.495 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.495 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:32:51.495 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.244473+0000", 2024-04-24T15:32:51.495 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:32:51.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.495 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.495 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:32:51.495 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.244471+0000", 2024-04-24T15:32:51.495 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999986 2024-04-24T15:32:51.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.495 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.496 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:32:51.496 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.244477+0000", 2024-04-24T15:32:51.496 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7819999999999999e-06 2024-04-24T15:32:51.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.496 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.496 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:32:51.496 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.244487+0000", 2024-04-24T15:32:51.496 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0247e-05 2024-04-24T15:32:51.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.496 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.496 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:32:51.496 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.244499+0000", 2024-04-24T15:32:51.496 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1324e-05 2024-04-24T15:32:51.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.496 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.497 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:51.497 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.244545+0000", 2024-04-24T15:32:51.497 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6186000000000001e-05 2024-04-24T15:32:51.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.497 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.497 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-04-24T15:32:51.497 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.244553+0000", 2024-04-24T15:32:51.497 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5380000000000002e-06 2024-04-24T15:32:51.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.497 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.497 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:51.497 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.631596+0000", 2024-04-24T15:32:51.497 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.38704289600000003 2024-04-24T15:32:51.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.498 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.498 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-04-24T15:32:51.498 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.631606+0000", 2024-04-24T15:32:51.498 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6390000000000004e-06 2024-04-24T15:32:51.498 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.498 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.498 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:32:51.498 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:48.658831+0000", 2024-04-24T15:32:51.498 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0272250540000001 2024-04-24T15:32:51.498 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.498 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.498 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:32:51.498 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:48.658991+0000", 2024-04-24T15:32:51.498 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015994800000000001 2024-04-24T15:32:51.498 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:32:51.499 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:32:51.499 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:32:51.499 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:48.659135+0000", 2024-04-24T15:32:51.499 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000144456 2024-04-24T15:32:51.499 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:51.499 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:51.499 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:51.499 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:32:51.499 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:32:51.499 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:32:51.500 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-24T15:32:51.545 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:32:51.546+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:32:51.608 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:32:51.608+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:32:51.637 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:32:51.637 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:32:51.637 INFO:teuthology.orchestra.run.smithi151.stdout: "num_ops": 0 2024-04-24T15:32:51.637 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:32:51.649 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-24T15:32:51.733 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:32:51.734+0000 7f23abcf6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 44021) UID: 0 2024-04-24T15:32:51.769 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #9, image image_client.0-clone9 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent10 2024-04-24T15:32:51.787 INFO:teuthology.orchestra.run.smithi096.stdout:2228 read 0x12f47c4 thru 0x12fe40b (0x9c48 bytes) 2024-04-24T15:32:51.787 INFO:teuthology.orchestra.run.smithi096.stdout:2229 write 0x2d4d29f thru 0x2d5b99f (0xe701 bytes) 2024-04-24T15:32:51.801 INFO:teuthology.orchestra.run.smithi096.stdout:2230 read 0xdfb5b0 thru 0xe097ec (0xe23d bytes) 2024-04-24T15:32:51.810 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:32:51.810+0000 7f23abcf6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 44021) UID: 0 2024-04-24T15:32:51.814 INFO:teuthology.orchestra.run.smithi096.stdout:2231 trunc from 0x40a9f73 to 0xe70dec6 2024-04-24T15:32:51.817 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:32:51.817 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:32:51.817 INFO:teuthology.orchestra.run.smithi151.stdout: "complaint_time": 30, 2024-04-24T15:32:51.817 INFO:teuthology.orchestra.run.smithi151.stdout: "num_blocked_ops": 0 2024-04-24T15:32:51.817 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:32:51.819 INFO:teuthology.orchestra.run.smithi096.stdout:2232 read 0xddd4c46 thru 0xddd6ac1 (0x1e7c bytes) 2024-04-24T15:32:51.819 INFO:teuthology.orchestra.run.smithi096.stdout:2235 write 0xb115537 thru 0xb11e903 (0x93cd bytes) 2024-04-24T15:32:51.827 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-24T15:32:51.838 INFO:teuthology.orchestra.run.smithi096.stdout:2236 trunc from 0xe70dec6 to 0x147a3d6 2024-04-24T15:32:51.876 INFO:teuthology.orchestra.run.smithi096.stdout:2238 writesame 0xdcd2ab0 thru 0xdcd732b data_size 0x2(0x487c bytes) 2024-04-24T15:32:51.876 INFO:teuthology.orchestra.run.smithi096.stdout:2239 punch from 0x2c2f0cf to 0x2c38383, (0x92b4 bytes) 2024-04-24T15:32:51.876 INFO:teuthology.orchestra.run.smithi096.stdout:2242 write 0x3a6451a thru 0x3a66eef (0x29d6 bytes) 2024-04-24T15:32:51.877 INFO:teuthology.orchestra.run.smithi096.stdout:2243 writesame 0x91d95fc thru 0x91dc5a1 data_size 0x39(0x2fa6 bytes) 2024-04-24T15:32:51.878 INFO:teuthology.orchestra.run.smithi096.stdout:2244 read 0x5ccf542 thru 0x5cd79f3 (0x84b2 bytes) 2024-04-24T15:32:51.878 INFO:teuthology.orchestra.run.smithi096.stdout:2245 read 0x890d0e1 thru 0x89125e2 (0x5502 bytes) 2024-04-24T15:32:51.894 INFO:teuthology.orchestra.run.smithi096.stdout:2246 trunc from 0xdcd732c to 0x9fbefd 2024-04-24T15:32:51.911 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:32:51.912+0000 7f23abcf6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 44021) UID: 0 2024-04-24T15:32:51.922 INFO:teuthology.orchestra.run.smithi096.stdout:2247 write 0x88e0b49 thru 0x88ef88f (0xed47 bytes) 2024-04-24T15:32:51.924 INFO:teuthology.orchestra.run.smithi096.stdout:2248 read 0x40754e thru 0x411eee (0xa9a1 bytes) 2024-04-24T15:32:51.924 INFO:teuthology.orchestra.run.smithi096.stdout:2249 writesame 0xda3d91 thru 0xdb1133 data_size 0x1(0xd3a3 bytes) 2024-04-24T15:32:51.947 INFO:teuthology.orchestra.run.smithi096.stdout:2251 write 0x957b5a thru 0x9598ab (0x1d52 bytes) 2024-04-24T15:32:51.948 INFO:teuthology.orchestra.run.smithi096.stdout:2252 write 0x6032b14 thru 0x60406c4 (0xdbb1 bytes) 2024-04-24T15:32:51.949 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:51.950 INFO:teuthology.orchestra.run.smithi096.stdout:2254 write 0xc78867 thru 0xc8167b (0x8e15 bytes) 2024-04-24T15:32:51.951 INFO:teuthology.orchestra.run.smithi096.stdout:2255 write 0x5e0cbda thru 0x5e0db26 (0xf4d bytes) 2024-04-24T15:32:51.953 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:51.953 INFO:teuthology.orchestra.run.smithi096.stdout:2257 flatten 2024-04-24T15:32:51.958 INFO:teuthology.orchestra.run.smithi096.stdout:2258 clone 12 order 24 su 4194304 sc 7 2024-04-24T15:32:51.995 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:32:51.996 INFO:teuthology.orchestra.run.smithi151.stdout: "size": 20, 2024-04-24T15:32:51.996 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 600, 2024-04-24T15:32:51.996 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [ 2024-04-24T15:32:51.996 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:51.996 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:2507 4.e 4:729aaa1b:::rbd_data.114a1f60cb66.000000000000029a:head [set-alloc-hint object_size 262144 write_size 262144,write 233472~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-24T15:32:51.996 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:23.072715+0000", 2024-04-24T15:32:51.996 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 28.921896534999998, 2024-04-24T15:32:51.996 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0065961370000000002, 2024-04-24T15:32:51.996 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:51.996 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:51.996 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:51.996 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:51.996 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:51.996 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 2507 2024-04-24T15:32:51.997 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:51.997 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:51.997 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:51.997 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:51.997 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.072715+0000", 2024-04-24T15:32:51.997 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:51.997 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:51.997 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:51.997 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:51.997 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.072715+0000", 2024-04-24T15:32:51.997 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:51.997 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:51.997 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:51.997 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:51.998 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.072713+0000", 2024-04-24T15:32:51.998 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999981 2024-04-24T15:32:51.998 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:51.998 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:51.998 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:51.998 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.072727+0000", 2024-04-24T15:32:51.998 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3549e-05 2024-04-24T15:32:51.998 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:51.998 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:51.998 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:51.999 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.072739+0000", 2024-04-24T15:32:51.999 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.1973999999999999e-05 2024-04-24T15:32:51.999 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:51.999 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:51.999 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:51.999 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.072746+0000", 2024-04-24T15:32:51.999 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2849999999999997e-06 2024-04-24T15:32:51.999 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:51.999 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:51.999 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:51.999 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.072798+0000", 2024-04-24T15:32:52.000 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.2289e-05 2024-04-24T15:32:52.000 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.000 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.000 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op must wait for map", 2024-04-24T15:32:52.000 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.072806+0000", 2024-04-24T15:32:52.000 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.8469999999999996e-06 2024-04-24T15:32:52.000 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.000 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.000 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:52.000 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.074503+0000", 2024-04-24T15:32:52.000 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0016967029999999999 2024-04-24T15:32:52.000 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.001 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.001 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:52.001 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.074593+0000", 2024-04-24T15:32:52.001 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.0313000000000001e-05 2024-04-24T15:32:52.001 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.001 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.001 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 6", 2024-04-24T15:32:52.001 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.074732+0000", 2024-04-24T15:32:52.001 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000138376 2024-04-24T15:32:52.001 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.001 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.001 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:52.001 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.078538+0000", 2024-04-24T15:32:52.001 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0038068749999999999 2024-04-24T15:32:52.001 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.001 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.002 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:52.002 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.079232+0000", 2024-04-24T15:32:52.002 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00069326900000000005 2024-04-24T15:32:52.002 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.002 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.002 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:52.002 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.079273+0000", 2024-04-24T15:32:52.002 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.1236000000000003e-05 2024-04-24T15:32:52.002 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.002 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.002 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:52.002 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.079311+0000", 2024-04-24T15:32:52.002 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8497000000000001e-05 2024-04-24T15:32:52.003 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.003 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:52.003 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.003 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.003 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.003 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_repop(client.4426.0:2563 4.1d e34/27 4:b841be81:::rbd_data.114a1f60cb66.00000000000001d7:head v 34'17, mlcod=32'15)", 2024-04-24T15:32:52.003 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.108404+0000", 2024-04-24T15:32:52.003 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.886208003, 2024-04-24T15:32:52.003 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0073539469999999996, 2024-04-24T15:32:52.003 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:52.003 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:52.003 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:52.003 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.003 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:52.004 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.108404+0000", 2024-04-24T15:32:52.004 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.004 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.004 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.004 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:52.004 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.108404+0000", 2024-04-24T15:32:52.004 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.004 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.004 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.004 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:52.004 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.108403+0000", 2024-04-24T15:32:52.004 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:52.004 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.004 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.004 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:52.005 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.108420+0000", 2024-04-24T15:32:52.005 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.6980999999999999e-05 2024-04-24T15:32:52.005 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.005 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.005 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:52.005 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.108428+0000", 2024-04-24T15:32:52.005 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0560000000000005e-06 2024-04-24T15:32:52.005 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.005 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.005 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:52.005 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.108432+0000", 2024-04-24T15:32:52.005 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.4060000000000001e-06 2024-04-24T15:32:52.005 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.005 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.006 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:52.006 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.108462+0000", 2024-04-24T15:32:52.006 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.008e-05 2024-04-24T15:32:52.006 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.006 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.006 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:52.006 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.108475+0000", 2024-04-24T15:32:52.006 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3018e-05 2024-04-24T15:32:52.006 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.006 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.006 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:52.006 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.115726+0000", 2024-04-24T15:32:52.006 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0072510040000000001 2024-04-24T15:32:52.006 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.007 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.007 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:52.007 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.115758+0000", 2024-04-24T15:32:52.007 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.2092999999999999e-05 2024-04-24T15:32:52.007 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.007 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:52.007 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.007 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.007 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.007 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:2620 4.18 4:1e938922:::rbd_data.114a1f60cb66.0000000000000028:head [stat,write 163840~6144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:52.007 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.151073+0000", 2024-04-24T15:32:52.007 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.843539289999999, 2024-04-24T15:32:52.007 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0063989210000000001, 2024-04-24T15:32:52.007 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:52.008 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:52.008 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:52.008 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:52.008 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:52.008 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 2620 2024-04-24T15:32:52.008 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.008 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:52.008 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.008 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:52.008 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.151073+0000", 2024-04-24T15:32:52.008 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.008 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.008 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.008 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:52.009 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.151073+0000", 2024-04-24T15:32:52.009 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.009 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.009 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.009 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:52.009 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.151072+0000", 2024-04-24T15:32:52.009 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:52.009 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.009 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.009 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:52.009 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.151083+0000", 2024-04-24T15:32:52.009 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0801999999999999e-05 2024-04-24T15:32:52.009 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.009 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.009 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:52.010 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.151086+0000", 2024-04-24T15:32:52.010 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.3900000000000002e-06 2024-04-24T15:32:52.010 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.010 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.010 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:52.010 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.151089+0000", 2024-04-24T15:32:52.010 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.294e-06 2024-04-24T15:32:52.010 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.010 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.010 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:52.010 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.152198+0000", 2024-04-24T15:32:52.010 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.001109407 2024-04-24T15:32:52.011 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.011 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.011 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:52.011 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.152273+0000", 2024-04-24T15:32:52.011 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.4419999999999996e-05 2024-04-24T15:32:52.011 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.011 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.011 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 6", 2024-04-24T15:32:52.011 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.152396+0000", 2024-04-24T15:32:52.011 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00012322400000000001 2024-04-24T15:32:52.011 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.011 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.011 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:52.011 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.157242+0000", 2024-04-24T15:32:52.011 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0048462369999999998 2024-04-24T15:32:52.011 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.012 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.012 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:52.012 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.157414+0000", 2024-04-24T15:32:52.012 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000172411 2024-04-24T15:32:52.012 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.012 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.012 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:52.012 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.157451+0000", 2024-04-24T15:32:52.012 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.6269000000000001e-05 2024-04-24T15:32:52.012 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.012 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.012 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:52.012 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.157471+0000", 2024-04-24T15:32:52.012 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.0741e-05 2024-04-24T15:32:52.013 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.013 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:52.013 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.013 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.013 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.013 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:2623 4.18 4:1b4f38fa:::rbd_data.114a1f60cb66.000000000000002b:head [stat,write 161792~6144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:52.013 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.151108+0000", 2024-04-24T15:32:52.013 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.843504028000002, 2024-04-24T15:32:52.013 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0064461170000000003, 2024-04-24T15:32:52.013 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:52.013 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:52.013 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:52.013 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:52.013 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:52.014 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 2623 2024-04-24T15:32:52.014 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.014 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:52.014 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.014 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:52.014 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.151108+0000", 2024-04-24T15:32:52.014 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.014 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.014 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.014 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:52.014 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.151108+0000", 2024-04-24T15:32:52.014 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.014 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.015 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.015 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:52.015 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.151108+0000", 2024-04-24T15:32:52.015 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:52.015 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.015 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.015 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:52.015 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.151117+0000", 2024-04-24T15:32:52.015 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.8560000000000002e-06 2024-04-24T15:32:52.015 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.015 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.015 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:52.015 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.151122+0000", 2024-04-24T15:32:52.015 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5390000000000001e-06 2024-04-24T15:32:52.016 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.016 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.016 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:52.016 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.151124+0000", 2024-04-24T15:32:52.016 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.2800000000000002e-06 2024-04-24T15:32:52.016 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.016 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.016 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:52.016 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.152676+0000", 2024-04-24T15:32:52.016 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.001552124 2024-04-24T15:32:52.016 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.016 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.016 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:52.016 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.152759+0000", 2024-04-24T15:32:52.017 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.2688000000000005e-05 2024-04-24T15:32:52.017 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.017 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.017 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 6", 2024-04-24T15:32:52.017 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.152867+0000", 2024-04-24T15:32:52.017 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00010779400000000001 2024-04-24T15:32:52.017 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.017 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.017 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:52.017 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.157259+0000", 2024-04-24T15:32:52.017 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0043918230000000004 2024-04-24T15:32:52.017 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.017 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.017 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:52.018 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.157498+0000", 2024-04-24T15:32:52.018 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000239829 2024-04-24T15:32:52.018 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.018 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.018 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:52.018 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.157525+0000", 2024-04-24T15:32:52.018 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.6868000000000001e-05 2024-04-24T15:32:52.018 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.018 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.018 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:52.018 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.157554+0000", 2024-04-24T15:32:52.018 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8602000000000002e-05 2024-04-24T15:32:52.018 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.019 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:52.019 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.019 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.019 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.019 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:3618 4.18 4:1f7d02ce:::rbd_data.114a1f60cb66.0000000000000161:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:52.019 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.581260+0000", 2024-04-24T15:32:52.019 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.413351671000001, 2024-04-24T15:32:52.019 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0068184120000000003, 2024-04-24T15:32:52.019 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:52.019 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:52.019 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:52.019 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:52.019 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:52.019 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 3618 2024-04-24T15:32:52.020 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.020 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:52.020 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.020 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:52.020 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.581260+0000", 2024-04-24T15:32:52.020 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.020 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.020 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.020 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:52.020 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.581260+0000", 2024-04-24T15:32:52.020 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.020 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.020 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.020 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:52.021 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.581259+0000", 2024-04-24T15:32:52.021 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999999 2024-04-24T15:32:52.021 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.021 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.021 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:52.021 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.581265+0000", 2024-04-24T15:32:52.021 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.003e-06 2024-04-24T15:32:52.021 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.021 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.021 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:52.021 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.581278+0000", 2024-04-24T15:32:52.021 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.2483000000000001e-05 2024-04-24T15:32:52.021 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.021 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.021 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:52.022 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.581284+0000", 2024-04-24T15:32:52.022 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.7399999999999998e-06 2024-04-24T15:32:52.022 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.022 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.022 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:52.022 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.581783+0000", 2024-04-24T15:32:52.022 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00049892199999999995 2024-04-24T15:32:52.022 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.022 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.022 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:52.022 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.581902+0000", 2024-04-24T15:32:52.022 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000118272 2024-04-24T15:32:52.022 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.023 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.023 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 6", 2024-04-24T15:32:52.023 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.583068+0000", 2024-04-24T15:32:52.023 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0011665759999999999 2024-04-24T15:32:52.023 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.023 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.023 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:52.023 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.587081+0000", 2024-04-24T15:32:52.023 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.004012781 2024-04-24T15:32:52.023 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.023 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.023 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:52.023 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.587997+0000", 2024-04-24T15:32:52.023 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00091637700000000001 2024-04-24T15:32:52.023 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.024 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.024 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:52.024 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.588041+0000", 2024-04-24T15:32:52.024 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3619999999999999e-05 2024-04-24T15:32:52.024 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.024 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.024 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:52.024 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.588079+0000", 2024-04-24T15:32:52.024 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.7654000000000002e-05 2024-04-24T15:32:52.024 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.024 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:52.024 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.024 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.024 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.024 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:3767 4.9 4:908f8f18:::rbd_data.114a1f60cb66.000000000000001b:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:52.024 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.699557+0000", 2024-04-24T15:32:52.025 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.295054946, 2024-04-24T15:32:52.025 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0088231639999999997, 2024-04-24T15:32:52.025 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:52.025 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:52.025 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:52.025 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:52.025 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:52.025 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 3767 2024-04-24T15:32:52.025 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.025 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:52.025 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.025 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:52.025 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699557+0000", 2024-04-24T15:32:52.025 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.025 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.025 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.025 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:52.026 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699557+0000", 2024-04-24T15:32:52.026 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.026 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.026 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.026 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:52.026 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699556+0000", 2024-04-24T15:32:52.026 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999999 2024-04-24T15:32:52.026 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.026 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.026 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:52.026 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699561+0000", 2024-04-24T15:32:52.026 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.0440000000000003e-06 2024-04-24T15:32:52.026 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.026 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.027 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:52.027 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699571+0000", 2024-04-24T15:32:52.027 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0237999999999999e-05 2024-04-24T15:32:52.027 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.027 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.027 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:52.027 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699582+0000", 2024-04-24T15:32:52.027 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.1009000000000001e-05 2024-04-24T15:32:52.027 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.027 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.027 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:52.027 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699607+0000", 2024-04-24T15:32:52.027 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.5024999999999998e-05 2024-04-24T15:32:52.028 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.028 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.028 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:52.028 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699671+0000", 2024-04-24T15:32:52.028 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.3219999999999994e-05 2024-04-24T15:32:52.028 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.028 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.028 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:52.028 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.700952+0000", 2024-04-24T15:32:52.028 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.001281952 2024-04-24T15:32:52.028 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.028 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.028 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:52.028 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.705912+0000", 2024-04-24T15:32:52.028 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0049599889999999997 2024-04-24T15:32:52.029 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.029 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.029 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:52.029 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.708300+0000", 2024-04-24T15:32:52.029 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.002387934 2024-04-24T15:32:52.029 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.029 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.029 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:52.029 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.708345+0000", 2024-04-24T15:32:52.029 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4175000000000003e-05 2024-04-24T15:32:52.029 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.029 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.029 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:52.029 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.708380+0000", 2024-04-24T15:32:52.029 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5460999999999998e-05 2024-04-24T15:32:52.029 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.030 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:52.030 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.030 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.030 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.030 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:3770 4.13 4:cb3c841f:::rbd_data.114a1f60cb66.000000000000001e:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:52.030 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.699597+0000", 2024-04-24T15:32:52.030 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.295014481999999, 2024-04-24T15:32:52.030 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0083605450000000005, 2024-04-24T15:32:52.030 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:52.030 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:52.030 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:52.030 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:52.030 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:52.031 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 3770 2024-04-24T15:32:52.031 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.031 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:52.031 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.031 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:52.031 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699597+0000", 2024-04-24T15:32:52.031 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.031 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.031 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.031 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:52.031 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699597+0000", 2024-04-24T15:32:52.031 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.031 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.031 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.031 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:52.032 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699597+0000", 2024-04-24T15:32:52.032 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:52.032 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.032 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.032 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:52.032 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699600+0000", 2024-04-24T15:32:52.032 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.371e-06 2024-04-24T15:32:52.032 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.032 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.032 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:52.032 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699611+0000", 2024-04-24T15:32:52.032 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0552e-05 2024-04-24T15:32:52.032 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.032 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.032 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:52.033 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699615+0000", 2024-04-24T15:32:52.033 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5689999999999998e-06 2024-04-24T15:32:52.033 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.033 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.033 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:52.033 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699692+0000", 2024-04-24T15:32:52.033 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.6664999999999999e-05 2024-04-24T15:32:52.033 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.033 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.033 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:52.033 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699740+0000", 2024-04-24T15:32:52.033 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8032999999999999e-05 2024-04-24T15:32:52.033 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.033 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.033 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 6", 2024-04-24T15:32:52.034 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.700940+0000", 2024-04-24T15:32:52.034 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.001199978 2024-04-24T15:32:52.034 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.034 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.034 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:52.034 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.706397+0000", 2024-04-24T15:32:52.034 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0054569459999999998 2024-04-24T15:32:52.034 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.034 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.034 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:52.034 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.707870+0000", 2024-04-24T15:32:52.034 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.001473294 2024-04-24T15:32:52.034 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.034 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.035 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:52.035 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.707920+0000", 2024-04-24T15:32:52.035 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.994e-05 2024-04-24T15:32:52.035 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.035 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.035 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:52.035 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.707958+0000", 2024-04-24T15:32:52.035 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.7761e-05 2024-04-24T15:32:52.035 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.035 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:52.035 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.035 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.035 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.035 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:3773 4.e 4:7173e872:::rbd_data.114a1f60cb66.0000000000000021:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:52.036 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.699632+0000", 2024-04-24T15:32:52.036 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.294979918999999, 2024-04-24T15:32:52.036 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0068017260000000001, 2024-04-24T15:32:52.036 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:52.036 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:52.036 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:52.036 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:52.036 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:52.036 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 3773 2024-04-24T15:32:52.036 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.036 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:52.036 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.036 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:52.036 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699632+0000", 2024-04-24T15:32:52.036 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.036 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.037 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.037 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:52.037 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699632+0000", 2024-04-24T15:32:52.037 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.037 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.037 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.037 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:52.037 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699631+0000", 2024-04-24T15:32:52.037 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:52.037 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.037 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.037 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:52.037 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699639+0000", 2024-04-24T15:32:52.037 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7319999999999998e-06 2024-04-24T15:32:52.037 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.037 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.038 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:52.038 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699646+0000", 2024-04-24T15:32:52.038 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1330000000000001e-06 2024-04-24T15:32:52.038 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.038 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.038 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:52.038 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699651+0000", 2024-04-24T15:32:52.038 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.9200000000000003e-06 2024-04-24T15:32:52.038 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.038 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.038 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:52.038 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699689+0000", 2024-04-24T15:32:52.038 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.7979999999999999e-05 2024-04-24T15:32:52.041 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.042 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.042 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:52.042 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699759+0000", 2024-04-24T15:32:52.042 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.9468000000000006e-05 2024-04-24T15:32:52.042 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.042 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.042 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 6", 2024-04-24T15:32:52.042 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.701164+0000", 2024-04-24T15:32:52.042 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0014056629999999999 2024-04-24T15:32:52.042 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.042 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.042 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:52.042 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.705975+0000", 2024-04-24T15:32:52.043 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0048109340000000002 2024-04-24T15:32:52.043 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.043 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.043 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:52.043 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.706359+0000", 2024-04-24T15:32:52.043 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00038331699999999999 2024-04-24T15:32:52.043 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.043 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.043 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:52.043 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.706407+0000", 2024-04-24T15:32:52.043 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8544999999999997e-05 2024-04-24T15:32:52.043 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.043 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.043 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:52.043 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.706434+0000", 2024-04-24T15:32:52.044 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.6553000000000001e-05 2024-04-24T15:32:52.044 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.044 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:52.044 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.044 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.044 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.044 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:3794 4.18 4:1efc862d:::rbd_data.114a1f60cb66.0000000000000024:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:52.044 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.723446+0000", 2024-04-24T15:32:52.044 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.271165400000001, 2024-04-24T15:32:52.044 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.007572957, 2024-04-24T15:32:52.044 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:52.044 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:52.044 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:52.044 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:52.044 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:52.045 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 3794 2024-04-24T15:32:52.045 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.045 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:52.045 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.045 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:52.045 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723446+0000", 2024-04-24T15:32:52.045 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.045 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.045 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.045 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:52.045 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723446+0000", 2024-04-24T15:32:52.045 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.045 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.045 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.046 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:52.046 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723446+0000", 2024-04-24T15:32:52.046 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999999 2024-04-24T15:32:52.046 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.046 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.046 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:52.046 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723450+0000", 2024-04-24T15:32:52.046 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.1729999999999997e-06 2024-04-24T15:32:52.046 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.046 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.046 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:52.046 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723458+0000", 2024-04-24T15:32:52.046 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.653e-06 2024-04-24T15:32:52.046 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.047 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.047 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:52.047 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723465+0000", 2024-04-24T15:32:52.047 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.4450000000000001e-06 2024-04-24T15:32:52.047 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.047 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.047 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:52.047 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723501+0000", 2024-04-24T15:32:52.047 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5726000000000003e-05 2024-04-24T15:32:52.047 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.047 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.047 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:52.047 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723547+0000", 2024-04-24T15:32:52.047 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6479e-05 2024-04-24T15:32:52.047 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.048 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.048 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 6", 2024-04-24T15:32:52.048 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.724384+0000", 2024-04-24T15:32:52.048 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00083723800000000002 2024-04-24T15:32:52.048 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.048 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.048 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:52.048 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.730925+0000", 2024-04-24T15:32:52.048 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0065409179999999997 2024-04-24T15:32:52.048 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.048 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.048 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:52.048 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.730969+0000", 2024-04-24T15:32:52.048 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3869999999999998e-05 2024-04-24T15:32:52.049 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.049 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.049 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:52.049 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.730997+0000", 2024-04-24T15:32:52.049 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7783000000000001e-05 2024-04-24T15:32:52.049 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.049 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.049 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:52.049 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.731019+0000", 2024-04-24T15:32:52.049 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.2427999999999999e-05 2024-04-24T15:32:52.049 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.049 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:52.049 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.049 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.050 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.050 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:3797 4.18 4:1d8bdb81:::rbd_data.114a1f60cb66.0000000000000027:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:52.050 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.723479+0000", 2024-04-24T15:32:52.050 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.271132389999998, 2024-04-24T15:32:52.050 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.013378915, 2024-04-24T15:32:52.050 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:52.050 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:52.050 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:52.050 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:52.050 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:52.050 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 3797 2024-04-24T15:32:52.050 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.050 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:52.051 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.051 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:52.051 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723479+0000", 2024-04-24T15:32:52.051 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.051 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.051 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.051 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:52.051 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723479+0000", 2024-04-24T15:32:52.051 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.051 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.051 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.051 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:52.051 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723479+0000", 2024-04-24T15:32:52.051 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:52.051 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.052 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.052 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:52.052 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723482+0000", 2024-04-24T15:32:52.052 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5410000000000001e-06 2024-04-24T15:32:52.052 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.052 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.052 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:52.052 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723492+0000", 2024-04-24T15:32:52.052 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.6879999999999994e-06 2024-04-24T15:32:52.052 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.052 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.052 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:52.052 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723496+0000", 2024-04-24T15:32:52.053 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.34e-06 2024-04-24T15:32:52.053 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.053 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.053 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:52.053 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.728008+0000", 2024-04-24T15:32:52.053 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0045115049999999999 2024-04-24T15:32:52.053 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.053 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.053 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:52.053 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.728058+0000", 2024-04-24T15:32:52.053 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.0201000000000003e-05 2024-04-24T15:32:52.053 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.053 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.053 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 6", 2024-04-24T15:32:52.054 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.728499+0000", 2024-04-24T15:32:52.054 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00044129299999999999 2024-04-24T15:32:52.054 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.054 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.054 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:52.054 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.730962+0000", 2024-04-24T15:32:52.054 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0024628430000000002 2024-04-24T15:32:52.054 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.054 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.054 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:52.054 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.736810+0000", 2024-04-24T15:32:52.054 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0058474490000000002 2024-04-24T15:32:52.054 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.055 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.055 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:52.055 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.736837+0000", 2024-04-24T15:32:52.055 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7251999999999999e-05 2024-04-24T15:32:52.055 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.055 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.055 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:52.055 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.736858+0000", 2024-04-24T15:32:52.055 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.1387999999999999e-05 2024-04-24T15:32:52.055 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.055 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:52.055 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.055 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.055 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.055 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:3798 4.18 4:1e938922:::rbd_data.114a1f60cb66.0000000000000028:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:52.056 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.723506+0000", 2024-04-24T15:32:52.056 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.271105976000001, 2024-04-24T15:32:52.056 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.013398188, 2024-04-24T15:32:52.056 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:52.056 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:52.056 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:52.056 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:52.056 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:52.056 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 3798 2024-04-24T15:32:52.056 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.056 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:52.056 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.056 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:52.056 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723506+0000", 2024-04-24T15:32:52.056 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.057 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.057 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.057 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:52.057 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723506+0000", 2024-04-24T15:32:52.057 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.057 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.057 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.057 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:52.057 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723505+0000", 2024-04-24T15:32:52.057 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:52.057 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.057 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.057 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:52.058 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723509+0000", 2024-04-24T15:32:52.058 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.2349999999999999e-06 2024-04-24T15:32:52.058 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.058 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.058 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:52.058 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723515+0000", 2024-04-24T15:32:52.058 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.3239999999999999e-06 2024-04-24T15:32:52.058 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.058 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.058 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:52.058 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723519+0000", 2024-04-24T15:32:52.058 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8580000000000002e-06 2024-04-24T15:32:52.058 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.058 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.059 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:52.059 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.731045+0000", 2024-04-24T15:32:52.059 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0075259309999999996 2024-04-24T15:32:52.059 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.059 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.059 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:52.059 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.731090+0000", 2024-04-24T15:32:52.059 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4805999999999998e-05 2024-04-24T15:32:52.059 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.059 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.059 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 6", 2024-04-24T15:32:52.059 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.731571+0000", 2024-04-24T15:32:52.059 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00048136400000000002 2024-04-24T15:32:52.059 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.059 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.060 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:52.060 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.736784+0000", 2024-04-24T15:32:52.060 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0052135269999999999 2024-04-24T15:32:52.060 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.060 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.060 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:52.060 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.736865+0000", 2024-04-24T15:32:52.060 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0141000000000004e-05 2024-04-24T15:32:52.060 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.060 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.060 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:52.060 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.736887+0000", 2024-04-24T15:32:52.060 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.2194000000000001e-05 2024-04-24T15:32:52.060 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.060 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.061 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:52.061 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.736904+0000", 2024-04-24T15:32:52.061 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.7302000000000001e-05 2024-04-24T15:32:52.061 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.061 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:52.061 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.061 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.061 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.061 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:3801 4.18 4:1b4f38fa:::rbd_data.114a1f60cb66.000000000000002b:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:52.061 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.723629+0000", 2024-04-24T15:32:52.061 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.270982832000001, 2024-04-24T15:32:52.061 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.013474534, 2024-04-24T15:32:52.061 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:52.061 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:52.061 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:52.062 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:52.062 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:52.062 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 3801 2024-04-24T15:32:52.062 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.062 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:52.062 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.062 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:52.062 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723629+0000", 2024-04-24T15:32:52.062 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.062 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.062 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.062 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:52.062 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723629+0000", 2024-04-24T15:32:52.063 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.063 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.063 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.063 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:52.063 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723628+0000", 2024-04-24T15:32:52.063 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:52.063 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.063 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.063 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:52.063 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723632+0000", 2024-04-24T15:32:52.063 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.4690000000000002e-06 2024-04-24T15:32:52.063 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.063 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.063 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:52.064 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723640+0000", 2024-04-24T15:32:52.064 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3990000000000004e-06 2024-04-24T15:32:52.064 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.064 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.064 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:52.064 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723645+0000", 2024-04-24T15:32:52.064 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3699999999999997e-06 2024-04-24T15:32:52.064 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.064 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.064 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:52.064 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.733996+0000", 2024-04-24T15:32:52.064 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.010351111 2024-04-24T15:32:52.064 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.064 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.064 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:52.064 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.734045+0000", 2024-04-24T15:32:52.065 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.9166000000000003e-05 2024-04-24T15:32:52.065 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.065 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.065 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 6", 2024-04-24T15:32:52.065 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.734461+0000", 2024-04-24T15:32:52.065 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00041584 2024-04-24T15:32:52.065 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.065 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.065 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:52.065 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.736806+0000", 2024-04-24T15:32:52.065 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.002345232 2024-04-24T15:32:52.065 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.065 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.065 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:52.065 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.737065+0000", 2024-04-24T15:32:52.066 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00025913100000000001 2024-04-24T15:32:52.066 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.066 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.066 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:52.066 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.737086+0000", 2024-04-24T15:32:52.066 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.1180000000000001e-05 2024-04-24T15:32:52.066 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.066 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.066 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:52.066 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.737104+0000", 2024-04-24T15:32:52.066 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.7238999999999999e-05 2024-04-24T15:32:52.066 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.067 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:52.067 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.067 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.067 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.067 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:3896 4.1b 4:dd750b71:::rbd_data.114a1f60cb66.0000000000000058:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:52.067 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.797438+0000", 2024-04-24T15:32:52.067 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.197173607, 2024-04-24T15:32:52.067 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0070008420000000002, 2024-04-24T15:32:52.067 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:52.067 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:52.067 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:52.067 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:52.067 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:52.067 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 3896 2024-04-24T15:32:52.068 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.068 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:52.068 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.068 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:52.068 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797438+0000", 2024-04-24T15:32:52.068 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.068 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.068 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.068 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:52.068 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797438+0000", 2024-04-24T15:32:52.068 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.068 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.068 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.068 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:52.068 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797438+0000", 2024-04-24T15:32:52.068 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:52.069 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.069 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.069 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:52.069 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797441+0000", 2024-04-24T15:32:52.069 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8779999999999998e-06 2024-04-24T15:32:52.069 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.069 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.069 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:52.069 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797447+0000", 2024-04-24T15:32:52.069 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.6989999999999997e-06 2024-04-24T15:32:52.069 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.069 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.069 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:52.069 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797451+0000", 2024-04-24T15:32:52.069 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9689999999999996e-06 2024-04-24T15:32:52.070 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.070 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.070 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:52.070 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797915+0000", 2024-04-24T15:32:52.070 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00046327400000000001 2024-04-24T15:32:52.070 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.070 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.070 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:52.070 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797993+0000", 2024-04-24T15:32:52.070 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.8584000000000005e-05 2024-04-24T15:32:52.070 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.070 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.070 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:52.070 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.798090+0000", 2024-04-24T15:32:52.071 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.6638000000000005e-05 2024-04-24T15:32:52.071 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.071 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.071 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:52.071 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.799021+0000", 2024-04-24T15:32:52.071 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00093112299999999996 2024-04-24T15:32:52.071 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.071 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.071 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:52.071 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.804372+0000", 2024-04-24T15:32:52.071 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0053506789999999997 2024-04-24T15:32:52.071 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.071 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.071 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:52.071 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.804405+0000", 2024-04-24T15:32:52.072 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.3015999999999997e-05 2024-04-24T15:32:52.072 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.072 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.072 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:52.072 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.804439+0000", 2024-04-24T15:32:52.072 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.4431000000000003e-05 2024-04-24T15:32:52.072 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.072 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:52.072 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.072 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.072 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.072 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:3900 4.1a 4:5b94e8b4:::rbd_data.114a1f60cb66.000000000000006e:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:52.072 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.797456+0000", 2024-04-24T15:32:52.073 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.197155906999999, 2024-04-24T15:32:52.073 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.006973524, 2024-04-24T15:32:52.073 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:52.073 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:52.073 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:52.073 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:52.073 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:52.073 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 3900 2024-04-24T15:32:52.073 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.073 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:52.073 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.073 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:52.073 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797456+0000", 2024-04-24T15:32:52.073 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.073 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.074 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.074 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:52.074 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797456+0000", 2024-04-24T15:32:52.074 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.074 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.074 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.074 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:52.074 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797456+0000", 2024-04-24T15:32:52.074 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:52.074 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.074 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.074 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:52.075 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797458+0000", 2024-04-24T15:32:52.075 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.3209999999999999e-06 2024-04-24T15:32:52.075 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.075 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.075 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:52.075 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797464+0000", 2024-04-24T15:32:52.075 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.7579999999999998e-06 2024-04-24T15:32:52.075 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.075 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.075 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:52.075 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797467+0000", 2024-04-24T15:32:52.075 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.585e-06 2024-04-24T15:32:52.075 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.075 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.076 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:52.076 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797504+0000", 2024-04-24T15:32:52.076 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.7120999999999999e-05 2024-04-24T15:32:52.076 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.076 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.076 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:52.076 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797578+0000", 2024-04-24T15:32:52.076 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.3693000000000001e-05 2024-04-24T15:32:52.076 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.076 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.076 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:52.076 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797675+0000", 2024-04-24T15:32:52.076 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.7454000000000006e-05 2024-04-24T15:32:52.077 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.077 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.077 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:52.077 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.799033+0000", 2024-04-24T15:32:52.077 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.001357104 2024-04-24T15:32:52.077 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.077 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.077 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:52.077 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.804357+0000", 2024-04-24T15:32:52.077 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0053247490000000001 2024-04-24T15:32:52.077 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.077 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.077 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:52.077 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.804397+0000", 2024-04-24T15:32:52.078 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9573000000000002e-05 2024-04-24T15:32:52.078 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.078 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.078 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:52.078 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.804429+0000", 2024-04-24T15:32:52.078 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.2529e-05 2024-04-24T15:32:52.078 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.078 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:52.078 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.078 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.078 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.078 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:3923 4.9 4:94be6b6c:::rbd_data.114a1f60cb66.0000000000000097:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:52.078 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.806292+0000", 2024-04-24T15:32:52.078 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.188320045000001, 2024-04-24T15:32:52.079 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0062496729999999999, 2024-04-24T15:32:52.079 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:52.079 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:52.079 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:52.079 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:52.079 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:52.079 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 3923 2024-04-24T15:32:52.079 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.079 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:52.079 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.079 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:52.079 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.806292+0000", 2024-04-24T15:32:52.079 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.080 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.080 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.080 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:52.080 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.806292+0000", 2024-04-24T15:32:52.080 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.080 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.080 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.080 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:52.080 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.806291+0000", 2024-04-24T15:32:52.080 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999999 2024-04-24T15:32:52.080 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.080 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.080 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:52.080 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.806296+0000", 2024-04-24T15:32:52.081 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5730000000000001e-06 2024-04-24T15:32:52.081 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.081 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.081 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:52.081 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.806306+0000", 2024-04-24T15:32:52.081 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0409e-05 2024-04-24T15:32:52.081 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.081 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.081 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:52.081 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.806311+0000", 2024-04-24T15:32:52.081 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.7129999999999996e-06 2024-04-24T15:32:52.081 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.081 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.082 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:52.082 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.806350+0000", 2024-04-24T15:32:52.082 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9351999999999999e-05 2024-04-24T15:32:52.082 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.082 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.082 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:52.082 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.806431+0000", 2024-04-24T15:32:52.082 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.1050000000000005e-05 2024-04-24T15:32:52.082 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.082 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.082 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:52.082 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.806539+0000", 2024-04-24T15:32:52.082 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000107723 2024-04-24T15:32:52.083 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.083 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.083 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:52.083 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.809607+0000", 2024-04-24T15:32:52.083 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.003068468 2024-04-24T15:32:52.083 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.083 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.083 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:52.083 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.812493+0000", 2024-04-24T15:32:52.083 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0028862689999999999 2024-04-24T15:32:52.083 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.083 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.083 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:52.083 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.812521+0000", 2024-04-24T15:32:52.084 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7226e-05 2024-04-24T15:32:52.084 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.084 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.084 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:52.084 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.812541+0000", 2024-04-24T15:32:52.084 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.0729000000000001e-05 2024-04-24T15:32:52.084 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.084 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:52.084 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.084 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.084 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.084 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:3975 4.1b 4:d9227084:::rbd_data.114a1f60cb66.000000000000015f:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:52.084 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.827427+0000", 2024-04-24T15:32:52.084 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.167184600999999, 2024-04-24T15:32:52.085 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0061324140000000001, 2024-04-24T15:32:52.085 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:52.085 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:52.085 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:52.085 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:52.085 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:52.085 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 3975 2024-04-24T15:32:52.085 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.085 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:52.085 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.085 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:52.085 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.827427+0000", 2024-04-24T15:32:52.085 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.085 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.086 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.086 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:52.086 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.827427+0000", 2024-04-24T15:32:52.086 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.086 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.086 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.086 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:52.086 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.827427+0000", 2024-04-24T15:32:52.086 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:52.086 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.086 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.086 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:52.086 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.827431+0000", 2024-04-24T15:32:52.086 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2509999999999996e-06 2024-04-24T15:32:52.087 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.087 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.087 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:52.087 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.827440+0000", 2024-04-24T15:32:52.087 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.816e-06 2024-04-24T15:32:52.087 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.087 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.087 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:52.087 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.827445+0000", 2024-04-24T15:32:52.087 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.2059999999999998e-06 2024-04-24T15:32:52.087 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.087 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.087 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:52.087 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.827872+0000", 2024-04-24T15:32:52.088 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00042740299999999997 2024-04-24T15:32:52.088 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.088 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.088 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:52.088 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.827941+0000", 2024-04-24T15:32:52.088 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.8652000000000006e-05 2024-04-24T15:32:52.088 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.088 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.088 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:52.088 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.828048+0000", 2024-04-24T15:32:52.088 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000106941 2024-04-24T15:32:52.088 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.088 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.088 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:52.088 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.831351+0000", 2024-04-24T15:32:52.089 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0033031010000000001 2024-04-24T15:32:52.089 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.089 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.089 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:52.089 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.833495+0000", 2024-04-24T15:32:52.089 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0021436379999999998 2024-04-24T15:32:52.089 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.089 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.089 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:52.089 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.833526+0000", 2024-04-24T15:32:52.089 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.1436999999999997e-05 2024-04-24T15:32:52.089 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.089 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.089 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:52.090 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.833560+0000", 2024-04-24T15:32:52.090 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.3560999999999999e-05 2024-04-24T15:32:52.090 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.090 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:52.090 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.090 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.090 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.090 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:4010 4.1 4:813d513a:::rbd_data.114a1f60cb66.00000000000001c1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:52.090 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.845178+0000", 2024-04-24T15:32:52.090 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.149434252999999, 2024-04-24T15:32:52.090 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.007702108, 2024-04-24T15:32:52.090 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:52.090 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:52.091 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:52.091 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:52.091 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:52.091 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 4010 2024-04-24T15:32:52.091 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.091 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:52.091 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.091 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:52.091 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.845178+0000", 2024-04-24T15:32:52.091 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.091 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.091 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.091 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:52.091 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.845178+0000", 2024-04-24T15:32:52.092 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.092 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.092 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.092 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:52.092 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.845177+0000", 2024-04-24T15:32:52.092 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:52.092 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.092 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.092 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:52.092 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.845180+0000", 2024-04-24T15:32:52.092 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8119999999999999e-06 2024-04-24T15:32:52.092 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.092 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.092 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:52.092 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.845186+0000", 2024-04-24T15:32:52.093 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.9649999999999996e-06 2024-04-24T15:32:52.093 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.093 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.093 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:52.093 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.845189+0000", 2024-04-24T15:32:52.093 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.6040000000000001e-06 2024-04-24T15:32:52.093 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.093 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.093 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:52.093 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.845220+0000", 2024-04-24T15:32:52.093 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.0207000000000001e-05 2024-04-24T15:32:52.093 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.093 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.093 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:52.094 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.845262+0000", 2024-04-24T15:32:52.094 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2716000000000002e-05 2024-04-24T15:32:52.094 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.094 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.094 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 5", 2024-04-24T15:32:52.094 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.845326+0000", 2024-04-24T15:32:52.094 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.3490000000000004e-05 2024-04-24T15:32:52.094 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.094 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.094 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:52.094 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.849620+0000", 2024-04-24T15:32:52.094 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0042944769999999997 2024-04-24T15:32:52.094 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.095 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.095 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:52.095 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.852802+0000", 2024-04-24T15:32:52.095 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.003181798 2024-04-24T15:32:52.095 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.095 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.095 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:52.095 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.852846+0000", 2024-04-24T15:32:52.095 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3606000000000002e-05 2024-04-24T15:32:52.095 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.095 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.095 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:52.095 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.852880+0000", 2024-04-24T15:32:52.095 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.3908999999999997e-05 2024-04-24T15:32:52.095 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.096 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:52.096 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.096 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.096 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.096 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:4256 4.13 4:c8361c81:::rbd_data.114a1f60cb66.0000000000000316:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:52.096 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:25.054726+0000", 2024-04-24T15:32:52.096 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 26.939885809, 2024-04-24T15:32:52.096 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0079439720000000005, 2024-04-24T15:32:52.096 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:52.096 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:52.096 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:52.096 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:52.096 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:52.096 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 4256 2024-04-24T15:32:52.096 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.097 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:52.097 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.097 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:52.097 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.054726+0000", 2024-04-24T15:32:52.097 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.097 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.097 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.097 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:52.097 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.054726+0000", 2024-04-24T15:32:52.097 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.097 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.097 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.097 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:52.097 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.054725+0000", 2024-04-24T15:32:52.097 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999999 2024-04-24T15:32:52.098 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.098 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.098 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:52.098 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.054730+0000", 2024-04-24T15:32:52.098 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.4729999999999999e-06 2024-04-24T15:32:52.098 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.098 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.098 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:52.098 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.054742+0000", 2024-04-24T15:32:52.098 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.1489e-05 2024-04-24T15:32:52.098 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.098 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.098 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:52.099 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.054748+0000", 2024-04-24T15:32:52.099 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.879e-06 2024-04-24T15:32:52.099 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.099 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.099 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:52.099 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.054791+0000", 2024-04-24T15:32:52.099 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2673999999999998e-05 2024-04-24T15:32:52.099 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.099 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.099 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:52.099 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.054872+0000", 2024-04-24T15:32:52.099 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.1254999999999994e-05 2024-04-24T15:32:52.099 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.100 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.100 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 6", 2024-04-24T15:32:52.100 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.054978+0000", 2024-04-24T15:32:52.100 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00010624 2024-04-24T15:32:52.100 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.100 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.100 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:52.100 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.055893+0000", 2024-04-24T15:32:52.100 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00091453300000000003 2024-04-24T15:32:52.100 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.100 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.100 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:52.100 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.062620+0000", 2024-04-24T15:32:52.100 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0067271839999999998 2024-04-24T15:32:52.100 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.101 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.101 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:52.101 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.062645+0000", 2024-04-24T15:32:52.101 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.5491e-05 2024-04-24T15:32:52.101 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.101 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.101 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:52.101 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.062670+0000", 2024-04-24T15:32:52.101 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.474e-05 2024-04-24T15:32:52.101 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.101 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:52.101 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.101 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.101 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.102 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:7398 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-04-24T15:32:52.102 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:27.771937+0000", 2024-04-24T15:32:52.102 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 24.222674458, 2024-04-24T15:32:52.102 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.015783228, 2024-04-24T15:32:52.102 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:52.102 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:52.102 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:52.102 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:52.102 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:52.102 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 7398 2024-04-24T15:32:52.102 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.102 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:52.102 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.102 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:52.103 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:27.771937+0000", 2024-04-24T15:32:52.103 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.103 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.103 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.103 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:52.103 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:27.771937+0000", 2024-04-24T15:32:52.103 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.103 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.103 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.103 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:52.103 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:27.771936+0000", 2024-04-24T15:32:52.103 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999986 2024-04-24T15:32:52.103 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.103 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.104 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:52.104 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:27.771944+0000", 2024-04-24T15:32:52.104 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.8960000000000003e-06 2024-04-24T15:32:52.104 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.104 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.104 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:52.104 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:27.771955+0000", 2024-04-24T15:32:52.104 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0951000000000001e-05 2024-04-24T15:32:52.104 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.104 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.104 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:52.104 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:27.771959+0000", 2024-04-24T15:32:52.104 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.69e-06 2024-04-24T15:32:52.104 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.105 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.105 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:52.105 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:27.771998+0000", 2024-04-24T15:32:52.105 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8705000000000003e-05 2024-04-24T15:32:52.105 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.105 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.105 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:52.105 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:27.772054+0000", 2024-04-24T15:32:52.105 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.5420000000000001e-05 2024-04-24T15:32:52.105 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.105 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.105 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:52.105 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:27.774882+0000", 2024-04-24T15:32:52.105 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.002828344 2024-04-24T15:32:52.106 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.106 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.106 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:52.106 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:27.787163+0000", 2024-04-24T15:32:52.106 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.012281556000000001 2024-04-24T15:32:52.106 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.106 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.106 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:52.106 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:27.787651+0000", 2024-04-24T15:32:52.106 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00048708600000000002 2024-04-24T15:32:52.106 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.106 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.106 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:52.107 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:27.787691+0000", 2024-04-24T15:32:52.107 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0531999999999998e-05 2024-04-24T15:32:52.107 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.107 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.107 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:52.107 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:27.787721+0000", 2024-04-24T15:32:52.107 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.9518000000000001e-05 2024-04-24T15:32:52.107 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.107 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:52.107 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.107 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.107 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.107 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:7825 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-04-24T15:32:52.107 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:28.311523+0000", 2024-04-24T15:32:52.108 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 23.683088398999999, 2024-04-24T15:32:52.108 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.019381513, 2024-04-24T15:32:52.108 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:52.108 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:52.108 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:52.108 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:52.108 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:52.108 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 7825 2024-04-24T15:32:52.108 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.108 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:52.108 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.108 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:52.108 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:28.311523+0000", 2024-04-24T15:32:52.108 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.109 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.109 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.109 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:52.109 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:28.311523+0000", 2024-04-24T15:32:52.109 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.109 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.109 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.109 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:52.109 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:28.311523+0000", 2024-04-24T15:32:52.109 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:52.109 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.109 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.109 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:52.110 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:28.311527+0000", 2024-04-24T15:32:52.110 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6290000000000001e-06 2024-04-24T15:32:52.110 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.110 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.110 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:52.110 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:28.311538+0000", 2024-04-24T15:32:52.110 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0239e-05 2024-04-24T15:32:52.110 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.110 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.110 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:52.110 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:28.311542+0000", 2024-04-24T15:32:52.110 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3059999999999999e-06 2024-04-24T15:32:52.110 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.110 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.111 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:52.111 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:28.311621+0000", 2024-04-24T15:32:52.111 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.8671999999999995e-05 2024-04-24T15:32:52.111 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.111 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.111 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:52.111 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:28.311667+0000", 2024-04-24T15:32:52.111 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6842999999999999e-05 2024-04-24T15:32:52.111 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.111 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.111 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:52.111 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:28.315077+0000", 2024-04-24T15:32:52.111 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0034100889999999998 2024-04-24T15:32:52.111 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.111 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.111 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:52.112 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:28.330557+0000", 2024-04-24T15:32:52.112 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.015479100000000001 2024-04-24T15:32:52.112 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.112 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.112 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:52.112 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:28.330843+0000", 2024-04-24T15:32:52.112 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000286526 2024-04-24T15:32:52.112 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.112 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.112 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:52.112 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:28.330877+0000", 2024-04-24T15:32:52.112 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.4119999999999999e-05 2024-04-24T15:32:52.112 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.112 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.112 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:52.113 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:28.330905+0000", 2024-04-24T15:32:52.113 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7688999999999999e-05 2024-04-24T15:32:52.113 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.113 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:52.113 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.113 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.113 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:52.113 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:32:52.114 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-24T15:32:52.115 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:32:52.115+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:32:52.215 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:32:52.216+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:32:52.256 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 6] out_osds: [1, 4, 5, 7] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-04-24T15:32:52.256 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-24T15:32:52.256 INFO:tasks.thrashosds.thrasher:reset_purged_snaps_last 2024-04-24T15:32:52.256 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 reset_purged_snaps_last 2024-04-24T15:32:52.277 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:32:52.277 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:32:52.278 INFO:teuthology.orchestra.run.smithi151.stdout: "num_ops": 0 2024-04-24T15:32:52.278 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:32:52.287 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-24T15:32:52.330 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:32:52.331+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:32:52.409 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 reset_purged_snaps_last 2024-04-24T15:32:52.410 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:32:52.411 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:32:52.411 INFO:teuthology.orchestra.run.smithi151.stdout: "complaint_time": 30, 2024-04-24T15:32:52.411 INFO:teuthology.orchestra.run.smithi151.stdout: "num_blocked_ops": 0 2024-04-24T15:32:52.411 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:32:52.421 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-24T15:32:52.453 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:32:52.453+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:32:52.517 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:32:52.517+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:32:52.564 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 reset_purged_snaps_last 2024-04-24T15:32:52.587 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:32:52.587 INFO:teuthology.orchestra.run.smithi151.stdout: "size": 20, 2024-04-24T15:32:52.587 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 600, 2024-04-24T15:32:52.587 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [ 2024-04-24T15:32:52.587 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.587 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23613 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 14688256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:52.588 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.792522+0000", 2024-04-24T15:32:52.588 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 17.794829565000001, 2024-04-24T15:32:52.588 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040507356000000001, 2024-04-24T15:32:52.588 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:52.588 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:52.588 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:52.588 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:52.588 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:52.588 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23613 2024-04-24T15:32:52.588 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.588 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:52.588 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.588 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:52.588 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792522+0000", 2024-04-24T15:32:52.589 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.589 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.589 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.589 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:52.589 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792522+0000", 2024-04-24T15:32:52.589 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.589 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.589 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.589 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:52.589 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792522+0000", 2024-04-24T15:32:52.589 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:52.589 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.589 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.589 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:52.589 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792523+0000", 2024-04-24T15:32:52.590 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.576e-06 2024-04-24T15:32:52.590 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.590 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.590 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:52.590 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792526+0000", 2024-04-24T15:32:52.590 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7439999999999999e-06 2024-04-24T15:32:52.590 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.590 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.590 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:52.590 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792528+0000", 2024-04-24T15:32:52.590 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.0600000000000002e-06 2024-04-24T15:32:52.590 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.590 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.591 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:52.591 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.832864+0000", 2024-04-24T15:32:52.591 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040335742000000001 2024-04-24T15:32:52.591 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.591 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.591 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:52.591 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.832953+0000", 2024-04-24T15:32:52.591 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.9281000000000001e-05 2024-04-24T15:32:52.591 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.591 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.591 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:52.591 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.833029+0000", 2024-04-24T15:32:52.591 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.6259000000000003e-05 2024-04-24T15:32:52.591 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.591 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:52.592 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.592 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.592 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.592 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23614 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 14729216~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:52.592 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.792530+0000", 2024-04-24T15:32:52.592 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 17.794821300999999, 2024-04-24T15:32:52.592 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040633957999999998, 2024-04-24T15:32:52.592 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:52.592 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:52.592 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:52.592 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:52.592 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:52.592 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23614 2024-04-24T15:32:52.592 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.592 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:52.593 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.593 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:52.593 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792530+0000", 2024-04-24T15:32:52.593 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.593 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.593 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.593 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:52.593 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792530+0000", 2024-04-24T15:32:52.593 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.593 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.593 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.593 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:52.593 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792530+0000", 2024-04-24T15:32:52.593 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:32:52.593 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.594 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.594 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:52.594 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792531+0000", 2024-04-24T15:32:52.594 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.46e-06 2024-04-24T15:32:52.594 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.594 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.594 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:52.594 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792534+0000", 2024-04-24T15:32:52.594 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8959999999999999e-06 2024-04-24T15:32:52.594 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.594 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.594 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:52.594 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792536+0000", 2024-04-24T15:32:52.594 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.841e-06 2024-04-24T15:32:52.595 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.595 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.595 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:52.595 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.833060+0000", 2024-04-24T15:32:52.595 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040524125000000001 2024-04-24T15:32:52.595 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.595 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.595 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:52.595 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.833108+0000", 2024-04-24T15:32:52.595 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.7547999999999999e-05 2024-04-24T15:32:52.595 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.595 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.595 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:52.595 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.833164+0000", 2024-04-24T15:32:52.595 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6297e-05 2024-04-24T15:32:52.596 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.596 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:52.596 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.596 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.596 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.596 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23623 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15097856~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:52.596 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.793887+0000", 2024-04-24T15:32:52.596 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 17.79346434, 2024-04-24T15:32:52.596 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040432784999999999, 2024-04-24T15:32:52.596 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:52.596 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:52.596 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:52.596 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:52.596 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:52.596 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23623 2024-04-24T15:32:52.597 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.597 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:52.597 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.597 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:52.597 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793887+0000", 2024-04-24T15:32:52.597 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.597 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.597 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.597 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:52.597 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793887+0000", 2024-04-24T15:32:52.597 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.597 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.597 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.597 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:52.597 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793887+0000", 2024-04-24T15:32:52.598 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:32:52.598 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.598 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.598 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:52.598 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793888+0000", 2024-04-24T15:32:52.598 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3659999999999999e-06 2024-04-24T15:32:52.598 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.598 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.598 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:52.598 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793891+0000", 2024-04-24T15:32:52.598 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8389999999999998e-06 2024-04-24T15:32:52.598 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.598 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.598 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:52.599 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793893+0000", 2024-04-24T15:32:52.599 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.9549999999999999e-06 2024-04-24T15:32:52.599 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.599 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.599 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:52.599 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834239+0000", 2024-04-24T15:32:52.599 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040345487999999999 2024-04-24T15:32:52.599 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.599 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.599 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:52.599 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834280+0000", 2024-04-24T15:32:52.599 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.1878000000000001e-05 2024-04-24T15:32:52.599 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.599 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.599 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:52.600 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834320+0000", 2024-04-24T15:32:52.600 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9480000000000001e-05 2024-04-24T15:32:52.600 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.600 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:52.600 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.600 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.600 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.600 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23624 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15138816~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:52.600 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.794053+0000", 2024-04-24T15:32:52.600 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 17.793298392000001, 2024-04-24T15:32:52.600 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040370801999999997, 2024-04-24T15:32:52.600 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:52.600 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:52.600 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:52.601 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:52.601 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:52.601 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23624 2024-04-24T15:32:52.601 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.601 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:52.601 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.601 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:52.601 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794053+0000", 2024-04-24T15:32:52.601 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.601 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.601 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.601 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:52.601 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794053+0000", 2024-04-24T15:32:52.601 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.601 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.602 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.602 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:52.602 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793895+0000", 2024-04-24T15:32:52.602 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998422 2024-04-24T15:32:52.602 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.602 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.602 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:52.602 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794057+0000", 2024-04-24T15:32:52.602 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00016149400000000001 2024-04-24T15:32:52.602 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.602 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.602 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:52.602 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794065+0000", 2024-04-24T15:32:52.602 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.5259999999999993e-06 2024-04-24T15:32:52.603 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.603 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.603 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:52.603 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794071+0000", 2024-04-24T15:32:52.603 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6149999999999996e-06 2024-04-24T15:32:52.603 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.603 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.603 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:52.603 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834341+0000", 2024-04-24T15:32:52.603 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040270366000000002 2024-04-24T15:32:52.603 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.603 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.603 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:52.603 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834381+0000", 2024-04-24T15:32:52.604 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0102999999999999e-05 2024-04-24T15:32:52.604 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.604 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.604 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:52.604 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834424+0000", 2024-04-24T15:32:52.604 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2713000000000003e-05 2024-04-24T15:32:52.604 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.604 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:52.604 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.604 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.604 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.604 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23625 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15179776~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:52.604 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.794160+0000", 2024-04-24T15:32:52.604 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 17.79319151, 2024-04-24T15:32:52.604 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040413004000000002, 2024-04-24T15:32:52.605 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:52.605 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:52.605 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:52.605 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:52.605 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:52.605 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23625 2024-04-24T15:32:52.605 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.605 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:52.605 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.605 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:52.605 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794160+0000", 2024-04-24T15:32:52.605 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.605 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.605 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.605 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:52.606 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794160+0000", 2024-04-24T15:32:52.606 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.606 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.606 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.606 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:52.606 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794086+0000", 2024-04-24T15:32:52.606 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999256 2024-04-24T15:32:52.606 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.606 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.606 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:52.606 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794163+0000", 2024-04-24T15:32:52.606 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7896000000000001e-05 2024-04-24T15:32:52.606 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.606 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.607 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:52.607 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794171+0000", 2024-04-24T15:32:52.607 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.9500000000000001e-06 2024-04-24T15:32:52.607 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.607 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.607 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:52.607 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794179+0000", 2024-04-24T15:32:52.607 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7970000000000001e-06 2024-04-24T15:32:52.607 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.607 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.607 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:52.607 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834451+0000", 2024-04-24T15:32:52.607 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040271492999999998 2024-04-24T15:32:52.607 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.607 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.608 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:52.608 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834519+0000", 2024-04-24T15:32:52.608 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.7984000000000005e-05 2024-04-24T15:32:52.608 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.608 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.608 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:52.608 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834573+0000", 2024-04-24T15:32:52.608 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.4339999999999998e-05 2024-04-24T15:32:52.608 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.608 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:52.608 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.608 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.608 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.608 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23629 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15343616~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:52.608 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.794748+0000", 2024-04-24T15:32:52.609 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 17.792603708000001, 2024-04-24T15:32:52.609 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040419718, 2024-04-24T15:32:52.609 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:52.609 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:52.609 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:52.609 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:52.609 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:52.609 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23629 2024-04-24T15:32:52.609 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.609 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:52.609 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.609 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:52.609 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794748+0000", 2024-04-24T15:32:52.609 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.609 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.610 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.610 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:52.610 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794748+0000", 2024-04-24T15:32:52.610 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.610 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.610 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.610 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:52.610 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794673+0000", 2024-04-24T15:32:52.610 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999251 2024-04-24T15:32:52.610 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.610 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.610 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:52.610 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794751+0000", 2024-04-24T15:32:52.610 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.8480999999999996e-05 2024-04-24T15:32:52.611 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.611 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.611 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:52.611 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794760+0000", 2024-04-24T15:32:52.611 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3429999999999996e-06 2024-04-24T15:32:52.611 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.611 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.611 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:52.611 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794764+0000", 2024-04-24T15:32:52.611 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5600000000000004e-06 2024-04-24T15:32:52.611 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.611 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.611 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:52.611 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835040+0000", 2024-04-24T15:32:52.612 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040275188000000003 2024-04-24T15:32:52.612 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.612 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.612 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:52.612 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835105+0000", 2024-04-24T15:32:52.612 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.5992000000000001e-05 2024-04-24T15:32:52.612 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.612 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.612 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:52.612 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835167+0000", 2024-04-24T15:32:52.612 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.1981000000000002e-05 2024-04-24T15:32:52.612 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.612 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:52.612 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.612 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.612 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.613 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23630 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15384576~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:52.613 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.794839+0000", 2024-04-24T15:32:52.613 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 17.792512639000002, 2024-04-24T15:32:52.613 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040523894999999997, 2024-04-24T15:32:52.613 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:52.613 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:52.613 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:52.613 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:52.613 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:52.613 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23630 2024-04-24T15:32:52.613 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.613 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:52.613 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.613 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:52.614 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794839+0000", 2024-04-24T15:32:52.614 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.614 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.614 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.614 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:52.614 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794839+0000", 2024-04-24T15:32:52.614 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.614 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.614 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.614 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:52.614 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794768+0000", 2024-04-24T15:32:52.614 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999294 2024-04-24T15:32:52.614 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.615 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.615 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:52.615 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794841+0000", 2024-04-24T15:32:52.615 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2793000000000006e-05 2024-04-24T15:32:52.615 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.615 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.615 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:52.615 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794846+0000", 2024-04-24T15:32:52.615 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.7230000000000004e-06 2024-04-24T15:32:52.615 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.615 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.615 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:52.615 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794848+0000", 2024-04-24T15:32:52.615 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7659999999999999e-06 2024-04-24T15:32:52.616 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.616 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.616 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:52.616 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835223+0000", 2024-04-24T15:32:52.616 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040374275000000001 2024-04-24T15:32:52.616 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.616 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.616 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:52.616 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835295+0000", 2024-04-24T15:32:52.616 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2633999999999996e-05 2024-04-24T15:32:52.616 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.616 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.616 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:52.616 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835363+0000", 2024-04-24T15:32:52.617 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.7355999999999997e-05 2024-04-24T15:32:52.617 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.617 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:52.617 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.617 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.617 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.617 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23631 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15425536~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:52.617 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.794956+0000", 2024-04-24T15:32:52.617 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 17.792395798000001, 2024-04-24T15:32:52.617 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040562383, 2024-04-24T15:32:52.617 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:52.617 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:52.617 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:52.617 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:52.617 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:52.617 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23631 2024-04-24T15:32:52.618 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.618 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:52.618 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.618 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:52.618 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794956+0000", 2024-04-24T15:32:52.618 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.618 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.618 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.618 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:52.618 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794956+0000", 2024-04-24T15:32:52.618 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.618 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.618 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.619 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:52.619 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794859+0000", 2024-04-24T15:32:52.619 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999032 2024-04-24T15:32:52.619 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.619 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.619 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:52.619 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794959+0000", 2024-04-24T15:32:52.619 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000100207 2024-04-24T15:32:52.619 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.619 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.619 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:52.619 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794967+0000", 2024-04-24T15:32:52.619 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7640000000000005e-06 2024-04-24T15:32:52.619 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.619 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.620 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:52.620 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794971+0000", 2024-04-24T15:32:52.620 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.335e-06 2024-04-24T15:32:52.620 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.620 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.620 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:52.620 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835388+0000", 2024-04-24T15:32:52.620 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040417340000000003 2024-04-24T15:32:52.620 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.620 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.620 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:52.620 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835459+0000", 2024-04-24T15:32:52.620 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.0118999999999997e-05 2024-04-24T15:32:52.620 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.621 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.621 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:52.621 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835518+0000", 2024-04-24T15:32:52.621 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.9438999999999998e-05 2024-04-24T15:32:52.621 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.621 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:52.621 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.621 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.621 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.621 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23632 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15466496~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:52.621 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795060+0000", 2024-04-24T15:32:52.621 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 17.792291087999999, 2024-04-24T15:32:52.621 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040667993, 2024-04-24T15:32:52.622 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:52.622 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:52.622 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:52.622 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:52.622 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:52.622 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23632 2024-04-24T15:32:52.622 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.622 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:52.622 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.622 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:52.622 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795060+0000", 2024-04-24T15:32:52.622 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.622 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.622 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.623 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:52.623 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795060+0000", 2024-04-24T15:32:52.623 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.623 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.623 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.623 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:52.623 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794976+0000", 2024-04-24T15:32:52.623 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999156 2024-04-24T15:32:52.623 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.623 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.623 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:52.623 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795064+0000", 2024-04-24T15:32:52.623 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.7709999999999999e-05 2024-04-24T15:32:52.623 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.624 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.624 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:52.624 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795072+0000", 2024-04-24T15:32:52.624 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3769999999999996e-06 2024-04-24T15:32:52.624 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.624 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.624 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:52.624 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795078+0000", 2024-04-24T15:32:52.624 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.5079999999999996e-06 2024-04-24T15:32:52.624 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.624 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.624 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:52.624 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835541+0000", 2024-04-24T15:32:52.624 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040463440000000003 2024-04-24T15:32:52.624 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.625 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.625 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:52.625 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835657+0000", 2024-04-24T15:32:52.625 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00011572499999999999 2024-04-24T15:32:52.625 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.625 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.625 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:52.625 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835728+0000", 2024-04-24T15:32:52.625 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1402000000000005e-05 2024-04-24T15:32:52.625 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.625 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:52.625 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.625 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.625 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.626 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23633 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15507456~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:52.626 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795134+0000", 2024-04-24T15:32:52.626 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 17.792217433000001, 2024-04-24T15:32:52.626 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040748591000000001, 2024-04-24T15:32:52.626 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:52.626 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:52.626 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:52.626 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:52.626 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:52.626 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23633 2024-04-24T15:32:52.626 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.626 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:52.626 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.626 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:52.627 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795134+0000", 2024-04-24T15:32:52.627 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.627 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.627 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.627 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:52.627 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795134+0000", 2024-04-24T15:32:52.627 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.627 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.627 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.627 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:52.627 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795083+0000", 2024-04-24T15:32:52.627 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999949 2024-04-24T15:32:52.627 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.627 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.628 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:52.628 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795138+0000", 2024-04-24T15:32:52.628 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.4472000000000003e-05 2024-04-24T15:32:52.628 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.628 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.628 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:52.628 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795145+0000", 2024-04-24T15:32:52.628 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2779999999999996e-06 2024-04-24T15:32:52.628 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.628 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.628 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:52.628 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795149+0000", 2024-04-24T15:32:52.628 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.568e-06 2024-04-24T15:32:52.628 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.628 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.629 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:52.629 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835757+0000", 2024-04-24T15:32:52.629 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040608894999999999 2024-04-24T15:32:52.629 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.629 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.629 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:52.629 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835829+0000", 2024-04-24T15:32:52.629 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1879999999999996e-05 2024-04-24T15:32:52.629 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.629 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.629 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:52.629 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835883+0000", 2024-04-24T15:32:52.629 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.3346000000000001e-05 2024-04-24T15:32:52.629 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.629 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:52.630 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.630 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.630 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.630 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23634 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15548416~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:52.630 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795271+0000", 2024-04-24T15:32:52.630 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 17.792080399, 2024-04-24T15:32:52.630 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040755845999999998, 2024-04-24T15:32:52.630 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:52.630 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:52.630 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:52.630 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:52.630 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:52.630 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23634 2024-04-24T15:32:52.631 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.631 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:52.631 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.631 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:52.631 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795271+0000", 2024-04-24T15:32:52.631 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.631 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.631 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.631 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:52.631 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795271+0000", 2024-04-24T15:32:52.631 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.631 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.631 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.631 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:52.631 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795152+0000", 2024-04-24T15:32:52.632 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998813 2024-04-24T15:32:52.632 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.632 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.632 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:52.632 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795276+0000", 2024-04-24T15:32:52.632 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00012401799999999999 2024-04-24T15:32:52.632 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.632 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.632 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:52.632 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795281+0000", 2024-04-24T15:32:52.632 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8860000000000003e-06 2024-04-24T15:32:52.632 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.632 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.632 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:52.633 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795284+0000", 2024-04-24T15:32:52.633 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.0450000000000001e-06 2024-04-24T15:32:52.633 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.633 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.633 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:52.633 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835905+0000", 2024-04-24T15:32:52.633 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040621035999999999 2024-04-24T15:32:52.633 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.633 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.633 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:52.633 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835971+0000", 2024-04-24T15:32:52.633 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.5314000000000005e-05 2024-04-24T15:32:52.633 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.633 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.633 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:52.634 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836027+0000", 2024-04-24T15:32:52.634 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6305999999999999e-05 2024-04-24T15:32:52.634 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.634 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:52.634 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.634 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.634 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.634 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23635 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15589376~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:52.634 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795382+0000", 2024-04-24T15:32:52.634 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 17.791969626, 2024-04-24T15:32:52.634 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040805363999999997, 2024-04-24T15:32:52.634 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:52.634 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:52.634 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:52.635 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:52.635 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:52.635 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23635 2024-04-24T15:32:52.635 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.635 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:52.635 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.635 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:52.635 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795382+0000", 2024-04-24T15:32:52.635 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.635 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.635 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.635 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:52.635 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795382+0000", 2024-04-24T15:32:52.635 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.636 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.636 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.636 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:52.636 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795288+0000", 2024-04-24T15:32:52.636 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999056 2024-04-24T15:32:52.636 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.636 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.636 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:52.636 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795385+0000", 2024-04-24T15:32:52.636 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.7288999999999996e-05 2024-04-24T15:32:52.636 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.636 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.636 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:52.636 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795392+0000", 2024-04-24T15:32:52.636 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.452e-06 2024-04-24T15:32:52.637 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.637 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.637 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:52.637 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795397+0000", 2024-04-24T15:32:52.637 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2860000000000001e-06 2024-04-24T15:32:52.637 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.637 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.637 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:52.637 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836054+0000", 2024-04-24T15:32:52.637 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040657581999999998 2024-04-24T15:32:52.637 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.637 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.637 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:52.637 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836124+0000", 2024-04-24T15:32:52.637 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.0020999999999999e-05 2024-04-24T15:32:52.638 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.638 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.638 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:52.638 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836187+0000", 2024-04-24T15:32:52.638 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.3009999999999995e-05 2024-04-24T15:32:52.638 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.638 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:52.638 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.638 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.638 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.638 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23636 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15630336~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:52.638 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795610+0000", 2024-04-24T15:32:52.638 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 17.791741267999999, 2024-04-24T15:32:52.638 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040730942999999999, 2024-04-24T15:32:52.639 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:52.639 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:52.639 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:52.639 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:52.639 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:52.639 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23636 2024-04-24T15:32:52.639 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.639 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:52.639 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.639 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:52.639 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795610+0000", 2024-04-24T15:32:52.639 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.639 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.639 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.639 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:52.640 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795610+0000", 2024-04-24T15:32:52.640 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.640 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.640 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.640 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:52.640 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795401+0000", 2024-04-24T15:32:52.640 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9997911 2024-04-24T15:32:52.640 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.640 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.640 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:52.640 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795614+0000", 2024-04-24T15:32:52.640 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00021228600000000001 2024-04-24T15:32:52.640 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.640 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.640 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:52.641 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795622+0000", 2024-04-24T15:32:52.641 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3920000000000003e-06 2024-04-24T15:32:52.641 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.641 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.641 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:52.641 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795627+0000", 2024-04-24T15:32:52.641 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8199999999999996e-06 2024-04-24T15:32:52.641 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.641 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.641 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:52.641 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836211+0000", 2024-04-24T15:32:52.641 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040583803000000002 2024-04-24T15:32:52.641 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.642 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.642 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:52.642 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836284+0000", 2024-04-24T15:32:52.642 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.3617999999999999e-05 2024-04-24T15:32:52.642 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.642 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.642 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:52.642 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836341+0000", 2024-04-24T15:32:52.642 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6959999999999997e-05 2024-04-24T15:32:52.642 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.642 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:52.642 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.642 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.642 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.643 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23637 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15671296~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:52.643 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795699+0000", 2024-04-24T15:32:52.643 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 17.791652101, 2024-04-24T15:32:52.643 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040775556999999997, 2024-04-24T15:32:52.643 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:52.643 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:52.643 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:52.643 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:52.643 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:52.643 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23637 2024-04-24T15:32:52.643 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.643 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:52.643 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.643 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:52.644 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795699+0000", 2024-04-24T15:32:52.644 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.644 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.644 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.644 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:52.644 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795699+0000", 2024-04-24T15:32:52.644 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.644 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.644 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.644 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:52.644 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795632+0000", 2024-04-24T15:32:52.644 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999323 2024-04-24T15:32:52.644 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.644 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.644 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:52.645 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795704+0000", 2024-04-24T15:32:52.645 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2428000000000005e-05 2024-04-24T15:32:52.645 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.645 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.645 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:52.645 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795715+0000", 2024-04-24T15:32:52.645 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.1532000000000001e-05 2024-04-24T15:32:52.645 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.645 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.645 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:52.645 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795721+0000", 2024-04-24T15:32:52.645 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.8200000000000002e-06 2024-04-24T15:32:52.645 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.645 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.645 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:52.646 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836364+0000", 2024-04-24T15:32:52.646 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040642997 2024-04-24T15:32:52.646 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.646 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.646 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:52.646 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836435+0000", 2024-04-24T15:32:52.646 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.0821000000000005e-05 2024-04-24T15:32:52.646 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.646 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.646 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:52.646 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836475+0000", 2024-04-24T15:32:52.646 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9796999999999999e-05 2024-04-24T15:32:52.646 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.647 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:52.647 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.647 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.647 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.647 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23638 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15712256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:52.647 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795932+0000", 2024-04-24T15:32:52.647 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 17.791419071, 2024-04-24T15:32:52.647 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040668682999999997, 2024-04-24T15:32:52.647 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:52.647 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:52.647 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:52.647 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:52.647 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:52.647 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23638 2024-04-24T15:32:52.648 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.648 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:52.648 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.648 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:52.648 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795932+0000", 2024-04-24T15:32:52.648 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.648 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.648 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.648 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:52.649 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795932+0000", 2024-04-24T15:32:52.649 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.649 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.649 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.649 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:52.649 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795728+0000", 2024-04-24T15:32:52.649 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9997959 2024-04-24T15:32:52.649 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.649 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.649 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:52.649 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795936+0000", 2024-04-24T15:32:52.649 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000208128 2024-04-24T15:32:52.649 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.649 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.649 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:52.650 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795945+0000", 2024-04-24T15:32:52.650 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.7110000000000001e-06 2024-04-24T15:32:52.650 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.650 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.650 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:52.650 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795957+0000", 2024-04-24T15:32:52.650 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.1800999999999999e-05 2024-04-24T15:32:52.650 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.650 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.653 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:52.654 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836505+0000", 2024-04-24T15:32:52.654 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040547622999999998 2024-04-24T15:32:52.654 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.654 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.654 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:52.654 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836561+0000", 2024-04-24T15:32:52.654 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6598999999999998e-05 2024-04-24T15:32:52.654 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.654 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.655 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:52.655 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836601+0000", 2024-04-24T15:32:52.655 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9907000000000003e-05 2024-04-24T15:32:52.655 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.655 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:52.655 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.655 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.655 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.655 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23639 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15753216~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:52.655 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.796071+0000", 2024-04-24T15:32:52.655 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 17.79128081, 2024-04-24T15:32:52.655 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040648199000000003, 2024-04-24T15:32:52.655 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:52.655 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:52.656 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:52.656 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:52.656 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:52.656 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23639 2024-04-24T15:32:52.656 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.656 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:52.656 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.656 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:52.656 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796071+0000", 2024-04-24T15:32:52.656 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.656 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.656 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.656 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:52.656 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796071+0000", 2024-04-24T15:32:52.656 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.657 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.657 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.657 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:52.657 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795967+0000", 2024-04-24T15:32:52.657 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998965 2024-04-24T15:32:52.657 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.657 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.657 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:52.657 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796074+0000", 2024-04-24T15:32:52.657 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000107013 2024-04-24T15:32:52.657 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.657 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.657 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:52.657 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796081+0000", 2024-04-24T15:32:52.657 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2590000000000003e-06 2024-04-24T15:32:52.658 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.658 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.658 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:52.658 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796085+0000", 2024-04-24T15:32:52.658 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5269999999999999e-06 2024-04-24T15:32:52.658 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.658 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.658 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:52.658 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836624+0000", 2024-04-24T15:32:52.658 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040539324000000002 2024-04-24T15:32:52.658 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.658 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.658 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:52.659 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836667+0000", 2024-04-24T15:32:52.659 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2562999999999999e-05 2024-04-24T15:32:52.659 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.659 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.659 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:52.659 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836719+0000", 2024-04-24T15:32:52.659 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.2015999999999998e-05 2024-04-24T15:32:52.659 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.659 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:52.659 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.659 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.659 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.659 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23640 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15794176~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:52.659 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.796492+0000", 2024-04-24T15:32:52.660 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 17.790859051999998, 2024-04-24T15:32:52.660 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040420000999999997, 2024-04-24T15:32:52.660 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:52.660 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:52.660 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:52.660 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:52.660 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:52.660 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23640 2024-04-24T15:32:52.660 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.660 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:52.660 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.660 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:52.660 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796492+0000", 2024-04-24T15:32:52.660 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.660 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.661 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.661 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:52.661 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796492+0000", 2024-04-24T15:32:52.661 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.661 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.661 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.661 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:52.661 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796088+0000", 2024-04-24T15:32:52.661 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9995961 2024-04-24T15:32:52.661 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.661 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.661 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:52.661 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796496+0000", 2024-04-24T15:32:52.661 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00040785700000000002 2024-04-24T15:32:52.662 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.662 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.662 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:52.662 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796503+0000", 2024-04-24T15:32:52.662 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1049999999999997e-06 2024-04-24T15:32:52.662 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.662 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.662 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:52.662 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796508+0000", 2024-04-24T15:32:52.662 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.155e-06 2024-04-24T15:32:52.662 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.662 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.662 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:52.662 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836756+0000", 2024-04-24T15:32:52.663 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040248850000000003 2024-04-24T15:32:52.663 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.663 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.663 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:52.663 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836841+0000", 2024-04-24T15:32:52.663 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.4735999999999996e-05 2024-04-24T15:32:52.663 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.663 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.663 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:52.663 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836912+0000", 2024-04-24T15:32:52.663 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1229999999999994e-05 2024-04-24T15:32:52.663 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.663 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:52.663 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.664 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.664 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.664 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23641 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15835136~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:52.664 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.796658+0000", 2024-04-24T15:32:52.664 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 17.790692975999999, 2024-04-24T15:32:52.664 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040420706000000001, 2024-04-24T15:32:52.664 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:52.664 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:52.664 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:52.664 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:52.664 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:52.664 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23641 2024-04-24T15:32:52.664 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.664 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:52.664 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.665 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:52.665 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796658+0000", 2024-04-24T15:32:52.665 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.665 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.665 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.665 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:52.665 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796658+0000", 2024-04-24T15:32:52.665 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.665 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.665 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.665 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:52.665 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796512+0000", 2024-04-24T15:32:52.665 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998531 2024-04-24T15:32:52.665 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.665 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.666 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:52.666 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796669+0000", 2024-04-24T15:32:52.666 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000157577 2024-04-24T15:32:52.666 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.666 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.666 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:52.666 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796677+0000", 2024-04-24T15:32:52.666 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7800000000000001e-06 2024-04-24T15:32:52.666 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.666 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.666 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:52.666 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796681+0000", 2024-04-24T15:32:52.666 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.721e-06 2024-04-24T15:32:52.666 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.667 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.667 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:52.667 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836941+0000", 2024-04-24T15:32:52.667 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040259907999999997 2024-04-24T15:32:52.667 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.667 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.667 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:52.667 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837017+0000", 2024-04-24T15:32:52.667 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.6071999999999999e-05 2024-04-24T15:32:52.667 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.667 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.667 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:52.667 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837079+0000", 2024-04-24T15:32:52.667 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.2358000000000002e-05 2024-04-24T15:32:52.668 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.668 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:52.668 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.668 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.668 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.668 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23645 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15998976~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:52.668 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.797360+0000", 2024-04-24T15:32:52.668 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 17.789991478000001, 2024-04-24T15:32:52.668 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040387645999999999, 2024-04-24T15:32:52.668 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:52.668 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:52.668 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:52.668 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:52.668 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:52.669 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23645 2024-04-24T15:32:52.669 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.669 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:52.669 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.669 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:52.669 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797360+0000", 2024-04-24T15:32:52.669 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.669 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.669 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.669 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:52.669 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797360+0000", 2024-04-24T15:32:52.669 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.669 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.669 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.669 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:52.670 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797288+0000", 2024-04-24T15:32:52.670 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999928 2024-04-24T15:32:52.670 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.670 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.670 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:52.670 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797364+0000", 2024-04-24T15:32:52.670 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.5347999999999998e-05 2024-04-24T15:32:52.670 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.670 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.670 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:52.670 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797372+0000", 2024-04-24T15:32:52.670 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.2409999999999995e-06 2024-04-24T15:32:52.670 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.670 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.671 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:52.671 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797376+0000", 2024-04-24T15:32:52.671 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.515e-06 2024-04-24T15:32:52.671 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.671 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.671 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:52.671 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837586+0000", 2024-04-24T15:32:52.671 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040210047999999998 2024-04-24T15:32:52.671 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.671 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.671 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:52.671 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837672+0000", 2024-04-24T15:32:52.671 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.5997999999999999e-05 2024-04-24T15:32:52.671 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.672 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.672 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:52.672 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837748+0000", 2024-04-24T15:32:52.672 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.5328999999999997e-05 2024-04-24T15:32:52.672 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.672 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:52.672 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.672 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.672 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.672 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23646 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 16039936~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:52.672 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.797498+0000", 2024-04-24T15:32:52.672 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 17.789853377, 2024-04-24T15:32:52.672 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040433139, 2024-04-24T15:32:52.672 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:52.673 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:52.673 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:52.673 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:52.673 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:52.673 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23646 2024-04-24T15:32:52.673 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.673 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:52.673 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.673 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:52.673 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797498+0000", 2024-04-24T15:32:52.673 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.673 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.673 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.673 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:52.674 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797498+0000", 2024-04-24T15:32:52.674 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:52.674 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.674 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.674 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:52.674 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797381+0000", 2024-04-24T15:32:52.674 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998832 2024-04-24T15:32:52.674 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.674 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.674 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:52.674 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797501+0000", 2024-04-24T15:32:52.674 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000119759 2024-04-24T15:32:52.674 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.674 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.675 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:52.675 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797510+0000", 2024-04-24T15:32:52.675 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.4770000000000003e-06 2024-04-24T15:32:52.675 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.675 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.675 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:52.675 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797515+0000", 2024-04-24T15:32:52.675 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8330000000000001e-06 2024-04-24T15:32:52.675 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.675 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.675 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:52.675 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837777+0000", 2024-04-24T15:32:52.675 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040262502999999998 2024-04-24T15:32:52.675 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.676 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.676 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:52.676 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837860+0000", 2024-04-24T15:32:52.676 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3002000000000003e-05 2024-04-24T15:32:52.676 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:52.676 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:52.676 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:52.676 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837931+0000", 2024-04-24T15:32:52.676 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1222000000000004e-05 2024-04-24T15:32:52.676 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.676 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:52.676 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.676 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:52.676 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:52.677 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:32:52.677 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-24T15:32:52.678 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:32:52.678+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:32:52.679 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-24T15:32:52.778 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:32:52.779+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:32:52.791 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #10, image image_client.0-clone10 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent11 2024-04-24T15:32:52.847 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:32:52.847 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:32:52.847 INFO:teuthology.orchestra.run.smithi151.stdout: "num_ops": 0 2024-04-24T15:32:52.848 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:32:52.857 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-24T15:32:52.860 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 reset_purged_snaps_last 2024-04-24T15:32:52.901 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:32:52.901+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:32:52.903 INFO:teuthology.orchestra.run.smithi096.stdout:2259 trunc from 0x88ef890 to 0xe76d1b2 2024-04-24T15:32:52.910 INFO:teuthology.orchestra.run.smithi096.stdout:2261 read 0xd12315c thru 0xd125a9c (0x2941 bytes) 2024-04-24T15:32:52.910 INFO:teuthology.orchestra.run.smithi096.stdout:2264 write 0x51638a6 thru 0x5168c9c (0x53f7 bytes) 2024-04-24T15:32:52.913 INFO:teuthology.orchestra.run.smithi096.stdout:2269 writesame 0xe5eade6 thru 0xe5edab7 data_size 0x1(0x2cd2 bytes) 2024-04-24T15:32:52.921 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:52.921 INFO:teuthology.orchestra.run.smithi096.stdout:2272 write 0xe6ead15 thru 0xe6f1146 (0x6432 bytes) 2024-04-24T15:32:52.923 INFO:teuthology.orchestra.run.smithi096.stdout:2273 write 0x1e49842 thru 0x1e4b45a (0x1c19 bytes) 2024-04-24T15:32:52.938 INFO:teuthology.orchestra.run.smithi096.stdout:2274 writesame 0x4675395 thru 0x4680425 data_size 0x3d(0xb091 bytes) 2024-04-24T15:32:52.945 INFO:teuthology.orchestra.run.smithi096.stdout:2275 write 0x45cc3ed thru 0x45dba9b (0xf6af bytes) 2024-04-24T15:32:52.946 INFO:teuthology.orchestra.run.smithi096.stdout:2276 write 0x159f722 thru 0x15ab96a (0xc249 bytes) 2024-04-24T15:32:52.950 INFO:teuthology.orchestra.run.smithi096.stdout:2279 write 0xa89a0ea thru 0xa89fafb (0x5a12 bytes) 2024-04-24T15:32:52.954 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:52.954 INFO:teuthology.orchestra.run.smithi096.stdout:2283 write 0x6ad6404 thru 0x6ae047d (0xa07a bytes) 2024-04-24T15:32:52.955 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:52.961 INFO:teuthology.orchestra.run.smithi096.stdout:2286 trunc from 0xe76d1b2 to 0x9c54861 2024-04-24T15:32:52.981 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:32:52.981+0000 7f23abcf6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 44021) UID: 0 2024-04-24T15:32:52.985 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:32:52.985 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:32:52.985 INFO:teuthology.orchestra.run.smithi151.stdout: "complaint_time": 30, 2024-04-24T15:32:52.985 INFO:teuthology.orchestra.run.smithi151.stdout: "num_blocked_ops": 0 2024-04-24T15:32:52.985 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:32:52.985 INFO:teuthology.orchestra.run.smithi096.stdout:2287 read 0x584c87b thru 0x5854dff (0x8585 bytes) 2024-04-24T15:32:52.986 INFO:teuthology.orchestra.run.smithi096.stdout:2288 write 0xd93f1a thru 0xd97bd6 (0x3cbd bytes) 2024-04-24T15:32:52.987 INFO:teuthology.orchestra.run.smithi096.stdout:2289 write 0x7f15ad7 thru 0x7f1e2fe (0x8828 bytes) 2024-04-24T15:32:52.989 INFO:teuthology.orchestra.run.smithi096.stdout:2290 punch from 0x75cfebf to 0x75d0e23, (0xf64 bytes) 2024-04-24T15:32:52.995 INFO:teuthology.orchestra.run.smithi096.stdout:2291 writesame 0xb0fe577 thru 0xb100a12 data_size 0xc(0x249c bytes) 2024-04-24T15:32:52.996 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-24T15:32:52.996 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:52.996 INFO:teuthology.orchestra.run.smithi096.stdout:2295 write 0x70515fe thru 0x705e556 (0xcf59 bytes) 2024-04-24T15:32:52.998 INFO:teuthology.orchestra.run.smithi096.stdout:2296 punch from 0x5905978 to 0x5911301, (0xb989 bytes) 2024-04-24T15:32:52.998 INFO:teuthology.orchestra.run.smithi096.stdout:2297 punch from 0x61d9249 to 0x61e102d, (0x7de4 bytes) 2024-04-24T15:32:52.998 INFO:teuthology.orchestra.run.smithi096.stdout:2299 read 0xa51f55b thru 0xa524339 (0x4ddf bytes) 2024-04-24T15:32:53.010 INFO:teuthology.orchestra.run.smithi096.stdout:2300 trunc from 0xb100a13 to 0x29730e6 2024-04-24T15:32:53.028 INFO:teuthology.orchestra.run.smithi096.stdout:2303 read 0x765223 thru 0x7673d2 (0x21b0 bytes) 2024-04-24T15:32:53.028 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:53.028 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:53.028 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:53.028 INFO:teuthology.orchestra.run.smithi096.stdout:2307 read 0x72b6f6 thru 0x733fa1 (0x88ac bytes) 2024-04-24T15:32:53.028 INFO:teuthology.orchestra.run.smithi096.stdout:2308 writesame 0x1ce098e thru 0x1cf05a7 data_size 0x3d(0xfc1a bytes) 2024-04-24T15:32:53.030 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:53.040 INFO:teuthology.orchestra.run.smithi096.stdout:2310 writesame 0x63f362a thru 0x63fe965 data_size 0x2(0xb33c bytes) 2024-04-24T15:32:53.041 INFO:teuthology.orchestra.run.smithi096.stdout:2311 read 0x5181373 thru 0x5181d4a (0x9d8 bytes) 2024-04-24T15:32:53.041 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:53.042 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:53.042 INFO:teuthology.orchestra.run.smithi096.stdout:2315 read 0x1fc5d5e thru 0x1fd3c3c (0xdedf bytes) 2024-04-24T15:32:53.042 INFO:teuthology.orchestra.run.smithi096.stdout:2316 read 0x8748ee thru 0x8754e4 (0xbf7 bytes) 2024-04-24T15:32:53.051 INFO:teuthology.orchestra.run.smithi096.stdout:2317 writesame 0x96ffae9 thru 0x97080f0 data_size 0x4(0x8608 bytes) 2024-04-24T15:32:53.051 INFO:teuthology.orchestra.run.smithi096.stdout:2319 read 0x6d6a31a thru 0x6d73687 (0x936e bytes) 2024-04-24T15:32:53.051 INFO:teuthology.orchestra.run.smithi096.stdout:2320 write 0x3b19440 thru 0x3b2122c (0x7ded bytes) 2024-04-24T15:32:53.052 INFO:teuthology.orchestra.run.smithi096.stdout:2321 read 0x2433469 thru 0x2442081 (0xec19 bytes) 2024-04-24T15:32:53.053 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:53.053 INFO:teuthology.orchestra.run.smithi096.stdout:2323 punch from 0x63d35ef to 0x63dc9ed, (0x93fe bytes) 2024-04-24T15:32:53.053 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:53.053 INFO:teuthology.orchestra.run.smithi096.stdout:2325 read 0x6b81393 thru 0x6b8edea (0xda58 bytes) 2024-04-24T15:32:53.054 INFO:teuthology.orchestra.run.smithi096.stdout:2326 read 0x589a9f6 thru 0x589cfec (0x25f7 bytes) 2024-04-24T15:32:53.054 INFO:teuthology.orchestra.run.smithi096.stdout:2327 read 0x668d85 thru 0x676a30 (0xdcac bytes) 2024-04-24T15:32:53.055 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:53.055 INFO:teuthology.orchestra.run.smithi096.stdout:2330 read 0x77e2b80 thru 0x77e3f81 (0x1402 bytes) 2024-04-24T15:32:53.065 INFO:teuthology.orchestra.run.smithi096.stdout:2331 trunc from 0x97080f1 to 0x1dcd4a4 2024-04-24T15:32:53.072 INFO:teuthology.orchestra.run.smithi096.stdout:2333 read 0x983a4b thru 0x98ca7d (0x9033 bytes) 2024-04-24T15:32:53.072 INFO:teuthology.orchestra.run.smithi096.stdout:2334 writesame 0xdc6000 thru 0xdc8039 data_size 0x19(0x203a bytes) 2024-04-24T15:32:53.074 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:53.074 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:53.082 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:32:53.083+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:32:53.085 INFO:teuthology.orchestra.run.smithi096.stdout:2338 writesame 0x735eba9 thru 0x736bcd7 data_size 0x1(0xd12f bytes) 2024-04-24T15:32:53.093 INFO:teuthology.orchestra.run.smithi096.stdout:2339 write 0x9a533db thru 0x9a53bcf (0x7f5 bytes) 2024-04-24T15:32:53.096 INFO:teuthology.orchestra.run.smithi096.stdout:2341 punch from 0x26ed0bf to 0x26f5c94, (0x8bd5 bytes) 2024-04-24T15:32:53.096 INFO:teuthology.orchestra.run.smithi096.stdout:2343 write 0x70df6f5 thru 0x70e2d93 (0x369f bytes) 2024-04-24T15:32:53.098 INFO:teuthology.orchestra.run.smithi096.stdout:2345 writesame 0x639bd15 thru 0x639d0ba data_size 0x5(0x13a6 bytes) 2024-04-24T15:32:53.100 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:53.100 INFO:teuthology.orchestra.run.smithi096.stdout:2348 read 0x73eb002 thru 0x73f4122 (0x9121 bytes) 2024-04-24T15:32:53.108 INFO:teuthology.orchestra.run.smithi096.stdout:2349 trunc from 0x9a53bd0 to 0x35ae1e1 2024-04-24T15:32:53.118 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:53.118 INFO:teuthology.orchestra.run.smithi096.stdout:2352 write 0xa72edf thru 0xa8238a (0xf4ac bytes) 2024-04-24T15:32:53.128 INFO:teuthology.orchestra.run.smithi096.stdout:2353 writesame 0x6b355af thru 0x6b3c08e data_size 0x78(0x6ae0 bytes) 2024-04-24T15:32:53.129 INFO:teuthology.orchestra.run.smithi096.stdout:2354 punch from 0x1aabcf3 to 0x1abacca, (0xefd7 bytes) 2024-04-24T15:32:53.129 INFO:teuthology.orchestra.run.smithi096.stdout:2355 write 0x50c8986 thru 0x50d66b9 (0xdd34 bytes) 2024-04-24T15:32:53.131 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:53.137 INFO:teuthology.orchestra.run.smithi096.stdout:2357 write 0x878d0c1 thru 0x8794739 (0x7679 bytes) 2024-04-24T15:32:53.140 INFO:teuthology.orchestra.run.smithi096.stdout:2358 writesame 0x34f23d1 thru 0x34fc4ed data_size 0x5(0xa11d bytes) 2024-04-24T15:32:53.144 INFO:teuthology.orchestra.run.smithi096.stdout:2359 write 0x6733caa thru 0x6737bec (0x3f43 bytes) 2024-04-24T15:32:53.145 INFO:teuthology.orchestra.run.smithi096.stdout:2360 read 0x5f3c7c4 thru 0x5f412c9 (0x4b06 bytes) 2024-04-24T15:32:53.146 INFO:teuthology.orchestra.run.smithi096.stdout:2361 punch from 0x8783b4c to 0x8793306, (0xf7ba bytes) 2024-04-24T15:32:53.146 INFO:teuthology.orchestra.run.smithi096.stdout:2362 read 0x3e1a86 thru 0x3ec533 (0xaaae bytes) 2024-04-24T15:32:53.147 INFO:teuthology.orchestra.run.smithi096.stdout:2363 read 0x5f3676b thru 0x5f36b56 (0x3ec bytes) 2024-04-24T15:32:53.147 INFO:teuthology.orchestra.run.smithi096.stdout:2364 punch from 0x68536bd to 0x6863386, (0xfcc9 bytes) 2024-04-24T15:32:53.147 INFO:teuthology.orchestra.run.smithi096.stdout:2365 read 0x50e589b thru 0x50f428a (0xe9f0 bytes) 2024-04-24T15:32:53.147 INFO:teuthology.orchestra.run.smithi096.stdout:2367 write 0x43cb58c thru 0x43cd3af (0x1e24 bytes) 2024-04-24T15:32:53.148 INFO:teuthology.orchestra.run.smithi096.stdout:2368 trunc from 0x879473a to 0x91e7125 2024-04-24T15:32:53.161 INFO:teuthology.orchestra.run.smithi096.stdout:2369 trunc from 0x91e7125 to 0x245d68f 2024-04-24T15:32:53.165 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:32:53.165 INFO:teuthology.orchestra.run.smithi151.stdout: "size": 20, 2024-04-24T15:32:53.165 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 600, 2024-04-24T15:32:53.165 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [ 2024-04-24T15:32:53.165 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.165 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17317 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 11509760~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:53.165 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.224408+0000", 2024-04-24T15:32:53.165 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 18.940724023000001, 2024-04-24T15:32:53.165 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042604418999999998, 2024-04-24T15:32:53.165 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:53.166 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:53.166 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:53.166 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:53.166 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:53.166 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17317 2024-04-24T15:32:53.166 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.166 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:53.166 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.166 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:53.166 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224408+0000", 2024-04-24T15:32:53.166 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:53.166 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.166 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.166 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:53.166 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224408+0000", 2024-04-24T15:32:53.167 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:53.167 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.167 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.167 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:53.167 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224407+0000", 2024-04-24T15:32:53.167 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:53.167 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.167 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.167 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:53.167 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224409+0000", 2024-04-24T15:32:53.167 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.1059999999999998e-06 2024-04-24T15:32:53.167 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.167 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.167 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:53.167 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224415+0000", 2024-04-24T15:32:53.167 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6219999999999997e-06 2024-04-24T15:32:53.168 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.168 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.168 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:53.168 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224419+0000", 2024-04-24T15:32:53.168 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8340000000000001e-06 2024-04-24T15:32:53.168 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.168 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.168 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:53.168 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.266851+0000", 2024-04-24T15:32:53.168 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042431910000000003 2024-04-24T15:32:53.168 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.168 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.168 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:53.168 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.266931+0000", 2024-04-24T15:32:53.168 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0580000000000004e-05 2024-04-24T15:32:53.169 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.169 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.169 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:53.169 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267012+0000", 2024-04-24T15:32:53.169 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0698999999999994e-05 2024-04-24T15:32:53.169 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:53.169 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:53.169 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:53.169 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.169 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.169 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17318 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 11550720~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:53.169 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.224424+0000", 2024-04-24T15:32:53.169 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 18.940707655000001, 2024-04-24T15:32:53.169 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042771921999999997, 2024-04-24T15:32:53.169 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:53.170 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:53.170 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:53.170 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:53.170 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:53.170 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17318 2024-04-24T15:32:53.170 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.170 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:53.170 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.170 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:53.170 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224424+0000", 2024-04-24T15:32:53.170 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:53.170 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.170 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.170 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:53.170 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224424+0000", 2024-04-24T15:32:53.171 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:53.171 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.171 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.171 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:53.171 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224424+0000", 2024-04-24T15:32:53.171 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:53.171 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.171 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.171 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:53.171 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224434+0000", 2024-04-24T15:32:53.171 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.9699999999999994e-06 2024-04-24T15:32:53.171 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.171 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.171 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:53.171 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224443+0000", 2024-04-24T15:32:53.171 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.3689999999999996e-06 2024-04-24T15:32:53.172 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.172 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.172 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:53.172 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224447+0000", 2024-04-24T15:32:53.172 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.783e-06 2024-04-24T15:32:53.172 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.172 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.172 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:53.172 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267027+0000", 2024-04-24T15:32:53.172 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042579847999999997 2024-04-24T15:32:53.172 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.172 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.172 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:53.172 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267108+0000", 2024-04-24T15:32:53.172 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.1420000000000003e-05 2024-04-24T15:32:53.172 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.173 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.173 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:53.173 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267196+0000", 2024-04-24T15:32:53.173 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.7851999999999998e-05 2024-04-24T15:32:53.173 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:53.173 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:53.173 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:53.173 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.173 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.173 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17319 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 11591680~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:53.173 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.224453+0000", 2024-04-24T15:32:53.173 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 18.940678728999998, 2024-04-24T15:32:53.173 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042921783999999998, 2024-04-24T15:32:53.173 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:53.173 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:53.173 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:53.174 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:53.174 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:53.174 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17319 2024-04-24T15:32:53.174 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.174 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:53.174 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.174 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:53.174 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224453+0000", 2024-04-24T15:32:53.174 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:53.174 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.174 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.174 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:53.174 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224453+0000", 2024-04-24T15:32:53.174 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:53.175 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.175 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.175 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:53.175 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224452+0000", 2024-04-24T15:32:53.175 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:53.175 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.175 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.175 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:53.175 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224455+0000", 2024-04-24T15:32:53.175 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.289e-06 2024-04-24T15:32:53.175 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.175 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.175 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:53.175 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224461+0000", 2024-04-24T15:32:53.175 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.7300000000000002e-06 2024-04-24T15:32:53.175 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.176 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.176 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:53.176 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224464+0000", 2024-04-24T15:32:53.176 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.2890000000000002e-06 2024-04-24T15:32:53.176 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.176 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.176 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:53.176 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267218+0000", 2024-04-24T15:32:53.176 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042753696000000001 2024-04-24T15:32:53.176 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.176 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.176 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:53.176 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267295+0000", 2024-04-24T15:32:53.176 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7762999999999994e-05 2024-04-24T15:32:53.176 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.177 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.177 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:53.177 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267375+0000", 2024-04-24T15:32:53.177 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.9444999999999996e-05 2024-04-24T15:32:53.177 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:53.177 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:53.177 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:53.177 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.177 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.177 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17320 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 11632640~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:53.177 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.224468+0000", 2024-04-24T15:32:53.177 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 18.940663695000001, 2024-04-24T15:32:53.177 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043063615, 2024-04-24T15:32:53.177 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:53.177 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:53.178 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:53.178 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:53.178 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:53.178 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17320 2024-04-24T15:32:53.178 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.178 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:53.178 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.178 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:53.178 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224468+0000", 2024-04-24T15:32:53.178 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:53.178 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.178 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.178 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:53.178 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224468+0000", 2024-04-24T15:32:53.179 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:53.179 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.179 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.179 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:53.179 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224468+0000", 2024-04-24T15:32:53.179 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:53.179 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.179 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.179 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:53.179 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224470+0000", 2024-04-24T15:32:53.179 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.0600000000000002e-06 2024-04-24T15:32:53.179 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.179 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.179 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:53.180 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224475+0000", 2024-04-24T15:32:53.180 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.1059999999999999e-06 2024-04-24T15:32:53.180 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.180 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.180 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:53.180 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224478+0000", 2024-04-24T15:32:53.180 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.3100000000000001e-06 2024-04-24T15:32:53.180 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.180 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.180 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:53.180 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267386+0000", 2024-04-24T15:32:53.180 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042907529 2024-04-24T15:32:53.180 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.180 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.180 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:53.181 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267447+0000", 2024-04-24T15:32:53.181 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.1173000000000005e-05 2024-04-24T15:32:53.181 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.181 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.181 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:53.181 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267532+0000", 2024-04-24T15:32:53.181 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.4771999999999994e-05 2024-04-24T15:32:53.181 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:53.181 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:53.181 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:53.181 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.181 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.181 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17321 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 11673600~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:53.181 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.224499+0000", 2024-04-24T15:32:53.181 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 18.940632806, 2024-04-24T15:32:53.181 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043214447000000003, 2024-04-24T15:32:53.182 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:53.182 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:53.182 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:53.182 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:53.182 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:53.182 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17321 2024-04-24T15:32:53.182 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.182 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:53.182 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.182 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:53.182 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224499+0000", 2024-04-24T15:32:53.182 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:53.182 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.182 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.183 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:53.183 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224499+0000", 2024-04-24T15:32:53.183 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:53.183 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.183 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.183 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:53.183 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224499+0000", 2024-04-24T15:32:53.183 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:53.183 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.183 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.183 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:53.183 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224504+0000", 2024-04-24T15:32:53.183 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6869999999999999e-06 2024-04-24T15:32:53.183 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.183 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.183 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:53.184 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224509+0000", 2024-04-24T15:32:53.184 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2259999999999999e-06 2024-04-24T15:32:53.184 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.184 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.184 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:53.184 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224511+0000", 2024-04-24T15:32:53.184 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.418e-06 2024-04-24T15:32:53.184 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.184 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.184 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:53.184 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267552+0000", 2024-04-24T15:32:53.184 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043041455999999999 2024-04-24T15:32:53.184 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.184 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.184 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:53.185 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267629+0000", 2024-04-24T15:32:53.185 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7069000000000004e-05 2024-04-24T15:32:53.185 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.185 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.185 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:53.185 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267713+0000", 2024-04-24T15:32:53.185 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3843000000000003e-05 2024-04-24T15:32:53.185 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:53.185 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:53.185 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:53.185 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.185 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.185 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17322 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 11714560~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:53.185 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.224514+0000", 2024-04-24T15:32:53.185 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 18.940617568, 2024-04-24T15:32:53.186 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043339196000000003, 2024-04-24T15:32:53.186 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:53.186 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:53.186 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:53.186 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:53.186 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:53.186 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17322 2024-04-24T15:32:53.186 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.186 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:53.186 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.186 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:53.186 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224514+0000", 2024-04-24T15:32:53.186 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:53.186 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.187 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.187 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:53.187 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224514+0000", 2024-04-24T15:32:53.187 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:53.187 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.187 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.187 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:53.187 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224514+0000", 2024-04-24T15:32:53.187 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:53.187 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.187 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.187 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:53.187 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224515+0000", 2024-04-24T15:32:53.187 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.5570000000000001e-06 2024-04-24T15:32:53.188 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.188 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.188 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:53.188 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224518+0000", 2024-04-24T15:32:53.188 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.943e-06 2024-04-24T15:32:53.188 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.188 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.188 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:53.188 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224520+0000", 2024-04-24T15:32:53.188 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.9999999999999999e-06 2024-04-24T15:32:53.188 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.188 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.188 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:53.188 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267730+0000", 2024-04-24T15:32:53.188 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043209466000000002 2024-04-24T15:32:53.188 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.189 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.189 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:53.189 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267772+0000", 2024-04-24T15:32:53.189 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2453000000000002e-05 2024-04-24T15:32:53.189 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.189 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.189 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:53.189 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267853+0000", 2024-04-24T15:32:53.189 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.1094000000000007e-05 2024-04-24T15:32:53.189 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:53.189 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:53.189 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:53.189 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.189 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.189 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17323 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 11755520~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:53.189 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.224522+0000", 2024-04-24T15:32:53.190 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 18.940609228, 2024-04-24T15:32:53.190 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043506792000000002, 2024-04-24T15:32:53.190 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:53.190 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:53.190 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:53.190 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:53.190 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:53.190 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17323 2024-04-24T15:32:53.190 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.190 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:53.190 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.190 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:53.190 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224522+0000", 2024-04-24T15:32:53.190 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:53.190 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.191 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.191 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:53.191 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224522+0000", 2024-04-24T15:32:53.191 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:53.191 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.191 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.191 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:53.191 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224522+0000", 2024-04-24T15:32:53.191 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:32:53.191 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.191 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.191 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:53.191 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224524+0000", 2024-04-24T15:32:53.191 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3069999999999999e-06 2024-04-24T15:32:53.191 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.191 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.192 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:53.192 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224526+0000", 2024-04-24T15:32:53.192 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.773e-06 2024-04-24T15:32:53.192 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.192 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.192 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:53.192 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224529+0000", 2024-04-24T15:32:53.192 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.3e-06 2024-04-24T15:32:53.192 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.192 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.192 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:53.192 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267873+0000", 2024-04-24T15:32:53.192 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043344141000000003 2024-04-24T15:32:53.192 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.192 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.192 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:53.193 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267948+0000", 2024-04-24T15:32:53.193 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.4960000000000001e-05 2024-04-24T15:32:53.193 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.193 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.193 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:53.193 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.268029+0000", 2024-04-24T15:32:53.193 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.1520000000000006e-05 2024-04-24T15:32:53.193 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:53.193 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:53.193 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:53.193 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.193 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.193 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17328 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 11960320~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:53.193 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.226273+0000", 2024-04-24T15:32:53.193 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 18.938858242999999, 2024-04-24T15:32:53.194 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042524610999999997, 2024-04-24T15:32:53.194 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:53.194 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:53.194 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:53.194 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:53.194 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:53.194 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17328 2024-04-24T15:32:53.194 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.194 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:53.194 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.194 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:53.194 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226273+0000", 2024-04-24T15:32:53.194 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:53.194 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.194 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.195 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:53.195 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226273+0000", 2024-04-24T15:32:53.195 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:53.195 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.195 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.195 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:53.195 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226273+0000", 2024-04-24T15:32:53.195 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:32:53.195 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.195 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.195 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:53.195 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226275+0000", 2024-04-24T15:32:53.195 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.759e-06 2024-04-24T15:32:53.195 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.195 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.195 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:53.196 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226286+0000", 2024-04-24T15:32:53.196 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0841e-05 2024-04-24T15:32:53.196 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.196 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.196 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:53.196 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226290+0000", 2024-04-24T15:32:53.196 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.794e-06 2024-04-24T15:32:53.196 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.196 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.196 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:53.196 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.268693+0000", 2024-04-24T15:32:53.196 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042403519000000001 2024-04-24T15:32:53.196 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.196 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.196 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:53.197 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.268739+0000", 2024-04-24T15:32:53.197 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6149000000000001e-05 2024-04-24T15:32:53.197 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.197 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.197 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:53.197 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.268798+0000", 2024-04-24T15:32:53.197 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.8752999999999998e-05 2024-04-24T15:32:53.197 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:53.197 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:53.197 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:53.197 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.197 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.197 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17329 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 12001280~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:53.197 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.226295+0000", 2024-04-24T15:32:53.197 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 18.938837136, 2024-04-24T15:32:53.197 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042690771000000002, 2024-04-24T15:32:53.198 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:53.198 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:53.198 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:53.198 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:53.198 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:53.198 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17329 2024-04-24T15:32:53.198 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.198 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:53.198 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.198 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:53.198 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226295+0000", 2024-04-24T15:32:53.198 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:53.198 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.198 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.198 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:53.198 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226295+0000", 2024-04-24T15:32:53.199 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:53.199 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.199 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.199 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:53.199 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226294+0000", 2024-04-24T15:32:53.199 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:53.199 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.199 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.199 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:53.199 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226297+0000", 2024-04-24T15:32:53.199 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.0189999999999998e-06 2024-04-24T15:32:53.199 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.199 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.199 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:53.199 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226302+0000", 2024-04-24T15:32:53.199 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6260000000000003e-06 2024-04-24T15:32:53.200 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.200 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.200 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:53.200 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226304+0000", 2024-04-24T15:32:53.200 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.2230000000000001e-06 2024-04-24T15:32:53.200 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.200 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.200 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:53.200 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.268831+0000", 2024-04-24T15:32:53.200 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042527224000000002 2024-04-24T15:32:53.200 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.200 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.200 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:53.200 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.268904+0000", 2024-04-24T15:32:53.200 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.3168000000000001e-05 2024-04-24T15:32:53.200 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.200 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.201 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:53.201 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.268985+0000", 2024-04-24T15:32:53.201 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.1009999999999999e-05 2024-04-24T15:32:53.201 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:53.201 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:53.201 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:53.201 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.201 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.201 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17330 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 12042240~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:53.201 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.226307+0000", 2024-04-24T15:32:53.201 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 18.938825144999999, 2024-04-24T15:32:53.201 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042812009999999998, 2024-04-24T15:32:53.201 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:53.201 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:53.202 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:53.202 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:53.202 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:53.202 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17330 2024-04-24T15:32:53.202 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.202 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:53.202 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.202 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:53.202 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226307+0000", 2024-04-24T15:32:53.202 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:53.202 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.202 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.202 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:53.202 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226307+0000", 2024-04-24T15:32:53.202 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:53.202 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.203 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.203 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:53.203 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226306+0000", 2024-04-24T15:32:53.203 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:32:53.203 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.203 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.203 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:53.203 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226312+0000", 2024-04-24T15:32:53.203 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6180000000000003e-06 2024-04-24T15:32:53.203 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.203 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.203 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:53.203 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226316+0000", 2024-04-24T15:32:53.203 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2379999999999999e-06 2024-04-24T15:32:53.203 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.204 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.204 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:53.204 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226318+0000", 2024-04-24T15:32:53.204 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.3130000000000001e-06 2024-04-24T15:32:53.204 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.204 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.204 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:53.204 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269015+0000", 2024-04-24T15:32:53.204 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042696698999999998 2024-04-24T15:32:53.204 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.204 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.204 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:53.204 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269061+0000", 2024-04-24T15:32:53.204 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6193000000000003e-05 2024-04-24T15:32:53.204 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.205 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.205 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:53.205 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269119+0000", 2024-04-24T15:32:53.205 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.7169000000000001e-05 2024-04-24T15:32:53.205 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:53.205 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:53.205 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:53.205 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.205 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.205 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17331 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 12083200~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:53.205 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.226321+0000", 2024-04-24T15:32:53.205 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 18.938810713999999, 2024-04-24T15:32:53.205 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042976872999999999, 2024-04-24T15:32:53.205 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:53.205 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:53.205 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:53.206 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:53.206 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:53.206 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17331 2024-04-24T15:32:53.206 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.206 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:53.206 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.206 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:53.206 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226321+0000", 2024-04-24T15:32:53.206 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:53.206 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.206 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.206 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:53.206 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226321+0000", 2024-04-24T15:32:53.206 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:53.206 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.207 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.207 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:53.207 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226321+0000", 2024-04-24T15:32:53.207 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:32:53.207 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.207 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.207 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:53.207 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226322+0000", 2024-04-24T15:32:53.207 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.446e-06 2024-04-24T15:32:53.207 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.207 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.207 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:53.207 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226325+0000", 2024-04-24T15:32:53.207 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.004e-06 2024-04-24T15:32:53.207 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.207 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.208 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:53.208 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226327+0000", 2024-04-24T15:32:53.208 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.0990000000000001e-06 2024-04-24T15:32:53.208 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.208 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.208 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:53.208 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269137+0000", 2024-04-24T15:32:53.208 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042809381000000001 2024-04-24T15:32:53.208 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.208 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.208 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:53.208 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269217+0000", 2024-04-24T15:32:53.208 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.085e-05 2024-04-24T15:32:53.208 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.208 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.208 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:53.209 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269298+0000", 2024-04-24T15:32:53.209 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0317999999999999e-05 2024-04-24T15:32:53.209 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:53.209 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:53.209 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:53.209 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.209 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.209 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17332 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 12124160~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:53.209 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.226329+0000", 2024-04-24T15:32:53.209 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 18.938802226, 2024-04-24T15:32:53.209 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043084539999999998, 2024-04-24T15:32:53.209 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:53.209 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:53.209 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:53.209 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:53.209 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:53.209 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17332 2024-04-24T15:32:53.210 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.210 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:53.210 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.210 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:53.210 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226329+0000", 2024-04-24T15:32:53.210 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:53.210 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.210 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.210 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:53.210 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226329+0000", 2024-04-24T15:32:53.210 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:53.210 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.210 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.210 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:53.210 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226329+0000", 2024-04-24T15:32:53.210 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:32:53.211 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.211 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.211 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:53.211 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226331+0000", 2024-04-24T15:32:53.211 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.328e-06 2024-04-24T15:32:53.211 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.211 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.211 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:53.211 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226333+0000", 2024-04-24T15:32:53.211 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.9119999999999998e-06 2024-04-24T15:32:53.211 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.211 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.211 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:53.211 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226335+0000", 2024-04-24T15:32:53.211 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.9690000000000001e-06 2024-04-24T15:32:53.211 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.212 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.212 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:53.212 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269313+0000", 2024-04-24T15:32:53.212 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042977518999999999 2024-04-24T15:32:53.212 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.212 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.212 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:53.212 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269357+0000", 2024-04-24T15:32:53.212 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3748999999999997e-05 2024-04-24T15:32:53.212 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.212 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.212 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:53.212 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269414+0000", 2024-04-24T15:32:53.212 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.7265000000000001e-05 2024-04-24T15:32:53.212 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:53.212 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:53.213 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:53.213 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.213 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.213 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17333 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 12165120~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:53.213 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.226338+0000", 2024-04-24T15:32:53.213 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 18.938794089000002, 2024-04-24T15:32:53.213 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043264706999999999, 2024-04-24T15:32:53.213 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:53.213 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:53.213 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:53.213 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:53.213 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:53.213 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17333 2024-04-24T15:32:53.213 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.213 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:53.214 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.214 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:53.214 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226338+0000", 2024-04-24T15:32:53.214 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:53.214 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.214 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.214 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:53.214 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226338+0000", 2024-04-24T15:32:53.214 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:53.214 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.214 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.214 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:53.214 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226337+0000", 2024-04-24T15:32:53.214 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:32:53.214 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.214 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.215 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:53.215 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226339+0000", 2024-04-24T15:32:53.215 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.387e-06 2024-04-24T15:32:53.215 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.215 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.215 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:53.215 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226342+0000", 2024-04-24T15:32:53.215 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8059999999999999e-06 2024-04-24T15:32:53.215 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.215 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.215 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:53.215 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226344+0000", 2024-04-24T15:32:53.215 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.1390000000000002e-06 2024-04-24T15:32:53.216 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.216 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.216 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:53.216 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269445+0000", 2024-04-24T15:32:53.216 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043101457000000003 2024-04-24T15:32:53.216 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.216 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.216 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:53.216 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269520+0000", 2024-04-24T15:32:53.216 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.5278000000000007e-05 2024-04-24T15:32:53.216 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.216 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.216 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:53.216 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269602+0000", 2024-04-24T15:32:53.216 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.1847000000000004e-05 2024-04-24T15:32:53.216 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:53.217 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:53.217 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:53.217 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.217 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.217 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17334 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 12206080~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:53.217 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.226346+0000", 2024-04-24T15:32:53.217 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 18.938785673000002, 2024-04-24T15:32:53.217 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043468358999999998, 2024-04-24T15:32:53.217 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:53.217 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:53.217 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:53.217 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:53.217 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:53.217 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17334 2024-04-24T15:32:53.217 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.218 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:53.218 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.218 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:53.218 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226346+0000", 2024-04-24T15:32:53.218 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:53.218 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.218 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.218 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:53.218 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226346+0000", 2024-04-24T15:32:53.218 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:53.218 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.218 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.218 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:53.219 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226346+0000", 2024-04-24T15:32:53.219 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:32:53.219 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.219 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.219 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:53.219 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226347+0000", 2024-04-24T15:32:53.219 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.359e-06 2024-04-24T15:32:53.219 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.219 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.219 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:53.219 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226350+0000", 2024-04-24T15:32:53.219 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7800000000000001e-06 2024-04-24T15:32:53.219 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.219 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.219 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:53.220 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226352+0000", 2024-04-24T15:32:53.220 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.046e-06 2024-04-24T15:32:53.220 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.220 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.220 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:53.220 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269646+0000", 2024-04-24T15:32:53.220 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043294393 2024-04-24T15:32:53.220 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.220 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.220 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:53.220 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269732+0000", 2024-04-24T15:32:53.220 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.5716000000000004e-05 2024-04-24T15:32:53.220 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.221 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.221 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:53.221 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269814+0000", 2024-04-24T15:32:53.221 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.2270999999999999e-05 2024-04-24T15:32:53.221 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:53.221 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:53.221 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:53.221 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.221 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.221 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17342 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 12533760~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:53.221 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.228377+0000", 2024-04-24T15:32:53.221 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 18.936754779000001, 2024-04-24T15:32:53.221 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042573286000000002, 2024-04-24T15:32:53.221 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:53.221 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:53.222 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:53.222 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:53.222 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:53.222 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17342 2024-04-24T15:32:53.222 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.222 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:53.222 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.222 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:53.222 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228377+0000", 2024-04-24T15:32:53.222 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:53.222 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.222 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.222 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:53.222 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228377+0000", 2024-04-24T15:32:53.223 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:53.223 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.223 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.223 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:53.223 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228377+0000", 2024-04-24T15:32:53.223 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:53.223 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.223 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.223 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:53.223 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228389+0000", 2024-04-24T15:32:53.223 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.2262999999999999e-05 2024-04-24T15:32:53.223 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.223 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.223 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:53.224 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228398+0000", 2024-04-24T15:32:53.224 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.1079999999999999e-06 2024-04-24T15:32:53.224 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.224 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.224 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:53.224 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228402+0000", 2024-04-24T15:32:53.224 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8920000000000002e-06 2024-04-24T15:32:53.224 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.224 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.224 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:53.224 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.270842+0000", 2024-04-24T15:32:53.224 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042440193000000001 2024-04-24T15:32:53.224 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.224 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.224 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:53.225 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.270887+0000", 2024-04-24T15:32:53.225 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5359999999999999e-05 2024-04-24T15:32:53.225 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.225 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.225 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:53.225 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.270950+0000", 2024-04-24T15:32:53.225 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.2842999999999994e-05 2024-04-24T15:32:53.225 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:53.225 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:53.225 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:53.225 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.225 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.225 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17343 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 12574720~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:53.225 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.228407+0000", 2024-04-24T15:32:53.225 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 18.936724905999998, 2024-04-24T15:32:53.226 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042650325000000003, 2024-04-24T15:32:53.226 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:53.226 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:53.226 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:53.226 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:53.226 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:53.226 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17343 2024-04-24T15:32:53.226 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.226 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:53.226 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.226 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:53.226 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228407+0000", 2024-04-24T15:32:53.226 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:53.227 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.227 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.227 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:53.227 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228407+0000", 2024-04-24T15:32:53.227 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:53.227 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.227 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.227 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:53.227 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228406+0000", 2024-04-24T15:32:53.227 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:53.227 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.227 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.227 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:53.227 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228409+0000", 2024-04-24T15:32:53.228 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.3980000000000002e-06 2024-04-24T15:32:53.228 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.228 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.228 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:53.228 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228414+0000", 2024-04-24T15:32:53.228 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6590000000000004e-06 2024-04-24T15:32:53.228 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.228 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.228 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:53.228 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228418+0000", 2024-04-24T15:32:53.228 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.625e-06 2024-04-24T15:32:53.228 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.228 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.228 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:53.229 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.270966+0000", 2024-04-24T15:32:53.229 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042547709000000003 2024-04-24T15:32:53.229 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.229 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.229 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:53.229 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.271007+0000", 2024-04-24T15:32:53.229 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.1659000000000002e-05 2024-04-24T15:32:53.229 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.229 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.229 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:53.229 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.271057+0000", 2024-04-24T15:32:53.229 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.9679000000000003e-05 2024-04-24T15:32:53.229 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:53.230 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:53.230 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:53.230 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.230 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.230 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17344 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 12607488~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:53.230 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.228461+0000", 2024-04-24T15:32:53.230 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 18.936670918000001, 2024-04-24T15:32:53.230 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042713530999999999, 2024-04-24T15:32:53.230 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:53.230 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:53.230 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:53.230 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:53.230 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:53.230 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17344 2024-04-24T15:32:53.231 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.231 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:53.231 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.231 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:53.231 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228461+0000", 2024-04-24T15:32:53.231 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:53.231 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.231 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.231 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:53.231 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228461+0000", 2024-04-24T15:32:53.231 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:53.231 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.231 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.231 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:53.231 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228460+0000", 2024-04-24T15:32:53.232 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:53.232 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.232 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.232 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:53.232 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228464+0000", 2024-04-24T15:32:53.232 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.98e-06 2024-04-24T15:32:53.232 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.232 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.232 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:53.232 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228473+0000", 2024-04-24T15:32:53.232 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.0270000000000001e-06 2024-04-24T15:32:53.232 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.232 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.232 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:53.232 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228479+0000", 2024-04-24T15:32:53.233 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.5910000000000003e-06 2024-04-24T15:32:53.233 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.233 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.233 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:53.233 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.271081+0000", 2024-04-24T15:32:53.233 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042602025000000002 2024-04-24T15:32:53.233 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.233 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.233 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:53.233 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.271126+0000", 2024-04-24T15:32:53.233 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4904999999999998e-05 2024-04-24T15:32:53.233 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.233 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.233 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:53.233 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.271174+0000", 2024-04-24T15:32:53.234 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8603999999999997e-05 2024-04-24T15:32:53.234 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:53.234 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:53.234 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:53.234 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.234 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.234 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17345 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 12648448~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:53.234 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.228484+0000", 2024-04-24T15:32:53.234 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 18.936647382, 2024-04-24T15:32:53.234 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042811844000000002, 2024-04-24T15:32:53.234 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:53.234 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:53.234 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:53.234 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:53.235 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:53.235 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17345 2024-04-24T15:32:53.235 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.235 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:53.235 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.235 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:53.235 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228484+0000", 2024-04-24T15:32:53.235 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:53.235 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.235 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.235 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:53.235 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228484+0000", 2024-04-24T15:32:53.235 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:53.235 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.235 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.236 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:53.236 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228484+0000", 2024-04-24T15:32:53.236 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:53.236 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.236 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.236 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:53.236 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228487+0000", 2024-04-24T15:32:53.236 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.9950000000000001e-06 2024-04-24T15:32:53.236 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.236 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.236 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:53.236 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228493+0000", 2024-04-24T15:32:53.236 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.2870000000000001e-06 2024-04-24T15:32:53.236 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.236 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.237 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:53.237 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228497+0000", 2024-04-24T15:32:53.237 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.1219999999999997e-06 2024-04-24T15:32:53.237 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.237 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.237 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:53.237 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.271202+0000", 2024-04-24T15:32:53.237 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042704963999999998 2024-04-24T15:32:53.237 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.237 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.237 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:53.237 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.271250+0000", 2024-04-24T15:32:53.237 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8207000000000001e-05 2024-04-24T15:32:53.237 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.237 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.237 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:53.238 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.271296+0000", 2024-04-24T15:32:53.238 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5769999999999997e-05 2024-04-24T15:32:53.238 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:53.238 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:53.238 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:53.238 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.238 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.238 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17346 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 12689408~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:53.238 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.228502+0000", 2024-04-24T15:32:53.238 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 18.936629768, 2024-04-24T15:32:53.238 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042918317999999997, 2024-04-24T15:32:53.238 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:53.238 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:53.238 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:53.239 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:53.239 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:53.239 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17346 2024-04-24T15:32:53.239 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.239 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:53.239 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.239 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:53.239 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228502+0000", 2024-04-24T15:32:53.239 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:53.239 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.239 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.239 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:53.239 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228502+0000", 2024-04-24T15:32:53.239 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:53.240 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.240 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.240 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:53.240 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228502+0000", 2024-04-24T15:32:53.240 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:53.240 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.240 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.240 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:53.240 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228504+0000", 2024-04-24T15:32:53.240 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.497e-06 2024-04-24T15:32:53.240 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.240 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.240 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:53.240 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228510+0000", 2024-04-24T15:32:53.241 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.7429999999999999e-06 2024-04-24T15:32:53.241 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.241 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.241 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:53.241 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228514+0000", 2024-04-24T15:32:53.241 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0810000000000004e-06 2024-04-24T15:32:53.241 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.241 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.241 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:53.241 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.271330+0000", 2024-04-24T15:32:53.241 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042816616000000002 2024-04-24T15:32:53.241 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.241 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.241 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:53.241 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.271371+0000", 2024-04-24T15:32:53.242 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0222999999999998e-05 2024-04-24T15:32:53.242 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.242 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.242 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:53.242 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.271420+0000", 2024-04-24T15:32:53.242 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.9539000000000001e-05 2024-04-24T15:32:53.242 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:53.242 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:53.242 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:53.242 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.242 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.242 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17347 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 12730368~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:53.242 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.228518+0000", 2024-04-24T15:32:53.242 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 18.93661324, 2024-04-24T15:32:53.243 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043022429000000001, 2024-04-24T15:32:53.243 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:53.243 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:53.243 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:53.243 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:53.243 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:53.243 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17347 2024-04-24T15:32:53.243 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.243 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:53.243 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.243 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:53.243 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228518+0000", 2024-04-24T15:32:53.243 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:53.243 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.244 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.244 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:53.244 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228518+0000", 2024-04-24T15:32:53.244 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:53.244 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.244 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.244 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:53.244 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228518+0000", 2024-04-24T15:32:53.244 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:53.244 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.244 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.244 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:53.244 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228521+0000", 2024-04-24T15:32:53.244 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.4559999999999999e-06 2024-04-24T15:32:53.244 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.245 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.245 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:53.245 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228526+0000", 2024-04-24T15:32:53.245 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.7259999999999999e-06 2024-04-24T15:32:53.245 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.245 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.245 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:53.245 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228530+0000", 2024-04-24T15:32:53.245 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.658e-06 2024-04-24T15:32:53.245 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.245 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.245 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:53.245 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.271445+0000", 2024-04-24T15:32:53.245 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042915506999999999 2024-04-24T15:32:53.245 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.246 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.246 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:53.246 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.271490+0000", 2024-04-24T15:32:53.246 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4752000000000001e-05 2024-04-24T15:32:53.246 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:53.246 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:53.246 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:53.246 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.271541+0000", 2024-04-24T15:32:53.246 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.0699999999999999e-05 2024-04-24T15:32:53.246 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:53.246 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:53.246 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:53.246 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:53.246 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:53.247 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:32:53.247 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-24T15:32:53.249 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-24T15:32:53.250 INFO:teuthology.orchestra.run.smithi096.stdout:2370 write 0x7dbf53c thru 0x7dcd890 (0xe355 bytes) 2024-04-24T15:32:53.250 INFO:teuthology.orchestra.run.smithi096.stdout:2371 punch from 0x6a4f54d to 0x6a4fd5b, (0x80e bytes) 2024-04-24T15:32:53.250 INFO:teuthology.orchestra.run.smithi096.stdout:2372 write 0xeb09dda thru 0xeb0a277 (0x49e bytes) 2024-04-24T15:32:53.250 INFO:teuthology.orchestra.run.smithi096.stdout:2373 trunc from 0xeb0a278 to 0xaf4bb55 2024-04-24T15:32:53.250 INFO:teuthology.orchestra.run.smithi096.stdout:2374 read 0x474bf74 thru 0x475260f (0x669c bytes) 2024-04-24T15:32:53.250 INFO:teuthology.orchestra.run.smithi096.stdout:2375 write 0x13a7003 thru 0x13ad952 (0x6950 bytes) 2024-04-24T15:32:53.250 INFO:teuthology.orchestra.run.smithi096.stdout:2377 read 0x8c26afa thru 0x8c2fad2 (0x8fd9 bytes) 2024-04-24T15:32:53.250 INFO:teuthology.orchestra.run.smithi096.stdout:2378 writesame 0xc9d555e thru 0xc9d6940 data_size 0x1(0x13e3 bytes) 2024-04-24T15:32:53.250 INFO:teuthology.orchestra.run.smithi096.stdout:2380 trunc from 0xc9d6941 to 0x8cfb936 2024-04-24T15:32:53.250 INFO:teuthology.orchestra.run.smithi096.stdout:2382 writesame 0x369e7db thru 0x36a6d37 data_size 0x1(0x855d bytes) 2024-04-24T15:32:53.251 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:32:53.248+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:32:53.251 INFO:teuthology.orchestra.run.smithi096.stdout:2383 writesame 0x9a9e1 thru 0x9e5b3 data_size 0x5(0x3bd3 bytes) 2024-04-24T15:32:53.253 INFO:teuthology.orchestra.run.smithi096.stdout:2384 write 0xc3b22a thru 0xc435c9 (0x83a0 bytes) 2024-04-24T15:32:53.254 INFO:teuthology.orchestra.run.smithi096.stdout:2385 write 0xf3a9ae thru 0xf42184 (0x77d7 bytes) 2024-04-24T15:32:53.255 INFO:teuthology.orchestra.run.smithi096.stdout:2388 read 0x3e0aaec thru 0x3e0d13a (0x264f bytes) 2024-04-24T15:32:53.266 INFO:teuthology.orchestra.run.smithi096.stdout:2391 trunc from 0x8cfb936 to 0x8ee770 2024-04-24T15:32:53.288 INFO:teuthology.orchestra.run.smithi096.stdout:2392 punch from 0x6cc472 to 0x6ce38e, (0x1f1c bytes) 2024-04-24T15:32:53.288 INFO:teuthology.orchestra.run.smithi096.stdout:2393 punch from 0x68b12d to 0x698e68, (0xdd3b bytes) 2024-04-24T15:32:53.288 INFO:teuthology.orchestra.run.smithi096.stdout:2394 read 0x35ea7c thru 0x3679de (0x8f63 bytes) 2024-04-24T15:32:53.289 INFO:teuthology.orchestra.run.smithi096.stdout:2395 read 0x2ccf19 thru 0x2d3f10 (0x6ff8 bytes) 2024-04-24T15:32:53.296 INFO:teuthology.orchestra.run.smithi096.stdout:2396 write 0x2a6fa5d thru 0x2a77be3 (0x8187 bytes) 2024-04-24T15:32:53.301 INFO:teuthology.orchestra.run.smithi096.stdout:2397 trunc from 0x2a77be4 to 0x10e39f1 2024-04-24T15:32:53.315 INFO:teuthology.orchestra.run.smithi096.stdout:2398 writesame 0x3ff2f9e thru 0x3ff446a data_size 0x19(0x14cd bytes) 2024-04-24T15:32:53.316 INFO:teuthology.orchestra.run.smithi096.stdout:2399 punch from 0x228645c to 0x22961dd, (0xfd81 bytes) 2024-04-24T15:32:53.316 INFO:teuthology.orchestra.run.smithi096.stdout:2400 clone 13 order 21 su 65536 sc 6 2024-04-24T15:32:53.349 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:32:53.349+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:32:53.399 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-04-24T15:32:53.415 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:32:53.415 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:32:53.415 INFO:teuthology.orchestra.run.smithi151.stdout: "num_ops": 0 2024-04-24T15:32:53.415 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:32:53.417 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-04-24T15:32:53.425 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-24T15:32:53.435 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-04-24T15:32:53.449 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:32:53.449+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:32:53.453 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-04-24T15:32:53.472 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-04-24T15:32:53.490 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-04-24T15:32:53.507 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-04-24T15:32:53.526 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-04-24T15:32:53.527 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T15:32:53.527 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T15:32:53.527 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T15:32:53.527 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T15:32:53.527 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T15:32:53.527 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T15:32:53.527 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T15:32:53.527 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T15:32:53.579 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:32:53.579+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:32:53.589 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:53.589 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:53.599 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-24T15:32:53.650 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:32:53.651+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:32:53.665 INFO:teuthology.orchestra.run.smithi096.stderr:noscrub is set 2024-04-24T15:32:53.705 INFO:teuthology.orchestra.run.smithi096.stdout:truncating image image_client.0-clone12 from 0x3ff446b (overlap 0x8ee770) to 0x7252c0 2024-04-24T15:32:53.728 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:53.728 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:53.738 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-24T15:32:53.748 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #11, image image_client.0-clone11 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent12 2024-04-24T15:32:53.750 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:32:53.751+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:32:53.851 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:32:53.851+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:32:53.874 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:53.874 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:53.885 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-24T15:32:53.904 INFO:teuthology.orchestra.run.smithi096.stdout:2402 read 0x3b52116 thru 0x3b5450a (0x23f5 bytes) 2024-04-24T15:32:53.904 INFO:teuthology.orchestra.run.smithi096.stdout:2403 punch from 0x13a8ff to 0x147eb5, (0xd5b6 bytes) 2024-04-24T15:32:53.908 INFO:teuthology.orchestra.run.smithi096.stdout:2404 punch from 0x1ecc83c to 0x1ed0a33, (0x41f7 bytes) 2024-04-24T15:32:53.925 INFO:teuthology.orchestra.run.smithi096.stdout:2405 write 0xdcd8340 thru 0xdce6e89 (0xeb4a bytes) 2024-04-24T15:32:53.931 INFO:teuthology.orchestra.run.smithi096.stdout:2407 write 0xa84e794 thru 0xa853510 (0x4d7d bytes) 2024-04-24T15:32:53.938 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:53.938 INFO:teuthology.orchestra.run.smithi096.stdout:2409 write 0xb57510f thru 0xb575fac (0xe9e bytes) 2024-04-24T15:32:53.944 INFO:teuthology.orchestra.run.smithi096.stdout:2410 read 0x84a1c0c thru 0x84ab1b3 (0x95a8 bytes) 2024-04-24T15:32:53.944 INFO:teuthology.orchestra.run.smithi096.stdout:2411 punch from 0xcba6005 to 0xcba7c88, (0x1c83 bytes) 2024-04-24T15:32:53.944 INFO:teuthology.orchestra.run.smithi096.stdout:2414 read 0xdc31fa5 thru 0xdc41f7d (0xffd9 bytes) 2024-04-24T15:32:53.944 INFO:teuthology.orchestra.run.smithi096.stdout:2415 write 0x508eb98 thru 0x509c21f (0xd688 bytes) 2024-04-24T15:32:53.949 INFO:teuthology.orchestra.run.smithi096.stdout:2416 punch from 0x5737cf4 to 0x5738f7c, (0x1288 bytes) 2024-04-24T15:32:53.949 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:53.949 INFO:teuthology.orchestra.run.smithi096.stdout:2418 write 0x81a9c7d thru 0x81aa7d3 (0xb57 bytes) 2024-04-24T15:32:53.952 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:32:53.952+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:32:53.954 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:53.954 INFO:teuthology.orchestra.run.smithi096.stdout:2421 punch from 0x6efa265 to 0x6f06b91, (0xc92c bytes) 2024-04-24T15:32:53.954 INFO:teuthology.orchestra.run.smithi096.stdout:2422 read 0x8939903 thru 0x8943baf (0xa2ad bytes) 2024-04-24T15:32:53.956 INFO:teuthology.orchestra.run.smithi096.stdout:2423 trunc from 0xdce6e8a to 0xc713f51 2024-04-24T15:32:53.966 INFO:teuthology.orchestra.run.smithi096.stdout:2424 write 0x57a6476 thru 0x57b2cda (0xc865 bytes) 2024-04-24T15:32:53.973 INFO:teuthology.orchestra.run.smithi096.stdout:2425 read 0x6473097 thru 0x6479424 (0x638e bytes) 2024-04-24T15:32:53.973 INFO:teuthology.orchestra.run.smithi096.stdout:2427 write 0x2e5a187 thru 0x2e632cf (0x9149 bytes) 2024-04-24T15:32:53.983 INFO:teuthology.orchestra.run.smithi096.stdout:2431 writesame 0xb76feec thru 0xb77d6d3 data_size 0xbc(0xd7e8 bytes) 2024-04-24T15:32:53.989 INFO:teuthology.orchestra.run.smithi096.stdout:2432 write 0xba6002e thru 0xba6af9e (0xaf71 bytes) 2024-04-24T15:32:53.990 INFO:teuthology.orchestra.run.smithi096.stdout:2434 writesame 0x9041936 thru 0x9047296 data_size 0x1d(0x5961 bytes) 2024-04-24T15:32:53.993 INFO:teuthology.orchestra.run.smithi096.stdout:2437 read 0x95f00d9 thru 0x95fb175 (0xb09d bytes) 2024-04-24T15:32:53.994 INFO:teuthology.orchestra.run.smithi096.stdout:2438 write 0x1de1e1b thru 0x1dea559 (0x873f bytes) 2024-04-24T15:32:53.998 INFO:teuthology.orchestra.run.smithi096.stdout:2439 write 0x8fd8da3 thru 0x8fe1f19 (0x9177 bytes) 2024-04-24T15:32:54.003 INFO:teuthology.orchestra.run.smithi096.stdout:2440 write 0x10a6120 thru 0x10aa4a9 (0x438a bytes) 2024-04-24T15:32:54.009 INFO:teuthology.orchestra.run.smithi096.stdout:2442 read 0x94781f0 thru 0x947b9cf (0x37e0 bytes) 2024-04-24T15:32:54.009 INFO:teuthology.orchestra.run.smithi096.stdout:2443 read 0x489062c thru 0x489491c (0x42f1 bytes) 2024-04-24T15:32:54.009 INFO:teuthology.orchestra.run.smithi096.stdout:2444 write 0x339a523 thru 0x339e0b8 (0x3b96 bytes) 2024-04-24T15:32:54.014 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:54.014 INFO:teuthology.orchestra.run.smithi096.stdout:2446 read 0xb3c7c71 thru 0xb3cca9c (0x4e2c bytes) 2024-04-24T15:32:54.014 INFO:teuthology.orchestra.run.smithi096.stdout:2447 writesame 0x339f041 thru 0x33a4048 data_size 0x1a(0x5008 bytes) 2024-04-24T15:32:54.029 INFO:teuthology.orchestra.run.smithi096.stdout:2448 write 0xeb7e05a thru 0xeb81708 (0x36af bytes) 2024-04-24T15:32:54.034 INFO:teuthology.orchestra.run.smithi096.stdout:2449 punch from 0x6d794d1 to 0x6d86397, (0xcec6 bytes) 2024-04-24T15:32:54.034 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:54.035 INFO:teuthology.orchestra.run.smithi096.stdout:2451 write 0xab0d99f thru 0xab12433 (0x4a95 bytes) 2024-04-24T15:32:54.041 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:54.041 INFO:teuthology.orchestra.run.smithi096.stdout:2455 punch from 0x6c86251 to 0x6c88cbf, (0x2a6e bytes) 2024-04-24T15:32:54.041 INFO:teuthology.orchestra.run.smithi096.stdout:2456 read 0x9e3a306 thru 0x9e4497f (0xa67a bytes) 2024-04-24T15:32:54.045 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:54.045 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:54.049 INFO:teuthology.orchestra.run.smithi096.stdout:2457 trunc from 0xeb81709 to 0x75dc19f 2024-04-24T15:32:54.053 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:32:54.053+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:32:54.060 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-24T15:32:54.062 INFO:teuthology.orchestra.run.smithi096.stdout:2458 read 0x6278e95 thru 0x6287e8f (0xeffb bytes) 2024-04-24T15:32:54.062 INFO:teuthology.orchestra.run.smithi096.stdout:2459 read 0x72a31e thru 0x736c15 (0xc8f8 bytes) 2024-04-24T15:32:54.066 INFO:teuthology.orchestra.run.smithi096.stdout:2460 trunc from 0x75dc19f to 0xaaf1a13 2024-04-24T15:32:54.082 INFO:teuthology.orchestra.run.smithi096.stdout:2462 trunc from 0xaaf1a13 to 0x1bb10da 2024-04-24T15:32:54.121 INFO:teuthology.orchestra.run.smithi096.stdout:2464 write 0xcf0d546 thru 0xcf0e8b2 (0x136d bytes) 2024-04-24T15:32:54.125 INFO:teuthology.orchestra.run.smithi096.stdout:2466 punch from 0x2190b81 to 0x2191065, (0x4e4 bytes) 2024-04-24T15:32:54.130 INFO:teuthology.orchestra.run.smithi096.stdout:2469 writesame 0xd1d3591 thru 0xd1df959 data_size 0x3(0xc3c9 bytes) 2024-04-24T15:32:54.131 INFO:teuthology.orchestra.run.smithi096.stdout:2470 read 0x2d02539 thru 0x2d052d5 (0x2d9d bytes) 2024-04-24T15:32:54.131 INFO:teuthology.orchestra.run.smithi096.stdout:2471 read 0x514245a thru 0x514e46c (0xc013 bytes) 2024-04-24T15:32:54.132 INFO:teuthology.orchestra.run.smithi096.stdout:2473 trunc from 0xd1df95a to 0xb8705eb 2024-04-24T15:32:54.144 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:54.144 INFO:teuthology.orchestra.run.smithi096.stdout:2476 read 0xe3741 thru 0xeee62 (0xb722 bytes) 2024-04-24T15:32:54.147 INFO:teuthology.orchestra.run.smithi096.stdout:2478 trunc from 0xb8705eb to 0xa017b42 2024-04-24T15:32:54.152 INFO:teuthology.orchestra.run.smithi096.stdout:2479 punch from 0x56af784 to 0x56bcfc6, (0xd842 bytes) 2024-04-24T15:32:54.152 INFO:teuthology.orchestra.run.smithi096.stdout:2482 read 0x2bb0e87 thru 0x2bb96fa (0x8874 bytes) 2024-04-24T15:32:54.152 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:54.152 INFO:teuthology.orchestra.run.smithi096.stdout:2486 punch from 0xe234e6 to 0xe25e4f, (0x2969 bytes) 2024-04-24T15:32:54.152 INFO:teuthology.orchestra.run.smithi096.stdout:2487 punch from 0x6f2ebf4 to 0x6f3133e, (0x274a bytes) 2024-04-24T15:32:54.153 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:54.153 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:32:54.154+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:32:54.154 INFO:teuthology.orchestra.run.smithi096.stdout:2489 trunc from 0xa017b42 to 0x844b1c7 2024-04-24T15:32:54.159 INFO:teuthology.orchestra.run.smithi096.stdout:2490 trunc from 0x844b1c7 to 0x806b92d 2024-04-24T15:32:54.170 INFO:teuthology.orchestra.run.smithi096.stdout:2493 trunc from 0x806b92d to 0x38e2426 2024-04-24T15:32:54.190 INFO:teuthology.orchestra.run.smithi096.stdout:2496 write 0xc139d11 thru 0xc13bfd4 (0x22c4 bytes) 2024-04-24T15:32:54.193 INFO:teuthology.orchestra.run.smithi096.stdout:2497 write 0x83f5ece thru 0x83f9aed (0x3c20 bytes) 2024-04-24T15:32:54.196 INFO:teuthology.orchestra.run.smithi096.stdout:2498 write 0xc027ebb thru 0xc02d10e (0x5254 bytes) 2024-04-24T15:32:54.200 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:54.207 INFO:teuthology.orchestra.run.smithi096.stdout:2500 writesame 0xe073c9d thru 0xe080a2b data_size 0x9(0xcd8f bytes) 2024-04-24T15:32:54.207 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:54.207 INFO:teuthology.orchestra.run.smithi096.stdout:2503 read 0x700006a thru 0x7002f71 (0x2f08 bytes) 2024-04-24T15:32:54.208 INFO:teuthology.orchestra.run.smithi096.stdout:2504 write 0x9d4b906 thru 0x9d4c3f5 (0xaf0 bytes) 2024-04-24T15:32:54.210 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:54.210 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:54.211 INFO:teuthology.orchestra.run.smithi096.stdout:2505 read 0x316a6d1 thru 0x317a2ff (0xfc2f bytes) 2024-04-24T15:32:54.211 INFO:teuthology.orchestra.run.smithi096.stdout:2506 punch from 0x82c7968 to 0x82d0e7c, (0x9514 bytes) 2024-04-24T15:32:54.211 INFO:teuthology.orchestra.run.smithi096.stdout:2507 punch from 0x925660b to 0x9262ac1, (0xc4b6 bytes) 2024-04-24T15:32:54.211 INFO:teuthology.orchestra.run.smithi096.stdout:2513 read 0xbe77134 thru 0xbe79cb7 (0x2b84 bytes) 2024-04-24T15:32:54.211 INFO:teuthology.orchestra.run.smithi096.stdout:2514 punch from 0x64f018b to 0x64fd682, (0xd4f7 bytes) 2024-04-24T15:32:54.211 INFO:teuthology.orchestra.run.smithi096.stdout:2515 write 0xca5c8df thru 0xca6a590 (0xdcb2 bytes) 2024-04-24T15:32:54.216 INFO:teuthology.orchestra.run.smithi096.stdout:2516 write 0xa5d9615 thru 0xa5de416 (0x4e02 bytes) 2024-04-24T15:32:54.220 INFO:teuthology.orchestra.run.smithi096.stdout:2517 trunc from 0xe080a2c to 0xd327564 2024-04-24T15:32:54.222 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-24T15:32:54.225 INFO:teuthology.orchestra.run.smithi096.stdout:2518 write 0x22a383b thru 0x22ab7ec (0x7fb2 bytes) 2024-04-24T15:32:54.227 INFO:teuthology.orchestra.run.smithi096.stdout:2519 read 0x9b6cb50 thru 0x9b74f42 (0x83f3 bytes) 2024-04-24T15:32:54.241 INFO:teuthology.orchestra.run.smithi096.stdout:2520 trunc from 0xd327564 to 0x162a668 2024-04-24T15:32:54.255 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:32:54.255+0000 7f23abcf6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 44021) UID: 0 2024-04-24T15:32:54.265 INFO:teuthology.orchestra.run.smithi096.stdout:2521 write 0xc15b62 thru 0xc231ba (0xd659 bytes) 2024-04-24T15:32:54.276 INFO:teuthology.orchestra.run.smithi096.stdout:2522 write 0x5029557 thru 0x5031964 (0x840e bytes) 2024-04-24T15:32:54.292 INFO:teuthology.orchestra.run.smithi096.stdout:2523 writesame 0xb653453 thru 0xb6553a3 data_size 0x1(0x1f51 bytes) 2024-04-24T15:32:54.292 INFO:teuthology.orchestra.run.smithi096.stdout:2524 read 0x949eebf thru 0x94ab6bd (0xc7ff bytes) 2024-04-24T15:32:54.292 INFO:teuthology.orchestra.run.smithi096.stdout:2525 punch from 0xa17549e to 0xa183e1a, (0xe97c bytes) 2024-04-24T15:32:54.292 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:54.292 INFO:teuthology.orchestra.run.smithi096.stdout:2527 write 0x82ecf38 thru 0x82ef440 (0x2509 bytes) 2024-04-24T15:32:54.297 INFO:teuthology.orchestra.run.smithi096.stdout:2528 trunc from 0xb6553a4 to 0x8b92d91 2024-04-24T15:32:54.302 INFO:teuthology.orchestra.run.smithi096.stdout:2529 read 0x3f2468c thru 0x3f2ec57 (0xa5cc bytes) 2024-04-24T15:32:54.302 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:54.305 INFO:teuthology.orchestra.run.smithi096.stdout:2534 trunc from 0x8b92d91 to 0x62bd1de 2024-04-24T15:32:54.316 INFO:teuthology.orchestra.run.smithi096.stdout:2535 punch from 0x3c3ae69 to 0x3c47ea0, (0xd037 bytes) 2024-04-24T15:32:54.316 INFO:teuthology.orchestra.run.smithi096.stdout:2537 trunc from 0x62bd1de to 0x6d82f50 2024-04-24T15:32:54.322 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:54.329 INFO:teuthology.orchestra.run.smithi096.stdout:2540 trunc from 0x6d82f50 to 0x184bd13 2024-04-24T15:32:54.353 INFO:teuthology.orchestra.run.smithi096.stdout:2541 write 0x5f23ca0 thru 0x5f25c13 (0x1f74 bytes) 2024-04-24T15:32:54.356 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:32:54.356+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:32:54.356 INFO:teuthology.orchestra.run.smithi096.stdout:2543 trunc from 0x5f25c14 to 0x5bde82b 2024-04-24T15:32:54.370 INFO:teuthology.orchestra.run.smithi096.stdout:2545 trunc from 0x5bde82b to 0xbdd527e 2024-04-24T15:32:54.376 INFO:teuthology.orchestra.run.smithi096.stdout:2547 read 0x7afc7f2 thru 0x7b043cf (0x7bde bytes) 2024-04-24T15:32:54.377 INFO:teuthology.orchestra.run.smithi096.stdout:2548 write 0x1f6c6ea thru 0x1f6fcb6 (0x35cd bytes) 2024-04-24T15:32:54.377 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:54.377 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:54.390 INFO:teuthology.orchestra.run.smithi096.stdout:2550 trunc from 0xbdd527e to 0x289aaea 2024-04-24T15:32:54.395 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-24T15:32:54.410 INFO:teuthology.orchestra.run.smithi096.stdout:2552 write 0x7b630c9 thru 0x7b69d9b (0x6cd3 bytes) 2024-04-24T15:32:54.412 INFO:teuthology.orchestra.run.smithi096.stdout:2554 write 0x451ff8a thru 0x4527abf (0x7b36 bytes) 2024-04-24T15:32:54.416 INFO:teuthology.orchestra.run.smithi096.stdout:2555 punch from 0x9426f0 to 0x94e51a, (0xbe2a bytes) 2024-04-24T15:32:54.416 INFO:teuthology.orchestra.run.smithi096.stdout:2556 read 0xa37804 thru 0xa42b09 (0xb306 bytes) 2024-04-24T15:32:54.424 INFO:teuthology.orchestra.run.smithi096.stdout:2557 trunc from 0x7b69d9c to 0x1551d59 2024-04-24T15:32:54.440 INFO:teuthology.orchestra.run.smithi096.stdout:2560 read 0xbeb105 thru 0xbfa478 (0xf374 bytes) 2024-04-24T15:32:54.457 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:32:54.457+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:32:54.458 INFO:teuthology.orchestra.run.smithi096.stdout:2561 writesame 0xb23fa92 thru 0xb24ce79 data_size 0x4(0xd3e8 bytes) 2024-04-24T15:32:54.458 INFO:teuthology.orchestra.run.smithi096.stdout:2562 read 0x6cf046b thru 0x6cfff2b (0xfac1 bytes) 2024-04-24T15:32:54.458 INFO:teuthology.orchestra.run.smithi096.stdout:2563 read 0x81afaed thru 0x81bf371 (0xf885 bytes) 2024-04-24T15:32:54.458 INFO:teuthology.orchestra.run.smithi096.stdout:2566 writesame 0x76072f8 thru 0x7614c77 data_size 0xc0(0xd980 bytes) 2024-04-24T15:32:54.463 INFO:teuthology.orchestra.run.smithi096.stdout:2567 read 0x8c210ed thru 0x8c28a73 (0x7987 bytes) 2024-04-24T15:32:54.463 INFO:teuthology.orchestra.run.smithi096.stdout:2568 read 0x1ee1d4c thru 0x1eeb15c (0x9411 bytes) 2024-04-24T15:32:54.471 INFO:teuthology.orchestra.run.smithi096.stdout:2569 writesame 0xd1a5e52 thru 0xd1ab7b4 data_size 0x7(0x5963 bytes) 2024-04-24T15:32:54.480 INFO:teuthology.orchestra.run.smithi096.stdout:2570 trunc from 0xd1ab7b5 to 0x553c8b0 2024-04-24T15:32:54.506 INFO:teuthology.orchestra.run.smithi096.stdout:2571 writesame 0xe26c6c6 thru 0xe26fd61 data_size 0x14(0x369c bytes) 2024-04-24T15:32:54.506 INFO:teuthology.orchestra.run.smithi096.stdout:2572 write 0xd26d313 thru 0xd2765ce (0x92bc bytes) 2024-04-24T15:32:54.513 INFO:teuthology.orchestra.run.smithi096.stdout:2573 trunc from 0xe26fd62 to 0xb009954 2024-04-24T15:32:54.522 INFO:teuthology.orchestra.run.smithi096.stdout:2575 write 0xab17705 thru 0xab219ad (0xa2a9 bytes) 2024-04-24T15:32:54.525 INFO:teuthology.orchestra.run.smithi096.stdout:2576 read 0x12a26b1 thru 0x12aa278 (0x7bc8 bytes) 2024-04-24T15:32:54.526 INFO:teuthology.orchestra.run.smithi096.stdout:2577 write 0x97d5d8c thru 0x97dcf2e (0x71a3 bytes) 2024-04-24T15:32:54.534 INFO:teuthology.orchestra.run.smithi096.stdout:2578 writesame 0xb90b7f7 thru 0xb90e148 data_size 0x52(0x2952 bytes) 2024-04-24T15:32:54.534 INFO:teuthology.orchestra.run.smithi096.stdout:2579 punch from 0x4f34b03 to 0x4f3af43, (0x6440 bytes) 2024-04-24T15:32:54.534 INFO:teuthology.orchestra.run.smithi096.stdout:2580 write 0xa651b22 thru 0xa660b1e (0xeffd bytes) 2024-04-24T15:32:54.537 INFO:teuthology.orchestra.run.smithi096.stdout:2581 writesame 0xa8692c1 thru 0xa878f69 data_size 0x1(0xfca9 bytes) 2024-04-24T15:32:54.556 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:54.556 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:54.559 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:32:54.559+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:32:54.559 INFO:teuthology.orchestra.run.smithi096.stdout:2582 writesame 0xc223c75 thru 0xc2243f1 data_size 0x9(0x77d bytes) 2024-04-24T15:32:54.559 INFO:teuthology.orchestra.run.smithi096.stdout:2583 read 0xad4655f thru 0xad545b4 (0xe056 bytes) 2024-04-24T15:32:54.559 INFO:teuthology.orchestra.run.smithi096.stdout:2585 write 0x2ddd5f4 thru 0x2ddfd9a (0x27a7 bytes) 2024-04-24T15:32:54.568 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-24T15:32:54.568 INFO:teuthology.orchestra.run.smithi096.stdout:2586 write 0xcfa2a72 thru 0xcfaa5f7 (0x7b86 bytes) 2024-04-24T15:32:54.572 INFO:teuthology.orchestra.run.smithi096.stdout:2587 read 0x99b42f0 thru 0x99c1400 (0xd111 bytes) 2024-04-24T15:32:54.572 INFO:teuthology.orchestra.run.smithi096.stdout:2589 read 0xbdcfc9c thru 0xbddc444 (0xc7a9 bytes) 2024-04-24T15:32:54.572 INFO:teuthology.orchestra.run.smithi096.stdout:2590 read 0xbe416c3 thru 0xbe44f84 (0x38c2 bytes) 2024-04-24T15:32:54.572 INFO:teuthology.orchestra.run.smithi096.stdout:2591 read 0x1911d6e thru 0x191f5fa (0xd88d bytes) 2024-04-24T15:32:54.572 INFO:teuthology.orchestra.run.smithi096.stdout:2592 punch from 0xd0b197 to 0xd17965, (0xc7ce bytes) 2024-04-24T15:32:54.572 INFO:teuthology.orchestra.run.smithi096.stdout:2593 writesame 0xa0d8f0 thru 0xa0f588 data_size 0x1(0x1c99 bytes) 2024-04-24T15:32:54.588 INFO:teuthology.orchestra.run.smithi096.stdout:2594 write 0xeaa2637 thru 0xeab066e (0xe038 bytes) 2024-04-24T15:32:54.592 INFO:teuthology.orchestra.run.smithi096.stdout:2595 write 0x2b641ca thru 0x2b67f0c (0x3d43 bytes) 2024-04-24T15:32:54.594 INFO:teuthology.orchestra.run.smithi096.stdout:2597 read 0xab14548 thru 0xab18cc7 (0x4780 bytes) 2024-04-24T15:32:54.595 INFO:teuthology.orchestra.run.smithi096.stdout:2598 read 0x1a7e2fe thru 0x1a7f2df (0xfe2 bytes) 2024-04-24T15:32:54.595 INFO:teuthology.orchestra.run.smithi096.stdout:2599 write 0xc8d4c4 thru 0xc902b2 (0x2def bytes) 2024-04-24T15:32:54.597 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:54.597 INFO:teuthology.orchestra.run.smithi096.stdout:2601 write 0xdd9b121 thru 0xdda6fd2 (0xbeb2 bytes) 2024-04-24T15:32:54.601 INFO:teuthology.orchestra.run.smithi096.stdout:2602 write 0xcc1dd46 thru 0xcc20310 (0x25cb bytes) 2024-04-24T15:32:54.604 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:54.620 INFO:teuthology.orchestra.run.smithi096.stdout:2605 trunc from 0xeab066f to 0x1ba5f5c 2024-04-24T15:32:54.634 INFO:teuthology.orchestra.run.smithi096.stdout:2606 read 0x534ae7 thru 0x543907 (0xee21 bytes) 2024-04-24T15:32:54.636 INFO:teuthology.orchestra.run.smithi096.stdout:2607 trunc from 0x1ba5f5c to 0x327951e 2024-04-24T15:32:54.643 INFO:teuthology.orchestra.run.smithi096.stdout:2608 write 0x2d45c00 thru 0x2d4e778 (0x8b79 bytes) 2024-04-24T15:32:54.646 INFO:teuthology.orchestra.run.smithi096.stdout:2611 punch from 0xfcdf01 to 0xfd9044, (0xb143 bytes) 2024-04-24T15:32:54.659 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:32:54.659+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:32:54.665 INFO:teuthology.orchestra.run.smithi096.stdout:2613 writesame 0xe48eff9 thru 0xe49bcc3 data_size 0x1(0xcccb bytes) 2024-04-24T15:32:54.665 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:54.665 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:54.665 INFO:teuthology.orchestra.run.smithi096.stdout:2617 punch from 0x5356c3e to 0x535efc0, (0x8382 bytes) 2024-04-24T15:32:54.665 INFO:teuthology.orchestra.run.smithi096.stdout:2619 writesame 0xdb7125c thru 0xdb74490 data_size 0x1(0x3235 bytes) 2024-04-24T15:32:54.673 INFO:teuthology.orchestra.run.smithi096.stdout:2621 write 0xa673603 thru 0xa675860 (0x225e bytes) 2024-04-24T15:32:54.676 INFO:teuthology.orchestra.run.smithi096.stdout:2622 punch from 0x29e9c17 to 0x29f74e5, (0xd8ce bytes) 2024-04-24T15:32:54.677 INFO:teuthology.orchestra.run.smithi096.stdout:2623 write 0x363e59 thru 0x370bdb (0xcd83 bytes) 2024-04-24T15:32:54.688 INFO:teuthology.orchestra.run.smithi096.stdout:2625 read 0xbb7ec9f thru 0xbb8dae8 (0xee4a bytes) 2024-04-24T15:32:54.688 INFO:teuthology.orchestra.run.smithi096.stdout:2626 punch from 0x326fa8c to 0x3275b2d, (0x60a1 bytes) 2024-04-24T15:32:54.688 INFO:teuthology.orchestra.run.smithi096.stdout:2627 read 0x8b51dba thru 0x8b5aab1 (0x8cf8 bytes) 2024-04-24T15:32:54.688 INFO:teuthology.orchestra.run.smithi096.stdout:2628 read 0x37940d thru 0x388198 (0xed8c bytes) 2024-04-24T15:32:54.689 INFO:teuthology.orchestra.run.smithi096.stdout:2630 punch from 0x45615d9 to 0x4567d7a, (0x67a1 bytes) 2024-04-24T15:32:54.689 INFO:teuthology.orchestra.run.smithi096.stdout:2631 write 0xd5c9bb7 thru 0xd5d638a (0xc7d4 bytes) 2024-04-24T15:32:54.691 INFO:teuthology.orchestra.run.smithi096.stdout:2632 write 0x269cd86 thru 0x26abb78 (0xedf3 bytes) 2024-04-24T15:32:54.698 INFO:teuthology.orchestra.run.smithi096.stdout:2633 read 0x9f1e613 thru 0x9f2666b (0x8059 bytes) 2024-04-24T15:32:54.698 INFO:teuthology.orchestra.run.smithi096.stdout:2636 read 0xa5004f1 thru 0xa50d362 (0xce72 bytes) 2024-04-24T15:32:54.698 INFO:teuthology.orchestra.run.smithi096.stdout:2638 punch from 0xce408d7 to 0xce4161e, (0xd47 bytes) 2024-04-24T15:32:54.698 INFO:teuthology.orchestra.run.smithi096.stdout:2639 write 0x31cb716 thru 0x31d14c1 (0x5dac bytes) 2024-04-24T15:32:54.700 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:54.700 INFO:teuthology.orchestra.run.smithi096.stdout:2643 write 0x510142f thru 0x5101c51 (0x823 bytes) 2024-04-24T15:32:54.702 INFO:teuthology.orchestra.run.smithi096.stdout:2644 read 0x236957b thru 0x23711d4 (0x7c5a bytes) 2024-04-24T15:32:54.702 INFO:teuthology.orchestra.run.smithi096.stdout:2647 writesame 0x8ca9af2 thru 0x8cb8b77 data_size 0x11(0xf086 bytes) 2024-04-24T15:32:54.710 INFO:teuthology.orchestra.run.smithi096.stdout:2648 punch from 0xc12793c to 0xc127e72, (0x536 bytes) 2024-04-24T15:32:54.710 INFO:teuthology.orchestra.run.smithi096.stdout:2650 read 0x557bc00 thru 0x557ee59 (0x325a bytes) 2024-04-24T15:32:54.710 INFO:teuthology.orchestra.run.smithi096.stdout:2651 clone 14 order 20 su 65536 sc 10 2024-04-24T15:32:54.716 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:54.716 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:54.726 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-24T15:32:54.760 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:32:54.760+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:32:54.860 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:32:54.861+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:32:54.890 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:54.891 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:54.900 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-24T15:32:54.985 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:32:54.985+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:32:55.062 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:32:55.062+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:32:55.064 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:55.064 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:55.074 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-24T15:32:55.163 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:32:55.163+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:32:55.236 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:55.237 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:55.246 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-24T15:32:55.264 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:32:55.264+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:32:55.365 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:32:55.365+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:32:55.370 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:55.370 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:55.380 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-24T15:32:55.465 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:32:55.466+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:32:55.502 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:55.502 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:55.514 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-24T15:32:55.538 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-24T15:32:55.603 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:32:55.603+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:32:55.642 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:55.643 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:55.653 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-24T15:32:55.677 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-24T15:32:55.697 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:32:55.698+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:32:55.708 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-04-24T15:32:55.721 INFO:teuthology.orchestra.run.smithi096.stdout:truncating image image_client.0-clone13 from 0xe49bcc4 (overlap 0x1551d59) to 0x0 2024-04-24T15:32:55.735 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-04-24T15:32:55.760 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-04-24T15:32:55.768 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:32:55.768+0000 7f23abcf6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 44021) UID: 0 2024-04-24T15:32:55.781 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-04-24T15:32:55.794 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #12, image image_client.0-clone12 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent13 2024-04-24T15:32:55.798 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:55.798 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:55.805 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-04-24T15:32:55.807 INFO:teuthology.orchestra.run.smithi096.stdout:2655 write 0x1b6419d thru 0x1b6cdf9 (0x8c5d bytes) 2024-04-24T15:32:55.808 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-24T15:32:55.815 INFO:teuthology.orchestra.run.smithi096.stdout:2656 read 0x47b338a thru 0x47bc48c (0x9103 bytes) 2024-04-24T15:32:55.815 INFO:teuthology.orchestra.run.smithi096.stdout:2658 write 0x5bc9362 thru 0x5bd7f9a (0xec39 bytes) 2024-04-24T15:32:55.820 INFO:teuthology.orchestra.run.smithi096.stdout:2659 read 0x6330f67 thru 0x6336947 (0x59e1 bytes) 2024-04-24T15:32:55.820 INFO:teuthology.orchestra.run.smithi096.stdout:2660 write 0x6d274d8 thru 0x6d28a9f (0x15c8 bytes) 2024-04-24T15:32:55.823 INFO:teuthology.orchestra.run.smithi096.stdout:2662 punch from 0x4cec13b to 0x4cef03e, (0x2f03 bytes) 2024-04-24T15:32:55.823 INFO:teuthology.orchestra.run.smithi096.stdout:2663 writesame 0x57cf3cf thru 0x57d2d75 data_size 0x1(0x39a7 bytes) 2024-04-24T15:32:55.828 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-04-24T15:32:55.848 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-04-24T15:32:55.849 INFO:teuthology.orchestra.run.smithi096.stdout:2664 trunc from 0xe49bcc4 to 0x25f1879 2024-04-24T15:32:55.869 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:32:55.869+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:32:55.875 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-04-24T15:32:55.876 INFO:teuthology.orchestra.run.smithi096.stdout:2665 read 0x8f8726 thru 0x8fa714 (0x1fef bytes) 2024-04-24T15:32:55.878 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T15:32:55.878 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T15:32:55.878 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T15:32:55.878 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T15:32:55.878 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T15:32:55.878 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T15:32:55.878 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T15:32:55.878 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T15:32:55.883 INFO:teuthology.orchestra.run.smithi096.stdout:2666 write 0x315648f thru 0x31595d0 (0x3142 bytes) 2024-04-24T15:32:55.890 INFO:teuthology.orchestra.run.smithi096.stdout:2668 trunc from 0x31595d1 to 0x5b61309 2024-04-24T15:32:55.895 INFO:teuthology.orchestra.run.smithi096.stdout:2671 read 0x4b1c505 thru 0x4b1d8ea (0x13e6 bytes) 2024-04-24T15:32:55.895 INFO:teuthology.orchestra.run.smithi096.stdout:2672 read 0x53b0cf9 thru 0x53bb2cc (0xa5d4 bytes) 2024-04-24T15:32:55.895 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:55.904 INFO:teuthology.orchestra.run.smithi096.stdout:2676 writesame 0x815c1c4 thru 0x815f718 data_size 0x3(0x3555 bytes) 2024-04-24T15:32:55.904 INFO:teuthology.orchestra.run.smithi096.stdout:2677 read 0x1ea6809 thru 0x1eb42bd (0xdab5 bytes) 2024-04-24T15:32:55.905 INFO:teuthology.orchestra.run.smithi096.stdout:2678 writesame 0x1c0e257 thru 0x1c1c7e0 data_size 0xb(0xe58a bytes) 2024-04-24T15:32:55.910 INFO:teuthology.orchestra.run.smithi096.stdout:2680 punch from 0x32be9de to 0x32ca97c, (0xbf9e bytes) 2024-04-24T15:32:55.910 INFO:teuthology.orchestra.run.smithi096.stdout:2681 writesame 0x4cfac4e thru 0x4d08989 data_size 0x2(0xdd3c bytes) 2024-04-24T15:32:55.926 INFO:teuthology.orchestra.run.smithi096.stdout:2682 read 0x133ee15 thru 0x1348a0a (0x9bf6 bytes) 2024-04-24T15:32:55.927 INFO:teuthology.orchestra.run.smithi096.stdout:2684 trunc from 0x815f719 to 0xa22ba4d 2024-04-24T15:32:55.937 INFO:teuthology.orchestra.run.smithi096.stdout:2685 trunc from 0xa22ba4d to 0xeb7e9d9 2024-04-24T15:32:55.943 INFO:teuthology.orchestra.run.smithi096.stdout:2687 writesame 0x58c8ae3 thru 0x58cb29a data_size 0x29(0x27b8 bytes) 2024-04-24T15:32:55.946 INFO:teuthology.orchestra.run.smithi096.stdout:2689 read 0x24586b3 thru 0x24676dc (0xf02a bytes) 2024-04-24T15:32:55.946 INFO:teuthology.orchestra.run.smithi096.stdout:2690 write 0x792a260 thru 0x7936143 (0xbee4 bytes) 2024-04-24T15:32:55.950 INFO:teuthology.orchestra.run.smithi096.stdout:2691 writesame 0x3372efd thru 0x33787c2 data_size 0xb(0x58c6 bytes) 2024-04-24T15:32:55.962 INFO:teuthology.orchestra.run.smithi096.stdout:2692 trunc from 0xeb7e9d9 to 0x6e89d4b 2024-04-24T15:32:55.969 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:32:55.970+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:32:55.971 INFO:teuthology.orchestra.run.smithi096.stdout:2693 read 0x4c8d5c5 thru 0x4c91cbd (0x46f9 bytes) 2024-04-24T15:32:55.971 INFO:teuthology.orchestra.run.smithi096.stdout:2694 read 0x2edaef0 thru 0x2edf68f (0x47a0 bytes) 2024-04-24T15:32:55.971 INFO:teuthology.orchestra.run.smithi096.stdout:2695 read 0x1e9c11e thru 0x1ea4617 (0x84fa bytes) 2024-04-24T15:32:55.971 INFO:teuthology.orchestra.run.smithi096.stdout:2696 write 0x47bf225 thru 0x47c5b54 (0x6930 bytes) 2024-04-24T15:32:55.973 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:32:55.974 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:32:55.974 INFO:teuthology.orchestra.run.smithi151.stdout: "complaint_time": 30, 2024-04-24T15:32:55.974 INFO:teuthology.orchestra.run.smithi151.stdout: "num_blocked_ops": 0 2024-04-24T15:32:55.974 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:32:55.974 INFO:teuthology.orchestra.run.smithi096.stdout:2697 read 0x532e52a thru 0x5330b1c (0x25f3 bytes) 2024-04-24T15:32:55.975 INFO:teuthology.orchestra.run.smithi096.stdout:2698 write 0x33e6080 thru 0x33e7bb6 (0x1b37 bytes) 2024-04-24T15:32:55.977 INFO:teuthology.orchestra.run.smithi096.stdout:2699 read 0x3fe1e78 thru 0x3ff00af (0xe238 bytes) 2024-04-24T15:32:55.977 INFO:teuthology.orchestra.run.smithi096.stdout:2700 writesame 0xf228c9 thru 0xf2f146 data_size 0xb(0xc87e bytes) 2024-04-24T15:32:55.982 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:55.984 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-24T15:32:55.992 INFO:teuthology.orchestra.run.smithi096.stdout:2702 trunc from 0x6e89d4b to 0xe94587e 2024-04-24T15:32:55.996 INFO:teuthology.orchestra.run.smithi096.stdout:2703 read 0xd269987 thru 0xd2701dc (0x6856 bytes) 2024-04-24T15:32:55.996 INFO:teuthology.orchestra.run.smithi096.stdout:2706 read 0x3e32af thru 0x3f0d50 (0xdaa2 bytes) 2024-04-24T15:32:55.996 INFO:teuthology.orchestra.run.smithi096.stdout:2707 write 0x835a429 thru 0x835d73d (0x3315 bytes) 2024-04-24T15:32:55.999 INFO:teuthology.orchestra.run.smithi096.stdout:2708 punch from 0xbe57070 to 0xbe5a7dc, (0x376c bytes) 2024-04-24T15:32:55.999 INFO:teuthology.orchestra.run.smithi096.stdout:2709 read 0x7ba8c7e thru 0x7bab688 (0x2a0b bytes) 2024-04-24T15:32:55.999 INFO:teuthology.orchestra.run.smithi096.stdout:2710 write 0x5c1cd4a thru 0x5c288e1 (0xbb98 bytes) 2024-04-24T15:32:56.002 INFO:teuthology.orchestra.run.smithi096.stdout:2711 read 0x3691930 thru 0x3694550 (0x2c21 bytes) 2024-04-24T15:32:56.010 INFO:teuthology.orchestra.run.smithi096.stdout:2712 trunc from 0xe94587e to 0x8a328de 2024-04-24T15:32:56.015 INFO:teuthology.orchestra.run.smithi096.stdout:2713 read 0x4f63771 thru 0x4f675a1 (0x3e31 bytes) 2024-04-24T15:32:56.022 INFO:teuthology.orchestra.run.smithi096.stdout:2714 write 0xaac288a thru 0xaac7dc8 (0x553f bytes) 2024-04-24T15:32:56.025 INFO:teuthology.orchestra.run.smithi096.stdout:2716 writesame 0x7278904 thru 0x7279d07 data_size 0xe(0x1404 bytes) 2024-04-24T15:32:56.028 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:56.028 INFO:teuthology.orchestra.run.smithi096.stdout:2718 punch from 0x74e95ad to 0x74ebe3c, (0x288f bytes) 2024-04-24T15:32:56.029 INFO:teuthology.orchestra.run.smithi096.stdout:2720 trunc from 0xaac7dc9 to 0xbaa1044 2024-04-24T15:32:56.033 INFO:teuthology.orchestra.run.smithi096.stdout:2721 writesame 0x631916a thru 0x631f6ee data_size 0x1(0x6585 bytes) 2024-04-24T15:32:56.047 INFO:teuthology.orchestra.run.smithi096.stdout:2723 read 0x945ef07 thru 0x9460032 (0x112c bytes) 2024-04-24T15:32:56.047 INFO:teuthology.orchestra.run.smithi096.stdout:2724 punch from 0x2e0dba1 to 0x2e0ea29, (0xe88 bytes) 2024-04-24T15:32:56.048 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:56.054 INFO:teuthology.orchestra.run.smithi096.stdout:2726 writesame 0xccb286e thru 0xccbee86 data_size 0x2f(0xc619 bytes) 2024-04-24T15:32:56.054 INFO:teuthology.orchestra.run.smithi096.stdout:2727 punch from 0xbefd71f to 0xbf0bcfe, (0xe5df bytes) 2024-04-24T15:32:56.054 INFO:teuthology.orchestra.run.smithi096.stdout:2728 read 0x3e5d30e thru 0x3e6a662 (0xd355 bytes) 2024-04-24T15:32:56.054 INFO:teuthology.orchestra.run.smithi096.stdout:2729 write 0x480a0ec thru 0x481510a (0xb01f bytes) 2024-04-24T15:32:56.056 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:56.056 INFO:teuthology.orchestra.run.smithi096.stdout:2731 read 0xf85779 thru 0xf8a4d0 (0x4d58 bytes) 2024-04-24T15:32:56.056 INFO:teuthology.orchestra.run.smithi096.stdout:2732 writesame 0x9d65451 thru 0x9d7171c data_size 0x5b(0xc2cc bytes) 2024-04-24T15:32:56.061 INFO:teuthology.orchestra.run.smithi096.stdout:2733 write 0x708953 thru 0x70aa4b (0x20f9 bytes) 2024-04-24T15:32:56.064 INFO:teuthology.orchestra.run.smithi096.stdout:2734 read 0x963269e thru 0x963891e (0x6281 bytes) 2024-04-24T15:32:56.070 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:32:56.071+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:32:56.075 INFO:teuthology.orchestra.run.smithi096.stdout:2735 trunc from 0xccbee87 to 0x4049b12 2024-04-24T15:32:56.104 INFO:teuthology.orchestra.run.smithi096.stdout:2736 write 0x870e734 thru 0x8711f6a (0x3837 bytes) 2024-04-24T15:32:56.111 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:56.111 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:56.112 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:32:56.112 INFO:teuthology.orchestra.run.smithi151.stdout: "size": 20, 2024-04-24T15:32:56.112 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 600, 2024-04-24T15:32:56.112 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [ 2024-04-24T15:32:56.112 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.112 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:2507 4.e 4:729aaa1b:::rbd_data.114a1f60cb66.000000000000029a:head [set-alloc-hint object_size 262144 write_size 262144,write 233472~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-24T15:32:56.112 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:23.072715+0000", 2024-04-24T15:32:56.113 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 33.038669161999998, 2024-04-24T15:32:56.113 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0065961370000000002, 2024-04-24T15:32:56.113 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:56.113 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:56.113 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:56.113 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:56.113 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:56.113 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 2507 2024-04-24T15:32:56.113 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.113 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:56.113 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.113 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:56.113 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.072715+0000", 2024-04-24T15:32:56.113 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.113 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.113 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.114 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:56.114 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.072715+0000", 2024-04-24T15:32:56.114 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.114 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.114 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.114 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:56.114 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.072713+0000", 2024-04-24T15:32:56.114 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999981 2024-04-24T15:32:56.114 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.114 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.114 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:56.115 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.072727+0000", 2024-04-24T15:32:56.115 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3549e-05 2024-04-24T15:32:56.115 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.115 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.115 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:56.115 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.072739+0000", 2024-04-24T15:32:56.115 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.1973999999999999e-05 2024-04-24T15:32:56.115 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.115 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.115 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:56.115 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.072746+0000", 2024-04-24T15:32:56.115 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2849999999999997e-06 2024-04-24T15:32:56.115 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.115 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.116 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:56.116 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.072798+0000", 2024-04-24T15:32:56.116 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.2289e-05 2024-04-24T15:32:56.116 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.116 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.116 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op must wait for map", 2024-04-24T15:32:56.116 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.072806+0000", 2024-04-24T15:32:56.116 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.8469999999999996e-06 2024-04-24T15:32:56.116 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.116 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.116 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:56.117 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.074503+0000", 2024-04-24T15:32:56.117 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0016967029999999999 2024-04-24T15:32:56.117 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.117 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.117 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:56.117 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.074593+0000", 2024-04-24T15:32:56.117 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.0313000000000001e-05 2024-04-24T15:32:56.117 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.117 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.117 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 6", 2024-04-24T15:32:56.117 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.074732+0000", 2024-04-24T15:32:56.118 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000138376 2024-04-24T15:32:56.118 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.118 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.118 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:56.118 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.078538+0000", 2024-04-24T15:32:56.118 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0038068749999999999 2024-04-24T15:32:56.118 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.118 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.119 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:56.119 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.079232+0000", 2024-04-24T15:32:56.119 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00069326900000000005 2024-04-24T15:32:56.119 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.119 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.119 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:56.119 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.079273+0000", 2024-04-24T15:32:56.119 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.1236000000000003e-05 2024-04-24T15:32:56.120 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.120 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.120 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:56.120 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:23.079311+0000", 2024-04-24T15:32:56.120 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8497000000000001e-05 2024-04-24T15:32:56.120 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.120 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:56.120 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.121 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.121 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.121 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_repop(client.4426.0:2563 4.1d e34/27 4:b841be81:::rbd_data.114a1f60cb66.00000000000001d7:head v 34'17, mlcod=32'15)", 2024-04-24T15:32:56.121 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.108404+0000", 2024-04-24T15:32:56.121 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 32.002980630000003, 2024-04-24T15:32:56.121 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0073539469999999996, 2024-04-24T15:32:56.121 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:56.122 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:56.122 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:56.122 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.122 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:56.122 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.108404+0000", 2024-04-24T15:32:56.122 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.123 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.123 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.123 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:56.123 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.108404+0000", 2024-04-24T15:32:56.123 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.123 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.123 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.123 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:56.124 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.108403+0000", 2024-04-24T15:32:56.124 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:56.124 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.124 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.124 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:56.124 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.108420+0000", 2024-04-24T15:32:56.124 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.6980999999999999e-05 2024-04-24T15:32:56.124 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.125 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.125 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:56.125 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.108428+0000", 2024-04-24T15:32:56.125 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0560000000000005e-06 2024-04-24T15:32:56.125 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.125 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.125 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:56.125 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.108432+0000", 2024-04-24T15:32:56.125 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.4060000000000001e-06 2024-04-24T15:32:56.126 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.126 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.126 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:56.126 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.108462+0000", 2024-04-24T15:32:56.126 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.008e-05 2024-04-24T15:32:56.126 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.127 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.127 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:56.127 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.108475+0000", 2024-04-24T15:32:56.127 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3018e-05 2024-04-24T15:32:56.127 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.127 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.127 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:56.129 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.115726+0000", 2024-04-24T15:32:56.130 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0072510040000000001 2024-04-24T15:32:56.130 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.130 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.130 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:56.130 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.115758+0000", 2024-04-24T15:32:56.130 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.2092999999999999e-05 2024-04-24T15:32:56.131 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.131 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:56.131 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.131 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.131 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.131 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:2620 4.18 4:1e938922:::rbd_data.114a1f60cb66.0000000000000028:head [stat,write 163840~6144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:56.131 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.151073+0000", 2024-04-24T15:32:56.131 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 31.960311916999999, 2024-04-24T15:32:56.131 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0063989210000000001, 2024-04-24T15:32:56.132 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:56.132 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:56.132 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:56.132 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:56.132 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:56.132 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 2620 2024-04-24T15:32:56.132 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.132 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:56.132 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.132 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:56.132 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.151073+0000", 2024-04-24T15:32:56.132 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.132 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.132 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.133 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:56.133 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.151073+0000", 2024-04-24T15:32:56.133 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.133 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.133 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.133 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:56.133 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.151072+0000", 2024-04-24T15:32:56.133 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:56.133 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.133 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.133 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:56.133 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.151083+0000", 2024-04-24T15:32:56.133 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0801999999999999e-05 2024-04-24T15:32:56.133 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.133 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.134 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:56.134 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.151086+0000", 2024-04-24T15:32:56.134 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.3900000000000002e-06 2024-04-24T15:32:56.134 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.134 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.134 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:56.134 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.151089+0000", 2024-04-24T15:32:56.134 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.294e-06 2024-04-24T15:32:56.134 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.134 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.134 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:56.134 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.152198+0000", 2024-04-24T15:32:56.134 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.001109407 2024-04-24T15:32:56.135 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.135 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.135 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:56.135 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.152273+0000", 2024-04-24T15:32:56.135 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.4419999999999996e-05 2024-04-24T15:32:56.135 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.135 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.135 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 6", 2024-04-24T15:32:56.135 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.152396+0000", 2024-04-24T15:32:56.135 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00012322400000000001 2024-04-24T15:32:56.135 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.135 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.135 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:56.135 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.157242+0000", 2024-04-24T15:32:56.135 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0048462369999999998 2024-04-24T15:32:56.136 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.136 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.136 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:56.136 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.157414+0000", 2024-04-24T15:32:56.136 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000172411 2024-04-24T15:32:56.136 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.136 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.136 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:56.136 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.157451+0000", 2024-04-24T15:32:56.136 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.6269000000000001e-05 2024-04-24T15:32:56.136 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.136 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.136 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:56.136 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.157471+0000", 2024-04-24T15:32:56.137 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.0741e-05 2024-04-24T15:32:56.137 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.137 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:56.137 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.137 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.137 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.137 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:2623 4.18 4:1b4f38fa:::rbd_data.114a1f60cb66.000000000000002b:head [stat,write 161792~6144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:56.137 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.151108+0000", 2024-04-24T15:32:56.137 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 31.960276655000001, 2024-04-24T15:32:56.137 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0064461170000000003, 2024-04-24T15:32:56.137 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:56.137 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:56.137 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:56.137 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:56.137 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:56.138 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 2623 2024-04-24T15:32:56.138 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.138 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:56.138 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.138 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:56.138 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.151108+0000", 2024-04-24T15:32:56.138 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.138 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.138 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.138 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:56.138 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.151108+0000", 2024-04-24T15:32:56.138 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.138 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.139 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.139 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:56.139 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.151108+0000", 2024-04-24T15:32:56.139 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:56.139 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.139 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.139 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:56.139 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.151117+0000", 2024-04-24T15:32:56.139 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.8560000000000002e-06 2024-04-24T15:32:56.139 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.139 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.139 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:56.139 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.151122+0000", 2024-04-24T15:32:56.139 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5390000000000001e-06 2024-04-24T15:32:56.140 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.140 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.140 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:56.140 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.151124+0000", 2024-04-24T15:32:56.140 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.2800000000000002e-06 2024-04-24T15:32:56.140 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.140 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.140 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:56.140 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.152676+0000", 2024-04-24T15:32:56.140 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.001552124 2024-04-24T15:32:56.140 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.140 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.140 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:56.140 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.152759+0000", 2024-04-24T15:32:56.140 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.2688000000000005e-05 2024-04-24T15:32:56.141 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.141 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.141 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 6", 2024-04-24T15:32:56.141 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.152867+0000", 2024-04-24T15:32:56.141 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00010779400000000001 2024-04-24T15:32:56.141 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.141 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.141 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:56.141 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.157259+0000", 2024-04-24T15:32:56.141 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0043918230000000004 2024-04-24T15:32:56.141 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.141 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.141 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:56.141 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.157498+0000", 2024-04-24T15:32:56.141 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000239829 2024-04-24T15:32:56.142 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.142 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.142 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:56.142 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.157525+0000", 2024-04-24T15:32:56.142 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.6868000000000001e-05 2024-04-24T15:32:56.142 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.142 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.142 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:56.142 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.157554+0000", 2024-04-24T15:32:56.142 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8602000000000002e-05 2024-04-24T15:32:56.142 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.142 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:56.142 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.143 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.143 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.143 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:3618 4.18 4:1f7d02ce:::rbd_data.114a1f60cb66.0000000000000161:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:56.143 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.581260+0000", 2024-04-24T15:32:56.143 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 31.530124298, 2024-04-24T15:32:56.143 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0068184120000000003, 2024-04-24T15:32:56.143 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:56.143 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:56.143 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:56.143 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:56.143 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:56.143 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 3618 2024-04-24T15:32:56.143 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.143 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:56.143 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.144 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:56.144 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.581260+0000", 2024-04-24T15:32:56.144 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.144 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.144 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.144 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:56.144 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.581260+0000", 2024-04-24T15:32:56.144 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.144 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.144 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.144 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:56.144 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.581259+0000", 2024-04-24T15:32:56.144 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999999 2024-04-24T15:32:56.144 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.145 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.145 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:56.145 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.581265+0000", 2024-04-24T15:32:56.145 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.003e-06 2024-04-24T15:32:56.145 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.145 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.145 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:56.145 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.581278+0000", 2024-04-24T15:32:56.145 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.2483000000000001e-05 2024-04-24T15:32:56.145 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.145 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.145 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:56.145 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.581284+0000", 2024-04-24T15:32:56.145 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.7399999999999998e-06 2024-04-24T15:32:56.145 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.146 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.146 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:56.146 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.581783+0000", 2024-04-24T15:32:56.146 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00049892199999999995 2024-04-24T15:32:56.146 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.146 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.146 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:56.146 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.581902+0000", 2024-04-24T15:32:56.146 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000118272 2024-04-24T15:32:56.146 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.146 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.146 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 6", 2024-04-24T15:32:56.146 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.583068+0000", 2024-04-24T15:32:56.147 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0011665759999999999 2024-04-24T15:32:56.147 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.147 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.147 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:56.147 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.587081+0000", 2024-04-24T15:32:56.147 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.004012781 2024-04-24T15:32:56.147 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.147 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.147 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:56.147 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.587997+0000", 2024-04-24T15:32:56.147 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00091637700000000001 2024-04-24T15:32:56.147 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.147 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.147 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:56.147 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.588041+0000", 2024-04-24T15:32:56.148 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3619999999999999e-05 2024-04-24T15:32:56.148 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.148 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.148 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:56.148 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.588079+0000", 2024-04-24T15:32:56.148 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.7654000000000002e-05 2024-04-24T15:32:56.148 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.148 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:56.148 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.148 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.148 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.148 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:3767 4.9 4:908f8f18:::rbd_data.114a1f60cb66.000000000000001b:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:56.148 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.699557+0000", 2024-04-24T15:32:56.148 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 31.411827573, 2024-04-24T15:32:56.148 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0088231639999999997, 2024-04-24T15:32:56.149 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:56.149 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:56.149 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:56.149 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:56.149 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:56.149 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 3767 2024-04-24T15:32:56.149 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.149 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:56.149 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.149 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:56.149 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699557+0000", 2024-04-24T15:32:56.149 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.149 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.149 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.150 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:56.150 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699557+0000", 2024-04-24T15:32:56.150 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.150 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.150 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.150 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:56.150 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699556+0000", 2024-04-24T15:32:56.150 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999999 2024-04-24T15:32:56.150 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.150 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.150 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:56.150 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699561+0000", 2024-04-24T15:32:56.150 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.0440000000000003e-06 2024-04-24T15:32:56.150 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.151 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.151 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:56.151 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699571+0000", 2024-04-24T15:32:56.151 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0237999999999999e-05 2024-04-24T15:32:56.151 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.151 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.151 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:56.151 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699582+0000", 2024-04-24T15:32:56.151 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.1009000000000001e-05 2024-04-24T15:32:56.151 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.151 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.151 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:56.151 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699607+0000", 2024-04-24T15:32:56.151 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.5024999999999998e-05 2024-04-24T15:32:56.151 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.152 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.152 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:56.152 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699671+0000", 2024-04-24T15:32:56.152 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.3219999999999994e-05 2024-04-24T15:32:56.152 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.152 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.152 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:56.152 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.700952+0000", 2024-04-24T15:32:56.152 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.001281952 2024-04-24T15:32:56.152 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.152 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.152 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:56.152 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.705912+0000", 2024-04-24T15:32:56.152 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0049599889999999997 2024-04-24T15:32:56.153 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.153 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.153 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:56.153 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.708300+0000", 2024-04-24T15:32:56.153 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.002387934 2024-04-24T15:32:56.153 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.153 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.153 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:56.153 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.708345+0000", 2024-04-24T15:32:56.153 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4175000000000003e-05 2024-04-24T15:32:56.153 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.153 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.153 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:56.153 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.708380+0000", 2024-04-24T15:32:56.154 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5460999999999998e-05 2024-04-24T15:32:56.154 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.154 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:56.154 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.154 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.154 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.154 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:3770 4.13 4:cb3c841f:::rbd_data.114a1f60cb66.000000000000001e:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:56.154 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.699597+0000", 2024-04-24T15:32:56.154 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 31.411787108999999, 2024-04-24T15:32:56.154 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0083605450000000005, 2024-04-24T15:32:56.154 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:56.154 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:56.154 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:56.155 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:56.155 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:56.155 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 3770 2024-04-24T15:32:56.155 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.155 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:56.155 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.155 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:56.155 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699597+0000", 2024-04-24T15:32:56.155 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.155 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.155 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.155 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:56.155 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699597+0000", 2024-04-24T15:32:56.155 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.155 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.156 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.156 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:56.156 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699597+0000", 2024-04-24T15:32:56.156 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:56.156 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.156 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.156 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:56.156 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699600+0000", 2024-04-24T15:32:56.156 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.371e-06 2024-04-24T15:32:56.156 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.156 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.156 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:56.156 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699611+0000", 2024-04-24T15:32:56.156 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0552e-05 2024-04-24T15:32:56.156 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.156 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.157 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:56.157 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699615+0000", 2024-04-24T15:32:56.157 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5689999999999998e-06 2024-04-24T15:32:56.157 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.157 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.157 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:56.157 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699692+0000", 2024-04-24T15:32:56.157 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.6664999999999999e-05 2024-04-24T15:32:56.157 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.157 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.157 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:56.157 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699740+0000", 2024-04-24T15:32:56.157 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8032999999999999e-05 2024-04-24T15:32:56.157 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.157 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.157 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 6", 2024-04-24T15:32:56.158 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.700940+0000", 2024-04-24T15:32:56.158 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.001199978 2024-04-24T15:32:56.158 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.158 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.158 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:56.158 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.706397+0000", 2024-04-24T15:32:56.158 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0054569459999999998 2024-04-24T15:32:56.158 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.158 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.158 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:56.158 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.707870+0000", 2024-04-24T15:32:56.158 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.001473294 2024-04-24T15:32:56.158 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.159 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.159 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:56.159 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.707920+0000", 2024-04-24T15:32:56.159 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.994e-05 2024-04-24T15:32:56.159 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.159 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.159 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:56.159 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.707958+0000", 2024-04-24T15:32:56.159 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.7761e-05 2024-04-24T15:32:56.159 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.159 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:56.159 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.159 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.159 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.160 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:3773 4.e 4:7173e872:::rbd_data.114a1f60cb66.0000000000000021:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:56.160 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.699632+0000", 2024-04-24T15:32:56.160 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 31.411752545999999, 2024-04-24T15:32:56.160 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0068017260000000001, 2024-04-24T15:32:56.160 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:56.160 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:56.160 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:56.160 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:56.160 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:56.160 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 3773 2024-04-24T15:32:56.160 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.160 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:56.160 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.160 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:56.161 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699632+0000", 2024-04-24T15:32:56.161 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.161 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.161 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.161 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:56.161 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699632+0000", 2024-04-24T15:32:56.161 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.161 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.161 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.161 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:56.161 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699631+0000", 2024-04-24T15:32:56.161 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:56.161 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.161 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.162 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:56.162 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699639+0000", 2024-04-24T15:32:56.162 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7319999999999998e-06 2024-04-24T15:32:56.162 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.162 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.162 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:56.162 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699646+0000", 2024-04-24T15:32:56.162 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1330000000000001e-06 2024-04-24T15:32:56.162 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.162 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.162 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:56.162 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699651+0000", 2024-04-24T15:32:56.162 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.9200000000000003e-06 2024-04-24T15:32:56.163 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.163 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.163 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:56.163 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699689+0000", 2024-04-24T15:32:56.163 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.7979999999999999e-05 2024-04-24T15:32:56.163 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.163 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.163 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:56.163 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.699759+0000", 2024-04-24T15:32:56.163 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.9468000000000006e-05 2024-04-24T15:32:56.163 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.163 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.164 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 6", 2024-04-24T15:32:56.164 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.701164+0000", 2024-04-24T15:32:56.164 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0014056629999999999 2024-04-24T15:32:56.164 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.164 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.164 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:56.164 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.705975+0000", 2024-04-24T15:32:56.164 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0048109340000000002 2024-04-24T15:32:56.164 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.164 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.164 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:56.164 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.706359+0000", 2024-04-24T15:32:56.164 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00038331699999999999 2024-04-24T15:32:56.165 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.165 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.165 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:56.165 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.706407+0000", 2024-04-24T15:32:56.165 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8544999999999997e-05 2024-04-24T15:32:56.165 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.165 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.165 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:56.165 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.706434+0000", 2024-04-24T15:32:56.165 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.6553000000000001e-05 2024-04-24T15:32:56.165 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.165 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:56.165 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.165 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.166 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.166 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:3794 4.18 4:1efc862d:::rbd_data.114a1f60cb66.0000000000000024:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:56.166 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.723446+0000", 2024-04-24T15:32:56.166 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 31.387938027000001, 2024-04-24T15:32:56.166 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.007572957, 2024-04-24T15:32:56.166 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:56.166 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:56.166 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:56.166 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:56.166 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:56.166 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 3794 2024-04-24T15:32:56.166 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.167 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:56.167 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.167 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:56.167 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723446+0000", 2024-04-24T15:32:56.167 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.167 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.167 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.167 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:56.167 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723446+0000", 2024-04-24T15:32:56.167 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.167 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.167 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.167 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:56.167 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723446+0000", 2024-04-24T15:32:56.168 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999999 2024-04-24T15:32:56.168 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.168 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.168 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:56.168 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723450+0000", 2024-04-24T15:32:56.168 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.1729999999999997e-06 2024-04-24T15:32:56.168 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.168 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.168 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:56.168 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723458+0000", 2024-04-24T15:32:56.169 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.653e-06 2024-04-24T15:32:56.169 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.169 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.169 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:56.169 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723465+0000", 2024-04-24T15:32:56.169 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.4450000000000001e-06 2024-04-24T15:32:56.169 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.169 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.169 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:56.169 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723501+0000", 2024-04-24T15:32:56.169 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5726000000000003e-05 2024-04-24T15:32:56.169 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.169 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.169 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:56.170 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723547+0000", 2024-04-24T15:32:56.170 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6479e-05 2024-04-24T15:32:56.170 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.170 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.170 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 6", 2024-04-24T15:32:56.170 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.724384+0000", 2024-04-24T15:32:56.170 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00083723800000000002 2024-04-24T15:32:56.170 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.170 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.170 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:56.170 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.730925+0000", 2024-04-24T15:32:56.170 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0065409179999999997 2024-04-24T15:32:56.170 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.171 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.171 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:56.171 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.730969+0000", 2024-04-24T15:32:56.171 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3869999999999998e-05 2024-04-24T15:32:56.171 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.171 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.171 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:56.171 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.730997+0000", 2024-04-24T15:32:56.171 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7783000000000001e-05 2024-04-24T15:32:56.171 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.171 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.171 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:56.171 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.731019+0000", 2024-04-24T15:32:56.171 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.2427999999999999e-05 2024-04-24T15:32:56.171 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.172 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:56.172 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.172 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.172 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.172 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:3797 4.18 4:1d8bdb81:::rbd_data.114a1f60cb66.0000000000000027:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:56.172 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.723479+0000", 2024-04-24T15:32:56.172 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 31.387905017000001, 2024-04-24T15:32:56.172 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.013378915, 2024-04-24T15:32:56.172 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:56.172 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:56.172 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:56.172 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:56.172 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:56.172 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 3797 2024-04-24T15:32:56.173 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.173 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:56.173 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.173 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:56.173 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723479+0000", 2024-04-24T15:32:56.173 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.173 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.173 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.173 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:56.173 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723479+0000", 2024-04-24T15:32:56.173 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.173 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.173 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.173 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:56.173 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723479+0000", 2024-04-24T15:32:56.174 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:56.174 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.174 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.174 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:56.174 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723482+0000", 2024-04-24T15:32:56.174 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5410000000000001e-06 2024-04-24T15:32:56.174 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.174 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.174 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:56.174 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723492+0000", 2024-04-24T15:32:56.174 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.6879999999999994e-06 2024-04-24T15:32:56.174 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.174 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.175 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:56.175 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723496+0000", 2024-04-24T15:32:56.175 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.34e-06 2024-04-24T15:32:56.175 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.175 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.175 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:56.175 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.728008+0000", 2024-04-24T15:32:56.175 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0045115049999999999 2024-04-24T15:32:56.175 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.175 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.175 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:56.175 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.728058+0000", 2024-04-24T15:32:56.175 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.0201000000000003e-05 2024-04-24T15:32:56.175 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.175 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.176 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 6", 2024-04-24T15:32:56.176 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.728499+0000", 2024-04-24T15:32:56.176 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00044129299999999999 2024-04-24T15:32:56.176 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.176 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.176 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:56.176 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.730962+0000", 2024-04-24T15:32:56.176 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0024628430000000002 2024-04-24T15:32:56.176 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.176 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.176 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:56.176 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.736810+0000", 2024-04-24T15:32:56.176 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0058474490000000002 2024-04-24T15:32:56.176 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.177 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.177 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:56.177 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.736837+0000", 2024-04-24T15:32:56.177 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7251999999999999e-05 2024-04-24T15:32:56.177 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.177 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.177 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:56.177 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.736858+0000", 2024-04-24T15:32:56.177 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.1387999999999999e-05 2024-04-24T15:32:56.177 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.177 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:56.177 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.177 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.177 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.177 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:3798 4.18 4:1e938922:::rbd_data.114a1f60cb66.0000000000000028:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:56.178 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.723506+0000", 2024-04-24T15:32:56.178 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 31.387878603000001, 2024-04-24T15:32:56.178 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.013398188, 2024-04-24T15:32:56.178 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:56.178 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:56.178 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:56.178 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:56.178 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:56.178 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 3798 2024-04-24T15:32:56.178 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.178 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:56.178 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.178 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:56.178 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723506+0000", 2024-04-24T15:32:56.179 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.179 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.179 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.179 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:56.179 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723506+0000", 2024-04-24T15:32:56.179 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.179 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.179 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.179 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:56.179 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723505+0000", 2024-04-24T15:32:56.179 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:56.179 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.179 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.179 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:56.180 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723509+0000", 2024-04-24T15:32:56.180 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.2349999999999999e-06 2024-04-24T15:32:56.180 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.180 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.180 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:56.180 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723515+0000", 2024-04-24T15:32:56.180 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.3239999999999999e-06 2024-04-24T15:32:56.180 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.180 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.180 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:56.180 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723519+0000", 2024-04-24T15:32:56.180 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8580000000000002e-06 2024-04-24T15:32:56.181 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.181 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.181 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:56.181 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.731045+0000", 2024-04-24T15:32:56.181 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0075259309999999996 2024-04-24T15:32:56.181 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.181 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.181 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:56.181 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.731090+0000", 2024-04-24T15:32:56.181 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4805999999999998e-05 2024-04-24T15:32:56.181 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.181 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.181 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 6", 2024-04-24T15:32:56.181 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.731571+0000", 2024-04-24T15:32:56.182 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00048136400000000002 2024-04-24T15:32:56.182 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.182 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.182 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:56.182 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.736784+0000", 2024-04-24T15:32:56.182 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0052135269999999999 2024-04-24T15:32:56.182 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.182 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.182 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:56.182 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.736865+0000", 2024-04-24T15:32:56.182 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0141000000000004e-05 2024-04-24T15:32:56.182 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.182 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.182 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:56.183 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.736887+0000", 2024-04-24T15:32:56.183 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.2194000000000001e-05 2024-04-24T15:32:56.183 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.183 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.183 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:56.183 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.736904+0000", 2024-04-24T15:32:56.183 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.7302000000000001e-05 2024-04-24T15:32:56.183 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.183 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:56.183 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.183 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.183 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.183 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:3801 4.18 4:1b4f38fa:::rbd_data.114a1f60cb66.000000000000002b:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:56.183 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.723629+0000", 2024-04-24T15:32:56.184 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 31.387755459000001, 2024-04-24T15:32:56.184 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.013474534, 2024-04-24T15:32:56.184 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:56.184 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:56.184 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:56.184 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:56.184 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:56.184 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 3801 2024-04-24T15:32:56.184 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.184 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:56.184 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.184 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:56.184 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723629+0000", 2024-04-24T15:32:56.184 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.185 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.185 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.185 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:56.185 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723629+0000", 2024-04-24T15:32:56.185 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.185 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.185 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.185 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:56.185 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723628+0000", 2024-04-24T15:32:56.185 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:56.185 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.185 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.185 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:56.185 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723632+0000", 2024-04-24T15:32:56.185 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.4690000000000002e-06 2024-04-24T15:32:56.186 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.186 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.186 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:56.186 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723640+0000", 2024-04-24T15:32:56.186 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3990000000000004e-06 2024-04-24T15:32:56.186 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.186 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.186 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:56.186 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.723645+0000", 2024-04-24T15:32:56.186 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3699999999999997e-06 2024-04-24T15:32:56.186 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.186 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.187 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:56.187 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.733996+0000", 2024-04-24T15:32:56.187 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.010351111 2024-04-24T15:32:56.187 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.187 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.187 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:56.187 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.734045+0000", 2024-04-24T15:32:56.187 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.9166000000000003e-05 2024-04-24T15:32:56.187 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.187 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.187 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 6", 2024-04-24T15:32:56.187 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.734461+0000", 2024-04-24T15:32:56.187 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00041584 2024-04-24T15:32:56.187 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.188 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.188 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:56.188 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.736806+0000", 2024-04-24T15:32:56.188 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.002345232 2024-04-24T15:32:56.188 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.188 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.188 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:56.188 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.737065+0000", 2024-04-24T15:32:56.188 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00025913100000000001 2024-04-24T15:32:56.188 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.188 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.188 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:56.188 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.737086+0000", 2024-04-24T15:32:56.188 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.1180000000000001e-05 2024-04-24T15:32:56.189 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.189 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.189 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:56.189 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.737104+0000", 2024-04-24T15:32:56.189 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.7238999999999999e-05 2024-04-24T15:32:56.189 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.189 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:56.189 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.189 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.189 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.189 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:3896 4.1b 4:dd750b71:::rbd_data.114a1f60cb66.0000000000000058:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:56.189 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.797438+0000", 2024-04-24T15:32:56.189 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 31.313946233999999, 2024-04-24T15:32:56.189 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0070008420000000002, 2024-04-24T15:32:56.190 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:56.190 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:56.190 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:56.190 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:56.190 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:56.190 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 3896 2024-04-24T15:32:56.190 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.190 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:56.190 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.190 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:56.190 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797438+0000", 2024-04-24T15:32:56.190 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.190 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.191 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.191 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:56.191 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797438+0000", 2024-04-24T15:32:56.191 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.191 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.191 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.191 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:56.191 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797438+0000", 2024-04-24T15:32:56.191 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:56.191 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.191 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.191 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:56.191 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797441+0000", 2024-04-24T15:32:56.191 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8779999999999998e-06 2024-04-24T15:32:56.191 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.192 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.192 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:56.192 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797447+0000", 2024-04-24T15:32:56.192 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.6989999999999997e-06 2024-04-24T15:32:56.192 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.192 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.192 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:56.192 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797451+0000", 2024-04-24T15:32:56.192 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9689999999999996e-06 2024-04-24T15:32:56.192 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.192 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.192 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:56.192 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797915+0000", 2024-04-24T15:32:56.192 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00046327400000000001 2024-04-24T15:32:56.193 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.193 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.193 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:56.193 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797993+0000", 2024-04-24T15:32:56.193 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.8584000000000005e-05 2024-04-24T15:32:56.193 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.193 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.193 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:56.193 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.798090+0000", 2024-04-24T15:32:56.193 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.6638000000000005e-05 2024-04-24T15:32:56.193 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.193 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.193 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:56.193 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.799021+0000", 2024-04-24T15:32:56.194 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00093112299999999996 2024-04-24T15:32:56.194 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.194 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.194 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:56.194 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.804372+0000", 2024-04-24T15:32:56.194 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0053506789999999997 2024-04-24T15:32:56.194 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.194 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.194 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:56.194 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.804405+0000", 2024-04-24T15:32:56.194 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.3015999999999997e-05 2024-04-24T15:32:56.195 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.195 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.195 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:56.195 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.804439+0000", 2024-04-24T15:32:56.195 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.4431000000000003e-05 2024-04-24T15:32:56.195 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.195 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:56.195 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.195 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.195 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.195 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:3900 4.1a 4:5b94e8b4:::rbd_data.114a1f60cb66.000000000000006e:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:56.195 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.797456+0000", 2024-04-24T15:32:56.196 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 31.313928533999999, 2024-04-24T15:32:56.196 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.006973524, 2024-04-24T15:32:56.196 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:56.196 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:56.196 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:56.196 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:56.196 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:56.196 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 3900 2024-04-24T15:32:56.196 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.196 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:56.196 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.196 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:56.196 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797456+0000", 2024-04-24T15:32:56.196 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.196 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.197 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.197 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:56.197 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797456+0000", 2024-04-24T15:32:56.197 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.197 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.197 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.197 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:56.197 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797456+0000", 2024-04-24T15:32:56.197 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:56.197 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.197 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.197 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:56.197 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797458+0000", 2024-04-24T15:32:56.197 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.3209999999999999e-06 2024-04-24T15:32:56.198 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.198 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.198 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:56.198 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797464+0000", 2024-04-24T15:32:56.198 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.7579999999999998e-06 2024-04-24T15:32:56.198 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.198 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.198 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:56.198 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797467+0000", 2024-04-24T15:32:56.198 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.585e-06 2024-04-24T15:32:56.198 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.199 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.199 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:56.199 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797504+0000", 2024-04-24T15:32:56.199 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.7120999999999999e-05 2024-04-24T15:32:56.199 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.199 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.199 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:56.199 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797578+0000", 2024-04-24T15:32:56.199 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.3693000000000001e-05 2024-04-24T15:32:56.199 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.199 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.199 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:56.200 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.797675+0000", 2024-04-24T15:32:56.200 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.7454000000000006e-05 2024-04-24T15:32:56.200 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.200 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.200 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:56.200 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.799033+0000", 2024-04-24T15:32:56.200 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.001357104 2024-04-24T15:32:56.200 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.200 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.200 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:56.200 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.804357+0000", 2024-04-24T15:32:56.200 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0053247490000000001 2024-04-24T15:32:56.201 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.201 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.201 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:56.201 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.804397+0000", 2024-04-24T15:32:56.201 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9573000000000002e-05 2024-04-24T15:32:56.201 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.201 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.201 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:56.201 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.804429+0000", 2024-04-24T15:32:56.201 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.2529e-05 2024-04-24T15:32:56.201 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.201 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:56.201 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.201 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.201 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.202 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:3923 4.9 4:94be6b6c:::rbd_data.114a1f60cb66.0000000000000097:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:56.202 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.806292+0000", 2024-04-24T15:32:56.202 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 31.305092672000001, 2024-04-24T15:32:56.202 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0062496729999999999, 2024-04-24T15:32:56.202 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:56.202 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:56.202 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:56.202 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:56.202 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:56.202 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 3923 2024-04-24T15:32:56.202 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.202 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:56.203 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.203 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:56.203 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.806292+0000", 2024-04-24T15:32:56.203 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.203 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.203 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.203 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:56.203 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.806292+0000", 2024-04-24T15:32:56.203 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.203 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.203 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.203 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:56.203 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.806291+0000", 2024-04-24T15:32:56.203 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999999 2024-04-24T15:32:56.204 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.204 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.204 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:56.204 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.806296+0000", 2024-04-24T15:32:56.204 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5730000000000001e-06 2024-04-24T15:32:56.204 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.204 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.204 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:56.204 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.806306+0000", 2024-04-24T15:32:56.204 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0409e-05 2024-04-24T15:32:56.204 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.204 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.204 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:56.204 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.806311+0000", 2024-04-24T15:32:56.204 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.7129999999999996e-06 2024-04-24T15:32:56.204 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.205 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.205 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:56.205 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.806350+0000", 2024-04-24T15:32:56.205 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9351999999999999e-05 2024-04-24T15:32:56.205 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.205 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.205 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:56.205 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.806431+0000", 2024-04-24T15:32:56.205 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.1050000000000005e-05 2024-04-24T15:32:56.205 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.205 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.205 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:56.205 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.806539+0000", 2024-04-24T15:32:56.205 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000107723 2024-04-24T15:32:56.205 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.205 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.205 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:56.206 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.809607+0000", 2024-04-24T15:32:56.206 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.003068468 2024-04-24T15:32:56.206 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.206 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.206 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:56.206 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.812493+0000", 2024-04-24T15:32:56.206 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0028862689999999999 2024-04-24T15:32:56.206 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.206 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.206 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:56.206 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.812521+0000", 2024-04-24T15:32:56.206 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7226e-05 2024-04-24T15:32:56.206 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.206 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.207 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:56.207 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.812541+0000", 2024-04-24T15:32:56.207 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.0729000000000001e-05 2024-04-24T15:32:56.207 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.207 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:56.207 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.207 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.207 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.207 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:3975 4.1b 4:d9227084:::rbd_data.114a1f60cb66.000000000000015f:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:56.207 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.827427+0000", 2024-04-24T15:32:56.207 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 31.283957227999998, 2024-04-24T15:32:56.207 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0061324140000000001, 2024-04-24T15:32:56.207 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:56.207 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:56.207 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:56.207 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:56.208 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:56.208 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 3975 2024-04-24T15:32:56.208 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.208 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:56.208 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.208 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:56.208 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.827427+0000", 2024-04-24T15:32:56.208 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.208 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.208 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.208 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:56.208 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.827427+0000", 2024-04-24T15:32:56.208 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.208 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.208 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.208 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:56.209 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.827427+0000", 2024-04-24T15:32:56.209 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:56.209 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.209 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.209 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:56.209 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.827431+0000", 2024-04-24T15:32:56.209 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2509999999999996e-06 2024-04-24T15:32:56.209 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.209 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.209 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:56.209 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.827440+0000", 2024-04-24T15:32:56.209 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.816e-06 2024-04-24T15:32:56.209 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.209 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.209 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:56.210 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.827445+0000", 2024-04-24T15:32:56.210 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.2059999999999998e-06 2024-04-24T15:32:56.210 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.210 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.210 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:56.210 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.827872+0000", 2024-04-24T15:32:56.210 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00042740299999999997 2024-04-24T15:32:56.210 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.210 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.210 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:56.210 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.827941+0000", 2024-04-24T15:32:56.210 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.8652000000000006e-05 2024-04-24T15:32:56.210 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.211 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.211 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:56.211 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.828048+0000", 2024-04-24T15:32:56.211 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000106941 2024-04-24T15:32:56.211 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.211 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.211 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:56.211 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.831351+0000", 2024-04-24T15:32:56.211 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0033031010000000001 2024-04-24T15:32:56.211 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.211 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.211 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:56.211 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.833495+0000", 2024-04-24T15:32:56.211 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0021436379999999998 2024-04-24T15:32:56.211 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.211 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.212 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:56.212 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.833526+0000", 2024-04-24T15:32:56.212 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.1436999999999997e-05 2024-04-24T15:32:56.212 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.212 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.212 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:56.212 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.833560+0000", 2024-04-24T15:32:56.212 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.3560999999999999e-05 2024-04-24T15:32:56.212 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.212 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:56.212 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.212 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.212 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.212 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:4010 4.1 4:813d513a:::rbd_data.114a1f60cb66.00000000000001c1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:56.212 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:24.845178+0000", 2024-04-24T15:32:56.212 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 31.266206879999999, 2024-04-24T15:32:56.213 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.007702108, 2024-04-24T15:32:56.213 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:56.213 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:56.213 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:56.213 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:56.213 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:56.213 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 4010 2024-04-24T15:32:56.213 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.213 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:56.213 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.213 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:56.213 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.845178+0000", 2024-04-24T15:32:56.213 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.214 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.214 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.214 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:56.214 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.845178+0000", 2024-04-24T15:32:56.214 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.214 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.214 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.214 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:56.215 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.845177+0000", 2024-04-24T15:32:56.215 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:56.215 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.215 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.215 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:56.215 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.845180+0000", 2024-04-24T15:32:56.215 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8119999999999999e-06 2024-04-24T15:32:56.215 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.215 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.216 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:56.216 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.845186+0000", 2024-04-24T15:32:56.216 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.9649999999999996e-06 2024-04-24T15:32:56.216 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.216 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.216 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:56.216 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.845189+0000", 2024-04-24T15:32:56.216 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.6040000000000001e-06 2024-04-24T15:32:56.216 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.217 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.217 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:56.217 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.845220+0000", 2024-04-24T15:32:56.217 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.0207000000000001e-05 2024-04-24T15:32:56.217 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.217 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.217 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:56.218 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.845262+0000", 2024-04-24T15:32:56.218 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2716000000000002e-05 2024-04-24T15:32:56.218 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.218 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.218 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 5", 2024-04-24T15:32:56.218 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.845326+0000", 2024-04-24T15:32:56.218 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.3490000000000004e-05 2024-04-24T15:32:56.218 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.219 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.219 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:56.219 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.849620+0000", 2024-04-24T15:32:56.219 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0042944769999999997 2024-04-24T15:32:56.219 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.219 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.219 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:56.219 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.852802+0000", 2024-04-24T15:32:56.220 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.003181798 2024-04-24T15:32:56.220 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.220 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.220 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:56.220 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.852846+0000", 2024-04-24T15:32:56.220 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3606000000000002e-05 2024-04-24T15:32:56.220 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.220 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.220 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:56.221 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:24.852880+0000", 2024-04-24T15:32:56.221 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.3908999999999997e-05 2024-04-24T15:32:56.221 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.221 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:56.221 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.221 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.221 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.221 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:4256 4.13 4:c8361c81:::rbd_data.114a1f60cb66.0000000000000316:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T15:32:56.222 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:25.054726+0000", 2024-04-24T15:32:56.222 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 31.056658435999999, 2024-04-24T15:32:56.222 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0079439720000000005, 2024-04-24T15:32:56.222 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:56.222 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:56.222 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:56.222 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:56.222 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:56.223 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 4256 2024-04-24T15:32:56.223 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.223 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:56.223 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.223 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:56.223 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.054726+0000", 2024-04-24T15:32:56.223 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.223 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.224 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.224 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:56.224 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.054726+0000", 2024-04-24T15:32:56.224 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.224 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.224 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.224 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:56.224 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.054725+0000", 2024-04-24T15:32:56.224 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999999 2024-04-24T15:32:56.225 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.225 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.225 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:56.225 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.054730+0000", 2024-04-24T15:32:56.225 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.4729999999999999e-06 2024-04-24T15:32:56.225 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.225 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.225 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:56.225 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.054742+0000", 2024-04-24T15:32:56.225 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.1489e-05 2024-04-24T15:32:56.226 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.226 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.226 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:56.226 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.054748+0000", 2024-04-24T15:32:56.226 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.879e-06 2024-04-24T15:32:56.226 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.226 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.226 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:56.227 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.054791+0000", 2024-04-24T15:32:56.227 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2673999999999998e-05 2024-04-24T15:32:56.227 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.227 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.227 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:56.227 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.054872+0000", 2024-04-24T15:32:56.227 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.1254999999999994e-05 2024-04-24T15:32:56.227 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.227 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.228 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 6", 2024-04-24T15:32:56.228 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.054978+0000", 2024-04-24T15:32:56.228 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00010624 2024-04-24T15:32:56.228 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.228 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.228 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:56.228 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.055893+0000", 2024-04-24T15:32:56.228 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00091453300000000003 2024-04-24T15:32:56.228 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.229 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.229 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:56.229 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.062620+0000", 2024-04-24T15:32:56.229 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0067271839999999998 2024-04-24T15:32:56.229 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.229 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.229 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:56.229 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.062645+0000", 2024-04-24T15:32:56.229 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.5491e-05 2024-04-24T15:32:56.229 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.229 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.230 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:56.230 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:25.062670+0000", 2024-04-24T15:32:56.230 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.474e-05 2024-04-24T15:32:56.230 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.230 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:56.230 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.230 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.230 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.230 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:7398 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-04-24T15:32:56.230 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:27.771937+0000", 2024-04-24T15:32:56.230 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 28.339447085, 2024-04-24T15:32:56.230 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.015783228, 2024-04-24T15:32:56.230 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:56.231 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:56.231 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:56.231 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:56.231 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:56.231 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 7398 2024-04-24T15:32:56.231 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.231 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:56.231 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.231 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:56.231 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:27.771937+0000", 2024-04-24T15:32:56.231 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.231 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.231 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.231 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:56.231 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:27.771937+0000", 2024-04-24T15:32:56.231 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.232 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.232 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.232 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:56.232 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:27.771936+0000", 2024-04-24T15:32:56.232 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999986 2024-04-24T15:32:56.232 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.232 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.232 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:56.232 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:27.771944+0000", 2024-04-24T15:32:56.232 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.8960000000000003e-06 2024-04-24T15:32:56.232 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.232 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.232 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:56.232 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:27.771955+0000", 2024-04-24T15:32:56.232 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0951000000000001e-05 2024-04-24T15:32:56.232 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.233 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.233 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:56.233 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:27.771959+0000", 2024-04-24T15:32:56.233 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.69e-06 2024-04-24T15:32:56.233 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.233 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.233 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:56.233 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:27.771998+0000", 2024-04-24T15:32:56.233 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8705000000000003e-05 2024-04-24T15:32:56.233 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.233 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.233 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:56.233 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:27.772054+0000", 2024-04-24T15:32:56.233 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.5420000000000001e-05 2024-04-24T15:32:56.233 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.233 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.234 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:56.234 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:27.774882+0000", 2024-04-24T15:32:56.234 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.002828344 2024-04-24T15:32:56.234 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.234 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.234 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:56.234 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:27.787163+0000", 2024-04-24T15:32:56.234 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.012281556000000001 2024-04-24T15:32:56.234 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.234 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.234 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:56.234 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:27.787651+0000", 2024-04-24T15:32:56.235 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00048708600000000002 2024-04-24T15:32:56.235 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.235 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.235 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:56.235 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:27.787691+0000", 2024-04-24T15:32:56.235 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0531999999999998e-05 2024-04-24T15:32:56.235 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.235 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.235 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:56.235 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:27.787721+0000", 2024-04-24T15:32:56.235 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.9518000000000001e-05 2024-04-24T15:32:56.235 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.235 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:56.235 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.235 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.235 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.236 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:7825 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-04-24T15:32:56.236 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:28.311523+0000", 2024-04-24T15:32:56.236 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.799861025999999, 2024-04-24T15:32:56.236 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.019381513, 2024-04-24T15:32:56.236 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:56.236 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:32:56.236 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:56.236 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:56.236 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:56.236 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 7825 2024-04-24T15:32:56.236 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.236 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:56.236 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.236 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:56.236 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:28.311523+0000", 2024-04-24T15:32:56.237 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.237 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.237 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.237 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:56.237 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:28.311523+0000", 2024-04-24T15:32:56.237 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.237 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.237 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.237 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:56.237 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:28.311523+0000", 2024-04-24T15:32:56.237 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:56.237 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.237 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.237 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:56.238 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:28.311527+0000", 2024-04-24T15:32:56.238 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6290000000000001e-06 2024-04-24T15:32:56.238 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.238 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.238 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:56.238 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:28.311538+0000", 2024-04-24T15:32:56.238 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0239e-05 2024-04-24T15:32:56.238 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.238 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.238 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:56.238 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:28.311542+0000", 2024-04-24T15:32:56.238 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3059999999999999e-06 2024-04-24T15:32:56.238 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.239 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.239 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:56.239 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:28.311621+0000", 2024-04-24T15:32:56.239 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.8671999999999995e-05 2024-04-24T15:32:56.239 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.239 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.239 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:56.239 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:28.311667+0000", 2024-04-24T15:32:56.239 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6842999999999999e-05 2024-04-24T15:32:56.239 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.239 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.239 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 3", 2024-04-24T15:32:56.239 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:28.315077+0000", 2024-04-24T15:32:56.239 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0034100889999999998 2024-04-24T15:32:56.240 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.240 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.240 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:32:56.240 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:28.330557+0000", 2024-04-24T15:32:56.240 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.015479100000000001 2024-04-24T15:32:56.240 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.240 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.240 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:32:56.240 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:28.330843+0000", 2024-04-24T15:32:56.240 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000286526 2024-04-24T15:32:56.240 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.240 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.240 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:32:56.241 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:28.330877+0000", 2024-04-24T15:32:56.241 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.4119999999999999e-05 2024-04-24T15:32:56.241 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.241 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.241 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:56.241 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:28.330905+0000", 2024-04-24T15:32:56.241 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7688999999999999e-05 2024-04-24T15:32:56.241 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.241 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:56.241 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.241 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.241 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:56.241 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:32:56.243 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-24T15:32:56.246 INFO:teuthology.orchestra.run.smithi096.stdout:2742 trunc from 0x8711f6b to 0xe9ffd89 2024-04-24T15:32:56.247 INFO:teuthology.orchestra.run.smithi096.stdout:2743 read 0x56005e0 thru 0x5601759 (0x117a bytes) 2024-04-24T15:32:56.247 INFO:teuthology.orchestra.run.smithi096.stdout:2744 writesame 0x688416e thru 0x68917b7 data_size 0x52(0xd64a bytes) 2024-04-24T15:32:56.247 INFO:teuthology.orchestra.run.smithi096.stdout:2746 writesame 0xbcc7688 thru 0xbcd0c93 data_size 0x16(0x960c bytes) 2024-04-24T15:32:56.247 INFO:teuthology.orchestra.run.smithi096.stdout:2748 punch from 0x31fc578 to 0x31fdfc4, (0x1a4c bytes) 2024-04-24T15:32:56.247 INFO:teuthology.orchestra.run.smithi096.stdout:2749 read 0xb734f2f thru 0xb741db0 (0xce82 bytes) 2024-04-24T15:32:56.247 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:56.247 INFO:teuthology.orchestra.run.smithi096.stdout:2751 write 0x926532a thru 0x9274b3e (0xf815 bytes) 2024-04-24T15:32:56.247 INFO:teuthology.orchestra.run.smithi096.stdout:2752 read 0x4abfa46 thru 0x4acb6c6 (0xbc81 bytes) 2024-04-24T15:32:56.247 INFO:teuthology.orchestra.run.smithi096.stdout:2753 writesame 0x7866d5e thru 0x7868b79 data_size 0x4(0x1e1c bytes) 2024-04-24T15:32:56.247 INFO:teuthology.orchestra.run.smithi096.stdout:2755 read 0xaf9b50a thru 0xafa797e (0xc475 bytes) 2024-04-24T15:32:56.247 INFO:teuthology.orchestra.run.smithi096.stdout:2756 writesame 0x2545951 thru 0x2554c05 data_size 0x8b(0xf2b5 bytes) 2024-04-24T15:32:56.247 INFO:teuthology.orchestra.run.smithi096.stdout:2758 trunc from 0xe9ffd89 to 0x457ca61 2024-04-24T15:32:56.247 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:56.247 INFO:teuthology.orchestra.run.smithi096.stdout:2760 read 0x3a601c5 thru 0x3a693be (0x91fa bytes) 2024-04-24T15:32:56.248 INFO:teuthology.orchestra.run.smithi096.stdout:2761 punch from 0x1f5b9fd to 0x1f6238c, (0x698f bytes) 2024-04-24T15:32:56.248 INFO:teuthology.orchestra.run.smithi096.stdout:2765 punch from 0xde6dfa to 0xdf5ef2, (0xf0f8 bytes) 2024-04-24T15:32:56.248 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:56.248 INFO:teuthology.orchestra.run.smithi096.stdout:2768 read 0x36a99ec thru 0x36b78c8 (0xdedd bytes) 2024-04-24T15:32:56.248 INFO:teuthology.orchestra.run.smithi096.stdout:2770 trunc from 0x457ca61 to 0xbf40d67 2024-04-24T15:32:56.248 INFO:teuthology.orchestra.run.smithi096.stdout:2771 write 0xa1191a9 thru 0xa1265e1 (0xd439 bytes) 2024-04-24T15:32:56.248 INFO:teuthology.orchestra.run.smithi096.stdout:2772 write 0xb747a7e thru 0xb74f379 (0x78fc bytes) 2024-04-24T15:32:56.248 INFO:teuthology.orchestra.run.smithi096.stdout:2773 read 0x75a6651 thru 0x75b4ffa (0xe9aa bytes) 2024-04-24T15:32:56.248 INFO:teuthology.orchestra.run.smithi096.stdout:2774 read 0x974c153 thru 0x9759dce (0xdc7c bytes) 2024-04-24T15:32:56.248 INFO:teuthology.orchestra.run.smithi096.stdout:2775 write 0xba60569 thru 0xba6d721 (0xd1b9 bytes) 2024-04-24T15:32:56.248 INFO:teuthology.orchestra.run.smithi096.stdout:2776 write 0x73d374e thru 0x73d7bed (0x44a0 bytes) 2024-04-24T15:32:56.248 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:56.248 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:56.248 INFO:teuthology.orchestra.run.smithi096.stdout:2779 write 0x1e7b65b thru 0x1e81e35 (0x67db bytes) 2024-04-24T15:32:56.248 INFO:teuthology.orchestra.run.smithi096.stdout:2781 punch from 0x417f270 to 0x4181e0f, (0x2b9f bytes) 2024-04-24T15:32:56.248 INFO:teuthology.orchestra.run.smithi096.stdout:2782 read 0xb001a40 thru 0xb009db6 (0x8377 bytes) 2024-04-24T15:32:56.249 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:56.249 INFO:teuthology.orchestra.run.smithi096.stdout:2785 writesame 0x4354c6e thru 0x4363088 data_size 0x11(0xe41b bytes) 2024-04-24T15:32:56.249 INFO:teuthology.orchestra.run.smithi096.stdout:2786 writesame 0xbd2eb92 thru 0xbd376bc data_size 0x1(0x8b2b bytes) 2024-04-24T15:32:56.249 INFO:teuthology.orchestra.run.smithi096.stdout:2787 trunc from 0xbf40d67 to 0x723a5ac 2024-04-24T15:32:56.249 INFO:teuthology.orchestra.run.smithi096.stdout:2790 trunc from 0x723a5ac to 0x9b0284b 2024-04-24T15:32:56.249 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:32:56.243+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:32:56.250 INFO:teuthology.orchestra.run.smithi096.stdout:2792 write 0x269e212 thru 0x26adcf9 (0xfae8 bytes) 2024-04-24T15:32:56.250 INFO:teuthology.orchestra.run.smithi096.stdout:2793 read 0x37a4ed6 thru 0x37adce4 (0x8e0f bytes) 2024-04-24T15:32:56.250 INFO:teuthology.orchestra.run.smithi096.stdout:2794 writesame 0x2b7afa3 thru 0x2b7f6cc data_size 0x1(0x472a bytes) 2024-04-24T15:32:56.254 INFO:teuthology.orchestra.run.smithi096.stdout:2796 writesame 0x5a64ece thru 0x5a701e3 data_size 0x51(0xb316 bytes) 2024-04-24T15:32:56.258 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:56.258 INFO:teuthology.orchestra.run.smithi096.stdout:2798 read 0x8da733a thru 0x8da92ca (0x1f91 bytes) 2024-04-24T15:32:56.258 INFO:teuthology.orchestra.run.smithi096.stdout:2799 write 0x20304b9 thru 0x2035fd8 (0x5b20 bytes) 2024-04-24T15:32:56.259 INFO:teuthology.orchestra.run.smithi096.stdout:2800 read 0x885776e thru 0x8864652 (0xcee5 bytes) 2024-04-24T15:32:56.260 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:56.260 INFO:teuthology.orchestra.run.smithi096.stdout:2803 writesame 0x41099f6 thru 0x4115f15 data_size 0x98(0xc520 bytes) 2024-04-24T15:32:56.265 INFO:teuthology.orchestra.run.smithi096.stdout:2805 read 0x495c51a thru 0x496bc2b (0xf712 bytes) 2024-04-24T15:32:56.265 INFO:teuthology.orchestra.run.smithi096.stdout:2806 read 0x40fead8 thru 0x41072e5 (0x880e bytes) 2024-04-24T15:32:56.274 INFO:teuthology.orchestra.run.smithi096.stdout:2809 write 0xd26094f thru 0xd262bda (0x228c bytes) 2024-04-24T15:32:56.277 INFO:teuthology.orchestra.run.smithi096.stdout:2810 writesame 0xb9bcef5 thru 0xb9bf115 data_size 0x1(0x2221 bytes) 2024-04-24T15:32:56.290 INFO:teuthology.orchestra.run.smithi096.stdout:2811 writesame 0xedbbf4e thru 0xedbd618 data_size 0x5(0x16cb bytes) 2024-04-24T15:32:56.290 INFO:teuthology.orchestra.run.smithi096.stdout:2812 writesame 0x105b8 thru 0x1d699 data_size 0x1(0xd0e2 bytes) 2024-04-24T15:32:56.325 INFO:teuthology.orchestra.run.smithi096.stdout:2813 writesame 0x8bc983d thru 0x8bd1d3a data_size 0x2(0x84fe bytes) 2024-04-24T15:32:56.334 INFO:teuthology.orchestra.run.smithi096.stdout:2814 read 0x4407f58 thru 0x44135f2 (0xb69b bytes) 2024-04-24T15:32:56.335 INFO:teuthology.orchestra.run.smithi096.stdout:2815 punch from 0x3f12f6 to 0x3f69ed, (0x56f7 bytes) 2024-04-24T15:32:56.335 INFO:teuthology.orchestra.run.smithi096.stdout:2816 writesame 0x64648f3 thru 0x6465d32 data_size 0x48(0x1440 bytes) 2024-04-24T15:32:56.338 INFO:teuthology.orchestra.run.smithi096.stdout:2817 punch from 0x36c9419 to 0x36d1172, (0x7d59 bytes) 2024-04-24T15:32:56.338 INFO:teuthology.orchestra.run.smithi096.stdout:2818 read 0x85aa1d9 thru 0x85aae73 (0xc9b bytes) 2024-04-24T15:32:56.338 INFO:teuthology.orchestra.run.smithi096.stdout:2819 read 0x44a30a2 thru 0x44b27df (0xf73e bytes) 2024-04-24T15:32:56.338 INFO:teuthology.orchestra.run.smithi096.stdout:2822 write 0xb7d7978 thru 0xb7e46d1 (0xcd5a bytes) 2024-04-24T15:32:56.343 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:32:56.344+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:32:56.350 INFO:teuthology.orchestra.run.smithi096.stdout:2823 trunc from 0xedbd619 to 0x703149d 2024-04-24T15:32:56.366 INFO:teuthology.orchestra.run.smithi096.stdout:2824 writesame 0x81aa9c7 thru 0x81b89d6 data_size 0x30(0xe010 bytes) 2024-04-24T15:32:56.366 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:56.366 INFO:teuthology.orchestra.run.smithi096.stdout:2827 read 0x9c0dc3 thru 0x9cf9c6 (0xec04 bytes) 2024-04-24T15:32:56.366 INFO:teuthology.orchestra.run.smithi096.stdout:2828 read 0x81051b5 thru 0x810a414 (0x5260 bytes) 2024-04-24T15:32:56.374 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:32:56.374 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:32:56.374 INFO:teuthology.orchestra.run.smithi151.stdout: "num_ops": 0 2024-04-24T15:32:56.374 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:32:56.378 INFO:teuthology.orchestra.run.smithi096.stdout:2830 writesame 0xe165252 thru 0xe16e91e data_size 0x7(0x96cd bytes) 2024-04-24T15:32:56.378 INFO:teuthology.orchestra.run.smithi096.stdout:2831 writesame 0x20f280c thru 0x20ffcfb data_size 0x8(0xd4f0 bytes) 2024-04-24T15:32:56.382 INFO:teuthology.orchestra.run.smithi096.stdout:2832 write 0x71d7828 thru 0x71de84e (0x7027 bytes) 2024-04-24T15:32:56.384 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-24T15:32:56.385 INFO:teuthology.orchestra.run.smithi096.stdout:2834 read 0x54ebf73 thru 0x54f3773 (0x7801 bytes) 2024-04-24T15:32:56.386 INFO:teuthology.orchestra.run.smithi096.stdout:2836 read 0x16d2295 thru 0x16de639 (0xc3a5 bytes) 2024-04-24T15:32:56.386 INFO:teuthology.orchestra.run.smithi096.stdout:2837 read 0x731304f thru 0x7317de5 (0x4d97 bytes) 2024-04-24T15:32:56.391 INFO:teuthology.orchestra.run.smithi096.stdout:2838 writesame 0xe6da2c0 thru 0xe6dade1 data_size 0x19(0xb22 bytes) 2024-04-24T15:32:56.391 INFO:teuthology.orchestra.run.smithi096.stdout:2839 write 0xd8555ac thru 0xd85831d (0x2d72 bytes) 2024-04-24T15:32:56.394 INFO:teuthology.orchestra.run.smithi096.stdout:2840 punch from 0xa71f934 to 0xa72dc0c, (0xe2d8 bytes) 2024-04-24T15:32:56.394 INFO:teuthology.orchestra.run.smithi096.stdout:2842 write 0x36c524e thru 0x36c77c8 (0x257b bytes) 2024-04-24T15:32:56.396 INFO:teuthology.orchestra.run.smithi096.stdout:2844 read 0x7a1f90e thru 0x7a250ec (0x57df bytes) 2024-04-24T15:32:56.396 INFO:teuthology.orchestra.run.smithi096.stdout:2845 read 0x5e7558c thru 0x5e7db23 (0x8598 bytes) 2024-04-24T15:32:56.396 INFO:teuthology.orchestra.run.smithi096.stdout:2846 punch from 0x1ecab6b to 0x1ed1ea3, (0x7338 bytes) 2024-04-24T15:32:56.396 INFO:teuthology.orchestra.run.smithi096.stdout:2849 write 0x427700d thru 0x4278623 (0x1617 bytes) 2024-04-24T15:32:56.404 INFO:teuthology.orchestra.run.smithi096.stdout:2850 writesame 0xebef873 thru 0xebf3ba4 data_size 0x5e(0x4332 bytes) 2024-04-24T15:32:56.418 INFO:teuthology.orchestra.run.smithi096.stdout:2853 trunc from 0xebf3ba5 to 0x3a6cb1b 2024-04-24T15:32:56.440 INFO:teuthology.orchestra.run.smithi096.stdout:2855 write 0x4eb1cd9 thru 0x4ec11de (0xf506 bytes) 2024-04-24T15:32:56.445 INFO:teuthology.orchestra.run.smithi096.stdout:2856 read 0x3f8e2cd thru 0x3f9213a (0x3e6e bytes) 2024-04-24T15:32:56.445 INFO:teuthology.orchestra.run.smithi096.stdout:2857 read 0x4320e83 thru 0x4328556 (0x76d4 bytes) 2024-04-24T15:32:56.452 INFO:teuthology.orchestra.run.smithi096.stdout:2858 trunc from 0x4ec11df to 0xa87adc9 2024-04-24T15:32:56.456 INFO:teuthology.orchestra.run.smithi096.stdout:2859 read 0x47b9c7e thru 0x47c9073 (0xf3f6 bytes) 2024-04-24T15:32:56.456 INFO:teuthology.orchestra.run.smithi096.stdout:2861 punch from 0x7b425c6 to 0x7b4b8bd, (0x92f7 bytes) 2024-04-24T15:32:56.460 INFO:teuthology.orchestra.run.smithi096.stdout:2865 write 0xa953771 thru 0xa9547a6 (0x1036 bytes) 2024-04-24T15:32:56.462 INFO:teuthology.orchestra.run.smithi096.stdout:2866 writesame 0x55fb788 thru 0x5605e1f data_size 0xc(0xa698 bytes) 2024-04-24T15:32:56.466 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:56.467 INFO:teuthology.orchestra.run.smithi096.stdout:2868 trunc from 0xa9547a7 to 0x9fd5995 2024-04-24T15:32:56.467 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:32:56.468+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:32:56.477 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:56.480 INFO:teuthology.orchestra.run.smithi096.stdout:2870 trunc from 0x9fd5995 to 0xbf6a71a 2024-04-24T15:32:56.486 INFO:teuthology.orchestra.run.smithi096.stdout:2871 writesame 0x7bb2734 thru 0x7bbff42 data_size 0x3(0xd80f bytes) 2024-04-24T15:32:56.496 INFO:teuthology.orchestra.run.smithi096.stdout:2872 write 0x1c72b49 thru 0x1c756d0 (0x2b88 bytes) 2024-04-24T15:32:56.502 INFO:teuthology.orchestra.run.smithi096.stdout:2873 trunc from 0xbf6a71a to 0xe23e492 2024-04-24T15:32:56.506 INFO:teuthology.orchestra.run.smithi096.stdout:2876 write 0x2a3b736 thru 0x2a4a2e2 (0xebad bytes) 2024-04-24T15:32:56.510 INFO:teuthology.orchestra.run.smithi096.stdout:2877 read 0x77577ed thru 0x77648a2 (0xd0b6 bytes) 2024-04-24T15:32:56.510 INFO:teuthology.orchestra.run.smithi096.stdout:2878 read 0xbc5c4f2 thru 0xbc65eea (0x99f9 bytes) 2024-04-24T15:32:56.510 INFO:teuthology.orchestra.run.smithi096.stdout:2879 writesame 0x3e93c2e thru 0x3e987b0 data_size 0x1(0x4b83 bytes) 2024-04-24T15:32:56.521 INFO:teuthology.orchestra.run.smithi096.stdout:2882 write 0xab71fb1 thru 0xab7964b (0x769b bytes) 2024-04-24T15:32:56.524 INFO:teuthology.orchestra.run.smithi096.stdout:2884 punch from 0x96511d to 0x96d273, (0x8156 bytes) 2024-04-24T15:32:56.524 INFO:teuthology.orchestra.run.smithi096.stdout:2885 write 0xb6a923d thru 0xb6b2b81 (0x9945 bytes) 2024-04-24T15:32:56.545 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:32:56.546+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:32:56.548 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:32:56.548 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:32:56.548 INFO:teuthology.orchestra.run.smithi151.stdout: "complaint_time": 30, 2024-04-24T15:32:56.548 INFO:teuthology.orchestra.run.smithi151.stdout: "num_blocked_ops": 0 2024-04-24T15:32:56.548 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:32:56.548 INFO:teuthology.orchestra.run.smithi096.stdout:2886 trunc from 0xe23e492 to 0x2ffd9f3 2024-04-24T15:32:56.559 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-24T15:32:56.559 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:56.571 INFO:teuthology.orchestra.run.smithi096.stdout:2890 writesame 0x946104d thru 0x946c1d3 data_size 0x1(0xb187 bytes) 2024-04-24T15:32:56.571 INFO:teuthology.orchestra.run.smithi096.stdout:2891 read 0x82b2fb9 thru 0x82b4f68 (0x1fb0 bytes) 2024-04-24T15:32:56.581 INFO:teuthology.orchestra.run.smithi096.stdout:2894 trunc from 0x946c1d4 to 0x9766e9 2024-04-24T15:32:56.618 INFO:teuthology.orchestra.run.smithi096.stdout:2896 writesame 0x1e18a42 thru 0x1e23c3d data_size 0x6(0xb1fc bytes) 2024-04-24T15:32:56.630 INFO:teuthology.orchestra.run.smithi096.stdout:2897 write 0x7b63826 thru 0x7b6f384 (0xbb5f bytes) 2024-04-24T15:32:56.636 INFO:teuthology.orchestra.run.smithi096.stdout:2898 trunc from 0x7b6f385 to 0xa0caf24 2024-04-24T15:32:56.641 INFO:teuthology.orchestra.run.smithi096.stdout:2899 write 0x3580b61 thru 0x3582e8d (0x232d bytes) 2024-04-24T15:32:56.643 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:56.647 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:32:56.647+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:32:56.649 INFO:teuthology.orchestra.run.smithi096.stdout:2902 trunc from 0xa0caf24 to 0xecb92c5 2024-04-24T15:32:56.655 INFO:teuthology.orchestra.run.smithi096.stdout:2903 read 0x14a1b43 thru 0x14a5170 (0x362e bytes) 2024-04-24T15:32:56.655 INFO:teuthology.orchestra.run.smithi096.stdout:2905 punch from 0x31e1e16 to 0x31e97f1, (0x79db bytes) 2024-04-24T15:32:56.655 INFO:teuthology.orchestra.run.smithi096.stdout:2907 punch from 0xd2e7b79 to 0xd2f64bd, (0xe944 bytes) 2024-04-24T15:32:56.655 INFO:teuthology.orchestra.run.smithi096.stdout:2909 writesame 0xa18c089 thru 0xa19ab47 data_size 0x65(0xeabf bytes) 2024-04-24T15:32:56.661 INFO:teuthology.orchestra.run.smithi096.stdout:2910 write 0x8649610 thru 0x864f191 (0x5b82 bytes) 2024-04-24T15:32:56.665 INFO:teuthology.orchestra.run.smithi096.stdout:2911 writesame 0x9cfe0de thru 0x9d0bcf4 data_size 0x7(0xdc17 bytes) 2024-04-24T15:32:56.671 INFO:teuthology.orchestra.run.smithi096.stdout:2912 punch from 0xe13b9f to 0xe1f394, (0xb7f5 bytes) 2024-04-24T15:32:56.671 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:56.677 INFO:teuthology.orchestra.run.smithi096.stdout:2914 trunc from 0xecb92c5 to 0x8f57492 2024-04-24T15:32:56.687 INFO:teuthology.orchestra.run.smithi096.stdout:2915 punch from 0x79042ad to 0x7905883, (0x15d6 bytes) 2024-04-24T15:32:56.687 INFO:teuthology.orchestra.run.smithi096.stdout:2916 write 0x505f40e thru 0x5063afd (0x46f0 bytes) 2024-04-24T15:32:56.690 INFO:teuthology.orchestra.run.smithi096.stdout:2917 read 0x7ae287c thru 0x7aedd47 (0xb4cc bytes) 2024-04-24T15:32:56.698 INFO:teuthology.orchestra.run.smithi096.stdout:2918 write 0xbb72035 thru 0xbb7341a (0x13e6 bytes) 2024-04-24T15:32:56.700 INFO:teuthology.orchestra.run.smithi096.stdout:2919 write 0x9cd6518 thru 0x9cda887 (0x4370 bytes) 2024-04-24T15:32:56.702 INFO:teuthology.orchestra.run.smithi096.stdout:2920 read 0xad7f205 thru 0xad87da4 (0x8ba0 bytes) 2024-04-24T15:32:56.710 INFO:teuthology.orchestra.run.smithi096.stdout:2921 trunc from 0xbb7341b to 0x5c440cc 2024-04-24T15:32:56.722 INFO:teuthology.orchestra.run.smithi096.stderr:nodeep-scrub is set 2024-04-24T15:32:56.727 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:32:56.727 INFO:teuthology.orchestra.run.smithi151.stdout: "size": 20, 2024-04-24T15:32:56.727 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 600, 2024-04-24T15:32:56.727 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [ 2024-04-24T15:32:56.727 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.727 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23613 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 14688256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:56.727 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.792522+0000", 2024-04-24T15:32:56.728 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 21.934752243999998, 2024-04-24T15:32:56.728 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040507356000000001, 2024-04-24T15:32:56.728 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:56.728 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:56.728 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:56.728 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:56.728 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:56.728 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23613 2024-04-24T15:32:56.728 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.728 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:56.728 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.728 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:56.728 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792522+0000", 2024-04-24T15:32:56.728 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.729 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.729 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.729 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:56.729 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792522+0000", 2024-04-24T15:32:56.729 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.729 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.729 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.729 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:56.729 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792522+0000", 2024-04-24T15:32:56.729 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:56.729 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.729 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.729 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:56.729 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792523+0000", 2024-04-24T15:32:56.729 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.576e-06 2024-04-24T15:32:56.730 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.730 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.730 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:56.730 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792526+0000", 2024-04-24T15:32:56.730 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7439999999999999e-06 2024-04-24T15:32:56.730 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.730 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.730 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:56.730 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792528+0000", 2024-04-24T15:32:56.730 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.0600000000000002e-06 2024-04-24T15:32:56.730 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.730 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.730 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:56.730 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.832864+0000", 2024-04-24T15:32:56.731 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040335742000000001 2024-04-24T15:32:56.731 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.731 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.731 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:56.731 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.832953+0000", 2024-04-24T15:32:56.731 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.9281000000000001e-05 2024-04-24T15:32:56.731 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.731 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.731 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:56.731 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.833029+0000", 2024-04-24T15:32:56.731 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.6259000000000003e-05 2024-04-24T15:32:56.731 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.731 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:56.731 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.731 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.732 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.732 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23614 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 14729216~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:56.732 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.792530+0000", 2024-04-24T15:32:56.732 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 21.93474398, 2024-04-24T15:32:56.732 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040633957999999998, 2024-04-24T15:32:56.732 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:56.732 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:56.732 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:56.732 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:56.732 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:56.732 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23614 2024-04-24T15:32:56.732 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.732 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:56.732 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.732 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:56.732 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792530+0000", 2024-04-24T15:32:56.732 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.733 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.733 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.733 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:56.733 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792530+0000", 2024-04-24T15:32:56.733 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.733 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.733 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.733 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:56.733 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792530+0000", 2024-04-24T15:32:56.733 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:32:56.733 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.733 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.733 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:56.733 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792531+0000", 2024-04-24T15:32:56.733 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.46e-06 2024-04-24T15:32:56.733 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.734 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.734 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:56.734 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792534+0000", 2024-04-24T15:32:56.734 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8959999999999999e-06 2024-04-24T15:32:56.734 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.734 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.734 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:56.734 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792536+0000", 2024-04-24T15:32:56.734 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.841e-06 2024-04-24T15:32:56.734 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.734 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.734 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:56.734 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.833060+0000", 2024-04-24T15:32:56.734 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040524125000000001 2024-04-24T15:32:56.735 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.735 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.735 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:56.735 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.833108+0000", 2024-04-24T15:32:56.735 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.7547999999999999e-05 2024-04-24T15:32:56.735 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.735 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.735 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:56.735 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.833164+0000", 2024-04-24T15:32:56.735 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6297e-05 2024-04-24T15:32:56.735 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.735 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:56.735 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.735 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.735 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.735 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23623 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15097856~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:56.735 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.793887+0000", 2024-04-24T15:32:56.736 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 21.933387019000001, 2024-04-24T15:32:56.736 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040432784999999999, 2024-04-24T15:32:56.736 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:56.736 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:56.736 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:56.736 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:56.736 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:56.736 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23623 2024-04-24T15:32:56.736 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.736 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:56.736 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.736 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:56.736 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793887+0000", 2024-04-24T15:32:56.736 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.736 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.736 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.737 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:56.737 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793887+0000", 2024-04-24T15:32:56.737 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.737 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.737 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.737 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:56.737 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793887+0000", 2024-04-24T15:32:56.737 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:32:56.737 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.737 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.737 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:56.737 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793888+0000", 2024-04-24T15:32:56.737 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3659999999999999e-06 2024-04-24T15:32:56.737 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.737 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.737 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:56.737 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793891+0000", 2024-04-24T15:32:56.738 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8389999999999998e-06 2024-04-24T15:32:56.738 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.738 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.738 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:56.738 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793893+0000", 2024-04-24T15:32:56.738 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.9549999999999999e-06 2024-04-24T15:32:56.738 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.738 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.738 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:56.738 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834239+0000", 2024-04-24T15:32:56.738 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040345487999999999 2024-04-24T15:32:56.738 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.738 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.738 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:56.739 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834280+0000", 2024-04-24T15:32:56.739 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.1878000000000001e-05 2024-04-24T15:32:56.739 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.739 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.739 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:56.739 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834320+0000", 2024-04-24T15:32:56.739 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9480000000000001e-05 2024-04-24T15:32:56.739 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.739 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:56.739 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.739 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.739 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.739 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23624 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15138816~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:56.739 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.794053+0000", 2024-04-24T15:32:56.739 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 21.933221070999998, 2024-04-24T15:32:56.739 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040370801999999997, 2024-04-24T15:32:56.740 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:56.740 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:56.740 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:56.740 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:56.740 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:56.740 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23624 2024-04-24T15:32:56.740 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.740 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:56.740 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.740 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:56.740 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794053+0000", 2024-04-24T15:32:56.740 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.740 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.740 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.740 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:56.741 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794053+0000", 2024-04-24T15:32:56.741 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.741 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.741 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.741 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:56.741 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793895+0000", 2024-04-24T15:32:56.741 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998422 2024-04-24T15:32:56.741 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.741 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.741 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:56.741 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794057+0000", 2024-04-24T15:32:56.741 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00016149400000000001 2024-04-24T15:32:56.741 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.741 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.741 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:56.742 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794065+0000", 2024-04-24T15:32:56.742 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.5259999999999993e-06 2024-04-24T15:32:56.742 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.742 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.742 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:56.742 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794071+0000", 2024-04-24T15:32:56.742 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6149999999999996e-06 2024-04-24T15:32:56.742 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.742 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.742 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:56.742 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834341+0000", 2024-04-24T15:32:56.742 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040270366000000002 2024-04-24T15:32:56.742 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.742 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.743 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:56.743 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834381+0000", 2024-04-24T15:32:56.743 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0102999999999999e-05 2024-04-24T15:32:56.743 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.743 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.743 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:56.743 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834424+0000", 2024-04-24T15:32:56.743 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2713000000000003e-05 2024-04-24T15:32:56.743 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.743 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:56.743 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.743 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.743 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.743 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23625 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15179776~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:56.744 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.794160+0000", 2024-04-24T15:32:56.744 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 21.933114189000001, 2024-04-24T15:32:56.744 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040413004000000002, 2024-04-24T15:32:56.744 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:56.744 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:56.744 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:56.744 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:56.744 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:56.744 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23625 2024-04-24T15:32:56.744 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.744 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:56.744 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.744 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:56.744 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794160+0000", 2024-04-24T15:32:56.745 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.745 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.745 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.745 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:56.745 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794160+0000", 2024-04-24T15:32:56.745 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.745 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.745 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.745 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:56.745 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794086+0000", 2024-04-24T15:32:56.745 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999256 2024-04-24T15:32:56.745 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.745 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.745 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:56.745 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794163+0000", 2024-04-24T15:32:56.746 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7896000000000001e-05 2024-04-24T15:32:56.746 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.746 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.746 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:56.746 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794171+0000", 2024-04-24T15:32:56.746 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.9500000000000001e-06 2024-04-24T15:32:56.746 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.746 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.746 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:56.746 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794179+0000", 2024-04-24T15:32:56.746 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7970000000000001e-06 2024-04-24T15:32:56.746 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.746 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.746 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:56.747 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834451+0000", 2024-04-24T15:32:56.747 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040271492999999998 2024-04-24T15:32:56.747 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.747 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.747 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:56.747 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834519+0000", 2024-04-24T15:32:56.747 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.7984000000000005e-05 2024-04-24T15:32:56.747 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.747 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.747 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:56.747 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834573+0000", 2024-04-24T15:32:56.747 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.4339999999999998e-05 2024-04-24T15:32:56.747 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.747 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:56.748 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.748 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.748 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.748 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23629 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15343616~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:56.748 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.794748+0000", 2024-04-24T15:32:56.748 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 21.932526386999999, 2024-04-24T15:32:56.748 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040419718, 2024-04-24T15:32:56.748 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:56.748 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:56.748 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:56.748 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:56.748 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:56.748 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23629 2024-04-24T15:32:56.748 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.748 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:56.749 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.749 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:56.749 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794748+0000", 2024-04-24T15:32:56.749 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.749 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.749 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.749 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:56.749 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794748+0000", 2024-04-24T15:32:56.749 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.749 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.749 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.749 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:56.749 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794673+0000", 2024-04-24T15:32:56.749 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999251 2024-04-24T15:32:56.749 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.749 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.750 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:56.750 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794751+0000", 2024-04-24T15:32:56.750 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.8480999999999996e-05 2024-04-24T15:32:56.750 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.750 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.750 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:56.750 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794760+0000", 2024-04-24T15:32:56.750 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3429999999999996e-06 2024-04-24T15:32:56.750 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.750 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.750 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:56.750 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794764+0000", 2024-04-24T15:32:56.750 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5600000000000004e-06 2024-04-24T15:32:56.751 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.751 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.751 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:56.751 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835040+0000", 2024-04-24T15:32:56.751 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040275188000000003 2024-04-24T15:32:56.751 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.751 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.751 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:56.751 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835105+0000", 2024-04-24T15:32:56.751 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.5992000000000001e-05 2024-04-24T15:32:56.751 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.751 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.751 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:56.751 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835167+0000", 2024-04-24T15:32:56.751 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.1981000000000002e-05 2024-04-24T15:32:56.751 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.752 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:56.752 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.752 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.752 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.752 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23630 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15384576~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:56.752 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.794839+0000", 2024-04-24T15:32:56.752 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 21.932435318, 2024-04-24T15:32:56.752 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040523894999999997, 2024-04-24T15:32:56.752 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:56.752 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:56.752 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:56.752 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:56.752 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:56.752 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23630 2024-04-24T15:32:56.752 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.752 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:56.753 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.753 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:56.753 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794839+0000", 2024-04-24T15:32:56.753 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.753 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.753 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.753 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:56.753 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794839+0000", 2024-04-24T15:32:56.753 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.753 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.753 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.753 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:56.753 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794768+0000", 2024-04-24T15:32:56.753 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999294 2024-04-24T15:32:56.753 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.754 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.754 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:56.754 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794841+0000", 2024-04-24T15:32:56.754 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2793000000000006e-05 2024-04-24T15:32:56.754 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.754 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.754 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:56.754 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794846+0000", 2024-04-24T15:32:56.754 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.7230000000000004e-06 2024-04-24T15:32:56.754 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.754 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.754 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:56.754 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794848+0000", 2024-04-24T15:32:56.754 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7659999999999999e-06 2024-04-24T15:32:56.755 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.755 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.755 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:56.755 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835223+0000", 2024-04-24T15:32:56.755 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040374275000000001 2024-04-24T15:32:56.755 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.755 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.755 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:56.755 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835295+0000", 2024-04-24T15:32:56.755 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2633999999999996e-05 2024-04-24T15:32:56.755 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.755 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.755 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:56.755 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835363+0000", 2024-04-24T15:32:56.755 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.7355999999999997e-05 2024-04-24T15:32:56.755 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.756 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:56.756 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.756 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.756 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.756 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23631 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15425536~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:56.756 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.794956+0000", 2024-04-24T15:32:56.756 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 21.932318476999999, 2024-04-24T15:32:56.756 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040562383, 2024-04-24T15:32:56.756 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:56.756 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:56.756 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:56.756 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:56.756 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:56.756 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23631 2024-04-24T15:32:56.756 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.757 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:56.757 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.757 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:56.757 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794956+0000", 2024-04-24T15:32:56.757 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.757 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.757 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.757 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:56.757 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794956+0000", 2024-04-24T15:32:56.757 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.757 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.757 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.757 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:56.757 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794859+0000", 2024-04-24T15:32:56.758 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999032 2024-04-24T15:32:56.758 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.758 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.758 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:56.758 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794959+0000", 2024-04-24T15:32:56.758 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000100207 2024-04-24T15:32:56.758 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.758 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.758 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:56.758 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794967+0000", 2024-04-24T15:32:56.758 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7640000000000005e-06 2024-04-24T15:32:56.758 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.758 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.758 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:56.759 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794971+0000", 2024-04-24T15:32:56.759 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.335e-06 2024-04-24T15:32:56.759 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.759 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.759 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:56.759 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835388+0000", 2024-04-24T15:32:56.759 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040417340000000003 2024-04-24T15:32:56.759 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.759 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.759 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:56.759 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835459+0000", 2024-04-24T15:32:56.759 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.0118999999999997e-05 2024-04-24T15:32:56.759 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.759 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.759 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:56.759 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835518+0000", 2024-04-24T15:32:56.760 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.9438999999999998e-05 2024-04-24T15:32:56.760 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.760 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:56.760 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.760 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.760 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.760 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23632 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15466496~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:56.760 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795060+0000", 2024-04-24T15:32:56.760 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 21.932213767, 2024-04-24T15:32:56.760 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040667993, 2024-04-24T15:32:56.760 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:56.760 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:56.760 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:56.760 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:56.760 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:56.760 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23632 2024-04-24T15:32:56.761 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.761 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:56.761 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.761 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:56.761 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795060+0000", 2024-04-24T15:32:56.761 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.761 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.761 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.761 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:56.761 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795060+0000", 2024-04-24T15:32:56.761 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.761 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.761 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.761 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:56.761 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794976+0000", 2024-04-24T15:32:56.761 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999156 2024-04-24T15:32:56.762 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.762 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.762 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:56.762 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795064+0000", 2024-04-24T15:32:56.762 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.7709999999999999e-05 2024-04-24T15:32:56.762 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.762 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.762 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:56.762 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795072+0000", 2024-04-24T15:32:56.762 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3769999999999996e-06 2024-04-24T15:32:56.762 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.762 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.762 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:56.762 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795078+0000", 2024-04-24T15:32:56.762 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.5079999999999996e-06 2024-04-24T15:32:56.763 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.763 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.763 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:56.763 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835541+0000", 2024-04-24T15:32:56.763 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040463440000000003 2024-04-24T15:32:56.763 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.763 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.763 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:56.763 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835657+0000", 2024-04-24T15:32:56.763 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00011572499999999999 2024-04-24T15:32:56.763 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.763 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.763 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:56.763 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835728+0000", 2024-04-24T15:32:56.764 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1402000000000005e-05 2024-04-24T15:32:56.764 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.764 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:56.764 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.764 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.764 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.764 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23633 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15507456~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:56.764 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795134+0000", 2024-04-24T15:32:56.764 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 21.932140111999999, 2024-04-24T15:32:56.764 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040748591000000001, 2024-04-24T15:32:56.764 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:56.764 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:56.764 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:56.764 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:56.764 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:56.764 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23633 2024-04-24T15:32:56.765 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.765 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:56.765 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.765 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:56.765 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795134+0000", 2024-04-24T15:32:56.765 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.765 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.765 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.765 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:56.765 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795134+0000", 2024-04-24T15:32:56.765 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.765 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.765 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.765 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:56.765 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795083+0000", 2024-04-24T15:32:56.766 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999949 2024-04-24T15:32:56.766 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.766 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.766 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:56.766 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795138+0000", 2024-04-24T15:32:56.766 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.4472000000000003e-05 2024-04-24T15:32:56.766 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.766 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.766 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:56.766 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795145+0000", 2024-04-24T15:32:56.766 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2779999999999996e-06 2024-04-24T15:32:56.766 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.766 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.766 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:56.767 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795149+0000", 2024-04-24T15:32:56.767 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.568e-06 2024-04-24T15:32:56.767 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.767 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.767 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:56.767 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835757+0000", 2024-04-24T15:32:56.767 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040608894999999999 2024-04-24T15:32:56.767 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.767 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.767 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:56.767 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835829+0000", 2024-04-24T15:32:56.767 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1879999999999996e-05 2024-04-24T15:32:56.767 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.767 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.767 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:56.767 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835883+0000", 2024-04-24T15:32:56.768 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.3346000000000001e-05 2024-04-24T15:32:56.768 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.768 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:56.768 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.768 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.768 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.768 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23634 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15548416~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:56.768 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795271+0000", 2024-04-24T15:32:56.768 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 21.932003078000001, 2024-04-24T15:32:56.768 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040755845999999998, 2024-04-24T15:32:56.768 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:56.768 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:56.768 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:56.768 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:56.768 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:56.768 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23634 2024-04-24T15:32:56.769 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.769 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:56.769 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.769 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:56.769 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795271+0000", 2024-04-24T15:32:56.769 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.769 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.769 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.769 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:56.769 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795271+0000", 2024-04-24T15:32:56.769 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.769 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.769 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.769 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:56.769 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795152+0000", 2024-04-24T15:32:56.770 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998813 2024-04-24T15:32:56.770 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.770 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.770 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:56.770 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795276+0000", 2024-04-24T15:32:56.770 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00012401799999999999 2024-04-24T15:32:56.770 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.770 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.770 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:56.770 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795281+0000", 2024-04-24T15:32:56.770 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8860000000000003e-06 2024-04-24T15:32:56.770 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.770 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.770 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:56.771 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795284+0000", 2024-04-24T15:32:56.771 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.0450000000000001e-06 2024-04-24T15:32:56.771 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.771 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.771 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:56.771 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835905+0000", 2024-04-24T15:32:56.771 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040621035999999999 2024-04-24T15:32:56.771 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.771 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.771 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:56.771 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835971+0000", 2024-04-24T15:32:56.771 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.5314000000000005e-05 2024-04-24T15:32:56.771 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.771 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.771 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:56.771 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836027+0000", 2024-04-24T15:32:56.772 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6305999999999999e-05 2024-04-24T15:32:56.772 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.772 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:56.772 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.772 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.772 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.772 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23635 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15589376~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:56.772 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795382+0000", 2024-04-24T15:32:56.772 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 21.931892305000002, 2024-04-24T15:32:56.772 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040805363999999997, 2024-04-24T15:32:56.772 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:56.772 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:56.772 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:56.772 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:56.772 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:56.772 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23635 2024-04-24T15:32:56.773 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.773 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:56.773 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.773 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:56.773 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795382+0000", 2024-04-24T15:32:56.773 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.773 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.773 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.773 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:56.773 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795382+0000", 2024-04-24T15:32:56.773 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.773 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.773 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.773 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:56.773 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795288+0000", 2024-04-24T15:32:56.774 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999056 2024-04-24T15:32:56.774 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.774 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.774 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:56.774 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795385+0000", 2024-04-24T15:32:56.774 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.7288999999999996e-05 2024-04-24T15:32:56.774 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.774 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.774 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:56.774 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795392+0000", 2024-04-24T15:32:56.774 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.452e-06 2024-04-24T15:32:56.774 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.774 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.774 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:56.774 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795397+0000", 2024-04-24T15:32:56.774 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2860000000000001e-06 2024-04-24T15:32:56.775 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.775 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.775 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:56.775 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836054+0000", 2024-04-24T15:32:56.775 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040657581999999998 2024-04-24T15:32:56.775 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.775 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.775 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:56.775 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836124+0000", 2024-04-24T15:32:56.775 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.0020999999999999e-05 2024-04-24T15:32:56.775 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.775 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.775 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:56.776 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836187+0000", 2024-04-24T15:32:56.776 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.3009999999999995e-05 2024-04-24T15:32:56.776 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.776 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:56.776 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.776 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.776 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.776 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23636 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15630336~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:56.776 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795610+0000", 2024-04-24T15:32:56.776 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 21.931663947000001, 2024-04-24T15:32:56.776 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040730942999999999, 2024-04-24T15:32:56.776 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:56.776 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:56.776 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:56.776 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:56.776 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:56.777 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23636 2024-04-24T15:32:56.777 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.777 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:56.777 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.777 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:56.777 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795610+0000", 2024-04-24T15:32:56.777 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.777 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.777 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.777 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:56.777 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795610+0000", 2024-04-24T15:32:56.778 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.778 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.778 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.778 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:56.778 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795401+0000", 2024-04-24T15:32:56.778 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9997911 2024-04-24T15:32:56.778 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.778 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.778 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:56.778 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795614+0000", 2024-04-24T15:32:56.778 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00021228600000000001 2024-04-24T15:32:56.778 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.778 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.779 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:56.779 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795622+0000", 2024-04-24T15:32:56.779 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3920000000000003e-06 2024-04-24T15:32:56.779 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.779 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.779 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:56.779 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795627+0000", 2024-04-24T15:32:56.779 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8199999999999996e-06 2024-04-24T15:32:56.779 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.779 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.779 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:56.779 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836211+0000", 2024-04-24T15:32:56.779 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040583803000000002 2024-04-24T15:32:56.779 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.779 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.780 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:56.780 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836284+0000", 2024-04-24T15:32:56.780 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.3617999999999999e-05 2024-04-24T15:32:56.780 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.780 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.780 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:56.780 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836341+0000", 2024-04-24T15:32:56.780 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6959999999999997e-05 2024-04-24T15:32:56.780 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.780 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:56.780 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.780 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.780 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.780 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23637 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15671296~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:56.780 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795699+0000", 2024-04-24T15:32:56.780 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 21.931574779999998, 2024-04-24T15:32:56.781 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040775556999999997, 2024-04-24T15:32:56.781 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:56.781 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:56.781 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:56.781 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:56.781 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:56.781 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23637 2024-04-24T15:32:56.781 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.781 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:56.781 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.781 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:56.781 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795699+0000", 2024-04-24T15:32:56.781 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.781 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.781 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.782 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:56.782 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795699+0000", 2024-04-24T15:32:56.782 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.782 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.782 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.782 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:56.782 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795632+0000", 2024-04-24T15:32:56.782 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999323 2024-04-24T15:32:56.782 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.782 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.782 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:56.782 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795704+0000", 2024-04-24T15:32:56.782 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2428000000000005e-05 2024-04-24T15:32:56.783 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.783 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.783 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:56.783 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795715+0000", 2024-04-24T15:32:56.783 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.1532000000000001e-05 2024-04-24T15:32:56.783 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.783 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.783 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:56.783 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795721+0000", 2024-04-24T15:32:56.783 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.8200000000000002e-06 2024-04-24T15:32:56.783 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.783 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.783 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:56.783 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836364+0000", 2024-04-24T15:32:56.783 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040642997 2024-04-24T15:32:56.783 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.784 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.784 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:56.784 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836435+0000", 2024-04-24T15:32:56.784 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.0821000000000005e-05 2024-04-24T15:32:56.784 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.784 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.784 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:56.784 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836475+0000", 2024-04-24T15:32:56.784 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9796999999999999e-05 2024-04-24T15:32:56.784 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.784 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:56.784 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.784 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.784 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.784 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23638 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15712256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:56.784 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795932+0000", 2024-04-24T15:32:56.785 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 21.931341750000001, 2024-04-24T15:32:56.785 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040668682999999997, 2024-04-24T15:32:56.785 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:56.785 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:56.785 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:56.785 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:56.785 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:56.785 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23638 2024-04-24T15:32:56.785 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.785 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:56.785 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.785 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:56.785 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795932+0000", 2024-04-24T15:32:56.785 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.785 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.786 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.786 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:56.786 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795932+0000", 2024-04-24T15:32:56.786 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.786 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.786 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.786 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:56.786 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795728+0000", 2024-04-24T15:32:56.786 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9997959 2024-04-24T15:32:56.786 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.786 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.786 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:56.786 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795936+0000", 2024-04-24T15:32:56.786 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000208128 2024-04-24T15:32:56.787 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.787 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.787 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:56.787 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795945+0000", 2024-04-24T15:32:56.787 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.7110000000000001e-06 2024-04-24T15:32:56.787 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.787 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.787 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:56.787 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795957+0000", 2024-04-24T15:32:56.787 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.1800999999999999e-05 2024-04-24T15:32:56.787 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.787 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.787 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:56.787 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836505+0000", 2024-04-24T15:32:56.787 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040547622999999998 2024-04-24T15:32:56.787 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.788 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.788 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:56.788 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836561+0000", 2024-04-24T15:32:56.788 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6598999999999998e-05 2024-04-24T15:32:56.788 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.788 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.788 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:56.788 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836601+0000", 2024-04-24T15:32:56.788 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9907000000000003e-05 2024-04-24T15:32:56.788 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.788 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:56.788 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.788 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.788 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.788 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23639 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15753216~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:56.789 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.796071+0000", 2024-04-24T15:32:56.789 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 21.931203489000001, 2024-04-24T15:32:56.789 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040648199000000003, 2024-04-24T15:32:56.789 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:56.789 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:56.789 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:56.789 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:56.789 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:56.789 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23639 2024-04-24T15:32:56.789 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.789 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:56.789 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.789 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:56.789 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796071+0000", 2024-04-24T15:32:56.790 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.790 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.790 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.790 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:56.790 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796071+0000", 2024-04-24T15:32:56.790 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.790 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.790 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.790 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:56.790 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795967+0000", 2024-04-24T15:32:56.790 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998965 2024-04-24T15:32:56.790 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.790 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.790 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:56.790 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796074+0000", 2024-04-24T15:32:56.791 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000107013 2024-04-24T15:32:56.791 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.791 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.791 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:56.791 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796081+0000", 2024-04-24T15:32:56.791 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2590000000000003e-06 2024-04-24T15:32:56.791 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.791 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.791 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:56.791 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796085+0000", 2024-04-24T15:32:56.791 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5269999999999999e-06 2024-04-24T15:32:56.791 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.791 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.791 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:56.791 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836624+0000", 2024-04-24T15:32:56.791 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040539324000000002 2024-04-24T15:32:56.792 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.792 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.792 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:56.792 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836667+0000", 2024-04-24T15:32:56.792 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2562999999999999e-05 2024-04-24T15:32:56.792 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.792 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.792 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:56.792 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836719+0000", 2024-04-24T15:32:56.792 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.2015999999999998e-05 2024-04-24T15:32:56.792 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.792 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:56.792 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.792 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.793 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.793 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23640 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15794176~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:56.793 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.796492+0000", 2024-04-24T15:32:56.793 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 21.930781731, 2024-04-24T15:32:56.793 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040420000999999997, 2024-04-24T15:32:56.793 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:56.793 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:56.793 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:56.793 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:56.793 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:56.793 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23640 2024-04-24T15:32:56.793 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.793 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:56.794 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.794 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:56.794 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796492+0000", 2024-04-24T15:32:56.794 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.794 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.794 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.794 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:56.794 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796492+0000", 2024-04-24T15:32:56.794 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.794 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.794 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.794 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:56.794 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796088+0000", 2024-04-24T15:32:56.795 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9995961 2024-04-24T15:32:56.795 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.795 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.795 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:56.795 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796496+0000", 2024-04-24T15:32:56.795 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00040785700000000002 2024-04-24T15:32:56.795 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.795 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.795 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:56.795 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796503+0000", 2024-04-24T15:32:56.795 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1049999999999997e-06 2024-04-24T15:32:56.795 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.795 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.795 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:56.796 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796508+0000", 2024-04-24T15:32:56.796 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.155e-06 2024-04-24T15:32:56.796 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.796 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.796 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:56.796 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836756+0000", 2024-04-24T15:32:56.796 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040248850000000003 2024-04-24T15:32:56.796 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.796 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.796 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:56.796 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836841+0000", 2024-04-24T15:32:56.796 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.4735999999999996e-05 2024-04-24T15:32:56.796 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.796 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.797 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:56.797 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836912+0000", 2024-04-24T15:32:56.797 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1229999999999994e-05 2024-04-24T15:32:56.797 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.797 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:56.797 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.797 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.797 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.797 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23641 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15835136~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:56.797 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.796658+0000", 2024-04-24T15:32:56.797 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 21.930615655, 2024-04-24T15:32:56.797 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040420706000000001, 2024-04-24T15:32:56.797 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:56.797 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:56.797 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:56.798 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:56.798 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:56.798 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23641 2024-04-24T15:32:56.798 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.798 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:56.798 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.798 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:56.798 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796658+0000", 2024-04-24T15:32:56.798 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.798 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.798 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.798 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:56.798 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796658+0000", 2024-04-24T15:32:56.799 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.799 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.799 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.799 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:56.799 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796512+0000", 2024-04-24T15:32:56.799 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998531 2024-04-24T15:32:56.799 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.799 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.799 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:56.799 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796669+0000", 2024-04-24T15:32:56.799 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000157577 2024-04-24T15:32:56.799 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.799 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.800 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:56.800 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796677+0000", 2024-04-24T15:32:56.800 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7800000000000001e-06 2024-04-24T15:32:56.800 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.800 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.800 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:56.800 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796681+0000", 2024-04-24T15:32:56.800 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.721e-06 2024-04-24T15:32:56.800 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.800 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.800 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:56.800 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836941+0000", 2024-04-24T15:32:56.800 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040259907999999997 2024-04-24T15:32:56.800 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.801 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.801 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:56.801 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837017+0000", 2024-04-24T15:32:56.801 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.6071999999999999e-05 2024-04-24T15:32:56.801 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.801 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.801 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:56.801 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837079+0000", 2024-04-24T15:32:56.801 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.2358000000000002e-05 2024-04-24T15:32:56.801 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.801 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:56.801 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.801 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.801 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.801 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23645 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15998976~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:56.802 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.797360+0000", 2024-04-24T15:32:56.802 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 21.929914156999999, 2024-04-24T15:32:56.802 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040387645999999999, 2024-04-24T15:32:56.802 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:56.802 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:56.802 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:56.802 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:56.802 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:56.802 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23645 2024-04-24T15:32:56.802 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.802 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:56.802 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.802 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:56.803 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797360+0000", 2024-04-24T15:32:56.803 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.803 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.803 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.803 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:56.803 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797360+0000", 2024-04-24T15:32:56.803 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.803 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.803 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.803 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:56.803 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797288+0000", 2024-04-24T15:32:56.803 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999928 2024-04-24T15:32:56.803 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.803 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.803 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:56.804 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797364+0000", 2024-04-24T15:32:56.804 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.5347999999999998e-05 2024-04-24T15:32:56.804 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.804 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.804 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:56.804 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797372+0000", 2024-04-24T15:32:56.804 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.2409999999999995e-06 2024-04-24T15:32:56.804 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.804 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.804 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:56.804 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797376+0000", 2024-04-24T15:32:56.804 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.515e-06 2024-04-24T15:32:56.804 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.804 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.805 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:56.805 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837586+0000", 2024-04-24T15:32:56.805 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040210047999999998 2024-04-24T15:32:56.805 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.805 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.805 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:56.805 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837672+0000", 2024-04-24T15:32:56.805 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.5997999999999999e-05 2024-04-24T15:32:56.805 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.805 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.805 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:56.805 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837748+0000", 2024-04-24T15:32:56.805 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.5328999999999997e-05 2024-04-24T15:32:56.805 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.805 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:56.805 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.806 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.806 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.806 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23646 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 16039936~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:56.806 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.797498+0000", 2024-04-24T15:32:56.806 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 21.929776056000001, 2024-04-24T15:32:56.806 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040433139, 2024-04-24T15:32:56.806 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:56.806 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:56.806 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:56.806 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:56.806 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:56.806 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23646 2024-04-24T15:32:56.806 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.807 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:56.807 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.807 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:56.807 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797498+0000", 2024-04-24T15:32:56.807 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.807 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.807 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.807 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:56.807 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797498+0000", 2024-04-24T15:32:56.807 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:56.807 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.807 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.807 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:56.807 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797381+0000", 2024-04-24T15:32:56.808 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998832 2024-04-24T15:32:56.808 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.808 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.808 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:56.808 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797501+0000", 2024-04-24T15:32:56.808 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000119759 2024-04-24T15:32:56.808 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.808 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.808 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:56.808 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797510+0000", 2024-04-24T15:32:56.808 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.4770000000000003e-06 2024-04-24T15:32:56.808 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.808 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.808 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:56.808 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797515+0000", 2024-04-24T15:32:56.808 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8330000000000001e-06 2024-04-24T15:32:56.809 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.809 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.809 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:56.809 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837777+0000", 2024-04-24T15:32:56.809 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040262502999999998 2024-04-24T15:32:56.809 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.809 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.809 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:56.809 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837860+0000", 2024-04-24T15:32:56.809 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3002000000000003e-05 2024-04-24T15:32:56.809 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:56.809 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:56.809 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:56.809 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837931+0000", 2024-04-24T15:32:56.809 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1222000000000004e-05 2024-04-24T15:32:56.810 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.810 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:56.810 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.810 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:56.810 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:56.810 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:32:56.811 INFO:teuthology.orchestra.run.smithi096.stdout:2922 trunc from 0x5c440cc to 0xec0c7ac 2024-04-24T15:32:56.812 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-24T15:32:56.813 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:32:56.811+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:32:56.814 INFO:teuthology.orchestra.run.smithi096.stdout:2924 writesame 0x188ea74 thru 0x189d444 data_size 0x11(0xe9d1 bytes) 2024-04-24T15:32:56.814 INFO:teuthology.orchestra.run.smithi096.stdout:2925 trunc from 0xec0c7ac to 0x688eb85 2024-04-24T15:32:56.814 INFO:teuthology.orchestra.run.smithi096.stdout:2927 write 0x5b73205 thru 0x5b7b7cc (0x85c8 bytes) 2024-04-24T15:32:56.814 INFO:teuthology.orchestra.run.smithi096.stdout:2928 write 0x3b87514 thru 0x3b93525 (0xc012 bytes) 2024-04-24T15:32:56.815 INFO:teuthology.orchestra.run.smithi096.stdout:2929 write 0x945ac10 thru 0x945b294 (0x685 bytes) 2024-04-24T15:32:56.815 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:56.815 INFO:teuthology.orchestra.run.smithi096.stdout:2931 writesame 0x254f7aa thru 0x255a56d data_size 0x2c(0xadc4 bytes) 2024-04-24T15:32:56.815 INFO:teuthology.orchestra.run.smithi096.stdout:2932 punch from 0x92e7413 to 0x92f2c0e, (0xb7fb bytes) 2024-04-24T15:32:56.815 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:56.815 INFO:teuthology.orchestra.run.smithi096.stdout:2934 write 0x19bf383 thru 0x19cefda (0xfc58 bytes) 2024-04-24T15:32:56.815 INFO:teuthology.orchestra.run.smithi096.stdout:2936 trunc from 0x945b295 to 0x9ad3c55 2024-04-24T15:32:56.815 INFO:teuthology.orchestra.run.smithi096.stdout:2937 writesame 0x5c427fd thru 0x5c44ace data_size 0x1(0x22d2 bytes) 2024-04-24T15:32:56.815 INFO:teuthology.orchestra.run.smithi096.stdout:2939 read 0x11280c8 thru 0x112901a (0xf53 bytes) 2024-04-24T15:32:56.815 INFO:teuthology.orchestra.run.smithi096.stdout:2940 punch from 0xafd41f to 0xb00dfe, (0x39df bytes) 2024-04-24T15:32:56.815 INFO:teuthology.orchestra.run.smithi096.stdout:2941 write 0xa90dd69 thru 0xa9181b6 (0xa44e bytes) 2024-04-24T15:32:56.816 INFO:teuthology.orchestra.run.smithi096.stdout:2942 read 0x114f53a thru 0x115c9fd (0xd4c4 bytes) 2024-04-24T15:32:56.816 INFO:teuthology.orchestra.run.smithi096.stdout:2946 trunc from 0xa9181b7 to 0xa620d02 2024-04-24T15:32:56.821 INFO:teuthology.orchestra.run.smithi096.stdout:2948 read 0x2b35d95 thru 0x2b3cf1e (0x718a bytes) 2024-04-24T15:32:56.821 INFO:teuthology.orchestra.run.smithi096.stdout:2949 punch from 0x1ef99f4 to 0x1efe487, (0x4a93 bytes) 2024-04-24T15:32:56.821 INFO:teuthology.orchestra.run.smithi096.stdout:2950 write 0x1da5e87 thru 0x1dadd2a (0x7ea4 bytes) 2024-04-24T15:32:56.823 INFO:teuthology.orchestra.run.smithi096.stdout:2951 read 0x6dd4124 thru 0x6ddb468 (0x7345 bytes) 2024-04-24T15:32:56.823 INFO:teuthology.orchestra.run.smithi096.stdout:2952 read 0x6c389e9 thru 0x6c43f19 (0xb531 bytes) 2024-04-24T15:32:56.826 INFO:teuthology.orchestra.run.smithi096.stdout:2953 trunc from 0xa620d02 to 0xcb1b0c3 2024-04-24T15:32:56.830 INFO:teuthology.orchestra.run.smithi096.stdout:2954 read 0xac3869 thru 0xad05ad (0xcd45 bytes) 2024-04-24T15:32:56.831 INFO:teuthology.orchestra.run.smithi096.stdout:2955 writesame 0x4936f73 thru 0x4945def data_size 0x3(0xee7d bytes) 2024-04-24T15:32:56.843 INFO:teuthology.orchestra.run.smithi096.stdout:2956 writesame 0xcfac3cb thru 0xcfb6472 data_size 0x2(0xa0a8 bytes) 2024-04-24T15:32:56.849 INFO:teuthology.orchestra.run.smithi096.stdout:2957 write 0xe2ac339 thru 0xe2b091e (0x45e6 bytes) 2024-04-24T15:32:56.853 INFO:teuthology.orchestra.run.smithi096.stdout:2958 punch from 0x2ab56ca to 0x2abc8bc, (0x71f2 bytes) 2024-04-24T15:32:56.857 INFO:teuthology.orchestra.run.smithi096.stdout:2959 writesame 0xe2faa13 thru 0xe304ae8 data_size 0x22(0xa0d6 bytes) 2024-04-24T15:32:56.857 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:56.857 INFO:teuthology.orchestra.run.smithi096.stdout:2963 write 0x305f07e thru 0x306da2c (0xe9af bytes) 2024-04-24T15:32:56.861 INFO:teuthology.orchestra.run.smithi096.stdout:2965 write 0x684fdd5 thru 0x685ea67 (0xec93 bytes) 2024-04-24T15:32:56.866 INFO:teuthology.orchestra.run.smithi096.stdout:2967 write 0x5afce3f thru 0x5b0884c (0xba0e bytes) 2024-04-24T15:32:56.869 INFO:teuthology.orchestra.run.smithi096.stdout:2970 punch from 0x4f560d0 to 0x4f632fa, (0xd22a bytes) 2024-04-24T15:32:56.869 INFO:teuthology.orchestra.run.smithi096.stdout:2971 write 0x13539f0 thru 0x1360626 (0xcc37 bytes) 2024-04-24T15:32:56.872 INFO:teuthology.orchestra.run.smithi096.stdout:2972 read 0x346ae2f thru 0x34761e0 (0xb3b2 bytes) 2024-04-24T15:32:56.873 INFO:teuthology.orchestra.run.smithi096.stdout:2973 write 0x1be7021 thru 0x1beb995 (0x4975 bytes) 2024-04-24T15:32:56.875 INFO:teuthology.orchestra.run.smithi096.stdout:2974 read 0x6a04930 thru 0x6a0d5ec (0x8cbd bytes) 2024-04-24T15:32:56.876 INFO:teuthology.orchestra.run.smithi096.stdout:2976 trunc from 0xe304ae9 to 0xda5ef52 2024-04-24T15:32:56.884 INFO:teuthology.orchestra.run.smithi096.stdout:2978 read 0xa6c5a3d thru 0xa6d135f (0xb923 bytes) 2024-04-24T15:32:56.884 INFO:teuthology.orchestra.run.smithi096.stdout:2979 punch from 0x28f3086 to 0x28fce6c, (0x9de6 bytes) 2024-04-24T15:32:56.884 INFO:teuthology.orchestra.run.smithi096.stdout:2980 writesame 0xb4feb0c thru 0xb50949e data_size 0x1(0xa993 bytes) 2024-04-24T15:32:56.903 INFO:teuthology.orchestra.run.smithi096.stdout:2981 punch from 0xaa3a5f to 0xaa82f2, (0x4893 bytes) 2024-04-24T15:32:56.903 INFO:teuthology.orchestra.run.smithi096.stdout:2982 writesame 0x750018a thru 0x750434a data_size 0x3(0x41c1 bytes) 2024-04-24T15:32:56.907 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:56.908 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:56.908 INFO:teuthology.orchestra.run.smithi096.stdout:2988 punch from 0x757726a to 0x7577d0f, (0xaa5 bytes) 2024-04-24T15:32:56.912 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:32:56.912+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:32:56.912 INFO:teuthology.orchestra.run.smithi096.stdout:2989 write 0xdde9834 thru 0xddeffa6 (0x6773 bytes) 2024-04-24T15:32:56.915 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:56.915 INFO:teuthology.orchestra.run.smithi096.stdout:2991 write 0x85272c4 thru 0x8535494 (0xe1d1 bytes) 2024-04-24T15:32:56.922 INFO:teuthology.orchestra.run.smithi096.stdout:2992 trunc from 0xddeffa7 to 0xb0a6f88 2024-04-24T15:32:56.930 INFO:teuthology.orchestra.run.smithi096.stdout:2993 clone 15 order 18 su 16384 sc 7 2024-04-24T15:32:56.977 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:32:56.977 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:32:56.977 INFO:teuthology.orchestra.run.smithi151.stdout: "num_ops": 0 2024-04-24T15:32:56.977 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:32:56.987 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-24T15:32:57.013 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:32:57.014+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:32:57.114 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:32:57.115+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:32:57.150 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:32:57.150 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:32:57.150 INFO:teuthology.orchestra.run.smithi151.stdout: "complaint_time": 30, 2024-04-24T15:32:57.150 INFO:teuthology.orchestra.run.smithi151.stdout: "num_blocked_ops": 0 2024-04-24T15:32:57.150 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:32:57.160 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-24T15:32:57.215 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:32:57.216+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:32:57.316 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:32:57.317+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:32:57.325 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:32:57.325 INFO:teuthology.orchestra.run.smithi151.stdout: "size": 20, 2024-04-24T15:32:57.325 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 600, 2024-04-24T15:32:57.325 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [ 2024-04-24T15:32:57.325 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.325 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17317 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 11509760~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:57.325 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.224408+0000", 2024-04-24T15:32:57.325 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 23.100322254999998, 2024-04-24T15:32:57.325 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042604418999999998, 2024-04-24T15:32:57.325 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:57.326 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:57.326 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:57.326 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:57.326 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:57.326 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17317 2024-04-24T15:32:57.326 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.326 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:57.326 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.326 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:57.326 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224408+0000", 2024-04-24T15:32:57.326 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.326 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.327 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.327 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:57.327 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224408+0000", 2024-04-24T15:32:57.327 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.327 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.327 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.327 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:57.327 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224407+0000", 2024-04-24T15:32:57.327 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:57.327 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.327 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.327 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:57.327 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224409+0000", 2024-04-24T15:32:57.327 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.1059999999999998e-06 2024-04-24T15:32:57.328 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.328 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.328 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:57.328 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224415+0000", 2024-04-24T15:32:57.328 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6219999999999997e-06 2024-04-24T15:32:57.328 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.328 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.328 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:57.328 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224419+0000", 2024-04-24T15:32:57.328 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8340000000000001e-06 2024-04-24T15:32:57.328 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.329 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.329 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:57.329 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.266851+0000", 2024-04-24T15:32:57.329 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042431910000000003 2024-04-24T15:32:57.329 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.329 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.329 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:57.329 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.266931+0000", 2024-04-24T15:32:57.329 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0580000000000004e-05 2024-04-24T15:32:57.329 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.329 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.329 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:57.329 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267012+0000", 2024-04-24T15:32:57.330 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0698999999999994e-05 2024-04-24T15:32:57.330 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.330 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:57.330 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.330 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.330 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.330 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17318 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 11550720~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:57.330 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.224424+0000", 2024-04-24T15:32:57.330 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 23.100305887000001, 2024-04-24T15:32:57.330 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042771921999999997, 2024-04-24T15:32:57.330 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:57.330 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:57.330 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:57.331 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:57.331 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:57.331 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17318 2024-04-24T15:32:57.331 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.331 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:57.331 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.331 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:57.331 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224424+0000", 2024-04-24T15:32:57.331 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.331 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.331 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.331 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:57.331 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224424+0000", 2024-04-24T15:32:57.331 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.332 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.332 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.332 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:57.332 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224424+0000", 2024-04-24T15:32:57.332 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:57.332 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.332 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.332 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:57.332 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224434+0000", 2024-04-24T15:32:57.332 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.9699999999999994e-06 2024-04-24T15:32:57.332 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.332 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.332 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:57.332 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224443+0000", 2024-04-24T15:32:57.333 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.3689999999999996e-06 2024-04-24T15:32:57.333 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.333 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.333 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:57.333 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224447+0000", 2024-04-24T15:32:57.333 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.783e-06 2024-04-24T15:32:57.333 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.333 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.333 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:57.333 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267027+0000", 2024-04-24T15:32:57.333 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042579847999999997 2024-04-24T15:32:57.333 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.333 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.333 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:57.334 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267108+0000", 2024-04-24T15:32:57.334 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.1420000000000003e-05 2024-04-24T15:32:57.334 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.334 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.334 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:57.334 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267196+0000", 2024-04-24T15:32:57.334 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.7851999999999998e-05 2024-04-24T15:32:57.334 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.334 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:57.334 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.334 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.334 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.334 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17319 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 11591680~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:57.335 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.224453+0000", 2024-04-24T15:32:57.335 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 23.100276960999999, 2024-04-24T15:32:57.335 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042921783999999998, 2024-04-24T15:32:57.335 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:57.335 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:57.335 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:57.335 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:57.335 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:57.335 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17319 2024-04-24T15:32:57.335 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.335 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:57.335 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.335 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:57.335 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224453+0000", 2024-04-24T15:32:57.335 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.335 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.336 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.336 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:57.336 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224453+0000", 2024-04-24T15:32:57.336 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.336 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.336 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.336 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:57.336 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224452+0000", 2024-04-24T15:32:57.336 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:57.336 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.336 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.336 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:57.336 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224455+0000", 2024-04-24T15:32:57.336 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.289e-06 2024-04-24T15:32:57.336 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.337 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.337 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:57.337 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224461+0000", 2024-04-24T15:32:57.337 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.7300000000000002e-06 2024-04-24T15:32:57.337 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.337 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.337 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:57.337 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224464+0000", 2024-04-24T15:32:57.337 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.2890000000000002e-06 2024-04-24T15:32:57.337 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.337 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.337 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:57.337 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267218+0000", 2024-04-24T15:32:57.337 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042753696000000001 2024-04-24T15:32:57.337 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.337 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.338 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:57.338 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267295+0000", 2024-04-24T15:32:57.338 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7762999999999994e-05 2024-04-24T15:32:57.338 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.338 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.338 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:57.338 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267375+0000", 2024-04-24T15:32:57.338 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.9444999999999996e-05 2024-04-24T15:32:57.338 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.338 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:57.338 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.338 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.338 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.338 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17320 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 11632640~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:57.339 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.224468+0000", 2024-04-24T15:32:57.339 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 23.100261926999998, 2024-04-24T15:32:57.339 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043063615, 2024-04-24T15:32:57.339 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:57.339 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:57.339 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:57.339 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:57.339 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:57.339 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17320 2024-04-24T15:32:57.339 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.339 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:57.339 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.339 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:57.340 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224468+0000", 2024-04-24T15:32:57.340 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.340 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.340 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.340 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:57.340 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224468+0000", 2024-04-24T15:32:57.340 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.340 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.340 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.340 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:57.340 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224468+0000", 2024-04-24T15:32:57.340 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:57.340 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.340 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.340 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:57.341 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224470+0000", 2024-04-24T15:32:57.341 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.0600000000000002e-06 2024-04-24T15:32:57.341 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.341 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.341 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:57.341 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224475+0000", 2024-04-24T15:32:57.341 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.1059999999999999e-06 2024-04-24T15:32:57.341 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.341 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.341 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:57.341 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224478+0000", 2024-04-24T15:32:57.341 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.3100000000000001e-06 2024-04-24T15:32:57.341 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.342 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.342 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:57.342 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267386+0000", 2024-04-24T15:32:57.342 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042907529 2024-04-24T15:32:57.342 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.342 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.342 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:57.342 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267447+0000", 2024-04-24T15:32:57.342 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.1173000000000005e-05 2024-04-24T15:32:57.342 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.342 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.342 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:57.343 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267532+0000", 2024-04-24T15:32:57.343 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.4771999999999994e-05 2024-04-24T15:32:57.343 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.343 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:57.343 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.343 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.343 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.343 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17321 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 11673600~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:57.343 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.224499+0000", 2024-04-24T15:32:57.343 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 23.100231038, 2024-04-24T15:32:57.343 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043214447000000003, 2024-04-24T15:32:57.343 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:57.343 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:57.343 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:57.344 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:57.344 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:57.344 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17321 2024-04-24T15:32:57.344 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.344 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:57.344 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.344 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:57.344 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224499+0000", 2024-04-24T15:32:57.344 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.344 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.344 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.344 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:57.344 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224499+0000", 2024-04-24T15:32:57.345 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.345 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.345 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.345 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:57.345 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224499+0000", 2024-04-24T15:32:57.345 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:57.345 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.345 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.345 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:57.345 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224504+0000", 2024-04-24T15:32:57.345 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6869999999999999e-06 2024-04-24T15:32:57.345 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.345 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.346 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:57.346 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224509+0000", 2024-04-24T15:32:57.346 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2259999999999999e-06 2024-04-24T15:32:57.346 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.346 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.346 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:57.346 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224511+0000", 2024-04-24T15:32:57.346 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.418e-06 2024-04-24T15:32:57.346 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.346 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.346 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:57.346 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267552+0000", 2024-04-24T15:32:57.346 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043041455999999999 2024-04-24T15:32:57.347 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.347 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.347 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:57.347 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267629+0000", 2024-04-24T15:32:57.347 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7069000000000004e-05 2024-04-24T15:32:57.347 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.347 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.347 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:57.347 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267713+0000", 2024-04-24T15:32:57.347 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3843000000000003e-05 2024-04-24T15:32:57.347 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.347 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:57.347 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.347 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.347 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.348 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17322 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 11714560~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:57.348 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.224514+0000", 2024-04-24T15:32:57.348 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 23.100215800000001, 2024-04-24T15:32:57.348 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043339196000000003, 2024-04-24T15:32:57.348 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:57.348 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:57.348 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:57.348 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:57.348 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:57.348 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17322 2024-04-24T15:32:57.348 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.348 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:57.348 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.348 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:57.348 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224514+0000", 2024-04-24T15:32:57.349 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.349 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.349 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.349 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:57.349 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224514+0000", 2024-04-24T15:32:57.349 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.349 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.349 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.349 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:57.349 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224514+0000", 2024-04-24T15:32:57.349 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:57.349 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.349 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.349 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:57.349 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224515+0000", 2024-04-24T15:32:57.350 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.5570000000000001e-06 2024-04-24T15:32:57.350 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.350 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.350 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:57.350 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224518+0000", 2024-04-24T15:32:57.350 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.943e-06 2024-04-24T15:32:57.350 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.350 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.350 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:57.350 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224520+0000", 2024-04-24T15:32:57.350 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.9999999999999999e-06 2024-04-24T15:32:57.350 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.350 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.350 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:57.350 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267730+0000", 2024-04-24T15:32:57.351 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043209466000000002 2024-04-24T15:32:57.351 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.351 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.351 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:57.351 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267772+0000", 2024-04-24T15:32:57.351 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2453000000000002e-05 2024-04-24T15:32:57.351 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.351 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.351 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:57.351 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267853+0000", 2024-04-24T15:32:57.351 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.1094000000000007e-05 2024-04-24T15:32:57.351 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.351 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:57.352 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.352 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.352 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.352 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17323 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 11755520~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:57.352 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.224522+0000", 2024-04-24T15:32:57.352 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 23.10020746, 2024-04-24T15:32:57.352 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043506792000000002, 2024-04-24T15:32:57.352 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:57.352 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:57.352 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:57.352 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:57.352 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:57.352 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17323 2024-04-24T15:32:57.352 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.352 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:57.353 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.353 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:57.353 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224522+0000", 2024-04-24T15:32:57.353 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.353 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.353 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.353 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:57.353 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224522+0000", 2024-04-24T15:32:57.353 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.353 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.353 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.353 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:57.353 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224522+0000", 2024-04-24T15:32:57.353 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:32:57.353 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.354 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.354 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:57.354 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224524+0000", 2024-04-24T15:32:57.354 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3069999999999999e-06 2024-04-24T15:32:57.354 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.354 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.354 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:57.354 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224526+0000", 2024-04-24T15:32:57.354 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.773e-06 2024-04-24T15:32:57.354 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.354 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.354 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:57.354 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224529+0000", 2024-04-24T15:32:57.355 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.3e-06 2024-04-24T15:32:57.355 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.355 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.355 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:57.355 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267873+0000", 2024-04-24T15:32:57.355 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043344141000000003 2024-04-24T15:32:57.355 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.355 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.355 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:57.355 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267948+0000", 2024-04-24T15:32:57.355 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.4960000000000001e-05 2024-04-24T15:32:57.355 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.355 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.355 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:57.357 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.268029+0000", 2024-04-24T15:32:57.357 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.1520000000000006e-05 2024-04-24T15:32:57.358 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.358 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:57.358 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.358 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.358 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.358 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17328 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 11960320~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:57.358 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.226273+0000", 2024-04-24T15:32:57.358 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 23.098456474999999, 2024-04-24T15:32:57.358 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042524610999999997, 2024-04-24T15:32:57.358 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:57.358 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:57.358 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:57.358 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:57.359 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:57.359 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17328 2024-04-24T15:32:57.359 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.359 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:57.359 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.359 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:57.359 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226273+0000", 2024-04-24T15:32:57.359 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.359 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.359 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.359 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:57.359 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226273+0000", 2024-04-24T15:32:57.359 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.360 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.360 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.360 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:57.360 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226273+0000", 2024-04-24T15:32:57.360 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:32:57.360 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.360 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.360 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:57.360 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226275+0000", 2024-04-24T15:32:57.360 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.759e-06 2024-04-24T15:32:57.360 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.360 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.360 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:57.361 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226286+0000", 2024-04-24T15:32:57.361 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0841e-05 2024-04-24T15:32:57.361 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.361 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.361 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:57.361 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226290+0000", 2024-04-24T15:32:57.361 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.794e-06 2024-04-24T15:32:57.361 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.361 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.361 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:57.361 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.268693+0000", 2024-04-24T15:32:57.361 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042403519000000001 2024-04-24T15:32:57.361 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.361 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.362 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:57.362 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.268739+0000", 2024-04-24T15:32:57.362 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6149000000000001e-05 2024-04-24T15:32:57.362 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.362 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.362 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:57.362 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.268798+0000", 2024-04-24T15:32:57.362 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.8752999999999998e-05 2024-04-24T15:32:57.362 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.362 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:57.362 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.362 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.362 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.363 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17329 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 12001280~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:57.363 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.226295+0000", 2024-04-24T15:32:57.363 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 23.098435368000001, 2024-04-24T15:32:57.363 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042690771000000002, 2024-04-24T15:32:57.363 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:57.363 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:57.363 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:57.363 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:57.363 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:57.363 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17329 2024-04-24T15:32:57.363 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.363 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:57.363 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.363 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:57.364 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226295+0000", 2024-04-24T15:32:57.364 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.364 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.364 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.364 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:57.364 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226295+0000", 2024-04-24T15:32:57.364 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.364 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.364 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.364 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:57.364 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226294+0000", 2024-04-24T15:32:57.364 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:57.364 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.364 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.365 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:57.365 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226297+0000", 2024-04-24T15:32:57.365 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.0189999999999998e-06 2024-04-24T15:32:57.365 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.365 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.365 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:57.365 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226302+0000", 2024-04-24T15:32:57.365 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6260000000000003e-06 2024-04-24T15:32:57.365 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.365 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.365 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:57.365 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226304+0000", 2024-04-24T15:32:57.365 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.2230000000000001e-06 2024-04-24T15:32:57.365 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.366 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.366 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:57.366 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.268831+0000", 2024-04-24T15:32:57.366 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042527224000000002 2024-04-24T15:32:57.366 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.366 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.366 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:57.366 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.268904+0000", 2024-04-24T15:32:57.366 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.3168000000000001e-05 2024-04-24T15:32:57.366 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.366 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.366 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:57.367 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.268985+0000", 2024-04-24T15:32:57.367 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.1009999999999999e-05 2024-04-24T15:32:57.367 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.367 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:57.367 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.367 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.367 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.367 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17330 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 12042240~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:57.367 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.226307+0000", 2024-04-24T15:32:57.367 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 23.098423377, 2024-04-24T15:32:57.367 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042812009999999998, 2024-04-24T15:32:57.367 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:57.367 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:57.367 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:57.368 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:57.368 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:57.368 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17330 2024-04-24T15:32:57.368 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.368 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:57.368 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.368 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:57.368 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226307+0000", 2024-04-24T15:32:57.368 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.368 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.368 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.368 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:57.368 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226307+0000", 2024-04-24T15:32:57.368 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.368 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.369 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.369 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:57.369 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226306+0000", 2024-04-24T15:32:57.369 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:32:57.369 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.369 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.369 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:57.369 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226312+0000", 2024-04-24T15:32:57.369 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6180000000000003e-06 2024-04-24T15:32:57.369 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.369 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.369 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:57.369 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226316+0000", 2024-04-24T15:32:57.370 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2379999999999999e-06 2024-04-24T15:32:57.370 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.370 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.370 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:57.370 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226318+0000", 2024-04-24T15:32:57.370 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.3130000000000001e-06 2024-04-24T15:32:57.370 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.370 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.370 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:57.370 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269015+0000", 2024-04-24T15:32:57.370 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042696698999999998 2024-04-24T15:32:57.370 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.370 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.370 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:57.371 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269061+0000", 2024-04-24T15:32:57.371 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6193000000000003e-05 2024-04-24T15:32:57.371 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.371 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.371 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:57.371 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269119+0000", 2024-04-24T15:32:57.371 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.7169000000000001e-05 2024-04-24T15:32:57.371 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.371 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:57.371 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.371 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.371 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.371 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17331 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 12083200~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:57.372 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.226321+0000", 2024-04-24T15:32:57.372 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 23.098408945999999, 2024-04-24T15:32:57.372 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042976872999999999, 2024-04-24T15:32:57.372 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:57.372 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:57.372 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:57.372 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:57.372 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:57.372 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17331 2024-04-24T15:32:57.372 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.372 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:57.372 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.372 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:57.372 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226321+0000", 2024-04-24T15:32:57.373 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.373 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.373 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.373 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:57.373 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226321+0000", 2024-04-24T15:32:57.373 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.373 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.373 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.373 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:57.373 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226321+0000", 2024-04-24T15:32:57.373 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:32:57.373 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.373 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.373 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:57.373 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226322+0000", 2024-04-24T15:32:57.374 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.446e-06 2024-04-24T15:32:57.374 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.374 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.374 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:57.374 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226325+0000", 2024-04-24T15:32:57.374 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.004e-06 2024-04-24T15:32:57.374 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.374 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.374 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:57.374 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226327+0000", 2024-04-24T15:32:57.374 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.0990000000000001e-06 2024-04-24T15:32:57.374 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.374 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.374 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:57.375 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269137+0000", 2024-04-24T15:32:57.375 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042809381000000001 2024-04-24T15:32:57.375 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.375 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.375 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:57.375 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269217+0000", 2024-04-24T15:32:57.375 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.085e-05 2024-04-24T15:32:57.375 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.375 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.375 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:57.375 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269298+0000", 2024-04-24T15:32:57.375 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0317999999999999e-05 2024-04-24T15:32:57.376 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.376 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:57.376 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.376 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.376 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.376 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17332 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 12124160~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:57.376 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.226329+0000", 2024-04-24T15:32:57.376 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 23.098400458, 2024-04-24T15:32:57.376 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043084539999999998, 2024-04-24T15:32:57.376 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:57.376 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:57.376 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:57.376 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:57.376 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:57.376 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17332 2024-04-24T15:32:57.377 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.377 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:57.377 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.377 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:57.377 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226329+0000", 2024-04-24T15:32:57.377 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.377 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.377 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.377 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:57.377 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226329+0000", 2024-04-24T15:32:57.377 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.377 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.377 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.378 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:57.378 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226329+0000", 2024-04-24T15:32:57.378 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:32:57.378 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.378 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.378 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:57.378 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226331+0000", 2024-04-24T15:32:57.378 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.328e-06 2024-04-24T15:32:57.378 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.378 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.378 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:57.378 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226333+0000", 2024-04-24T15:32:57.378 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.9119999999999998e-06 2024-04-24T15:32:57.378 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.379 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.379 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:57.379 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226335+0000", 2024-04-24T15:32:57.379 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.9690000000000001e-06 2024-04-24T15:32:57.379 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.379 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.379 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:57.379 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269313+0000", 2024-04-24T15:32:57.379 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042977518999999999 2024-04-24T15:32:57.379 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.379 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.379 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:57.379 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269357+0000", 2024-04-24T15:32:57.379 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3748999999999997e-05 2024-04-24T15:32:57.379 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.380 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.380 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:57.380 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269414+0000", 2024-04-24T15:32:57.380 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.7265000000000001e-05 2024-04-24T15:32:57.380 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.380 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:57.380 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.380 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.380 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.380 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17333 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 12165120~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:57.380 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.226338+0000", 2024-04-24T15:32:57.380 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 23.098392320999999, 2024-04-24T15:32:57.380 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043264706999999999, 2024-04-24T15:32:57.380 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:57.380 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:57.381 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:57.381 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:57.381 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:57.381 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17333 2024-04-24T15:32:57.381 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.381 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:57.381 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.381 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:57.381 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226338+0000", 2024-04-24T15:32:57.381 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.381 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.381 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.381 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:57.381 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226338+0000", 2024-04-24T15:32:57.382 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.382 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.382 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.382 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:57.382 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226337+0000", 2024-04-24T15:32:57.382 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:32:57.382 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.382 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.382 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:57.382 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226339+0000", 2024-04-24T15:32:57.382 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.387e-06 2024-04-24T15:32:57.382 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.382 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.382 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:57.383 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226342+0000", 2024-04-24T15:32:57.383 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8059999999999999e-06 2024-04-24T15:32:57.383 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.383 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.383 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:57.383 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226344+0000", 2024-04-24T15:32:57.383 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.1390000000000002e-06 2024-04-24T15:32:57.383 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.383 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.383 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:57.383 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269445+0000", 2024-04-24T15:32:57.383 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043101457000000003 2024-04-24T15:32:57.383 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.384 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.384 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:57.384 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269520+0000", 2024-04-24T15:32:57.384 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.5278000000000007e-05 2024-04-24T15:32:57.384 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.384 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.384 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:57.384 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269602+0000", 2024-04-24T15:32:57.384 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.1847000000000004e-05 2024-04-24T15:32:57.384 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.384 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:57.384 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.384 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.384 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.385 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17334 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 12206080~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:57.385 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.226346+0000", 2024-04-24T15:32:57.385 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 23.098383904999999, 2024-04-24T15:32:57.385 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043468358999999998, 2024-04-24T15:32:57.385 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:57.385 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:57.385 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:57.385 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:57.385 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:57.385 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17334 2024-04-24T15:32:57.385 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.385 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:57.386 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.386 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:57.386 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226346+0000", 2024-04-24T15:32:57.386 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.386 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.386 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.386 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:57.386 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226346+0000", 2024-04-24T15:32:57.386 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.386 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.386 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.386 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:57.386 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226346+0000", 2024-04-24T15:32:57.387 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:32:57.387 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.387 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.387 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:57.387 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226347+0000", 2024-04-24T15:32:57.387 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.359e-06 2024-04-24T15:32:57.387 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.387 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.387 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:57.387 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226350+0000", 2024-04-24T15:32:57.387 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7800000000000001e-06 2024-04-24T15:32:57.387 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.387 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.387 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:57.387 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226352+0000", 2024-04-24T15:32:57.388 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.046e-06 2024-04-24T15:32:57.388 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.388 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.388 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:57.388 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269646+0000", 2024-04-24T15:32:57.388 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043294393 2024-04-24T15:32:57.388 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.388 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.388 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:57.388 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269732+0000", 2024-04-24T15:32:57.388 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.5716000000000004e-05 2024-04-24T15:32:57.388 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.388 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.388 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:57.388 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269814+0000", 2024-04-24T15:32:57.389 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.2270999999999999e-05 2024-04-24T15:32:57.389 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.389 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:57.389 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.389 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.389 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.389 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17342 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 12533760~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:57.389 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.228377+0000", 2024-04-24T15:32:57.389 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 23.096353011000001, 2024-04-24T15:32:57.389 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042573286000000002, 2024-04-24T15:32:57.389 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:57.389 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:57.389 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:57.389 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:57.389 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:57.389 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17342 2024-04-24T15:32:57.390 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.390 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:57.390 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.390 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:57.390 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228377+0000", 2024-04-24T15:32:57.390 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.390 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.390 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.390 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:57.390 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228377+0000", 2024-04-24T15:32:57.390 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.390 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.391 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.391 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:57.391 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228377+0000", 2024-04-24T15:32:57.391 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:57.391 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.391 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.391 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:57.391 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228389+0000", 2024-04-24T15:32:57.391 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.2262999999999999e-05 2024-04-24T15:32:57.391 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.391 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.392 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:57.392 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228398+0000", 2024-04-24T15:32:57.392 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.1079999999999999e-06 2024-04-24T15:32:57.392 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.392 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.392 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:57.392 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228402+0000", 2024-04-24T15:32:57.392 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8920000000000002e-06 2024-04-24T15:32:57.392 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.392 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.392 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:57.392 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.270842+0000", 2024-04-24T15:32:57.392 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042440193000000001 2024-04-24T15:32:57.393 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.393 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.393 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:57.393 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.270887+0000", 2024-04-24T15:32:57.393 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5359999999999999e-05 2024-04-24T15:32:57.393 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.393 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.393 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:57.393 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.270950+0000", 2024-04-24T15:32:57.393 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.2842999999999994e-05 2024-04-24T15:32:57.393 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.393 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:57.393 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.393 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.394 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.394 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17343 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 12574720~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:57.394 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.228407+0000", 2024-04-24T15:32:57.394 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 23.096323137999999, 2024-04-24T15:32:57.394 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042650325000000003, 2024-04-24T15:32:57.394 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:57.394 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:57.394 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:57.394 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:57.394 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:57.394 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17343 2024-04-24T15:32:57.394 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.394 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:57.395 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.395 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:57.395 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228407+0000", 2024-04-24T15:32:57.395 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.395 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.395 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.395 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:57.395 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228407+0000", 2024-04-24T15:32:57.395 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.395 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.395 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.395 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:57.396 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228406+0000", 2024-04-24T15:32:57.396 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:57.396 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.396 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.396 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:57.396 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228409+0000", 2024-04-24T15:32:57.396 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.3980000000000002e-06 2024-04-24T15:32:57.396 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.396 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.396 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:57.396 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228414+0000", 2024-04-24T15:32:57.396 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6590000000000004e-06 2024-04-24T15:32:57.396 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.396 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.397 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:57.397 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228418+0000", 2024-04-24T15:32:57.397 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.625e-06 2024-04-24T15:32:57.397 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.397 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.397 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:57.397 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.270966+0000", 2024-04-24T15:32:57.397 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042547709000000003 2024-04-24T15:32:57.397 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.397 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.397 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:57.397 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.271007+0000", 2024-04-24T15:32:57.398 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.1659000000000002e-05 2024-04-24T15:32:57.398 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.398 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.398 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:57.398 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.271057+0000", 2024-04-24T15:32:57.398 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.9679000000000003e-05 2024-04-24T15:32:57.398 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.398 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:57.398 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.398 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.398 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.398 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17344 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 12607488~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:57.398 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.228461+0000", 2024-04-24T15:32:57.399 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 23.096269150000001, 2024-04-24T15:32:57.399 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042713530999999999, 2024-04-24T15:32:57.399 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:57.399 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:57.399 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:57.399 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:57.399 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:57.399 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17344 2024-04-24T15:32:57.399 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.399 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:57.399 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.399 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:57.399 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228461+0000", 2024-04-24T15:32:57.399 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.399 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.400 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.400 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:57.400 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228461+0000", 2024-04-24T15:32:57.400 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.400 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.400 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.400 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:57.400 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228460+0000", 2024-04-24T15:32:57.400 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:57.400 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.400 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.400 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:57.400 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228464+0000", 2024-04-24T15:32:57.400 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.98e-06 2024-04-24T15:32:57.401 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.401 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.401 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:57.401 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228473+0000", 2024-04-24T15:32:57.401 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.0270000000000001e-06 2024-04-24T15:32:57.401 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.401 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.401 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:57.401 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228479+0000", 2024-04-24T15:32:57.401 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.5910000000000003e-06 2024-04-24T15:32:57.401 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.401 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.402 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:57.402 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.271081+0000", 2024-04-24T15:32:57.402 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042602025000000002 2024-04-24T15:32:57.402 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.402 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.402 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:57.402 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.271126+0000", 2024-04-24T15:32:57.402 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4904999999999998e-05 2024-04-24T15:32:57.402 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.402 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.402 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:57.402 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.271174+0000", 2024-04-24T15:32:57.403 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8603999999999997e-05 2024-04-24T15:32:57.403 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.403 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:57.403 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.403 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.403 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.403 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17345 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 12648448~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:57.403 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.228484+0000", 2024-04-24T15:32:57.403 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 23.096245614000001, 2024-04-24T15:32:57.403 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042811844000000002, 2024-04-24T15:32:57.403 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:57.403 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:57.403 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:57.404 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:57.404 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:57.404 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17345 2024-04-24T15:32:57.404 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.404 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:57.404 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.404 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:57.404 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228484+0000", 2024-04-24T15:32:57.404 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.404 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.404 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.404 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:57.404 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228484+0000", 2024-04-24T15:32:57.405 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.405 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.405 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.405 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:57.405 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228484+0000", 2024-04-24T15:32:57.405 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:57.405 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.405 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.405 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:57.405 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228487+0000", 2024-04-24T15:32:57.405 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.9950000000000001e-06 2024-04-24T15:32:57.405 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.405 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.406 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:57.406 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228493+0000", 2024-04-24T15:32:57.406 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.2870000000000001e-06 2024-04-24T15:32:57.406 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.406 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.406 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:57.406 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228497+0000", 2024-04-24T15:32:57.406 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.1219999999999997e-06 2024-04-24T15:32:57.406 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.406 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.406 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:57.406 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.271202+0000", 2024-04-24T15:32:57.406 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042704963999999998 2024-04-24T15:32:57.407 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.407 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.407 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:57.407 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.271250+0000", 2024-04-24T15:32:57.407 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8207000000000001e-05 2024-04-24T15:32:57.407 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.407 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.407 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:57.407 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.271296+0000", 2024-04-24T15:32:57.407 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5769999999999997e-05 2024-04-24T15:32:57.407 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.407 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:57.407 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.408 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.408 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.408 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17346 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 12689408~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:57.408 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.228502+0000", 2024-04-24T15:32:57.408 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 23.096228, 2024-04-24T15:32:57.408 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042918317999999997, 2024-04-24T15:32:57.408 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:57.408 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:57.408 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:57.408 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:57.408 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:57.408 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17346 2024-04-24T15:32:57.409 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.409 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:57.409 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.409 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:57.409 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228502+0000", 2024-04-24T15:32:57.409 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.409 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.409 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.409 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:57.409 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228502+0000", 2024-04-24T15:32:57.409 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.409 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.409 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.409 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:57.410 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228502+0000", 2024-04-24T15:32:57.410 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:57.410 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.410 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.410 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:57.410 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228504+0000", 2024-04-24T15:32:57.410 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.497e-06 2024-04-24T15:32:57.410 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.410 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.410 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:57.410 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228510+0000", 2024-04-24T15:32:57.410 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.7429999999999999e-06 2024-04-24T15:32:57.410 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.411 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.411 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:57.411 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228514+0000", 2024-04-24T15:32:57.411 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0810000000000004e-06 2024-04-24T15:32:57.411 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.411 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.411 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:57.411 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.271330+0000", 2024-04-24T15:32:57.411 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042816616000000002 2024-04-24T15:32:57.411 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.411 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.411 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:57.411 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.271371+0000", 2024-04-24T15:32:57.411 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0222999999999998e-05 2024-04-24T15:32:57.412 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.412 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.412 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:57.412 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.271420+0000", 2024-04-24T15:32:57.412 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.9539000000000001e-05 2024-04-24T15:32:57.412 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.412 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:57.412 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.412 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.412 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.412 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17347 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 12730368~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:57.412 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.228518+0000", 2024-04-24T15:32:57.412 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 23.096211472, 2024-04-24T15:32:57.412 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043022429000000001, 2024-04-24T15:32:57.413 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:57.413 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:57.413 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:57.413 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:57.413 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:57.413 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17347 2024-04-24T15:32:57.413 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.413 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:57.413 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.413 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:57.413 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228518+0000", 2024-04-24T15:32:57.413 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.413 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.413 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.414 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:57.414 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228518+0000", 2024-04-24T15:32:57.414 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.414 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.414 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.414 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:57.414 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228518+0000", 2024-04-24T15:32:57.414 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:57.414 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.414 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.414 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:57.414 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228521+0000", 2024-04-24T15:32:57.414 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.4559999999999999e-06 2024-04-24T15:32:57.415 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.415 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.415 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:57.415 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228526+0000", 2024-04-24T15:32:57.415 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.7259999999999999e-06 2024-04-24T15:32:57.415 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.415 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.415 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:57.415 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228530+0000", 2024-04-24T15:32:57.415 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.658e-06 2024-04-24T15:32:57.415 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.415 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.415 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:57.415 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.271445+0000", 2024-04-24T15:32:57.416 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042915506999999999 2024-04-24T15:32:57.416 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.416 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.416 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:57.416 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.271490+0000", 2024-04-24T15:32:57.416 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4752000000000001e-05 2024-04-24T15:32:57.416 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.416 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.416 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:57.416 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.271541+0000", 2024-04-24T15:32:57.416 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.0699999999999999e-05 2024-04-24T15:32:57.416 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.416 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:57.416 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.417 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.417 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:57.417 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:32:57.418 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-24T15:32:57.419 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:32:57.418+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:32:57.518 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:32:57.519+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:32:57.582 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:32:57.582 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:32:57.582 INFO:teuthology.orchestra.run.smithi151.stdout: "num_ops": 0 2024-04-24T15:32:57.582 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:32:57.592 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-24T15:32:57.619 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:32:57.620+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:32:57.715 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:32:57.715 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:32:57.715 INFO:teuthology.orchestra.run.smithi151.stdout: "complaint_time": 30, 2024-04-24T15:32:57.715 INFO:teuthology.orchestra.run.smithi151.stdout: "num_blocked_ops": 0 2024-04-24T15:32:57.715 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:32:57.720 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:32:57.720+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:32:57.726 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-24T15:32:57.787 INFO:teuthology.orchestra.run.smithi096.stdout:truncating image image_client.0-clone14 from 0xb0a6f88 (overlap 0x9766e9) to 0x0 2024-04-24T15:32:57.821 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:32:57.821+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:32:57.854 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:32:57.854 INFO:teuthology.orchestra.run.smithi151.stdout: "size": 20, 2024-04-24T15:32:57.854 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 600, 2024-04-24T15:32:57.854 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [ 2024-04-24T15:32:57.855 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.855 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18006 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12419072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:57.855 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.207911+0000", 2024-04-24T15:32:57.855 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 23.646053773999999, 2024-04-24T15:32:57.855 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042899496000000002, 2024-04-24T15:32:57.855 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:57.855 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:57.855 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:57.855 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:57.855 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:57.855 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18006 2024-04-24T15:32:57.855 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.855 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:57.856 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.856 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:57.856 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207911+0000", 2024-04-24T15:32:57.856 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.856 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.856 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.856 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:57.856 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207911+0000", 2024-04-24T15:32:57.856 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.856 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.856 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.856 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:57.856 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207910+0000", 2024-04-24T15:32:57.857 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:57.857 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.857 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.857 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:57.857 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207913+0000", 2024-04-24T15:32:57.857 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.2309999999999999e-06 2024-04-24T15:32:57.857 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.857 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.857 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:57.857 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207918+0000", 2024-04-24T15:32:57.857 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.3399999999999997e-06 2024-04-24T15:32:57.857 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.858 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.858 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:57.858 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207922+0000", 2024-04-24T15:32:57.858 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5439999999999999e-06 2024-04-24T15:32:57.858 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.858 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.858 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:57.858 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250726+0000", 2024-04-24T15:32:57.858 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042804129000000003 2024-04-24T15:32:57.858 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.858 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.858 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:57.859 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250765+0000", 2024-04-24T15:32:57.859 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9554000000000001e-05 2024-04-24T15:32:57.859 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.859 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.859 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:57.859 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250810+0000", 2024-04-24T15:32:57.859 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5027000000000001e-05 2024-04-24T15:32:57.859 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.859 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:57.859 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.859 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.859 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.859 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18007 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12460032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:57.860 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.207961+0000", 2024-04-24T15:32:57.860 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 23.646003932999999, 2024-04-24T15:32:57.860 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042965470999999998, 2024-04-24T15:32:57.860 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:57.860 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:57.860 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:57.860 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:57.860 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:57.860 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18007 2024-04-24T15:32:57.860 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.860 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:57.860 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.860 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:57.861 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207961+0000", 2024-04-24T15:32:57.861 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.861 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.861 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.861 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:57.861 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207961+0000", 2024-04-24T15:32:57.861 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.861 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.861 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.861 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:57.861 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207960+0000", 2024-04-24T15:32:57.861 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:57.861 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.862 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.862 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:57.862 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207963+0000", 2024-04-24T15:32:57.862 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.3759999999999999e-06 2024-04-24T15:32:57.862 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.862 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.862 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:57.862 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207970+0000", 2024-04-24T15:32:57.862 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.0489999999999998e-06 2024-04-24T15:32:57.862 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.862 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.862 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:57.863 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207975+0000", 2024-04-24T15:32:57.863 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.1640000000000003e-06 2024-04-24T15:32:57.863 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.863 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.863 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:57.863 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250834+0000", 2024-04-24T15:32:57.863 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042859060999999997 2024-04-24T15:32:57.863 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.863 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.863 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:57.863 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250874+0000", 2024-04-24T15:32:57.863 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0674999999999999e-05 2024-04-24T15:32:57.864 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.864 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.864 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:57.864 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250926+0000", 2024-04-24T15:32:57.864 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.1693000000000001e-05 2024-04-24T15:32:57.864 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.864 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:57.864 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.864 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.864 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.864 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18008 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12500992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:57.864 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208030+0000", 2024-04-24T15:32:57.865 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 23.645934839999999, 2024-04-24T15:32:57.865 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043003337000000003, 2024-04-24T15:32:57.865 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:57.865 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:57.865 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:57.865 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:57.865 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:57.865 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18008 2024-04-24T15:32:57.865 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.865 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:57.865 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.865 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:57.865 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208030+0000", 2024-04-24T15:32:57.866 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.866 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.866 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.866 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:57.866 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208030+0000", 2024-04-24T15:32:57.866 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.866 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.866 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.866 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:57.866 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208029+0000", 2024-04-24T15:32:57.866 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:57.866 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.866 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.866 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:57.867 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208032+0000", 2024-04-24T15:32:57.867 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.117e-06 2024-04-24T15:32:57.867 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.867 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.867 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:57.867 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208039+0000", 2024-04-24T15:32:57.867 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.0129999999999996e-06 2024-04-24T15:32:57.867 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.867 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.867 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:57.867 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208043+0000", 2024-04-24T15:32:57.867 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0600000000000001e-06 2024-04-24T15:32:57.867 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.867 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.868 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:57.868 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250948+0000", 2024-04-24T15:32:57.868 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042904577999999999 2024-04-24T15:32:57.868 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.868 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.868 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:57.868 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250989+0000", 2024-04-24T15:32:57.868 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0978e-05 2024-04-24T15:32:57.868 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.868 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.868 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:57.868 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251033+0000", 2024-04-24T15:32:57.869 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4161e-05 2024-04-24T15:32:57.869 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.869 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:57.869 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.869 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.869 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.869 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18009 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12541952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:57.869 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208093+0000", 2024-04-24T15:32:57.869 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 23.645871094, 2024-04-24T15:32:57.869 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043043451000000003, 2024-04-24T15:32:57.869 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:57.869 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:57.869 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:57.869 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:57.870 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:57.870 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18009 2024-04-24T15:32:57.870 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.870 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:57.870 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.870 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:57.870 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208093+0000", 2024-04-24T15:32:57.870 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.870 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.870 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.870 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:57.870 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208093+0000", 2024-04-24T15:32:57.871 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.871 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.871 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.871 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:57.871 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208093+0000", 2024-04-24T15:32:57.871 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:57.871 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.871 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.871 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:57.871 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208096+0000", 2024-04-24T15:32:57.871 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.439e-06 2024-04-24T15:32:57.871 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.871 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.872 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:57.872 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208103+0000", 2024-04-24T15:32:57.872 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1019999999999999e-06 2024-04-24T15:32:57.872 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.872 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.872 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:57.872 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208107+0000", 2024-04-24T15:32:57.872 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0910000000000003e-06 2024-04-24T15:32:57.872 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.872 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.872 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:57.872 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251054+0000", 2024-04-24T15:32:57.872 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042946826 2024-04-24T15:32:57.872 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.873 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.873 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:57.873 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251092+0000", 2024-04-24T15:32:57.873 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.7941999999999997e-05 2024-04-24T15:32:57.873 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.873 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.873 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:57.873 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251137+0000", 2024-04-24T15:32:57.873 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4638000000000002e-05 2024-04-24T15:32:57.873 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.873 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:57.873 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.873 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.873 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.873 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18010 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12615680~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:57.874 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208202+0000", 2024-04-24T15:32:57.874 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 23.645762263000002, 2024-04-24T15:32:57.874 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043046549000000003, 2024-04-24T15:32:57.874 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:57.874 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:57.874 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:57.874 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:57.874 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:57.874 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18010 2024-04-24T15:32:57.874 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.874 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:57.874 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.874 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:57.875 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208202+0000", 2024-04-24T15:32:57.875 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.875 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.875 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.875 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:57.875 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208202+0000", 2024-04-24T15:32:57.875 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.875 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.875 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.875 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:57.875 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208202+0000", 2024-04-24T15:32:57.875 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:57.875 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.875 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.876 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:57.876 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208205+0000", 2024-04-24T15:32:57.876 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.2490000000000001e-06 2024-04-24T15:32:57.876 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.876 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.876 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:57.876 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208215+0000", 2024-04-24T15:32:57.876 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.9070000000000002e-06 2024-04-24T15:32:57.876 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.876 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.876 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:57.876 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208220+0000", 2024-04-24T15:32:57.876 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8849999999999998e-06 2024-04-24T15:32:57.877 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.877 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.877 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:57.877 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251158+0000", 2024-04-24T15:32:57.877 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042938416 2024-04-24T15:32:57.877 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.877 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.877 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:57.877 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251197+0000", 2024-04-24T15:32:57.877 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9341000000000002e-05 2024-04-24T15:32:57.877 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.877 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.878 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:57.878 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251249+0000", 2024-04-24T15:32:57.878 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.1391000000000003e-05 2024-04-24T15:32:57.878 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.878 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:57.878 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.878 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.878 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.878 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18011 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12656640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:57.878 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208258+0000", 2024-04-24T15:32:57.878 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 23.645706417, 2024-04-24T15:32:57.878 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043102708000000003, 2024-04-24T15:32:57.878 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:57.879 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:57.879 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:57.879 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:57.879 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:57.879 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18011 2024-04-24T15:32:57.879 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.879 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:57.879 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.879 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:57.879 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208258+0000", 2024-04-24T15:32:57.879 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.879 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.879 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.880 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:57.880 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208258+0000", 2024-04-24T15:32:57.880 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.880 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.880 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.880 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:57.880 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208258+0000", 2024-04-24T15:32:57.880 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:57.880 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.880 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.880 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:57.880 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208260+0000", 2024-04-24T15:32:57.880 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7810000000000001e-06 2024-04-24T15:32:57.881 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.881 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.881 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:57.881 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208267+0000", 2024-04-24T15:32:57.881 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.7610000000000002e-06 2024-04-24T15:32:57.881 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.881 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.881 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:57.881 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208271+0000", 2024-04-24T15:32:57.881 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0890000000000002e-06 2024-04-24T15:32:57.881 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.881 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.881 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:57.881 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251271+0000", 2024-04-24T15:32:57.881 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043000159000000003 2024-04-24T15:32:57.882 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.882 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.882 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:57.882 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251315+0000", 2024-04-24T15:32:57.882 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3446999999999999e-05 2024-04-24T15:32:57.882 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.882 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.882 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:57.882 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251361+0000", 2024-04-24T15:32:57.882 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5952000000000003e-05 2024-04-24T15:32:57.882 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.882 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:57.882 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.882 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.883 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.883 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18012 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12697600~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:57.883 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208289+0000", 2024-04-24T15:32:57.883 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 23.645675373, 2024-04-24T15:32:57.883 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043174774999999999, 2024-04-24T15:32:57.883 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:57.883 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:57.883 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:57.883 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:57.883 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:57.883 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18012 2024-04-24T15:32:57.883 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.883 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:57.883 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.883 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:57.884 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208289+0000", 2024-04-24T15:32:57.884 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.884 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.884 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.884 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:57.884 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208289+0000", 2024-04-24T15:32:57.884 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.884 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.884 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.884 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:57.884 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208289+0000", 2024-04-24T15:32:57.884 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:57.884 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.884 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.885 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:57.885 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208291+0000", 2024-04-24T15:32:57.885 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.542e-06 2024-04-24T15:32:57.885 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.885 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.885 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:57.885 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208298+0000", 2024-04-24T15:32:57.885 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.3899999999999998e-06 2024-04-24T15:32:57.885 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.885 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.885 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:57.885 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208302+0000", 2024-04-24T15:32:57.885 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.019e-06 2024-04-24T15:32:57.886 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.886 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.886 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:57.886 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251382+0000", 2024-04-24T15:32:57.886 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043080790000000001 2024-04-24T15:32:57.886 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.886 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.886 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:57.886 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251421+0000", 2024-04-24T15:32:57.886 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8473000000000002e-05 2024-04-24T15:32:57.886 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.886 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.886 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:57.886 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251464+0000", 2024-04-24T15:32:57.887 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3133999999999997e-05 2024-04-24T15:32:57.887 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.887 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:57.887 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.887 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.887 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.887 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18013 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12738560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:57.887 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208423+0000", 2024-04-24T15:32:57.887 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 23.645541264999999, 2024-04-24T15:32:57.887 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043145554000000003, 2024-04-24T15:32:57.887 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:57.887 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:57.887 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:57.887 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:57.887 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:57.888 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18013 2024-04-24T15:32:57.888 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.888 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:57.888 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.888 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:57.888 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208423+0000", 2024-04-24T15:32:57.888 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.888 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.888 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.888 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:57.888 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208423+0000", 2024-04-24T15:32:57.888 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.888 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.888 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.888 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:57.888 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208423+0000", 2024-04-24T15:32:57.889 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:57.889 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.889 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.889 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:57.889 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208426+0000", 2024-04-24T15:32:57.889 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.0539999999999999e-06 2024-04-24T15:32:57.889 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.889 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.889 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:57.889 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208433+0000", 2024-04-24T15:32:57.889 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.43e-06 2024-04-24T15:32:57.889 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.889 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.889 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:57.890 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208437+0000", 2024-04-24T15:32:57.890 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5590000000000001e-06 2024-04-24T15:32:57.890 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.890 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.890 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:57.890 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251485+0000", 2024-04-24T15:32:57.890 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043048259999999998 2024-04-24T15:32:57.890 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.890 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.890 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:57.890 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251525+0000", 2024-04-24T15:32:57.890 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9916999999999997e-05 2024-04-24T15:32:57.890 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.891 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.891 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:57.891 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251569+0000", 2024-04-24T15:32:57.891 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3939000000000001e-05 2024-04-24T15:32:57.891 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.891 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:57.891 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.891 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.891 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.891 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18015 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12779520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:57.891 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208630+0000", 2024-04-24T15:32:57.891 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 23.645334739999999, 2024-04-24T15:32:57.891 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043047328000000003, 2024-04-24T15:32:57.891 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:57.891 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:57.892 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:57.892 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:57.892 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:57.892 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18015 2024-04-24T15:32:57.892 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.892 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:57.892 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.892 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:57.892 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208630+0000", 2024-04-24T15:32:57.892 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.892 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.892 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.892 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:57.892 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208630+0000", 2024-04-24T15:32:57.892 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.892 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.893 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.893 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:57.893 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208629+0000", 2024-04-24T15:32:57.893 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999999 2024-04-24T15:32:57.893 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.893 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.893 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:57.893 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208634+0000", 2024-04-24T15:32:57.893 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8960000000000002e-06 2024-04-24T15:32:57.893 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.893 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.893 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:57.893 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208645+0000", 2024-04-24T15:32:57.893 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0939e-05 2024-04-24T15:32:57.894 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.894 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.894 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:57.894 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208651+0000", 2024-04-24T15:32:57.894 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.896e-06 2024-04-24T15:32:57.894 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.894 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.894 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:57.894 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251591+0000", 2024-04-24T15:32:57.894 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042939862000000002 2024-04-24T15:32:57.894 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.894 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.894 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:57.895 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251631+0000", 2024-04-24T15:32:57.895 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0216000000000003e-05 2024-04-24T15:32:57.895 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.895 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.895 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:57.895 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251677+0000", 2024-04-24T15:32:57.895 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6390999999999997e-05 2024-04-24T15:32:57.895 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.895 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:57.895 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.895 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.895 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.895 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18016 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12820480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:57.895 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208660+0000", 2024-04-24T15:32:57.895 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 23.645304951, 2024-04-24T15:32:57.896 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043127510000000001, 2024-04-24T15:32:57.896 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:57.896 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:57.896 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:57.896 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:57.896 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:57.896 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18016 2024-04-24T15:32:57.896 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.896 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:57.896 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.896 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:57.896 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208660+0000", 2024-04-24T15:32:57.896 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.896 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.897 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.897 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:57.897 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208660+0000", 2024-04-24T15:32:57.897 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.897 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.897 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.897 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:57.897 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208659+0000", 2024-04-24T15:32:57.897 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:57.897 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.897 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.897 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:57.897 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208662+0000", 2024-04-24T15:32:57.898 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.0979999999999998e-06 2024-04-24T15:32:57.898 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.898 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.898 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:57.898 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208669+0000", 2024-04-24T15:32:57.898 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.4470000000000003e-06 2024-04-24T15:32:57.898 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.898 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.898 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:57.898 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208673+0000", 2024-04-24T15:32:57.898 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2810000000000002e-06 2024-04-24T15:32:57.898 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.898 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.899 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:57.899 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251699+0000", 2024-04-24T15:32:57.899 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043025728999999999 2024-04-24T15:32:57.899 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.899 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.899 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:57.899 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251743+0000", 2024-04-24T15:32:57.899 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4469999999999999e-05 2024-04-24T15:32:57.899 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.899 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.899 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:57.899 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251787+0000", 2024-04-24T15:32:57.899 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3949999999999998e-05 2024-04-24T15:32:57.899 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.899 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:57.899 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.900 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.900 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.900 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18017 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12861440~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:57.900 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208721+0000", 2024-04-24T15:32:57.900 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 23.645243503, 2024-04-24T15:32:57.900 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043167749999999998, 2024-04-24T15:32:57.900 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:57.900 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:57.900 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:57.900 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:57.900 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:57.900 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18017 2024-04-24T15:32:57.900 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.900 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:57.900 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.900 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:57.901 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208721+0000", 2024-04-24T15:32:57.901 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.901 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.901 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.901 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:57.901 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208721+0000", 2024-04-24T15:32:57.901 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.901 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.901 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.901 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:57.901 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208720+0000", 2024-04-24T15:32:57.901 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:57.901 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.901 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.901 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:57.901 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208724+0000", 2024-04-24T15:32:57.901 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.3919999999999999e-06 2024-04-24T15:32:57.902 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.902 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.902 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:57.902 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208729+0000", 2024-04-24T15:32:57.902 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.3480000000000003e-06 2024-04-24T15:32:57.902 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.902 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.902 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:57.902 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208732+0000", 2024-04-24T15:32:57.902 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.9529999999999999e-06 2024-04-24T15:32:57.902 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.902 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.902 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:57.902 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251807+0000", 2024-04-24T15:32:57.902 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043074492999999998 2024-04-24T15:32:57.903 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.903 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.903 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:57.903 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251846+0000", 2024-04-24T15:32:57.903 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8958999999999997e-05 2024-04-24T15:32:57.903 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.903 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.903 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:57.903 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251889+0000", 2024-04-24T15:32:57.903 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3253000000000001e-05 2024-04-24T15:32:57.903 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.903 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:57.903 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.903 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.903 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.903 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18018 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12902400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:57.903 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208752+0000", 2024-04-24T15:32:57.904 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 23.64521289, 2024-04-24T15:32:57.904 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043241012000000002, 2024-04-24T15:32:57.904 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:57.904 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:57.904 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:57.904 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:57.904 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:57.904 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18018 2024-04-24T15:32:57.904 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.904 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:57.904 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.904 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:57.904 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208752+0000", 2024-04-24T15:32:57.904 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.905 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.905 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.905 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:57.905 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208752+0000", 2024-04-24T15:32:57.905 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.905 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.905 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.905 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:57.905 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208751+0000", 2024-04-24T15:32:57.905 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:57.905 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.905 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.905 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:57.906 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208753+0000", 2024-04-24T15:32:57.906 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.894e-06 2024-04-24T15:32:57.906 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.906 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.906 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:57.906 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208757+0000", 2024-04-24T15:32:57.906 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.856e-06 2024-04-24T15:32:57.906 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.906 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.906 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:57.906 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208760+0000", 2024-04-24T15:32:57.906 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.6749999999999998e-06 2024-04-24T15:32:57.906 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.906 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.906 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:57.907 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251909+0000", 2024-04-24T15:32:57.907 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043149145999999999 2024-04-24T15:32:57.907 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.907 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.907 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:57.907 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251948+0000", 2024-04-24T15:32:57.907 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9159000000000002e-05 2024-04-24T15:32:57.907 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.907 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.907 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:57.907 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251993+0000", 2024-04-24T15:32:57.907 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4638999999999997e-05 2024-04-24T15:32:57.907 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.907 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:57.907 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.907 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.908 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.908 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18019 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12943360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:57.908 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208938+0000", 2024-04-24T15:32:57.908 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 23.645026599000001, 2024-04-24T15:32:57.908 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043162272000000002, 2024-04-24T15:32:57.908 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:57.908 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:57.908 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:57.908 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:57.908 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:57.908 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18019 2024-04-24T15:32:57.908 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.908 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:57.908 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.908 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:57.908 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208938+0000", 2024-04-24T15:32:57.908 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.909 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.909 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.909 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:57.909 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208938+0000", 2024-04-24T15:32:57.909 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.909 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.909 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.909 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:57.909 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208762+0000", 2024-04-24T15:32:57.909 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999824 2024-04-24T15:32:57.909 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.909 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.909 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:57.909 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208942+0000", 2024-04-24T15:32:57.909 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000180008 2024-04-24T15:32:57.909 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.909 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.910 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:57.910 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208950+0000", 2024-04-24T15:32:57.910 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.1550000000000007e-06 2024-04-24T15:32:57.910 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.910 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.910 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:57.910 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208956+0000", 2024-04-24T15:32:57.910 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.7910000000000001e-06 2024-04-24T15:32:57.910 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.910 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.910 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:57.910 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252014+0000", 2024-04-24T15:32:57.910 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043058427000000003 2024-04-24T15:32:57.910 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.910 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.910 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:57.911 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252054+0000", 2024-04-24T15:32:57.911 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0071999999999999e-05 2024-04-24T15:32:57.911 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.911 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.911 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:57.911 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252100+0000", 2024-04-24T15:32:57.911 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5917000000000001e-05 2024-04-24T15:32:57.911 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.911 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:57.911 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.911 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.911 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.911 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18020 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12984320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:57.911 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.209092+0000", 2024-04-24T15:32:57.912 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 23.644872600999999, 2024-04-24T15:32:57.912 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043111295000000001, 2024-04-24T15:32:57.912 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:57.912 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:57.912 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:57.912 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:57.912 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:57.912 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18020 2024-04-24T15:32:57.912 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.912 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:57.912 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.912 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:57.912 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209092+0000", 2024-04-24T15:32:57.912 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.913 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.913 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.913 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:57.913 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209092+0000", 2024-04-24T15:32:57.913 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.913 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.913 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.913 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:57.913 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208964+0000", 2024-04-24T15:32:57.913 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998722 2024-04-24T15:32:57.913 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.913 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.913 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:57.913 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209095+0000", 2024-04-24T15:32:57.913 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000130648 2024-04-24T15:32:57.914 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.914 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.914 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:57.914 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209103+0000", 2024-04-24T15:32:57.914 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0320000000000003e-06 2024-04-24T15:32:57.914 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.914 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.914 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:57.914 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209108+0000", 2024-04-24T15:32:57.914 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8690000000000003e-06 2024-04-24T15:32:57.914 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.914 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.914 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:57.914 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252121+0000", 2024-04-24T15:32:57.915 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043013323999999999 2024-04-24T15:32:57.915 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.915 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.915 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:57.915 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252160+0000", 2024-04-24T15:32:57.915 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8376e-05 2024-04-24T15:32:57.915 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.915 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.915 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:57.915 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252203+0000", 2024-04-24T15:32:57.915 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.371e-05 2024-04-24T15:32:57.915 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.915 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:57.915 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.915 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.916 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.916 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18021 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 13025280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:57.916 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.209132+0000", 2024-04-24T15:32:57.916 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 23.644832243, 2024-04-24T15:32:57.916 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043183903000000003, 2024-04-24T15:32:57.916 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:57.916 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:57.916 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:57.916 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:57.916 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:57.916 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18021 2024-04-24T15:32:57.916 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.916 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:57.916 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.916 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:57.916 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209132+0000", 2024-04-24T15:32:57.917 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.917 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.917 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.917 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:57.917 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209132+0000", 2024-04-24T15:32:57.917 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.917 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.917 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.917 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:57.917 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209132+0000", 2024-04-24T15:32:57.917 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:57.917 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.917 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.917 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:57.917 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209134+0000", 2024-04-24T15:32:57.917 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.891e-06 2024-04-24T15:32:57.917 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.918 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.918 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:57.918 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209138+0000", 2024-04-24T15:32:57.918 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0659999999999997e-06 2024-04-24T15:32:57.918 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.918 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.918 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:57.918 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209140+0000", 2024-04-24T15:32:57.918 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.5339999999999998e-06 2024-04-24T15:32:57.918 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.918 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.918 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:57.918 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252232+0000", 2024-04-24T15:32:57.918 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043091142999999998 2024-04-24T15:32:57.918 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.919 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.919 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:57.919 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252270+0000", 2024-04-24T15:32:57.919 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8872000000000003e-05 2024-04-24T15:32:57.919 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.919 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.919 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:57.919 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252316+0000", 2024-04-24T15:32:57.919 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5788000000000003e-05 2024-04-24T15:32:57.919 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.919 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:57.919 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.919 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.919 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.919 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18022 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 13066240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:57.919 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.209411+0000", 2024-04-24T15:32:57.919 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 23.644553694999999, 2024-04-24T15:32:57.920 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043016616000000001, 2024-04-24T15:32:57.920 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:57.920 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:57.920 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:57.920 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:57.920 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:57.920 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18022 2024-04-24T15:32:57.920 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.920 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:57.920 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.920 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:57.920 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209411+0000", 2024-04-24T15:32:57.920 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.920 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.920 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.920 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:57.920 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209411+0000", 2024-04-24T15:32:57.921 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.921 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.921 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.921 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:57.921 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209143+0000", 2024-04-24T15:32:57.921 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999732 2024-04-24T15:32:57.921 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.921 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.921 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:57.921 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209415+0000", 2024-04-24T15:32:57.921 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000271873 2024-04-24T15:32:57.921 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.921 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.921 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:57.921 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209423+0000", 2024-04-24T15:32:57.921 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3080000000000008e-06 2024-04-24T15:32:57.921 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.922 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.922 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:57.922 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209428+0000", 2024-04-24T15:32:57.922 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.5040000000000002e-06 2024-04-24T15:32:57.922 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.922 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.922 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:57.922 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252339+0000", 2024-04-24T15:32:57.922 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042910419999999998 2024-04-24T15:32:57.922 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.922 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.922 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:57.923 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252380+0000", 2024-04-24T15:32:57.923 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0991000000000001e-05 2024-04-24T15:32:57.923 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.923 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.923 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:57.923 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252427+0000", 2024-04-24T15:32:57.923 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.7647e-05 2024-04-24T15:32:57.923 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.923 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:57.923 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.923 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.923 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.923 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18023 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 13107200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:57.923 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.209594+0000", 2024-04-24T15:32:57.923 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 23.644370181999999, 2024-04-24T15:32:57.924 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042938543000000003, 2024-04-24T15:32:57.924 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:57.924 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:57.924 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:57.924 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:57.924 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:57.924 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18023 2024-04-24T15:32:57.924 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.924 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:57.924 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.924 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:57.924 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209594+0000", 2024-04-24T15:32:57.924 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.924 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.925 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.925 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:57.925 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209594+0000", 2024-04-24T15:32:57.925 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.925 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.925 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.925 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:57.925 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209440+0000", 2024-04-24T15:32:57.925 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999846 2024-04-24T15:32:57.925 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.925 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.925 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:57.925 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209598+0000", 2024-04-24T15:32:57.925 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00015731100000000001 2024-04-24T15:32:57.926 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.926 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.926 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:57.926 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209604+0000", 2024-04-24T15:32:57.926 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.2910000000000004e-06 2024-04-24T15:32:57.926 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.926 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.926 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:57.926 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209607+0000", 2024-04-24T15:32:57.926 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5389999999999999e-06 2024-04-24T15:32:57.926 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.926 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.926 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:57.926 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252449+0000", 2024-04-24T15:32:57.927 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042841833000000003 2024-04-24T15:32:57.927 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.927 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.927 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:57.927 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252489+0000", 2024-04-24T15:32:57.927 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9898000000000003e-05 2024-04-24T15:32:57.927 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.927 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.927 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:57.927 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252533+0000", 2024-04-24T15:32:57.927 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3702999999999998e-05 2024-04-24T15:32:57.927 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.927 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:57.927 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.927 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.927 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.928 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18025 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 13189120~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:32:57.928 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.209832+0000", 2024-04-24T15:32:57.928 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 23.644132680999999, 2024-04-24T15:32:57.928 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042913556999999998, 2024-04-24T15:32:57.928 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:57.928 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:57.928 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:57.928 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:57.928 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:57.928 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18025 2024-04-24T15:32:57.928 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.928 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:57.928 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.928 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:57.929 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209832+0000", 2024-04-24T15:32:57.929 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.929 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.929 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.929 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:57.929 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209832+0000", 2024-04-24T15:32:57.929 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.929 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.929 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.929 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:57.929 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209774+0000", 2024-04-24T15:32:57.929 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999418 2024-04-24T15:32:57.929 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.929 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.930 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:57.930 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209835+0000", 2024-04-24T15:32:57.930 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.1377000000000005e-05 2024-04-24T15:32:57.930 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.930 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.930 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:57.930 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209843+0000", 2024-04-24T15:32:57.930 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.2330000000000006e-06 2024-04-24T15:32:57.930 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.930 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.930 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:57.930 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209848+0000", 2024-04-24T15:32:57.930 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6149999999999999e-06 2024-04-24T15:32:57.930 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.931 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.931 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:57.931 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252658+0000", 2024-04-24T15:32:57.931 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042810002 2024-04-24T15:32:57.931 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.931 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.931 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:57.931 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252697+0000", 2024-04-24T15:32:57.931 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8757999999999997e-05 2024-04-24T15:32:57.931 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.931 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.931 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:57.931 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252745+0000", 2024-04-24T15:32:57.931 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.863e-05 2024-04-24T15:32:57.932 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.932 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:57.932 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.932 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.932 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.932 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:24749 4.8 4:137f5701:::rbd_header.114ae15b9a20:head [watch reconnect cookie 140672595753696 gen 11] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-04-24T15:32:57.932 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:37.628703+0000", 2024-04-24T15:32:57.932 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 20.225261062000001, 2024-04-24T15:32:57.932 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0102918700000001, 2024-04-24T15:32:57.932 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:57.932 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:57.932 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:57.932 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:57.932 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:57.933 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 24749 2024-04-24T15:32:57.933 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.933 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:57.933 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.933 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:57.933 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628703+0000", 2024-04-24T15:32:57.933 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.933 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.933 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.933 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:57.933 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628703+0000", 2024-04-24T15:32:57.933 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.933 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.934 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.934 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:57.934 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628703+0000", 2024-04-24T15:32:57.934 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999999 2024-04-24T15:32:57.934 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.934 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.934 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:57.934 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628708+0000", 2024-04-24T15:32:57.934 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.4589999999999997e-06 2024-04-24T15:32:57.934 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.934 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.934 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:57.934 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628721+0000", 2024-04-24T15:32:57.934 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.2601e-05 2024-04-24T15:32:57.935 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.935 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.935 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:57.935 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628728+0000", 2024-04-24T15:32:57.935 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.9059999999999996e-06 2024-04-24T15:32:57.935 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.935 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.935 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:57.935 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628789+0000", 2024-04-24T15:32:57.935 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.1672000000000002e-05 2024-04-24T15:32:57.935 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.935 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.935 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting_for_map not empty", 2024-04-24T15:32:57.935 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628797+0000", 2024-04-24T15:32:57.936 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.8189999999999992e-06 2024-04-24T15:32:57.936 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.936 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.936 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:57.936 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.631255+0000", 2024-04-24T15:32:57.936 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0024577610000000001 2024-04-24T15:32:57.936 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.936 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.936 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for peered", 2024-04-24T15:32:57.936 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.631263+0000", 2024-04-24T15:32:57.936 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.4109999999999996e-06 2024-04-24T15:32:57.936 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.936 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.937 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:57.937 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:38.637560+0000", 2024-04-24T15:32:57.937 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0062970339999999 2024-04-24T15:32:57.937 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.937 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.937 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:57.937 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:38.637747+0000", 2024-04-24T15:32:57.937 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00018693400000000001 2024-04-24T15:32:57.937 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.937 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.937 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:57.937 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:38.638995+0000", 2024-04-24T15:32:57.937 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0012481300000000001 2024-04-24T15:32:57.938 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.938 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:57.938 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.938 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.938 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.938 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:24750 4.8 4:137f5701:::rbd_header.114ae15b9a20:head [notify cookie 140672594434416] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-04-24T15:32:57.938 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:37.628736+0000", 2024-04-24T15:32:57.938 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 20.225228802, 2024-04-24T15:32:57.938 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0096094309999999, 2024-04-24T15:32:57.938 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:32:57.938 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:32:57.938 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:32:57.938 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:32:57.939 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:32:57.939 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 24750 2024-04-24T15:32:57.939 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.939 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:32:57.939 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.939 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:32:57.939 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628736+0000", 2024-04-24T15:32:57.939 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.939 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.939 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.939 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:32:57.939 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628736+0000", 2024-04-24T15:32:57.939 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:32:57.940 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.940 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.940 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:32:57.940 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628735+0000", 2024-04-24T15:32:57.940 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:32:57.940 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.940 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.940 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:32:57.940 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628739+0000", 2024-04-24T15:32:57.940 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5109999999999999e-06 2024-04-24T15:32:57.940 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.940 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.940 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:32:57.940 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628745+0000", 2024-04-24T15:32:57.941 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.8300000000000001e-06 2024-04-24T15:32:57.941 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.941 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.941 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:32:57.941 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628749+0000", 2024-04-24T15:32:57.941 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.7110000000000003e-06 2024-04-24T15:32:57.941 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.941 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.941 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:57.941 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628826+0000", 2024-04-24T15:32:57.941 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.6678e-05 2024-04-24T15:32:57.941 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.941 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.941 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting_for_map not empty", 2024-04-24T15:32:57.941 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628834+0000", 2024-04-24T15:32:57.941 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.6190000000000002e-06 2024-04-24T15:32:57.942 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.942 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.942 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:57.942 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.631305+0000", 2024-04-24T15:32:57.942 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0024714400000000001 2024-04-24T15:32:57.942 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.942 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.942 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for peered", 2024-04-24T15:32:57.942 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.631313+0000", 2024-04-24T15:32:57.942 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.6529999999999994e-06 2024-04-24T15:32:57.942 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.942 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.942 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:32:57.942 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:38.638068+0000", 2024-04-24T15:32:57.943 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0067548369999999 2024-04-24T15:32:57.943 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.943 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.943 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:32:57.943 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:38.638158+0000", 2024-04-24T15:32:57.943 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.0079000000000003e-05 2024-04-24T15:32:57.943 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:32:57.943 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:32:57.943 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:32:57.943 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:38.638345+0000", 2024-04-24T15:32:57.943 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00018757100000000001 2024-04-24T15:32:57.943 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.943 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:57.943 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.943 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:32:57.943 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:32:57.944 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:32:57.944 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-24T15:32:57.946 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-24T15:32:57.947 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #13, image image_client.0-clone13 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent14 2024-04-24T15:32:57.947 INFO:teuthology.orchestra.run.smithi096.stdout:2994 write 0xe138459 thru 0xe13da48 (0x55f0 bytes) 2024-04-24T15:32:57.947 INFO:teuthology.orchestra.run.smithi096.stdout:2996 read 0x706e4c9 thru 0x706f1f9 (0xd31 bytes) 2024-04-24T15:32:57.947 INFO:teuthology.orchestra.run.smithi096.stdout:2997 write 0x17ea2f9 thru 0x17f48ac (0xa5b4 bytes) 2024-04-24T15:32:57.947 INFO:teuthology.orchestra.run.smithi096.stdout:3000 writesame 0x42cdd5a thru 0x42cdd7d data_size 0x2(0x24 bytes) 2024-04-24T15:32:57.947 INFO:teuthology.orchestra.run.smithi096.stdout:3003 write 0x6c924c5 thru 0x6c96d91 (0x48cd bytes) 2024-04-24T15:32:57.948 INFO:teuthology.orchestra.run.smithi096.stdout:3004 write 0xc70f02c thru 0xc711915 (0x28ea bytes) 2024-04-24T15:32:57.948 INFO:teuthology.orchestra.run.smithi096.stdout:3005 writesame 0x172d240 thru 0x172d55a data_size 0x5(0x31b bytes) 2024-04-24T15:32:57.948 INFO:teuthology.orchestra.run.smithi096.stdout:3006 read 0x24f9e0c thru 0x24fe54d (0x4742 bytes) 2024-04-24T15:32:57.948 INFO:teuthology.orchestra.run.smithi096.stdout:3007 write 0xc18ca43 thru 0xc193bca (0x7188 bytes) 2024-04-24T15:32:57.948 INFO:teuthology.orchestra.run.smithi096.stdout:3008 trunc from 0xe13da49 to 0x2aeda13 2024-04-24T15:32:57.948 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:32:57.945+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:32:57.964 INFO:teuthology.orchestra.run.smithi096.stdout:3009 write 0xaea027f thru 0xaead413 (0xd195 bytes) 2024-04-24T15:32:57.972 INFO:teuthology.orchestra.run.smithi096.stdout:3011 write 0x41c06fa thru 0x41c3741 (0x3048 bytes) 2024-04-24T15:32:57.975 INFO:teuthology.orchestra.run.smithi096.stdout:3013 punch from 0x4694954 to 0x46a0ecd, (0xc579 bytes) 2024-04-24T15:32:57.980 INFO:teuthology.orchestra.run.smithi096.stdout:3014 trunc from 0xaead414 to 0xea09bd7 2024-04-24T15:32:57.985 INFO:teuthology.orchestra.run.smithi096.stdout:3016 write 0x4c99043 thru 0x4c9a114 (0x10d2 bytes) 2024-04-24T15:32:57.989 INFO:teuthology.orchestra.run.smithi096.stdout:3017 write 0xa334765 thru 0xa344544 (0xfde0 bytes) 2024-04-24T15:32:58.007 INFO:teuthology.orchestra.run.smithi096.stdout:3018 trunc from 0xea09bd7 to 0x91e4acb 2024-04-24T15:32:58.021 INFO:teuthology.orchestra.run.smithi096.stdout:3020 trunc from 0x91e4acb to 0x9e3ba62 2024-04-24T15:32:58.036 INFO:teuthology.orchestra.run.smithi096.stdout:3021 write 0xe36285d thru 0xe3653de (0x2b82 bytes) 2024-04-24T15:32:58.040 INFO:teuthology.orchestra.run.smithi096.stdout:3022 write 0x507bb4c thru 0x508145c (0x5911 bytes) 2024-04-24T15:32:58.045 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:58.046 INFO:teuthology.orchestra.run.smithi096.stdout:3024 write 0xd8dd717 thru 0xd8e3b0c (0x63f6 bytes) 2024-04-24T15:32:58.046 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:32:58.046+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:32:58.049 INFO:teuthology.orchestra.run.smithi096.stdout:3025 writesame 0xd8f56e5 thru 0xd8fc53d data_size 0x29(0x6e59 bytes) 2024-04-24T15:32:58.068 INFO:teuthology.orchestra.run.smithi096.stdout:3027 trunc from 0xe3653df to 0xa5fc87 2024-04-24T15:32:58.101 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 6] out_osds: [1, 4, 5, 7] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-04-24T15:32:58.101 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-24T15:32:58.101 INFO:tasks.thrashosds.thrasher:Adding osd 4 2024-04-24T15:32:58.101 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 4 2024-04-24T15:32:58.102 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:32:58.102 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:32:58.103 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-04-24T15:32:58.103 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:32:58.105 INFO:teuthology.orchestra.run.smithi096.stdout:3031 read 0x3ca9a8 thru 0x3cbdb3 (0x140c bytes) 2024-04-24T15:32:58.105 INFO:teuthology.orchestra.run.smithi096.stdout:3033 read 0x64d332 thru 0x65663b (0x930a bytes) 2024-04-24T15:32:58.107 INFO:teuthology.orchestra.run.smithi096.stdout:3034 trunc from 0xa5fc87 to 0x21716cd 2024-04-24T15:32:58.116 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:58.117 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-24T15:32:58.121 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-04-24T15:32:58.125 INFO:teuthology.orchestra.run.smithi096.stdout:3039 writesame 0x4ce21d9 thru 0x4cea8da data_size 0xb(0x8702 bytes) 2024-04-24T15:32:58.125 INFO:teuthology.orchestra.run.smithi096.stdout:3041 punch from 0x374032a to 0x3749f2d, (0x9c03 bytes) 2024-04-24T15:32:58.125 INFO:teuthology.orchestra.run.smithi096.stdout:3042 write 0x39c82e3 thru 0x39d6cbf (0xe9dd bytes) 2024-04-24T15:32:58.135 INFO:teuthology.orchestra.run.smithi096.stdout:3043 punch from 0x33a94c9 to 0x33ad68c, (0x41c3 bytes) 2024-04-24T15:32:58.135 INFO:teuthology.orchestra.run.smithi096.stdout:3044 read 0x4ccd62 thru 0x4d315e (0x63fd bytes) 2024-04-24T15:32:58.135 INFO:teuthology.orchestra.run.smithi096.stdout:3045 write 0x49acb7c thru 0x49b42e8 (0x776d bytes) 2024-04-24T15:32:58.141 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:58.146 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:32:58.146+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:32:58.148 INFO:teuthology.orchestra.run.smithi096.stdout:3048 write 0x5d9009e thru 0x5d91f3e (0x1ea1 bytes) 2024-04-24T15:32:58.151 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-04-24T15:32:58.166 INFO:teuthology.orchestra.run.smithi096.stdout:3049 write 0xb9cd0cb thru 0xb9d5b5d (0x8a93 bytes) 2024-04-24T15:32:58.173 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-04-24T15:32:58.176 INFO:teuthology.orchestra.run.smithi096.stdout:3050 punch from 0xacd5985 to 0xace0ff7, (0xb672 bytes) 2024-04-24T15:32:58.189 INFO:teuthology.orchestra.run.smithi096.stdout:3051 trunc from 0xb9d5b5e to 0x15dd7f0 2024-04-24T15:32:58.212 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-04-24T15:32:58.222 INFO:teuthology.orchestra.run.smithi096.stdout:3053 writesame 0xd26a662 thru 0xd26f5e6 data_size 0x1(0x4f85 bytes) 2024-04-24T15:32:58.224 INFO:teuthology.orchestra.run.smithi096.stdout:3054 trunc from 0xd26f5e7 to 0xe5ce70b 2024-04-24T15:32:58.230 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:58.231 INFO:teuthology.orchestra.run.smithi096.stdout:3056 read 0xf43858 thru 0xf536fe (0xfea7 bytes) 2024-04-24T15:32:58.237 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-04-24T15:32:58.247 INFO:teuthology.orchestra.run.smithi096.stdout:3057 trunc from 0xe5ce70b to 0x2bd9314 2024-04-24T15:32:58.247 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:32:58.248+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:32:58.252 INFO:teuthology.orchestra.run.smithi096.stdout:3058 read 0xc1ce94 thru 0xc25d59 (0x8ec6 bytes) 2024-04-24T15:32:58.259 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-04-24T15:32:58.263 INFO:teuthology.orchestra.run.smithi096.stdout:3059 writesame 0x6b4ffe5 thru 0x6b5ef4c data_size 0x5e(0xef68 bytes) 2024-04-24T15:32:58.263 INFO:teuthology.orchestra.run.smithi096.stdout:3060 writesame 0x6c9377 thru 0x6d136d data_size 0x29(0x7ff7 bytes) 2024-04-24T15:32:58.283 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-04-24T15:32:58.287 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:58.287 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:58.292 INFO:teuthology.orchestra.run.smithi096.stdout:3061 write 0xd58bf1d thru 0xd596b15 (0xabf9 bytes) 2024-04-24T15:32:58.299 INFO:teuthology.orchestra.run.smithi096.stdout:3063 read 0x972a26d thru 0x972f722 (0x54b6 bytes) 2024-04-24T15:32:58.299 INFO:teuthology.orchestra.run.smithi096.stdout:3065 trunc from 0xd596b16 to 0xdb90beb 2024-04-24T15:32:58.300 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-24T15:32:58.305 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:58.305 INFO:teuthology.orchestra.run.smithi096.stdout:3068 read 0x525bba9 thru 0x5264c0f (0x9067 bytes) 2024-04-24T15:32:58.305 INFO:teuthology.orchestra.run.smithi096.stdout:3069 write 0xaaf7e0e thru 0xab00db7 (0x8faa bytes) 2024-04-24T15:32:58.309 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-04-24T15:32:58.312 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:58.312 INFO:teuthology.orchestra.run.smithi096.stdout:3072 punch from 0x75df911 to 0x75e6002, (0x66f1 bytes) 2024-04-24T15:32:58.312 INFO:teuthology.orchestra.run.smithi096.stdout:3073 read 0xc6de16d thru 0xc6eae62 (0xccf6 bytes) 2024-04-24T15:32:58.312 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:58.312 INFO:teuthology.orchestra.run.smithi096.stdout:3075 read 0x7ec179b thru 0x7ec2fdd (0x1843 bytes) 2024-04-24T15:32:58.312 INFO:teuthology.orchestra.run.smithi096.stdout:3076 write 0xb6daf28 thru 0xb6e4028 (0x9101 bytes) 2024-04-24T15:32:58.315 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T15:32:58.315 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T15:32:58.315 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T15:32:58.316 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T15:32:58.316 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T15:32:58.316 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T15:32:58.316 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T15:32:58.316 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T15:32:58.320 INFO:teuthology.orchestra.run.smithi096.stdout:3078 read 0x13b626c thru 0x13bd87c (0x7611 bytes) 2024-04-24T15:32:58.331 INFO:teuthology.orchestra.run.smithi096.stdout:3079 trunc from 0xdb90beb to 0x5ade7fd 2024-04-24T15:32:58.343 INFO:teuthology.orchestra.run.smithi096.stdout:3080 read 0x3d69036 thru 0x3d78b09 (0xfad4 bytes) 2024-04-24T15:32:58.348 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:32:58.348+0000 7f23abcf6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 44021) UID: 0 2024-04-24T15:32:58.349 INFO:teuthology.orchestra.run.smithi096.stdout:3081 trunc from 0x5ade7fd to 0x9d2388e 2024-04-24T15:32:58.355 INFO:teuthology.orchestra.run.smithi096.stdout:3084 read 0x2e06998 thru 0x2e133d8 (0xca41 bytes) 2024-04-24T15:32:58.355 INFO:teuthology.orchestra.run.smithi096.stdout:3085 punch from 0x5ac07 to 0x62a9d, (0x7e96 bytes) 2024-04-24T15:32:58.355 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:58.355 INFO:teuthology.orchestra.run.smithi096.stdout:3088 punch from 0x5fe16da to 0x5fe89f5, (0x731b bytes) 2024-04-24T15:32:58.355 INFO:teuthology.orchestra.run.smithi096.stdout:3090 punch from 0x112a42c to 0x1131248, (0x6e1c bytes) 2024-04-24T15:32:58.368 INFO:teuthology.orchestra.run.smithi096.stdout:3093 trunc from 0x9d2388e to 0x65bd9a 2024-04-24T15:32:58.400 INFO:teuthology.orchestra.run.smithi096.stdout:3095 write 0x5bc4540 thru 0x5bc5884 (0x1345 bytes) 2024-04-24T15:32:58.415 INFO:teuthology.orchestra.run.smithi096.stdout:3096 writesame 0x9c38a83 thru 0x9c47054 data_size 0x17(0xe5d2 bytes) 2024-04-24T15:32:58.415 INFO:teuthology.orchestra.run.smithi096.stdout:3097 writesame 0x99fb26c thru 0x99ff7dd data_size 0x3(0x4572 bytes) 2024-04-24T15:32:58.423 INFO:teuthology.orchestra.run.smithi096.stdout:3098 writesame 0x8771eac thru 0x8781c09 data_size 0x29(0xfd5e bytes) 2024-04-24T15:32:58.431 INFO:teuthology.orchestra.run.smithi096.stdout:3099 write 0x7f3e034 thru 0x7f4d172 (0xf13f bytes) 2024-04-24T15:32:58.442 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:58.442 INFO:teuthology.orchestra.run.smithi096.stdout:3102 punch from 0x1a8ccd5 to 0x1a91a80, (0x4dab bytes) 2024-04-24T15:32:58.442 INFO:teuthology.orchestra.run.smithi096.stdout:3103 writesame 0x41bf11a thru 0x41cb97f data_size 0x2(0xc866 bytes) 2024-04-24T15:32:58.448 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:32:58.449+0000 7f23abcf6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 44021) UID: 0 2024-04-24T15:32:58.455 INFO:teuthology.orchestra.run.smithi096.stdout:3104 writesame 0xc4f264 thru 0xc5c4d7 data_size 0x2(0xd274 bytes) 2024-04-24T15:32:58.471 INFO:teuthology.orchestra.run.smithi096.stdout:3106 read 0x8d33f10 thru 0x8d3e669 (0xa75a bytes) 2024-04-24T15:32:58.471 INFO:teuthology.orchestra.run.smithi096.stdout:3108 writesame 0x64db751 thru 0x64e19c8 data_size 0x17(0x6278 bytes) 2024-04-24T15:32:58.472 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:58.472 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:58.477 INFO:teuthology.orchestra.run.smithi096.stdout:3109 write 0x814a628 thru 0x814f5db (0x4fb4 bytes) 2024-04-24T15:32:58.484 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-24T15:32:58.487 INFO:teuthology.orchestra.run.smithi096.stdout:3111 read 0x5fb2bcf thru 0x5fbcda3 (0xa1d5 bytes) 2024-04-24T15:32:58.487 INFO:teuthology.orchestra.run.smithi096.stdout:3113 punch from 0x1b15e9e to 0x1b1b5a4, (0x5706 bytes) 2024-04-24T15:32:58.488 INFO:teuthology.orchestra.run.smithi096.stdout:3115 write 0x4e304fa thru 0x4e30557 (0x5e bytes) 2024-04-24T15:32:58.492 INFO:teuthology.orchestra.run.smithi096.stdout:3116 read 0x2ebfbcb thru 0x2ebfcce (0x104 bytes) 2024-04-24T15:32:58.499 INFO:teuthology.orchestra.run.smithi096.stdout:3117 write 0x9d9d10a thru 0x9da8bc4 (0xbabb bytes) 2024-04-24T15:32:58.508 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:58.509 INFO:teuthology.orchestra.run.smithi096.stdout:3119 write 0x77cc31f thru 0x77d9448 (0xd12a bytes) 2024-04-24T15:32:58.514 INFO:teuthology.orchestra.run.smithi096.stdout:3120 writesame 0x247504a thru 0x247b111 data_size 0x4c(0x60c8 bytes) 2024-04-24T15:32:58.520 INFO:teuthology.orchestra.run.smithi096.stdout:3121 punch from 0x97d205b to 0x97d879f, (0x6744 bytes) 2024-04-24T15:32:58.526 INFO:teuthology.orchestra.run.smithi096.stdout:3122 trunc from 0x9da8bc5 to 0x490abac 2024-04-24T15:32:58.543 INFO:teuthology.orchestra.run.smithi096.stdout:3123 read 0x16ead9a thru 0x16fa954 (0xfbbb bytes) 2024-04-24T15:32:58.543 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:58.543 INFO:teuthology.orchestra.run.smithi096.stdout:3125 read 0x379d511 thru 0x37a2683 (0x5173 bytes) 2024-04-24T15:32:58.543 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:32:58.543 INFO:teuthology.orchestra.run.smithi096.stdout:3127 read 0x42ada94 thru 0x42b8288 (0xa7f5 bytes) 2024-04-24T15:32:58.546 INFO:tasks.ceph.mon.a.smithi096.stderr:2024-04-24T15:32:58.546+0000 7fa88ddc8700 -1 mon.a@0(leader).osd e64 definitely_dead 0 2024-04-24T15:32:58.549 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:32:58.549+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:32:58.550 INFO:teuthology.orchestra.run.smithi096.stdout:3128 write 0x7a5cc48 thru 0x7a68084 (0xb43d bytes) 2024-04-24T15:32:58.556 INFO:teuthology.orchestra.run.smithi096.stdout:3129 read 0x4d6733 thru 0x4d73d6 (0xca4 bytes) 2024-04-24T15:32:58.556 INFO:teuthology.orchestra.run.smithi096.stdout:3130 write 0x3db48ce thru 0x3db84aa (0x3bdd bytes) 2024-04-24T15:32:58.570 INFO:teuthology.orchestra.run.smithi096.stdout:3131 writesame 0xbeecfcd thru 0xbef9911 data_size 0x4b(0xc945 bytes) 2024-04-24T15:32:58.570 INFO:teuthology.orchestra.run.smithi096.stdout:3134 punch from 0x41e713d to 0x41e9404, (0x22c7 bytes) 2024-04-24T15:32:58.571 INFO:teuthology.orchestra.run.smithi096.stdout:3135 write 0x7961d33 thru 0x796a4af (0x877d bytes) 2024-04-24T15:32:58.572 INFO:teuthology.orchestra.run.smithi096.stdout:3136 write 0x40071f3 thru 0x4013f5f (0xcd6d bytes) 2024-04-24T15:32:58.585 INFO:teuthology.orchestra.run.smithi096.stdout:3137 write 0xe0ba0e0 thru 0xe0be240 (0x4161 bytes) 2024-04-24T15:32:58.589 INFO:teuthology.orchestra.run.smithi096.stdout:3138 write 0x934b2e thru 0x93cea0 (0x8373 bytes) 2024-04-24T15:32:58.601 INFO:teuthology.orchestra.run.smithi096.stdout:3140 write 0x2195fc7 thru 0x21a40d1 (0xe10b bytes) 2024-04-24T15:32:58.614 INFO:teuthology.orchestra.run.smithi096.stdout:3144 write 0x2478fe5 thru 0x247f27a (0x6296 bytes) 2024-04-24T15:32:58.617 INFO:teuthology.orchestra.run.smithi096.stdout:3145 read 0x90f78f0 thru 0x90f9e48 (0x2559 bytes) 2024-04-24T15:32:58.617 INFO:teuthology.orchestra.run.smithi096.stdout:3146 writesame 0xb87c0ca thru 0xb87dea0 data_size 0x1(0x1dd7 bytes) 2024-04-24T15:32:58.623 INFO:teuthology.orchestra.run.smithi096.stdout:3147 write 0xabb748e thru 0xabc06eb (0x925e bytes) 2024-04-24T15:32:58.639 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:58.639 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:58.640 INFO:teuthology.orchestra.run.smithi096.stdout:3150 trunc from 0xe0be241 to 0x3fcdfba 2024-04-24T15:32:58.650 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:32:58.651+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:32:58.653 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-24T15:32:58.668 INFO:teuthology.orchestra.run.smithi096.stdout:3151 writesame 0xb5a1f96 thru 0xb5a7917 data_size 0x72(0x5982 bytes) 2024-04-24T15:32:58.668 INFO:teuthology.orchestra.run.smithi096.stdout:3152 punch from 0x8ac1ddd to 0x8ad01da, (0xe3fd bytes) 2024-04-24T15:32:58.676 INFO:teuthology.orchestra.run.smithi096.stdout:3153 writesame 0xdece4bb thru 0xded7be3 data_size 0x1(0x9729 bytes) 2024-04-24T15:32:58.676 INFO:teuthology.orchestra.run.smithi096.stdout:3154 write 0xd457195 thru 0xd45e566 (0x73d2 bytes) 2024-04-24T15:32:58.692 INFO:teuthology.orchestra.run.smithi096.stdout:3155 writesame 0xe6bb6cf thru 0xe6c1d60 data_size 0x13(0x6692 bytes) 2024-04-24T15:32:58.692 INFO:teuthology.orchestra.run.smithi096.stdout:3156 writesame 0x7755c55 thru 0x775e578 data_size 0x83(0x8924 bytes) 2024-04-24T15:32:58.702 INFO:teuthology.orchestra.run.smithi096.stdout:3157 write 0x2c503aa thru 0x2c558e1 (0x5538 bytes) 2024-04-24T15:32:58.719 INFO:teuthology.orchestra.run.smithi096.stdout:3158 trunc from 0xe6c1d61 to 0x5ade951 2024-04-24T15:32:58.744 INFO:teuthology.orchestra.run.smithi096.stderr:marked in osd.4. 2024-04-24T15:32:58.751 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:32:58.752+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:32:58.755 INFO:tasks.thrashosds.thrasher:Added osd 4 2024-04-24T15:32:58.807 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:58.807 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:58.813 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-24T15:32:58.816 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-24T15:32:58.858 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:32:58.858+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:32:58.952 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:32:58.952+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:32:58.990 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:58.990 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:58.999 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-24T15:32:59.083 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:32:59.083+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:32:59.154 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:32:59.154+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:32:59.164 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:59.164 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:59.174 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-24T15:32:59.258 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:32:59.258+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:32:59.337 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:59.337 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:59.347 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-24T15:32:59.356 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:32:59.356+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:32:59.456 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:32:59.456+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:32:59.512 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:59.512 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:59.522 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-24T15:32:59.557 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:32:59.558+0000 7f23abcf6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 44021) UID: 0 2024-04-24T15:32:59.658 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:32:59.659+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:32:59.685 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:59.685 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:59.695 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-24T15:32:59.750 INFO:teuthology.orchestra.run.smithi096.stderr:noscrub is unset 2024-04-24T15:32:59.759 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:32:59.760+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:32:59.861 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:32:59.861+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:32:59.862 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:32:59.862 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:32:59.872 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-24T15:32:59.962 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:32:59.962+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:00.036 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:00.036 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:00.046 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-24T15:33:00.087 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:33:00.087+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:33:00.164 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:33:00.164+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:33:00.182 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:00.182 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:00.192 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-24T15:33:00.265 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:33:00.265+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:33:00.315 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:00.315 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:00.325 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-24T15:33:00.332 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-24T15:33:00.376 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:00.376+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:00.467 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:00.467+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:00.471 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-04-24T15:33:00.489 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:00.489 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:00.489 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-04-24T15:33:00.498 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-24T15:33:00.506 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-04-24T15:33:00.524 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-04-24T15:33:00.542 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-04-24T15:33:00.560 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-04-24T15:33:00.578 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-04-24T15:33:00.582 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:00.583+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:00.597 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-04-24T15:33:00.598 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T15:33:00.598 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T15:33:00.598 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T15:33:00.599 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T15:33:00.599 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T15:33:00.599 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T15:33:00.599 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T15:33:00.599 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T15:33:00.663 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:00.663 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:33:00.663 INFO:teuthology.orchestra.run.smithi151.stdout: "num_ops": 0 2024-04-24T15:33:00.663 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:00.668 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:00.669+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:00.673 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-24T15:33:00.769 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:00.769+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:00.774 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:00.774 INFO:teuthology.orchestra.run.smithi096.stdout:3160 read 0x47ca1d1 thru 0x47d7bcd (0xd9fd bytes) 2024-04-24T15:33:00.774 INFO:teuthology.orchestra.run.smithi096.stdout:3161 punch from 0x16a714 to 0x171ea2, (0x778e bytes) 2024-04-24T15:33:00.774 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:00.784 INFO:teuthology.orchestra.run.smithi096.stdout:3165 write 0x995ab51 thru 0x99689bf (0xde6f bytes) 2024-04-24T15:33:00.792 INFO:teuthology.orchestra.run.smithi096.stdout:3166 read 0x4bf2fc4 thru 0x4c0084f (0xd88c bytes) 2024-04-24T15:33:00.793 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:00.793 INFO:teuthology.orchestra.run.smithi096.stdout:3168 trunc from 0x99689c0 to 0x8fe50bf 2024-04-24T15:33:00.803 INFO:teuthology.orchestra.run.smithi096.stdout:3170 read 0x8e9a9c5 thru 0x8ea4dd0 (0xa40c bytes) 2024-04-24T15:33:00.803 INFO:teuthology.orchestra.run.smithi096.stdout:3171 read 0x8b64fa7 thru 0x8b71fd4 (0xd02e bytes) 2024-04-24T15:33:00.803 INFO:teuthology.orchestra.run.smithi096.stdout:3172 write 0x70301e5 thru 0x7035412 (0x522e bytes) 2024-04-24T15:33:00.813 INFO:teuthology.orchestra.run.smithi096.stdout:3173 write 0xae78169 thru 0xae824df (0xa377 bytes) 2024-04-24T15:33:00.821 INFO:teuthology.orchestra.run.smithi096.stdout:3174 write 0x66c7793 thru 0x66cd2c9 (0x5b37 bytes) 2024-04-24T15:33:00.833 INFO:teuthology.orchestra.run.smithi096.stdout:3175 write 0xb8fbf60 thru 0xb90b31a (0xf3bb bytes) 2024-04-24T15:33:00.843 INFO:teuthology.orchestra.run.smithi096.stdout:3176 write 0x5bd251d thru 0x5bdf571 (0xd055 bytes) 2024-04-24T15:33:00.849 INFO:teuthology.orchestra.run.smithi096.stdout:3177 write 0xa9b6eca thru 0xa9c5c5a (0xed91 bytes) 2024-04-24T15:33:00.859 INFO:teuthology.orchestra.run.smithi096.stdout:3181 read 0x7af3d41 thru 0x7af4042 (0x302 bytes) 2024-04-24T15:33:00.859 INFO:teuthology.orchestra.run.smithi096.stdout:3182 read 0x79251bf thru 0x7930f18 (0xbd5a bytes) 2024-04-24T15:33:00.859 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:00.859 INFO:teuthology.orchestra.run.smithi096.stdout:3186 writesame 0x7b8614c thru 0x7b8fa93 data_size 0x78(0x9948 bytes) 2024-04-24T15:33:00.866 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:00.867 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:33:00.867 INFO:teuthology.orchestra.run.smithi151.stdout: "complaint_time": 30, 2024-04-24T15:33:00.867 INFO:teuthology.orchestra.run.smithi151.stdout: "num_blocked_ops": 0 2024-04-24T15:33:00.867 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:00.870 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:00.870+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:00.873 INFO:teuthology.orchestra.run.smithi096.stdout:3187 trunc from 0xb90b31b to 0x4408cdd 2024-04-24T15:33:00.878 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-24T15:33:00.896 INFO:teuthology.orchestra.run.smithi096.stdout:3189 write 0x77f0f6f thru 0x77ff5e9 (0xe67b bytes) 2024-04-24T15:33:00.903 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:00.903 INFO:teuthology.orchestra.run.smithi096.stdout:3192 read 0x49f9bf4 thru 0x49fe633 (0x4a40 bytes) 2024-04-24T15:33:00.913 INFO:teuthology.orchestra.run.smithi096.stdout:3195 write 0xa88001d thru 0xa88f23c (0xf220 bytes) 2024-04-24T15:33:00.931 INFO:teuthology.orchestra.run.smithi096.stdout:3197 trunc from 0xa88f23d to 0x41111e7 2024-04-24T15:33:00.942 INFO:teuthology.orchestra.run.smithi096.stdout:3198 write 0x65c4f1 thru 0x65d542 (0x1052 bytes) 2024-04-24T15:33:00.943 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:00.943 INFO:teuthology.orchestra.run.smithi096.stdout:3201 read 0x369ece0 thru 0x36a746c (0x878d bytes) 2024-04-24T15:33:00.944 INFO:teuthology.orchestra.run.smithi096.stdout:3202 punch from 0x1cedb1f to 0x1cfaef3, (0xd3d4 bytes) 2024-04-24T15:33:00.944 INFO:teuthology.orchestra.run.smithi096.stdout:3204 write 0x406d9da thru 0x4075ad4 (0x80fb bytes) 2024-04-24T15:33:00.949 INFO:teuthology.orchestra.run.smithi096.stdout:3205 trunc from 0x41111e7 to 0x2ae7cb5 2024-04-24T15:33:00.971 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:33:00.971+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:33:00.978 INFO:teuthology.orchestra.run.smithi096.stdout:3206 writesame 0xc96a159 thru 0xc978362 data_size 0x1(0xe20a bytes) 2024-04-24T15:33:00.982 INFO:teuthology.orchestra.run.smithi096.stdout:3207 write 0xc9efd58 thru 0xc9ff256 (0xf4ff bytes) 2024-04-24T15:33:00.995 INFO:teuthology.orchestra.run.smithi096.stdout:3208 read 0x7653cac thru 0x7659b70 (0x5ec5 bytes) 2024-04-24T15:33:00.995 INFO:teuthology.orchestra.run.smithi096.stdout:3209 writesame 0x3438c72 thru 0x343da8b data_size 0xd(0x4e1a bytes) 2024-04-24T15:33:01.000 INFO:teuthology.orchestra.run.smithi096.stdout:3210 write 0xa4fecc5 thru 0xa4fef14 (0x250 bytes) 2024-04-24T15:33:01.003 INFO:teuthology.orchestra.run.smithi096.stdout:3211 write 0x97b1c17 thru 0x97b1f2c (0x316 bytes) 2024-04-24T15:33:01.007 INFO:teuthology.orchestra.run.smithi096.stdout:3213 write 0x5516310 thru 0x55192ac (0x2f9d bytes) 2024-04-24T15:33:01.012 INFO:teuthology.orchestra.run.smithi096.stdout:3215 write 0x8580faa thru 0x85893a0 (0x83f7 bytes) 2024-04-24T15:33:01.020 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:01.020 INFO:teuthology.orchestra.run.smithi151.stdout: "size": 20, 2024-04-24T15:33:01.020 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 600, 2024-04-24T15:33:01.020 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [ 2024-04-24T15:33:01.020 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.020 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23613 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 14688256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:01.020 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.792522+0000", 2024-04-24T15:33:01.020 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 26.227363367999999, 2024-04-24T15:33:01.020 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040507356000000001, 2024-04-24T15:33:01.020 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:01.021 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:01.021 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:01.021 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:01.021 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:01.021 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23613 2024-04-24T15:33:01.021 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.021 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:01.021 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.021 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:01.021 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792522+0000", 2024-04-24T15:33:01.021 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.021 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.022 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.022 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:01.022 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792522+0000", 2024-04-24T15:33:01.022 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.022 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.022 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.022 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:01.022 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792522+0000", 2024-04-24T15:33:01.022 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:01.022 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.022 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.022 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:01.022 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792523+0000", 2024-04-24T15:33:01.023 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.576e-06 2024-04-24T15:33:01.023 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.023 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.023 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:01.023 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792526+0000", 2024-04-24T15:33:01.023 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7439999999999999e-06 2024-04-24T15:33:01.023 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.023 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.023 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:01.023 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792528+0000", 2024-04-24T15:33:01.023 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.0600000000000002e-06 2024-04-24T15:33:01.023 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.023 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.023 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:01.024 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.832864+0000", 2024-04-24T15:33:01.024 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040335742000000001 2024-04-24T15:33:01.024 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.024 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.024 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:01.024 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.832953+0000", 2024-04-24T15:33:01.024 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.9281000000000001e-05 2024-04-24T15:33:01.024 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.024 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.024 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:01.024 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.833029+0000", 2024-04-24T15:33:01.024 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.6259000000000003e-05 2024-04-24T15:33:01.024 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.024 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:01.025 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.025 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.025 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.025 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23614 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 14729216~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:01.025 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.792530+0000", 2024-04-24T15:33:01.025 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 26.227355104000001, 2024-04-24T15:33:01.025 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040633957999999998, 2024-04-24T15:33:01.025 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:01.025 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:01.025 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:01.025 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:01.025 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:01.025 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23614 2024-04-24T15:33:01.025 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.026 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:01.026 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.026 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:01.026 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792530+0000", 2024-04-24T15:33:01.026 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.026 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.026 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.026 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:01.026 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792530+0000", 2024-04-24T15:33:01.026 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.026 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.026 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.026 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:01.026 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792530+0000", 2024-04-24T15:33:01.027 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:01.027 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.027 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.027 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:01.027 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792531+0000", 2024-04-24T15:33:01.027 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.46e-06 2024-04-24T15:33:01.027 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.027 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.027 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:01.027 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792534+0000", 2024-04-24T15:33:01.027 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8959999999999999e-06 2024-04-24T15:33:01.027 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.028 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.028 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:01.028 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792536+0000", 2024-04-24T15:33:01.028 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.841e-06 2024-04-24T15:33:01.028 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.028 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.028 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:01.028 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.833060+0000", 2024-04-24T15:33:01.028 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040524125000000001 2024-04-24T15:33:01.028 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.028 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.028 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:01.028 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.833108+0000", 2024-04-24T15:33:01.028 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.7547999999999999e-05 2024-04-24T15:33:01.029 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.029 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.029 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:01.029 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.833164+0000", 2024-04-24T15:33:01.029 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6297e-05 2024-04-24T15:33:01.029 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.029 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:01.029 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.029 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.029 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.029 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23623 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15097856~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:01.029 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.793887+0000", 2024-04-24T15:33:01.029 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 26.225998143000002, 2024-04-24T15:33:01.029 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040432784999999999, 2024-04-24T15:33:01.029 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:01.029 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:01.030 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:01.030 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:01.030 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:01.030 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23623 2024-04-24T15:33:01.030 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.030 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:01.030 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.030 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:01.030 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793887+0000", 2024-04-24T15:33:01.030 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.030 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.030 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.030 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:01.031 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793887+0000", 2024-04-24T15:33:01.031 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.031 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.031 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.031 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:01.031 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793887+0000", 2024-04-24T15:33:01.031 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:01.031 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.031 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.031 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:01.031 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793888+0000", 2024-04-24T15:33:01.031 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3659999999999999e-06 2024-04-24T15:33:01.031 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.031 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.031 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:01.031 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793891+0000", 2024-04-24T15:33:01.032 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8389999999999998e-06 2024-04-24T15:33:01.032 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.032 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.032 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:01.032 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793893+0000", 2024-04-24T15:33:01.032 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.9549999999999999e-06 2024-04-24T15:33:01.032 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.032 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.032 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:01.032 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834239+0000", 2024-04-24T15:33:01.032 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040345487999999999 2024-04-24T15:33:01.032 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.032 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.032 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:01.032 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834280+0000", 2024-04-24T15:33:01.032 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.1878000000000001e-05 2024-04-24T15:33:01.033 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.033 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.033 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:01.033 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834320+0000", 2024-04-24T15:33:01.033 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9480000000000001e-05 2024-04-24T15:33:01.033 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.033 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:01.033 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.033 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.033 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.033 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23624 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15138816~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:01.033 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.794053+0000", 2024-04-24T15:33:01.033 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 26.225832194999999, 2024-04-24T15:33:01.033 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040370801999999997, 2024-04-24T15:33:01.033 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:01.034 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:01.034 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:01.034 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:01.034 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:01.034 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23624 2024-04-24T15:33:01.034 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.034 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:01.034 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.034 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:01.034 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794053+0000", 2024-04-24T15:33:01.034 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.034 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.034 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.035 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:01.035 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794053+0000", 2024-04-24T15:33:01.035 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.035 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.035 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.035 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:01.035 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793895+0000", 2024-04-24T15:33:01.035 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998422 2024-04-24T15:33:01.035 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.035 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.035 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:01.035 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794057+0000", 2024-04-24T15:33:01.035 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00016149400000000001 2024-04-24T15:33:01.035 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.036 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.036 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:01.036 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794065+0000", 2024-04-24T15:33:01.036 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.5259999999999993e-06 2024-04-24T15:33:01.036 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.036 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.036 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:01.036 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794071+0000", 2024-04-24T15:33:01.036 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6149999999999996e-06 2024-04-24T15:33:01.036 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.036 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.036 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:01.036 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834341+0000", 2024-04-24T15:33:01.036 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040270366000000002 2024-04-24T15:33:01.036 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.037 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.037 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:01.037 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834381+0000", 2024-04-24T15:33:01.037 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0102999999999999e-05 2024-04-24T15:33:01.037 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.037 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.037 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:01.037 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834424+0000", 2024-04-24T15:33:01.037 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2713000000000003e-05 2024-04-24T15:33:01.037 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.037 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:01.037 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.037 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.037 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.037 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23625 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15179776~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:01.038 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.794160+0000", 2024-04-24T15:33:01.038 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 26.225725313000002, 2024-04-24T15:33:01.038 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040413004000000002, 2024-04-24T15:33:01.038 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:01.038 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:01.038 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:01.038 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:01.038 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:01.038 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23625 2024-04-24T15:33:01.038 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.038 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:01.038 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.038 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:01.038 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794160+0000", 2024-04-24T15:33:01.039 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.039 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.039 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.039 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:01.039 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794160+0000", 2024-04-24T15:33:01.039 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.039 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.039 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.039 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:01.039 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794086+0000", 2024-04-24T15:33:01.039 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999256 2024-04-24T15:33:01.039 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.039 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.039 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:01.039 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794163+0000", 2024-04-24T15:33:01.040 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7896000000000001e-05 2024-04-24T15:33:01.040 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.040 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.040 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:01.040 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794171+0000", 2024-04-24T15:33:01.040 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.9500000000000001e-06 2024-04-24T15:33:01.040 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.040 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.040 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:01.040 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794179+0000", 2024-04-24T15:33:01.040 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7970000000000001e-06 2024-04-24T15:33:01.040 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.040 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.040 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:01.040 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834451+0000", 2024-04-24T15:33:01.040 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040271492999999998 2024-04-24T15:33:01.041 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.041 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.041 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:01.041 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834519+0000", 2024-04-24T15:33:01.041 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.7984000000000005e-05 2024-04-24T15:33:01.041 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.041 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.041 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:01.041 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834573+0000", 2024-04-24T15:33:01.041 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.4339999999999998e-05 2024-04-24T15:33:01.041 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.041 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:01.041 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.041 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.041 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.042 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23629 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15343616~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:01.042 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.794748+0000", 2024-04-24T15:33:01.042 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 26.225137511, 2024-04-24T15:33:01.042 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040419718, 2024-04-24T15:33:01.042 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:01.042 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:01.042 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:01.042 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:01.042 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:01.042 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23629 2024-04-24T15:33:01.042 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.042 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:01.042 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.042 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:01.043 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794748+0000", 2024-04-24T15:33:01.043 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.043 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.043 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.043 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:01.043 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794748+0000", 2024-04-24T15:33:01.043 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.043 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.043 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.043 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:01.043 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794673+0000", 2024-04-24T15:33:01.043 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999251 2024-04-24T15:33:01.043 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.043 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.043 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:01.044 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794751+0000", 2024-04-24T15:33:01.044 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.8480999999999996e-05 2024-04-24T15:33:01.044 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.044 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.044 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:01.044 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794760+0000", 2024-04-24T15:33:01.044 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3429999999999996e-06 2024-04-24T15:33:01.044 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.044 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.044 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:01.044 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794764+0000", 2024-04-24T15:33:01.044 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5600000000000004e-06 2024-04-24T15:33:01.044 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.044 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.044 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:01.044 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835040+0000", 2024-04-24T15:33:01.045 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040275188000000003 2024-04-24T15:33:01.045 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.045 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.045 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:01.045 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835105+0000", 2024-04-24T15:33:01.045 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.5992000000000001e-05 2024-04-24T15:33:01.045 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.045 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.045 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:01.045 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835167+0000", 2024-04-24T15:33:01.045 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.1981000000000002e-05 2024-04-24T15:33:01.045 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.045 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:01.045 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.046 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.046 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.046 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23630 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15384576~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:01.046 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.794839+0000", 2024-04-24T15:33:01.046 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 26.225046442, 2024-04-24T15:33:01.046 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040523894999999997, 2024-04-24T15:33:01.046 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:01.046 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:01.046 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:01.046 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:01.046 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:01.046 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23630 2024-04-24T15:33:01.046 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.046 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:01.046 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.047 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:01.047 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794839+0000", 2024-04-24T15:33:01.047 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.047 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.047 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.047 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:01.047 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794839+0000", 2024-04-24T15:33:01.047 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.047 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.047 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.047 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:01.047 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794768+0000", 2024-04-24T15:33:01.047 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999294 2024-04-24T15:33:01.048 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.048 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.048 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:01.048 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794841+0000", 2024-04-24T15:33:01.048 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2793000000000006e-05 2024-04-24T15:33:01.048 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.048 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.048 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:01.048 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794846+0000", 2024-04-24T15:33:01.048 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.7230000000000004e-06 2024-04-24T15:33:01.049 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.049 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.049 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:01.049 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794848+0000", 2024-04-24T15:33:01.049 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7659999999999999e-06 2024-04-24T15:33:01.049 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.049 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.049 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:01.049 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835223+0000", 2024-04-24T15:33:01.050 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040374275000000001 2024-04-24T15:33:01.050 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.050 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.050 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:01.050 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835295+0000", 2024-04-24T15:33:01.050 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2633999999999996e-05 2024-04-24T15:33:01.050 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.050 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.050 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:01.050 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835363+0000", 2024-04-24T15:33:01.050 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.7355999999999997e-05 2024-04-24T15:33:01.050 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.050 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:01.051 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.051 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.051 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.051 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23631 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15425536~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:01.051 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.794956+0000", 2024-04-24T15:33:01.051 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 26.224929600999999, 2024-04-24T15:33:01.051 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040562383, 2024-04-24T15:33:01.051 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:01.051 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:01.051 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:01.051 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:01.051 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:01.052 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23631 2024-04-24T15:33:01.052 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.052 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:01.052 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.052 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:01.052 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794956+0000", 2024-04-24T15:33:01.052 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.052 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.052 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.052 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:01.052 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794956+0000", 2024-04-24T15:33:01.052 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.052 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.053 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.053 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:01.053 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794859+0000", 2024-04-24T15:33:01.053 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999032 2024-04-24T15:33:01.053 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.053 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.053 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:01.053 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794959+0000", 2024-04-24T15:33:01.053 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000100207 2024-04-24T15:33:01.053 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.053 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.053 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:01.054 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794967+0000", 2024-04-24T15:33:01.054 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7640000000000005e-06 2024-04-24T15:33:01.054 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.054 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.054 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:01.054 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794971+0000", 2024-04-24T15:33:01.054 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.335e-06 2024-04-24T15:33:01.054 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.054 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.054 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:01.054 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835388+0000", 2024-04-24T15:33:01.054 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040417340000000003 2024-04-24T15:33:01.054 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.055 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.055 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:01.055 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835459+0000", 2024-04-24T15:33:01.055 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.0118999999999997e-05 2024-04-24T15:33:01.055 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.055 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.055 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:01.055 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835518+0000", 2024-04-24T15:33:01.055 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.9438999999999998e-05 2024-04-24T15:33:01.055 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.055 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:01.055 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.055 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.055 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.055 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23632 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15466496~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:01.056 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795060+0000", 2024-04-24T15:33:01.056 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 26.224824891000001, 2024-04-24T15:33:01.056 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040667993, 2024-04-24T15:33:01.056 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:01.056 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:01.056 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:01.056 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:01.056 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:01.056 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23632 2024-04-24T15:33:01.056 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.056 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:01.056 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.056 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:01.056 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795060+0000", 2024-04-24T15:33:01.056 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.056 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.057 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.057 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:01.057 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795060+0000", 2024-04-24T15:33:01.057 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.057 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.057 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.057 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:01.057 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794976+0000", 2024-04-24T15:33:01.057 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999156 2024-04-24T15:33:01.057 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.057 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.057 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:01.057 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795064+0000", 2024-04-24T15:33:01.057 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.7709999999999999e-05 2024-04-24T15:33:01.057 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.057 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.058 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:01.058 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795072+0000", 2024-04-24T15:33:01.058 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3769999999999996e-06 2024-04-24T15:33:01.058 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.058 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.058 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:01.058 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795078+0000", 2024-04-24T15:33:01.058 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.5079999999999996e-06 2024-04-24T15:33:01.058 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.058 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.058 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:01.058 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835541+0000", 2024-04-24T15:33:01.058 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040463440000000003 2024-04-24T15:33:01.058 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.058 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.059 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:01.059 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835657+0000", 2024-04-24T15:33:01.059 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00011572499999999999 2024-04-24T15:33:01.059 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.059 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.059 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:01.059 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835728+0000", 2024-04-24T15:33:01.059 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1402000000000005e-05 2024-04-24T15:33:01.059 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.059 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:01.059 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.059 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.059 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.059 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23633 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15507456~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:01.059 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795134+0000", 2024-04-24T15:33:01.059 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 26.224751235999999, 2024-04-24T15:33:01.059 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040748591000000001, 2024-04-24T15:33:01.060 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:01.060 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:01.060 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:01.060 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:01.060 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:01.060 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23633 2024-04-24T15:33:01.060 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.060 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:01.060 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.060 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:01.060 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795134+0000", 2024-04-24T15:33:01.060 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.060 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.060 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.060 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:01.060 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795134+0000", 2024-04-24T15:33:01.060 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.061 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.061 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.061 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:01.061 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795083+0000", 2024-04-24T15:33:01.061 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999949 2024-04-24T15:33:01.061 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.061 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.061 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:01.061 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795138+0000", 2024-04-24T15:33:01.061 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.4472000000000003e-05 2024-04-24T15:33:01.061 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.061 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.061 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:01.061 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795145+0000", 2024-04-24T15:33:01.061 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2779999999999996e-06 2024-04-24T15:33:01.062 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.062 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.062 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:01.062 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795149+0000", 2024-04-24T15:33:01.062 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.568e-06 2024-04-24T15:33:01.062 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.062 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.062 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:01.062 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835757+0000", 2024-04-24T15:33:01.062 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040608894999999999 2024-04-24T15:33:01.062 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.062 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.062 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:01.062 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835829+0000", 2024-04-24T15:33:01.062 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1879999999999996e-05 2024-04-24T15:33:01.063 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.063 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.063 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:01.063 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835883+0000", 2024-04-24T15:33:01.063 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.3346000000000001e-05 2024-04-24T15:33:01.063 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.063 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:01.063 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.063 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.063 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.063 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23634 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15548416~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:01.063 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795271+0000", 2024-04-24T15:33:01.063 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 26.224614202000001, 2024-04-24T15:33:01.063 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040755845999999998, 2024-04-24T15:33:01.063 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:01.063 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:01.064 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:01.064 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:01.064 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:01.064 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23634 2024-04-24T15:33:01.064 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.064 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:01.064 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.064 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:01.064 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795271+0000", 2024-04-24T15:33:01.064 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.064 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.064 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.064 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:01.064 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795271+0000", 2024-04-24T15:33:01.064 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.064 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.065 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.065 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:01.065 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795152+0000", 2024-04-24T15:33:01.065 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998813 2024-04-24T15:33:01.065 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.065 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.065 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:01.065 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795276+0000", 2024-04-24T15:33:01.065 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00012401799999999999 2024-04-24T15:33:01.065 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.065 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.065 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:01.065 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795281+0000", 2024-04-24T15:33:01.065 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8860000000000003e-06 2024-04-24T15:33:01.065 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.065 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.066 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:01.066 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795284+0000", 2024-04-24T15:33:01.066 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.0450000000000001e-06 2024-04-24T15:33:01.066 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.066 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.066 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:01.066 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835905+0000", 2024-04-24T15:33:01.066 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040621035999999999 2024-04-24T15:33:01.066 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.066 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.066 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:01.066 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835971+0000", 2024-04-24T15:33:01.066 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.5314000000000005e-05 2024-04-24T15:33:01.066 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.066 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.067 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:01.067 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836027+0000", 2024-04-24T15:33:01.067 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6305999999999999e-05 2024-04-24T15:33:01.067 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.067 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:01.067 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.067 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.067 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.067 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23635 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15589376~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:01.067 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795382+0000", 2024-04-24T15:33:01.067 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 26.224503428999999, 2024-04-24T15:33:01.067 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040805363999999997, 2024-04-24T15:33:01.067 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:01.067 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:01.067 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:01.068 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:01.068 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:01.068 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23635 2024-04-24T15:33:01.068 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.068 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:01.068 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.068 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:01.068 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795382+0000", 2024-04-24T15:33:01.068 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.068 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.068 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.068 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:01.068 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795382+0000", 2024-04-24T15:33:01.068 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.069 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.069 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.069 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:01.069 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795288+0000", 2024-04-24T15:33:01.069 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999056 2024-04-24T15:33:01.069 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.069 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.069 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:01.069 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795385+0000", 2024-04-24T15:33:01.069 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.7288999999999996e-05 2024-04-24T15:33:01.069 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.069 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.069 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:01.069 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795392+0000", 2024-04-24T15:33:01.069 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.452e-06 2024-04-24T15:33:01.069 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.070 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.070 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:01.070 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795397+0000", 2024-04-24T15:33:01.070 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2860000000000001e-06 2024-04-24T15:33:01.070 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.070 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.070 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:01.070 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836054+0000", 2024-04-24T15:33:01.070 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040657581999999998 2024-04-24T15:33:01.070 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.070 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.070 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:01.070 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836124+0000", 2024-04-24T15:33:01.070 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.0020999999999999e-05 2024-04-24T15:33:01.071 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.071 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.071 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:01.071 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836187+0000", 2024-04-24T15:33:01.071 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.3009999999999995e-05 2024-04-24T15:33:01.071 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.071 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:01.071 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.071 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.071 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.071 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23636 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15630336~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:01.071 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795610+0000", 2024-04-24T15:33:01.071 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 26.224275071000001, 2024-04-24T15:33:01.071 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040730942999999999, 2024-04-24T15:33:01.071 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:01.071 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:01.071 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:01.072 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:01.072 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:01.072 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23636 2024-04-24T15:33:01.072 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.072 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:01.072 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.072 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:01.072 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795610+0000", 2024-04-24T15:33:01.072 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.072 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.072 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.072 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:01.072 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795610+0000", 2024-04-24T15:33:01.072 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.072 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.072 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.072 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:01.073 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795401+0000", 2024-04-24T15:33:01.073 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9997911 2024-04-24T15:33:01.073 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.073 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.073 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:01.073 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795614+0000", 2024-04-24T15:33:01.073 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00021228600000000001 2024-04-24T15:33:01.073 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.073 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.073 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:01.073 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795622+0000", 2024-04-24T15:33:01.073 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3920000000000003e-06 2024-04-24T15:33:01.073 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.073 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.073 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:01.074 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795627+0000", 2024-04-24T15:33:01.074 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8199999999999996e-06 2024-04-24T15:33:01.074 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.074 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.074 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:01.074 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836211+0000", 2024-04-24T15:33:01.074 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040583803000000002 2024-04-24T15:33:01.074 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.074 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.074 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:01.074 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836284+0000", 2024-04-24T15:33:01.074 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.3617999999999999e-05 2024-04-24T15:33:01.074 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.074 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.075 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:01.075 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836341+0000", 2024-04-24T15:33:01.075 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6959999999999997e-05 2024-04-24T15:33:01.075 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.075 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:01.075 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.075 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.075 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.075 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23637 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15671296~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:01.075 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795699+0000", 2024-04-24T15:33:01.075 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 26.224185903999999, 2024-04-24T15:33:01.075 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040775556999999997, 2024-04-24T15:33:01.075 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:01.075 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:01.075 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:01.075 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:01.076 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:01.076 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23637 2024-04-24T15:33:01.076 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.076 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:01.076 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.076 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:01.076 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795699+0000", 2024-04-24T15:33:01.076 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.076 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.076 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.076 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:01.076 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795699+0000", 2024-04-24T15:33:01.076 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.076 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.077 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.077 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:01.077 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795632+0000", 2024-04-24T15:33:01.077 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999323 2024-04-24T15:33:01.077 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.077 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.077 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:01.077 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795704+0000", 2024-04-24T15:33:01.077 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2428000000000005e-05 2024-04-24T15:33:01.077 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.077 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.077 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:01.077 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795715+0000", 2024-04-24T15:33:01.077 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.1532000000000001e-05 2024-04-24T15:33:01.078 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.078 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.078 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:01.078 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795721+0000", 2024-04-24T15:33:01.078 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.8200000000000002e-06 2024-04-24T15:33:01.078 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.078 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.078 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:01.078 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836364+0000", 2024-04-24T15:33:01.078 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040642997 2024-04-24T15:33:01.078 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.078 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.078 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:01.078 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836435+0000", 2024-04-24T15:33:01.078 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.0821000000000005e-05 2024-04-24T15:33:01.079 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.079 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.079 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:01.079 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836475+0000", 2024-04-24T15:33:01.079 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9796999999999999e-05 2024-04-24T15:33:01.079 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.079 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:01.079 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.079 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.079 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.079 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23638 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15712256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:01.079 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795932+0000", 2024-04-24T15:33:01.079 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 26.223952873999998, 2024-04-24T15:33:01.079 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040668682999999997, 2024-04-24T15:33:01.079 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:01.080 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:01.080 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:01.080 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:01.080 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:01.080 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23638 2024-04-24T15:33:01.080 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.080 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:01.080 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.080 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:01.080 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795932+0000", 2024-04-24T15:33:01.080 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.080 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.080 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.080 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:01.080 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795932+0000", 2024-04-24T15:33:01.081 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.081 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.081 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.081 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:01.081 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795728+0000", 2024-04-24T15:33:01.081 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9997959 2024-04-24T15:33:01.081 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.081 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.081 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:01.081 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795936+0000", 2024-04-24T15:33:01.081 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000208128 2024-04-24T15:33:01.081 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.081 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.081 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:01.081 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795945+0000", 2024-04-24T15:33:01.082 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.7110000000000001e-06 2024-04-24T15:33:01.082 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.082 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.082 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:01.082 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795957+0000", 2024-04-24T15:33:01.082 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.1800999999999999e-05 2024-04-24T15:33:01.082 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.082 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.082 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:01.082 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836505+0000", 2024-04-24T15:33:01.082 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040547622999999998 2024-04-24T15:33:01.082 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.082 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.082 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:01.082 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836561+0000", 2024-04-24T15:33:01.082 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6598999999999998e-05 2024-04-24T15:33:01.083 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.083 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.083 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:01.083 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836601+0000", 2024-04-24T15:33:01.083 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9907000000000003e-05 2024-04-24T15:33:01.083 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.083 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:01.083 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.083 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.083 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.083 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23639 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15753216~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:01.083 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.796071+0000", 2024-04-24T15:33:01.083 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 26.223814612999998, 2024-04-24T15:33:01.083 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040648199000000003, 2024-04-24T15:33:01.083 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:01.084 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:01.084 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:01.084 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:01.084 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:01.084 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23639 2024-04-24T15:33:01.084 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.084 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:01.084 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.084 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:01.084 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796071+0000", 2024-04-24T15:33:01.084 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.084 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.084 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.084 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:01.085 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796071+0000", 2024-04-24T15:33:01.085 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.085 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.085 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.085 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:01.085 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795967+0000", 2024-04-24T15:33:01.085 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998965 2024-04-24T15:33:01.085 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.085 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.085 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:01.085 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796074+0000", 2024-04-24T15:33:01.085 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000107013 2024-04-24T15:33:01.085 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.085 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.085 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:01.086 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796081+0000", 2024-04-24T15:33:01.086 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2590000000000003e-06 2024-04-24T15:33:01.086 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.086 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.086 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:01.086 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796085+0000", 2024-04-24T15:33:01.086 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5269999999999999e-06 2024-04-24T15:33:01.086 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.086 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.086 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:01.086 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836624+0000", 2024-04-24T15:33:01.086 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040539324000000002 2024-04-24T15:33:01.087 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.087 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.087 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:01.087 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836667+0000", 2024-04-24T15:33:01.087 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2562999999999999e-05 2024-04-24T15:33:01.087 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.087 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.087 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:01.087 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836719+0000", 2024-04-24T15:33:01.087 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.2015999999999998e-05 2024-04-24T15:33:01.087 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.087 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:01.088 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.088 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.088 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.088 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23640 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15794176~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:01.088 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.796492+0000", 2024-04-24T15:33:01.088 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 26.223392855, 2024-04-24T15:33:01.088 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040420000999999997, 2024-04-24T15:33:01.088 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:01.088 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:01.088 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:01.088 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:01.088 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:01.089 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23640 2024-04-24T15:33:01.089 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.089 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:01.089 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.089 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:01.089 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796492+0000", 2024-04-24T15:33:01.089 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.089 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.089 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.089 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:01.090 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796492+0000", 2024-04-24T15:33:01.090 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.090 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.090 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.090 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:01.090 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796088+0000", 2024-04-24T15:33:01.090 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9995961 2024-04-24T15:33:01.090 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.090 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.090 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:01.090 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796496+0000", 2024-04-24T15:33:01.090 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00040785700000000002 2024-04-24T15:33:01.091 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.091 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.091 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:01.091 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796503+0000", 2024-04-24T15:33:01.091 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1049999999999997e-06 2024-04-24T15:33:01.091 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.091 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.092 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:01.092 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796508+0000", 2024-04-24T15:33:01.092 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.155e-06 2024-04-24T15:33:01.092 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.092 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.092 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:01.093 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836756+0000", 2024-04-24T15:33:01.093 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040248850000000003 2024-04-24T15:33:01.093 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.093 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.093 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:01.093 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836841+0000", 2024-04-24T15:33:01.094 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.4735999999999996e-05 2024-04-24T15:33:01.094 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.094 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.094 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:01.094 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836912+0000", 2024-04-24T15:33:01.094 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1229999999999994e-05 2024-04-24T15:33:01.094 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.094 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:01.094 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.095 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.095 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.095 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23641 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15835136~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:01.095 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.796658+0000", 2024-04-24T15:33:01.095 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 26.223226779000001, 2024-04-24T15:33:01.095 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040420706000000001, 2024-04-24T15:33:01.095 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:01.095 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:01.096 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:01.096 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:01.096 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:01.096 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23641 2024-04-24T15:33:01.096 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.096 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:01.096 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.096 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:01.096 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796658+0000", 2024-04-24T15:33:01.097 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.097 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.097 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.097 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:01.097 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796658+0000", 2024-04-24T15:33:01.097 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.097 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.098 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.098 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:01.098 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796512+0000", 2024-04-24T15:33:01.098 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998531 2024-04-24T15:33:01.099 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.099 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.099 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:01.099 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796669+0000", 2024-04-24T15:33:01.099 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000157577 2024-04-24T15:33:01.100 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.100 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.100 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:01.101 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796677+0000", 2024-04-24T15:33:01.101 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7800000000000001e-06 2024-04-24T15:33:01.101 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.101 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.101 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:01.102 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796681+0000", 2024-04-24T15:33:01.102 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.721e-06 2024-04-24T15:33:01.102 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.102 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.102 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:01.102 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836941+0000", 2024-04-24T15:33:01.102 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040259907999999997 2024-04-24T15:33:01.102 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.103 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.103 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:01.103 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837017+0000", 2024-04-24T15:33:01.103 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.6071999999999999e-05 2024-04-24T15:33:01.103 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.103 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.103 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:01.103 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837079+0000", 2024-04-24T15:33:01.103 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.2358000000000002e-05 2024-04-24T15:33:01.103 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.103 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:01.104 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.104 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.104 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.104 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23645 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15998976~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:01.104 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.797360+0000", 2024-04-24T15:33:01.104 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 26.222525280999999, 2024-04-24T15:33:01.104 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040387645999999999, 2024-04-24T15:33:01.104 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:01.104 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:01.104 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:01.104 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:01.104 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:01.104 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23645 2024-04-24T15:33:01.105 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.105 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:01.105 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.105 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:01.105 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797360+0000", 2024-04-24T15:33:01.105 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.105 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.105 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.105 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:01.105 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797360+0000", 2024-04-24T15:33:01.105 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.105 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.105 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.105 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:01.105 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797288+0000", 2024-04-24T15:33:01.106 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999928 2024-04-24T15:33:01.106 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.106 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.106 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:01.106 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797364+0000", 2024-04-24T15:33:01.106 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.5347999999999998e-05 2024-04-24T15:33:01.106 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.106 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.106 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:01.106 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797372+0000", 2024-04-24T15:33:01.106 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.2409999999999995e-06 2024-04-24T15:33:01.106 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.106 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.106 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:01.107 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797376+0000", 2024-04-24T15:33:01.107 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.515e-06 2024-04-24T15:33:01.107 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.107 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.107 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:01.107 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837586+0000", 2024-04-24T15:33:01.107 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040210047999999998 2024-04-24T15:33:01.107 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.107 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.107 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:01.107 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837672+0000", 2024-04-24T15:33:01.107 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.5997999999999999e-05 2024-04-24T15:33:01.107 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.107 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.108 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:01.108 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837748+0000", 2024-04-24T15:33:01.108 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.5328999999999997e-05 2024-04-24T15:33:01.108 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.108 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:01.108 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.108 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.108 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.108 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23646 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 16039936~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:01.108 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.797498+0000", 2024-04-24T15:33:01.108 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 26.222387179999998, 2024-04-24T15:33:01.108 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040433139, 2024-04-24T15:33:01.108 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:01.108 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:01.108 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:01.109 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:01.109 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:01.109 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23646 2024-04-24T15:33:01.109 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.109 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:01.109 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.109 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:01.109 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797498+0000", 2024-04-24T15:33:01.109 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.109 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.109 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.109 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:01.109 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797498+0000", 2024-04-24T15:33:01.109 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.109 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.110 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.110 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:01.110 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797381+0000", 2024-04-24T15:33:01.110 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998832 2024-04-24T15:33:01.110 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.110 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.110 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:01.110 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797501+0000", 2024-04-24T15:33:01.110 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000119759 2024-04-24T15:33:01.110 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.110 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.110 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:01.110 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797510+0000", 2024-04-24T15:33:01.110 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.4770000000000003e-06 2024-04-24T15:33:01.110 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.111 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.111 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:01.111 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797515+0000", 2024-04-24T15:33:01.111 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8330000000000001e-06 2024-04-24T15:33:01.111 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.111 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.111 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:01.111 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837777+0000", 2024-04-24T15:33:01.111 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040262502999999998 2024-04-24T15:33:01.111 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.111 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.111 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:01.111 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837860+0000", 2024-04-24T15:33:01.111 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3002000000000003e-05 2024-04-24T15:33:01.111 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.112 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.112 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:01.112 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837931+0000", 2024-04-24T15:33:01.112 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1222000000000004e-05 2024-04-24T15:33:01.112 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.112 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:01.112 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.112 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.112 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:01.112 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:01.115 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-24T15:33:01.116 INFO:teuthology.orchestra.run.smithi096.stdout:3217 write 0xe4268e5 thru 0xe42e557 (0x7c73 bytes) 2024-04-24T15:33:01.116 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:01.116 INFO:teuthology.orchestra.run.smithi096.stdout:3219 punch from 0xbe55840 to 0xbe5cc2b, (0x73eb bytes) 2024-04-24T15:33:01.116 INFO:teuthology.orchestra.run.smithi096.stdout:3220 trunc from 0xe42e558 to 0xdd00c6a 2024-04-24T15:33:01.116 INFO:teuthology.orchestra.run.smithi096.stdout:3221 writesame 0xbc94bde thru 0xbca2045 data_size 0x1c(0xd468 bytes) 2024-04-24T15:33:01.116 INFO:teuthology.orchestra.run.smithi096.stdout:3222 read 0xd38ce27 thru 0xd395fa9 (0x9183 bytes) 2024-04-24T15:33:01.116 INFO:teuthology.orchestra.run.smithi096.stdout:3223 read 0x65f2c4d thru 0x6600db3 (0xe167 bytes) 2024-04-24T15:33:01.116 INFO:teuthology.orchestra.run.smithi096.stdout:3224 punch from 0x420d6a0 to 0x420fff4, (0x2954 bytes) 2024-04-24T15:33:01.116 INFO:teuthology.orchestra.run.smithi096.stdout:3225 writesame 0x84e20b0 thru 0x84e335a data_size 0x3(0x12ab bytes) 2024-04-24T15:33:01.116 INFO:teuthology.orchestra.run.smithi096.stdout:3229 read 0x56bf402 thru 0x56c4fea (0x5be9 bytes) 2024-04-24T15:33:01.117 INFO:teuthology.orchestra.run.smithi096.stdout:3230 write 0x4dbc27e thru 0x4dc20ad (0x5e30 bytes) 2024-04-24T15:33:01.117 INFO:teuthology.orchestra.run.smithi096.stdout:3232 read 0x240aac1 thru 0x2417c2e (0xd16e bytes) 2024-04-24T15:33:01.117 INFO:teuthology.orchestra.run.smithi096.stdout:3233 punch from 0xbf8500a to 0xbf93b25, (0xeb1b bytes) 2024-04-24T15:33:01.117 INFO:teuthology.orchestra.run.smithi096.stdout:3235 read 0x7ea175b thru 0x7ea362e (0x1ed4 bytes) 2024-04-24T15:33:01.117 INFO:teuthology.orchestra.run.smithi096.stdout:3237 trunc from 0xdd00c6a to 0xc88d45 2024-04-24T15:33:01.117 INFO:teuthology.orchestra.run.smithi096.stdout:3238 write 0xb00e96e thru 0xb01095f (0x1ff2 bytes) 2024-04-24T15:33:01.117 INFO:teuthology.orchestra.run.smithi096.stdout:3239 write 0x373de91 thru 0x374aebd (0xd02d bytes) 2024-04-24T15:33:01.117 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:33:01.113+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:33:01.118 INFO:teuthology.orchestra.run.smithi096.stdout:3241 read 0x6ee8eee thru 0x6ef1e0b (0x8f1e bytes) 2024-04-24T15:33:01.118 INFO:teuthology.orchestra.run.smithi096.stdout:3242 punch from 0x44cd4ca to 0x44d4ee0, (0x7a16 bytes) 2024-04-24T15:33:01.118 INFO:teuthology.orchestra.run.smithi096.stdout:3244 write 0x1866440 thru 0x186bc6a (0x582b bytes) 2024-04-24T15:33:01.118 INFO:teuthology.orchestra.run.smithi096.stdout:3245 write 0x983fd4d thru 0x9846cd2 (0x6f86 bytes) 2024-04-24T15:33:01.123 INFO:teuthology.orchestra.run.smithi096.stdout:3247 read 0x4f7f363 thru 0x4f8bff2 (0xcc90 bytes) 2024-04-24T15:33:01.123 INFO:teuthology.orchestra.run.smithi096.stdout:3248 write 0x2a56d17 thru 0x2a5dfbf (0x72a9 bytes) 2024-04-24T15:33:01.133 INFO:teuthology.orchestra.run.smithi096.stdout:3250 write 0x42f4626 thru 0x42fe4e4 (0x9ebf bytes) 2024-04-24T15:33:01.144 INFO:teuthology.orchestra.run.smithi096.stdout:3252 writesame 0xbc19b54 thru 0xbc23cce data_size 0x1(0xa17b bytes) 2024-04-24T15:33:01.144 INFO:teuthology.orchestra.run.smithi096.stdout:3253 writesame 0xab83da7 thru 0xab91e0d data_size 0x27(0xe067 bytes) 2024-04-24T15:33:01.153 INFO:teuthology.orchestra.run.smithi096.stdout:3254 read 0xb91bd thru 0xc1596 (0x83da bytes) 2024-04-24T15:33:01.154 INFO:teuthology.orchestra.run.smithi096.stdout:3255 punch from 0x9456415 to 0x945beaf, (0x5a9a bytes) 2024-04-24T15:33:01.154 INFO:teuthology.orchestra.run.smithi096.stdout:3256 punch from 0x880471f to 0x880cf9c, (0x887d bytes) 2024-04-24T15:33:01.154 INFO:teuthology.orchestra.run.smithi096.stdout:3258 writesame 0x4b054a0 thru 0x4b09c0e data_size 0x1(0x476f bytes) 2024-04-24T15:33:01.163 INFO:teuthology.orchestra.run.smithi096.stdout:3259 read 0x8ecf954 thru 0x8edec28 (0xf2d5 bytes) 2024-04-24T15:33:01.163 INFO:teuthology.orchestra.run.smithi096.stdout:3260 writesame 0x82afaf2 thru 0x82b0c43 data_size 0x3(0x1152 bytes) 2024-04-24T15:33:01.166 INFO:teuthology.orchestra.run.smithi096.stdout:3262 writesame 0x748830e thru 0x7496528 data_size 0x1(0xe21b bytes) 2024-04-24T15:33:01.185 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:01.186 INFO:teuthology.orchestra.run.smithi096.stdout:3265 read 0x57009a thru 0x57edbf (0xed26 bytes) 2024-04-24T15:33:01.187 INFO:teuthology.orchestra.run.smithi096.stdout:3266 punch from 0xb44066d to 0xb447a77, (0x740a bytes) 2024-04-24T15:33:01.187 INFO:teuthology.orchestra.run.smithi096.stdout:3270 read 0x5632584 thru 0x563b4bf (0x8f3c bytes) 2024-04-24T15:33:01.187 INFO:teuthology.orchestra.run.smithi096.stdout:3272 punch from 0x4fd4783 to 0x4fdbbcb, (0x7448 bytes) 2024-04-24T15:33:01.187 INFO:teuthology.orchestra.run.smithi096.stdout:3273 read 0x43715a7 thru 0x43724e9 (0xf43 bytes) 2024-04-24T15:33:01.187 INFO:teuthology.orchestra.run.smithi096.stdout:3275 write 0x60508c thru 0x612f48 (0xdebd bytes) 2024-04-24T15:33:01.188 INFO:teuthology.orchestra.run.smithi096.stdout:3276 read 0xb2deb31 thru 0xb2e036b (0x183b bytes) 2024-04-24T15:33:01.188 INFO:teuthology.orchestra.run.smithi096.stdout:3277 read 0xb517751 thru 0xb51ab13 (0x33c3 bytes) 2024-04-24T15:33:01.188 INFO:teuthology.orchestra.run.smithi096.stdout:3278 write 0x8c901c1 thru 0x8c9ee6f (0xecaf bytes) 2024-04-24T15:33:01.198 INFO:teuthology.orchestra.run.smithi096.stdout:3283 write 0x1213fcd thru 0x121ae39 (0x6e6d bytes) 2024-04-24T15:33:01.203 INFO:teuthology.orchestra.run.smithi096.stdout:3284 write 0x6bbf7be thru 0x6bc6f84 (0x77c7 bytes) 2024-04-24T15:33:01.208 INFO:teuthology.orchestra.run.smithi096.stdout:3285 write 0x49f0e0c thru 0x49fd311 (0xc506 bytes) 2024-04-24T15:33:01.213 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:33:01.214+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:33:01.215 INFO:teuthology.orchestra.run.smithi096.stdout:3286 trunc from 0xbc23ccf to 0xcfac04e 2024-04-24T15:33:01.221 INFO:teuthology.orchestra.run.smithi096.stdout:3288 punch from 0x3132461 to 0x3138db4, (0x6953 bytes) 2024-04-24T15:33:01.221 INFO:teuthology.orchestra.run.smithi096.stdout:3289 write 0x7ba6213 thru 0x7badfe1 (0x7dcf bytes) 2024-04-24T15:33:01.225 INFO:teuthology.orchestra.run.smithi096.stdout:3290 write 0x39b9ac thru 0x3a3d5b (0x83b0 bytes) 2024-04-24T15:33:01.233 INFO:teuthology.orchestra.run.smithi096.stdout:3291 read 0x14b6e5e thru 0x14c400c (0xd1af bytes) 2024-04-24T15:33:01.233 INFO:teuthology.orchestra.run.smithi096.stdout:3292 read 0x4a18d42 thru 0x4a2356a (0xa829 bytes) 2024-04-24T15:33:01.241 INFO:teuthology.orchestra.run.smithi096.stdout:3294 writesame 0xe044745 thru 0xe04f67d data_size 0x1(0xaf39 bytes) 2024-04-24T15:33:01.241 INFO:teuthology.orchestra.run.smithi096.stdout:3295 write 0x128fd3a thru 0x129f0a4 (0xf36b bytes) 2024-04-24T15:33:01.251 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:01.251 INFO:teuthology.orchestra.run.smithi096.stdout:3299 writesame 0x81ceeb3 thru 0x81d1d8e data_size 0x2(0x2edc bytes) 2024-04-24T15:33:01.266 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:01.266 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:33:01.266 INFO:teuthology.orchestra.run.smithi151.stdout: "num_ops": 0 2024-04-24T15:33:01.266 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:01.269 INFO:teuthology.orchestra.run.smithi096.stdout:3300 trunc from 0xe04f67e to 0x192ee87 2024-04-24T15:33:01.279 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-24T15:33:01.297 INFO:teuthology.orchestra.run.smithi096.stdout:3303 writesame 0x4eed849 thru 0x4ef1b87 data_size 0xb(0x433f bytes) 2024-04-24T15:33:01.297 INFO:teuthology.orchestra.run.smithi096.stdout:3304 writesame 0x4dc9551 thru 0x4dd9302 data_size 0x2(0xfdb2 bytes) 2024-04-24T15:33:01.314 INFO:teuthology.orchestra.run.smithi096.stdout:3305 read 0x4134241 thru 0x413cca5 (0x8a65 bytes) 2024-04-24T15:33:01.314 INFO:teuthology.orchestra.run.smithi096.stdout:3306 write 0x4347819 thru 0x4348b93 (0x137b bytes) 2024-04-24T15:33:01.315 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:01.314+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:01.330 INFO:teuthology.orchestra.run.smithi096.stdout:3307 write 0xae56ce0 thru 0xae5b007 (0x4328 bytes) 2024-04-24T15:33:01.335 INFO:teuthology.orchestra.run.smithi096.stdout:3309 trunc from 0xae5b008 to 0xbc2f6f4 2024-04-24T15:33:01.340 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:01.340 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:01.340 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:01.340 INFO:teuthology.orchestra.run.smithi096.stdout:3315 read 0x4433938 thru 0x44347ca (0xe93 bytes) 2024-04-24T15:33:01.344 INFO:teuthology.orchestra.run.smithi096.stdout:3316 trunc from 0xbc2f6f4 to 0x86d1ae3 2024-04-24T15:33:01.365 INFO:teuthology.orchestra.run.smithi096.stdout:3317 writesame 0xeaa5a43 thru 0xeaad362 data_size 0x66(0x7920 bytes) 2024-04-24T15:33:01.377 INFO:teuthology.orchestra.run.smithi096.stdout:3318 trunc from 0xeaad363 to 0x488e252 2024-04-24T15:33:01.387 INFO:teuthology.orchestra.run.smithi096.stdout:3319 read 0x1dd8162 thru 0x1de42d4 (0xc173 bytes) 2024-04-24T15:33:01.387 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:01.387 INFO:teuthology.orchestra.run.smithi096.stdout:3321 write 0x168c88b thru 0x16987b4 (0xbf2a bytes) 2024-04-24T15:33:01.393 INFO:teuthology.orchestra.run.smithi096.stdout:3322 punch from 0x202b745 to 0x202efff, (0x38ba bytes) 2024-04-24T15:33:01.410 INFO:teuthology.orchestra.run.smithi096.stdout:3324 writesame 0xddc23a4 thru 0xddc7cc0 data_size 0x1(0x591d bytes) 2024-04-24T15:33:01.410 INFO:teuthology.orchestra.run.smithi096.stdout:3326 punch from 0xc70da6f to 0xc71b800, (0xdd91 bytes) 2024-04-24T15:33:01.410 INFO:teuthology.orchestra.run.smithi096.stdout:3327 writesame 0x7ac36a2 thru 0x7ac4813 data_size 0x1d(0x1172 bytes) 2024-04-24T15:33:01.414 INFO:teuthology.orchestra.run.smithi096.stdout:3328 read 0x176b397 thru 0x1772a46 (0x76b0 bytes) 2024-04-24T15:33:01.415 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:33:01.415+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:33:01.429 INFO:teuthology.orchestra.run.smithi096.stdout:3330 trunc from 0xddc7cc1 to 0x18b5fce 2024-04-24T15:33:01.449 INFO:teuthology.orchestra.run.smithi096.stdout:3331 write 0x444a354 thru 0x4453322 (0x8fcf bytes) 2024-04-24T15:33:01.453 INFO:teuthology.orchestra.run.smithi096.stdout:3332 punch from 0x7b94ff to 0x7c536a, (0xbe6b bytes) 2024-04-24T15:33:01.453 INFO:teuthology.orchestra.run.smithi096.stdout:3335 read 0x246d65a thru 0x246fbcc (0x2573 bytes) 2024-04-24T15:33:01.453 INFO:teuthology.orchestra.run.smithi096.stdout:3336 read 0x2249e59 thru 0x22538a0 (0x9a48 bytes) 2024-04-24T15:33:01.453 INFO:teuthology.orchestra.run.smithi096.stdout:3338 write 0x15bcf thru 0x221d6 (0xc608 bytes) 2024-04-24T15:33:01.459 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:01.459 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:33:01.459 INFO:teuthology.orchestra.run.smithi151.stdout: "complaint_time": 30, 2024-04-24T15:33:01.459 INFO:teuthology.orchestra.run.smithi151.stdout: "num_blocked_ops": 0 2024-04-24T15:33:01.459 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:01.463 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:01.464 INFO:teuthology.orchestra.run.smithi096.stdout:3340 read 0x404f847 thru 0x4059f0d (0xa6c7 bytes) 2024-04-24T15:33:01.464 INFO:teuthology.orchestra.run.smithi096.stdout:3341 read 0xd80f65 thru 0xd89a68 (0x8b04 bytes) 2024-04-24T15:33:01.471 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-24T15:33:01.480 INFO:teuthology.orchestra.run.smithi096.stdout:3343 write 0xd56d8fe thru 0xd574053 (0x6756 bytes) 2024-04-24T15:33:01.485 INFO:teuthology.orchestra.run.smithi096.stdout:3345 write 0x823c509 thru 0x8245ac6 (0x95be bytes) 2024-04-24T15:33:01.492 INFO:teuthology.orchestra.run.smithi096.stdout:3346 read 0x92192e3 thru 0x922154b (0x8269 bytes) 2024-04-24T15:33:01.492 INFO:teuthology.orchestra.run.smithi096.stdout:3347 read 0x9bec42d thru 0x9bf8f71 (0xcb45 bytes) 2024-04-24T15:33:01.492 INFO:teuthology.orchestra.run.smithi096.stdout:3348 punch from 0x74a38b0 to 0x74a4bd1, (0x1321 bytes) 2024-04-24T15:33:01.492 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:01.492 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:01.492 INFO:teuthology.orchestra.run.smithi096.stdout:3353 read 0x7fd9fc0 thru 0x7fe3160 (0x91a1 bytes) 2024-04-24T15:33:01.493 INFO:teuthology.orchestra.run.smithi096.stdout:3355 write 0x4bac0cd thru 0x4bb5cc2 (0x9bf6 bytes) 2024-04-24T15:33:01.506 INFO:teuthology.orchestra.run.smithi096.stdout:3356 writesame 0xde8d2cc thru 0xde9ced6 data_size 0x1(0xfc0b bytes) 2024-04-24T15:33:01.506 INFO:teuthology.orchestra.run.smithi096.stdout:3358 write 0x8fffbfb thru 0x9001fdd (0x23e3 bytes) 2024-04-24T15:33:01.511 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:01.516 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:33:01.516+0000 7f23abcf6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 44021) UID: 0 2024-04-24T15:33:01.525 INFO:teuthology.orchestra.run.smithi096.stdout:3360 trunc from 0xde9ced7 to 0x299548b 2024-04-24T15:33:01.538 INFO:teuthology.orchestra.run.smithi096.stdout:3361 read 0x251c525 thru 0x251cb09 (0x5e5 bytes) 2024-04-24T15:33:01.538 INFO:teuthology.orchestra.run.smithi096.stdout:3365 read 0xb20afd thru 0xb2c4c2 (0xb9c6 bytes) 2024-04-24T15:33:01.538 INFO:teuthology.orchestra.run.smithi096.stdout:3366 read 0x15e64ed thru 0x15ee8f5 (0x8409 bytes) 2024-04-24T15:33:01.538 INFO:teuthology.orchestra.run.smithi096.stdout:3367 read 0xfb1da8 thru 0xfb4083 (0x22dc bytes) 2024-04-24T15:33:01.548 INFO:teuthology.orchestra.run.smithi096.stdout:3368 write 0x6ec236c thru 0x6ec85aa (0x623f bytes) 2024-04-24T15:33:01.568 INFO:teuthology.orchestra.run.smithi096.stdout:3369 write 0x8e8d748 thru 0x8e992cf (0xbb88 bytes) 2024-04-24T15:33:01.582 INFO:teuthology.orchestra.run.smithi096.stdout:3371 punch from 0x68dfee5 to 0x68e1a96, (0x1bb1 bytes) 2024-04-24T15:33:01.582 INFO:teuthology.orchestra.run.smithi096.stdout:3372 read 0x572ab31 thru 0x5736a0e (0xbede bytes) 2024-04-24T15:33:01.590 INFO:teuthology.orchestra.run.smithi096.stdout:3373 write 0xc9191ae thru 0xc923926 (0xa779 bytes) 2024-04-24T15:33:01.598 INFO:teuthology.orchestra.run.smithi096.stdout:3374 write 0x7f73b61 thru 0x7f7bc8b (0x812b bytes) 2024-04-24T15:33:01.605 INFO:teuthology.orchestra.run.smithi096.stdout:3375 read 0x211a5dc thru 0x2125e37 (0xb85c bytes) 2024-04-24T15:33:01.605 INFO:teuthology.orchestra.run.smithi096.stdout:3376 read 0x24a5cb8 thru 0x24b1dc7 (0xc110 bytes) 2024-04-24T15:33:01.605 INFO:teuthology.orchestra.run.smithi096.stdout:3377 read 0x9b8ec26 thru 0x9b95956 (0x6d31 bytes) 2024-04-24T15:33:01.605 INFO:teuthology.orchestra.run.smithi096.stdout:3378 punch from 0x43fe671 to 0x440302b, (0x49ba bytes) 2024-04-24T15:33:01.609 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:01.610 INFO:teuthology.orchestra.run.smithi151.stdout: "size": 20, 2024-04-24T15:33:01.610 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 600, 2024-04-24T15:33:01.610 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [ 2024-04-24T15:33:01.610 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.610 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17317 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 11509760~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:01.610 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.224408+0000", 2024-04-24T15:33:01.610 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.385253486, 2024-04-24T15:33:01.610 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042604418999999998, 2024-04-24T15:33:01.610 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:01.610 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:01.610 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:01.610 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:01.611 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:01.611 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17317 2024-04-24T15:33:01.611 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.611 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:01.611 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.611 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:01.611 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224408+0000", 2024-04-24T15:33:01.611 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.611 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.611 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.611 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:01.611 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224408+0000", 2024-04-24T15:33:01.611 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.612 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.612 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.612 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:01.612 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224407+0000", 2024-04-24T15:33:01.612 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:01.612 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.612 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.612 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:01.612 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224409+0000", 2024-04-24T15:33:01.612 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.1059999999999998e-06 2024-04-24T15:33:01.612 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.612 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.612 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:01.612 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224415+0000", 2024-04-24T15:33:01.613 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6219999999999997e-06 2024-04-24T15:33:01.613 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.613 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.613 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:01.613 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224419+0000", 2024-04-24T15:33:01.613 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8340000000000001e-06 2024-04-24T15:33:01.613 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.613 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.613 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:01.613 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.266851+0000", 2024-04-24T15:33:01.613 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042431910000000003 2024-04-24T15:33:01.613 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.613 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.614 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:01.614 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.266931+0000", 2024-04-24T15:33:01.614 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0580000000000004e-05 2024-04-24T15:33:01.614 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.614 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.614 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:01.614 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267012+0000", 2024-04-24T15:33:01.614 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0698999999999994e-05 2024-04-24T15:33:01.614 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.614 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:01.614 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.614 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.614 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.614 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17318 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 11550720~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:01.615 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.224424+0000", 2024-04-24T15:33:01.615 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.385237117999999, 2024-04-24T15:33:01.615 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042771921999999997, 2024-04-24T15:33:01.615 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:01.615 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:01.615 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:01.615 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:01.615 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:01.615 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17318 2024-04-24T15:33:01.615 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.615 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:01.615 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.616 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:01.616 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224424+0000", 2024-04-24T15:33:01.616 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.616 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.616 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.616 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:01.616 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224424+0000", 2024-04-24T15:33:01.616 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.616 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.616 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.616 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:01.616 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224424+0000", 2024-04-24T15:33:01.616 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:01.616 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.616 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.617 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:01.617 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224434+0000", 2024-04-24T15:33:01.617 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.9699999999999994e-06 2024-04-24T15:33:01.617 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.617 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.617 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:01.617 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224443+0000", 2024-04-24T15:33:01.617 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.3689999999999996e-06 2024-04-24T15:33:01.617 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.617 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.617 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:01.617 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224447+0000", 2024-04-24T15:33:01.617 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.783e-06 2024-04-24T15:33:01.617 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.618 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.618 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:01.618 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267027+0000", 2024-04-24T15:33:01.618 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042579847999999997 2024-04-24T15:33:01.618 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.618 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.618 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:01.618 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267108+0000", 2024-04-24T15:33:01.618 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.1420000000000003e-05 2024-04-24T15:33:01.618 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.618 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.618 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:01.618 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267196+0000", 2024-04-24T15:33:01.618 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.7851999999999998e-05 2024-04-24T15:33:01.619 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.619 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:01.619 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.619 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.619 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.619 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17319 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 11591680~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:01.619 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.224453+0000", 2024-04-24T15:33:01.619 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.385208192, 2024-04-24T15:33:01.619 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042921783999999998, 2024-04-24T15:33:01.619 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:01.619 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:01.619 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:01.619 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:01.620 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:01.620 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17319 2024-04-24T15:33:01.620 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.620 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:01.620 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.620 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:01.620 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224453+0000", 2024-04-24T15:33:01.620 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.620 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.620 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.620 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:01.620 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224453+0000", 2024-04-24T15:33:01.620 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.620 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.621 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.621 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:01.621 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224452+0000", 2024-04-24T15:33:01.621 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:01.621 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.621 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.621 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:01.621 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224455+0000", 2024-04-24T15:33:01.621 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.289e-06 2024-04-24T15:33:01.621 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.621 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.621 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:01.621 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224461+0000", 2024-04-24T15:33:01.621 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.7300000000000002e-06 2024-04-24T15:33:01.621 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.622 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.622 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:01.622 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224464+0000", 2024-04-24T15:33:01.622 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.2890000000000002e-06 2024-04-24T15:33:01.622 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.622 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.622 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:01.622 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267218+0000", 2024-04-24T15:33:01.622 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042753696000000001 2024-04-24T15:33:01.622 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.622 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.622 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:01.622 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267295+0000", 2024-04-24T15:33:01.623 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7762999999999994e-05 2024-04-24T15:33:01.623 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.623 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.623 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:01.623 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267375+0000", 2024-04-24T15:33:01.623 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.9444999999999996e-05 2024-04-24T15:33:01.623 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.623 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:01.623 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.623 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.623 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.623 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17320 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 11632640~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:01.624 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.224468+0000", 2024-04-24T15:33:01.624 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.385193158, 2024-04-24T15:33:01.624 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043063615, 2024-04-24T15:33:01.624 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:01.624 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:01.624 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:01.624 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:01.624 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:01.624 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17320 2024-04-24T15:33:01.624 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.624 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:01.624 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.625 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:01.625 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224468+0000", 2024-04-24T15:33:01.625 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.625 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.625 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.625 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:01.625 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224468+0000", 2024-04-24T15:33:01.625 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.625 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.625 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.625 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:01.625 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224468+0000", 2024-04-24T15:33:01.625 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:01.625 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.626 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.626 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:01.626 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224470+0000", 2024-04-24T15:33:01.626 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.0600000000000002e-06 2024-04-24T15:33:01.626 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.626 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.626 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:01.626 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224475+0000", 2024-04-24T15:33:01.626 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.1059999999999999e-06 2024-04-24T15:33:01.626 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.626 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.627 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:01.627 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224478+0000", 2024-04-24T15:33:01.627 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.3100000000000001e-06 2024-04-24T15:33:01.627 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.627 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.627 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:01.627 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267386+0000", 2024-04-24T15:33:01.627 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042907529 2024-04-24T15:33:01.627 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.627 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.627 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:01.627 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267447+0000", 2024-04-24T15:33:01.627 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.1173000000000005e-05 2024-04-24T15:33:01.627 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.628 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.628 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:01.628 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267532+0000", 2024-04-24T15:33:01.628 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.4771999999999994e-05 2024-04-24T15:33:01.628 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.628 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:01.628 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.628 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.628 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.628 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17321 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 11673600~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:01.628 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.224499+0000", 2024-04-24T15:33:01.628 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.385162268999999, 2024-04-24T15:33:01.628 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043214447000000003, 2024-04-24T15:33:01.628 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:01.628 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:01.629 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:01.629 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:01.629 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:01.629 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17321 2024-04-24T15:33:01.629 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.629 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:01.629 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.629 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:01.629 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224499+0000", 2024-04-24T15:33:01.629 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.629 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.629 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.629 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:01.629 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224499+0000", 2024-04-24T15:33:01.630 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.630 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.630 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.630 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:01.630 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224499+0000", 2024-04-24T15:33:01.630 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:01.630 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.630 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.630 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:01.630 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224504+0000", 2024-04-24T15:33:01.630 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6869999999999999e-06 2024-04-24T15:33:01.630 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.630 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.630 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:01.631 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224509+0000", 2024-04-24T15:33:01.631 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2259999999999999e-06 2024-04-24T15:33:01.631 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.631 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.631 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:01.631 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224511+0000", 2024-04-24T15:33:01.631 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.418e-06 2024-04-24T15:33:01.631 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.631 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.631 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:01.631 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267552+0000", 2024-04-24T15:33:01.631 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043041455999999999 2024-04-24T15:33:01.631 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.632 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.632 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:01.632 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267629+0000", 2024-04-24T15:33:01.632 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7069000000000004e-05 2024-04-24T15:33:01.632 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.632 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.632 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:01.632 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267713+0000", 2024-04-24T15:33:01.632 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3843000000000003e-05 2024-04-24T15:33:01.632 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.632 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:01.633 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.633 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.633 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.633 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17322 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 11714560~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:01.633 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.224514+0000", 2024-04-24T15:33:01.633 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.385147030999999, 2024-04-24T15:33:01.633 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043339196000000003, 2024-04-24T15:33:01.633 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:01.633 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:01.633 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:01.633 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:01.633 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:01.633 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17322 2024-04-24T15:33:01.633 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.634 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:01.634 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.634 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:01.634 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224514+0000", 2024-04-24T15:33:01.634 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.634 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.634 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.634 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:01.634 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224514+0000", 2024-04-24T15:33:01.634 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.634 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.634 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.634 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:01.635 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224514+0000", 2024-04-24T15:33:01.635 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:01.635 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.635 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.635 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:01.635 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224515+0000", 2024-04-24T15:33:01.635 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.5570000000000001e-06 2024-04-24T15:33:01.635 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.635 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.635 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:01.635 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224518+0000", 2024-04-24T15:33:01.635 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.943e-06 2024-04-24T15:33:01.635 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.635 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.635 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:01.636 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224520+0000", 2024-04-24T15:33:01.636 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.9999999999999999e-06 2024-04-24T15:33:01.636 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.636 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.636 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:01.636 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267730+0000", 2024-04-24T15:33:01.636 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043209466000000002 2024-04-24T15:33:01.636 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.636 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.636 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:01.636 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267772+0000", 2024-04-24T15:33:01.636 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2453000000000002e-05 2024-04-24T15:33:01.636 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.636 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.637 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:01.637 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267853+0000", 2024-04-24T15:33:01.637 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.1094000000000007e-05 2024-04-24T15:33:01.637 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.637 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:01.637 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.637 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.637 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.637 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17323 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 11755520~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:01.637 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.224522+0000", 2024-04-24T15:33:01.637 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.385138691000002, 2024-04-24T15:33:01.638 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043506792000000002, 2024-04-24T15:33:01.638 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:01.638 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:01.638 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:01.638 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:01.638 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:01.638 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17323 2024-04-24T15:33:01.638 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.638 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:01.638 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.638 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:01.638 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224522+0000", 2024-04-24T15:33:01.639 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.639 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.639 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.639 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:01.639 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224522+0000", 2024-04-24T15:33:01.639 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.639 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.639 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.639 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:01.639 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224522+0000", 2024-04-24T15:33:01.639 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:01.639 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.639 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.639 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:01.639 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224524+0000", 2024-04-24T15:33:01.640 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3069999999999999e-06 2024-04-24T15:33:01.640 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.640 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.640 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:01.640 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224526+0000", 2024-04-24T15:33:01.640 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.773e-06 2024-04-24T15:33:01.640 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.640 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.640 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:01.640 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224529+0000", 2024-04-24T15:33:01.640 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.3e-06 2024-04-24T15:33:01.640 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.640 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.640 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:01.641 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267873+0000", 2024-04-24T15:33:01.641 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043344141000000003 2024-04-24T15:33:01.641 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.641 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.641 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:01.641 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267948+0000", 2024-04-24T15:33:01.641 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.4960000000000001e-05 2024-04-24T15:33:01.641 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.641 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.641 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:01.641 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.268029+0000", 2024-04-24T15:33:01.641 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.1520000000000006e-05 2024-04-24T15:33:01.641 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.641 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:01.642 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.642 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.642 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.642 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17328 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 11960320~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:01.642 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.226273+0000", 2024-04-24T15:33:01.642 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.383387706000001, 2024-04-24T15:33:01.642 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042524610999999997, 2024-04-24T15:33:01.642 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:01.642 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:01.642 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:01.642 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:01.642 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:01.642 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17328 2024-04-24T15:33:01.643 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.643 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:01.643 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.643 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:01.643 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226273+0000", 2024-04-24T15:33:01.643 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.643 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.643 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.643 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:01.643 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226273+0000", 2024-04-24T15:33:01.643 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.643 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.643 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.643 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:01.643 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226273+0000", 2024-04-24T15:33:01.644 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:01.644 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.644 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.644 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:01.644 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226275+0000", 2024-04-24T15:33:01.644 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.759e-06 2024-04-24T15:33:01.644 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.644 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.644 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:01.644 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226286+0000", 2024-04-24T15:33:01.644 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0841e-05 2024-04-24T15:33:01.644 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.644 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.644 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:01.645 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226290+0000", 2024-04-24T15:33:01.645 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.794e-06 2024-04-24T15:33:01.645 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.645 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.645 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:01.645 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.268693+0000", 2024-04-24T15:33:01.645 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042403519000000001 2024-04-24T15:33:01.645 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.645 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.645 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:01.645 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.268739+0000", 2024-04-24T15:33:01.645 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6149000000000001e-05 2024-04-24T15:33:01.645 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.645 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.646 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:01.646 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.268798+0000", 2024-04-24T15:33:01.646 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.8752999999999998e-05 2024-04-24T15:33:01.646 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.646 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:01.646 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.646 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.646 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.647 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17329 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 12001280~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:01.647 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.226295+0000", 2024-04-24T15:33:01.647 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.383366598999999, 2024-04-24T15:33:01.647 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042690771000000002, 2024-04-24T15:33:01.647 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:01.647 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:01.647 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:01.647 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:01.647 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:01.647 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17329 2024-04-24T15:33:01.647 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.647 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:01.648 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.648 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:01.648 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226295+0000", 2024-04-24T15:33:01.648 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.648 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.648 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.648 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:01.648 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226295+0000", 2024-04-24T15:33:01.648 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.648 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.648 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.648 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:01.648 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226294+0000", 2024-04-24T15:33:01.648 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:01.648 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.649 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.649 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:01.649 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226297+0000", 2024-04-24T15:33:01.649 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.0189999999999998e-06 2024-04-24T15:33:01.649 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.649 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.649 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:01.649 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226302+0000", 2024-04-24T15:33:01.649 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6260000000000003e-06 2024-04-24T15:33:01.649 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.649 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.649 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:01.650 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226304+0000", 2024-04-24T15:33:01.650 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.2230000000000001e-06 2024-04-24T15:33:01.650 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.650 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.650 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:01.650 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.268831+0000", 2024-04-24T15:33:01.650 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042527224000000002 2024-04-24T15:33:01.650 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.650 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.650 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:01.651 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.268904+0000", 2024-04-24T15:33:01.651 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.3168000000000001e-05 2024-04-24T15:33:01.651 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.651 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.651 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:01.651 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.268985+0000", 2024-04-24T15:33:01.651 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.1009999999999999e-05 2024-04-24T15:33:01.651 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.651 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:01.651 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.651 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.651 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.653 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17330 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 12042240~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:01.653 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.226307+0000", 2024-04-24T15:33:01.653 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.383354608000001, 2024-04-24T15:33:01.653 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042812009999999998, 2024-04-24T15:33:01.653 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:01.653 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:01.653 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:01.653 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:01.654 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:01.654 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17330 2024-04-24T15:33:01.654 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.654 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:01.654 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.654 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:01.654 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226307+0000", 2024-04-24T15:33:01.654 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.654 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.654 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.654 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:01.654 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226307+0000", 2024-04-24T15:33:01.654 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.654 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.655 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.655 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:01.655 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226306+0000", 2024-04-24T15:33:01.655 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:01.655 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.655 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.655 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:01.655 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226312+0000", 2024-04-24T15:33:01.655 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6180000000000003e-06 2024-04-24T15:33:01.655 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.655 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.656 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:01.656 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226316+0000", 2024-04-24T15:33:01.656 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2379999999999999e-06 2024-04-24T15:33:01.656 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.656 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.656 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:01.656 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226318+0000", 2024-04-24T15:33:01.656 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.3130000000000001e-06 2024-04-24T15:33:01.656 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.656 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.656 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:01.656 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269015+0000", 2024-04-24T15:33:01.656 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042696698999999998 2024-04-24T15:33:01.657 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.657 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.657 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:01.657 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269061+0000", 2024-04-24T15:33:01.657 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6193000000000003e-05 2024-04-24T15:33:01.657 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.657 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.657 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:01.657 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269119+0000", 2024-04-24T15:33:01.657 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.7169000000000001e-05 2024-04-24T15:33:01.657 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.657 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:01.657 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.657 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.658 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.658 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17331 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 12083200~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:01.658 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.226321+0000", 2024-04-24T15:33:01.658 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.383340177000001, 2024-04-24T15:33:01.658 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042976872999999999, 2024-04-24T15:33:01.658 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:01.658 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:01.658 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:01.658 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:01.658 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:01.658 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17331 2024-04-24T15:33:01.658 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.658 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:01.659 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.659 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:01.659 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226321+0000", 2024-04-24T15:33:01.659 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.659 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.659 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.659 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:01.659 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226321+0000", 2024-04-24T15:33:01.659 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.659 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.659 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.659 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:01.659 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226321+0000", 2024-04-24T15:33:01.660 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:01.660 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.660 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.660 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:01.660 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226322+0000", 2024-04-24T15:33:01.660 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.446e-06 2024-04-24T15:33:01.660 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.660 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.660 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:01.660 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226325+0000", 2024-04-24T15:33:01.660 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.004e-06 2024-04-24T15:33:01.660 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.660 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.661 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:01.661 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226327+0000", 2024-04-24T15:33:01.661 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.0990000000000001e-06 2024-04-24T15:33:01.661 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.661 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.661 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:01.661 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269137+0000", 2024-04-24T15:33:01.661 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042809381000000001 2024-04-24T15:33:01.661 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.661 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.661 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:01.661 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269217+0000", 2024-04-24T15:33:01.661 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.085e-05 2024-04-24T15:33:01.661 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.662 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.662 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:01.662 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269298+0000", 2024-04-24T15:33:01.662 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0317999999999999e-05 2024-04-24T15:33:01.662 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.662 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:01.662 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.662 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.662 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.662 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17332 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 12124160~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:01.662 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.226329+0000", 2024-04-24T15:33:01.662 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.383331688999998, 2024-04-24T15:33:01.662 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043084539999999998, 2024-04-24T15:33:01.663 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:01.663 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:01.663 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:01.663 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:01.663 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:01.663 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17332 2024-04-24T15:33:01.663 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.663 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:01.663 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.663 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:01.664 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226329+0000", 2024-04-24T15:33:01.664 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.664 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.664 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.664 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:01.664 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226329+0000", 2024-04-24T15:33:01.664 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.664 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.664 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.664 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:01.664 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226329+0000", 2024-04-24T15:33:01.664 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:01.664 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.665 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.665 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:01.665 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226331+0000", 2024-04-24T15:33:01.665 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.328e-06 2024-04-24T15:33:01.665 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.665 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.665 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:01.665 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226333+0000", 2024-04-24T15:33:01.665 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.9119999999999998e-06 2024-04-24T15:33:01.665 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.665 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.665 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:01.665 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226335+0000", 2024-04-24T15:33:01.665 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.9690000000000001e-06 2024-04-24T15:33:01.665 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.666 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.666 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:01.666 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269313+0000", 2024-04-24T15:33:01.666 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042977518999999999 2024-04-24T15:33:01.666 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.666 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.666 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:01.666 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269357+0000", 2024-04-24T15:33:01.666 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3748999999999997e-05 2024-04-24T15:33:01.666 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.666 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.666 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:01.666 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269414+0000", 2024-04-24T15:33:01.666 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.7265000000000001e-05 2024-04-24T15:33:01.666 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.667 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:01.667 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.667 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.667 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.667 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17333 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 12165120~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:01.667 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.226338+0000", 2024-04-24T15:33:01.667 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.383323552, 2024-04-24T15:33:01.667 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043264706999999999, 2024-04-24T15:33:01.667 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:01.667 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:01.667 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:01.667 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:01.667 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:01.667 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17333 2024-04-24T15:33:01.667 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.667 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:01.668 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.668 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:01.668 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226338+0000", 2024-04-24T15:33:01.668 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.668 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.668 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.668 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:01.668 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226338+0000", 2024-04-24T15:33:01.668 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.668 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.668 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.668 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:01.668 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226337+0000", 2024-04-24T15:33:01.668 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:01.668 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.668 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.668 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:01.669 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226339+0000", 2024-04-24T15:33:01.669 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.387e-06 2024-04-24T15:33:01.669 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.669 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.669 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:01.669 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226342+0000", 2024-04-24T15:33:01.669 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8059999999999999e-06 2024-04-24T15:33:01.669 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.669 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.669 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:01.669 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226344+0000", 2024-04-24T15:33:01.669 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.1390000000000002e-06 2024-04-24T15:33:01.669 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.669 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.669 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:01.670 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269445+0000", 2024-04-24T15:33:01.670 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043101457000000003 2024-04-24T15:33:01.670 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.670 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.670 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:01.670 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269520+0000", 2024-04-24T15:33:01.670 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.5278000000000007e-05 2024-04-24T15:33:01.670 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.670 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.670 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:01.670 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269602+0000", 2024-04-24T15:33:01.670 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.1847000000000004e-05 2024-04-24T15:33:01.670 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.671 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:01.671 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.671 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.671 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.671 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17334 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 12206080~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:01.671 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.226346+0000", 2024-04-24T15:33:01.671 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.383315136, 2024-04-24T15:33:01.671 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043468358999999998, 2024-04-24T15:33:01.671 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:01.671 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:01.671 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:01.671 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:01.671 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:01.671 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17334 2024-04-24T15:33:01.672 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.672 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:01.672 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.672 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:01.672 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226346+0000", 2024-04-24T15:33:01.672 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.672 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.672 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.672 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:01.672 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226346+0000", 2024-04-24T15:33:01.672 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.672 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.672 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.672 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:01.672 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226346+0000", 2024-04-24T15:33:01.673 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:01.673 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.673 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.673 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:01.673 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226347+0000", 2024-04-24T15:33:01.673 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.359e-06 2024-04-24T15:33:01.673 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.673 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.673 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:01.673 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226350+0000", 2024-04-24T15:33:01.673 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7800000000000001e-06 2024-04-24T15:33:01.673 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.673 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.673 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:01.674 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226352+0000", 2024-04-24T15:33:01.674 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.046e-06 2024-04-24T15:33:01.674 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.674 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.674 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:01.674 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269646+0000", 2024-04-24T15:33:01.674 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043294393 2024-04-24T15:33:01.674 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.674 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.674 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:01.674 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269732+0000", 2024-04-24T15:33:01.674 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.5716000000000004e-05 2024-04-24T15:33:01.674 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.674 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.674 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:01.675 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269814+0000", 2024-04-24T15:33:01.675 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.2270999999999999e-05 2024-04-24T15:33:01.675 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.675 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:01.675 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.675 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.675 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.675 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17342 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 12533760~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:01.675 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.228377+0000", 2024-04-24T15:33:01.675 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.381284242, 2024-04-24T15:33:01.675 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042573286000000002, 2024-04-24T15:33:01.675 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:01.675 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:01.676 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:01.676 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:01.676 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:01.676 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17342 2024-04-24T15:33:01.676 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.676 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:01.676 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.676 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:01.676 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228377+0000", 2024-04-24T15:33:01.676 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.676 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.676 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.676 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:01.677 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228377+0000", 2024-04-24T15:33:01.677 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.677 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.677 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.677 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:01.677 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228377+0000", 2024-04-24T15:33:01.677 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:01.677 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.677 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.677 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:01.677 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228389+0000", 2024-04-24T15:33:01.677 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.2262999999999999e-05 2024-04-24T15:33:01.677 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.677 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.677 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:01.678 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228398+0000", 2024-04-24T15:33:01.678 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.1079999999999999e-06 2024-04-24T15:33:01.678 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.678 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.678 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:01.678 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228402+0000", 2024-04-24T15:33:01.678 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8920000000000002e-06 2024-04-24T15:33:01.678 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.678 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.678 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:01.678 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.270842+0000", 2024-04-24T15:33:01.678 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042440193000000001 2024-04-24T15:33:01.678 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.678 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.679 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:01.679 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.270887+0000", 2024-04-24T15:33:01.679 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5359999999999999e-05 2024-04-24T15:33:01.679 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.679 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.679 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:01.679 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.270950+0000", 2024-04-24T15:33:01.680 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.2842999999999994e-05 2024-04-24T15:33:01.680 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.680 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:01.680 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.680 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.680 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.680 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17343 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 12574720~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:01.680 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.228407+0000", 2024-04-24T15:33:01.680 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.381254369000001, 2024-04-24T15:33:01.680 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042650325000000003, 2024-04-24T15:33:01.680 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:01.680 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:01.680 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:01.680 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:01.680 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:01.681 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17343 2024-04-24T15:33:01.681 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.681 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:01.681 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.681 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:01.681 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228407+0000", 2024-04-24T15:33:01.681 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.681 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.681 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.681 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:01.681 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228407+0000", 2024-04-24T15:33:01.681 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.681 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.681 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.682 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:01.682 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228406+0000", 2024-04-24T15:33:01.682 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:01.682 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.682 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.682 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:01.682 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228409+0000", 2024-04-24T15:33:01.682 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.3980000000000002e-06 2024-04-24T15:33:01.682 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.682 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.682 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:01.682 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228414+0000", 2024-04-24T15:33:01.682 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6590000000000004e-06 2024-04-24T15:33:01.683 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.683 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.683 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:01.683 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228418+0000", 2024-04-24T15:33:01.683 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.625e-06 2024-04-24T15:33:01.683 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.683 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.683 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:01.683 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.270966+0000", 2024-04-24T15:33:01.683 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042547709000000003 2024-04-24T15:33:01.683 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.683 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.683 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:01.684 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.271007+0000", 2024-04-24T15:33:01.684 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.1659000000000002e-05 2024-04-24T15:33:01.684 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.684 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.684 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:01.684 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.271057+0000", 2024-04-24T15:33:01.684 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.9679000000000003e-05 2024-04-24T15:33:01.684 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.684 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:01.684 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.684 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.684 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.684 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17344 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 12607488~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:01.685 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.228461+0000", 2024-04-24T15:33:01.685 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.381200380999999, 2024-04-24T15:33:01.685 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042713530999999999, 2024-04-24T15:33:01.685 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:01.685 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:01.685 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:01.685 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:01.685 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:01.685 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17344 2024-04-24T15:33:01.685 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.685 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:01.685 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.685 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:01.685 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228461+0000", 2024-04-24T15:33:01.686 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.686 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.686 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.686 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:01.686 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228461+0000", 2024-04-24T15:33:01.686 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.686 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.686 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.686 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:01.686 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228460+0000", 2024-04-24T15:33:01.686 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:01.686 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.686 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.687 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:01.687 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228464+0000", 2024-04-24T15:33:01.687 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.98e-06 2024-04-24T15:33:01.687 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.687 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.687 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:01.687 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228473+0000", 2024-04-24T15:33:01.687 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.0270000000000001e-06 2024-04-24T15:33:01.687 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.687 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.687 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:01.687 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228479+0000", 2024-04-24T15:33:01.687 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.5910000000000003e-06 2024-04-24T15:33:01.687 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.687 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.688 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:01.688 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.271081+0000", 2024-04-24T15:33:01.688 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042602025000000002 2024-04-24T15:33:01.688 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.688 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.688 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:01.688 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.271126+0000", 2024-04-24T15:33:01.688 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4904999999999998e-05 2024-04-24T15:33:01.688 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.688 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.688 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:01.688 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.271174+0000", 2024-04-24T15:33:01.688 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8603999999999997e-05 2024-04-24T15:33:01.689 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.689 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:01.689 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.689 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.689 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.689 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17345 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 12648448~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:01.689 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.228484+0000", 2024-04-24T15:33:01.689 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.381176844999999, 2024-04-24T15:33:01.689 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042811844000000002, 2024-04-24T15:33:01.689 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:01.689 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:01.689 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:01.689 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:01.690 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:01.690 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17345 2024-04-24T15:33:01.690 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.690 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:01.690 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.690 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:01.690 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228484+0000", 2024-04-24T15:33:01.690 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.690 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.690 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.690 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:01.690 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228484+0000", 2024-04-24T15:33:01.690 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.690 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.691 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.691 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:01.691 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228484+0000", 2024-04-24T15:33:01.691 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:01.691 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.691 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.691 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:01.691 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228487+0000", 2024-04-24T15:33:01.691 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.9950000000000001e-06 2024-04-24T15:33:01.691 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.691 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.691 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:01.691 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228493+0000", 2024-04-24T15:33:01.691 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.2870000000000001e-06 2024-04-24T15:33:01.691 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.692 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.692 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:01.692 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228497+0000", 2024-04-24T15:33:01.692 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.1219999999999997e-06 2024-04-24T15:33:01.692 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.692 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.692 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:01.692 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.271202+0000", 2024-04-24T15:33:01.692 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042704963999999998 2024-04-24T15:33:01.692 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.692 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.692 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:01.692 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.271250+0000", 2024-04-24T15:33:01.692 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8207000000000001e-05 2024-04-24T15:33:01.692 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.692 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.693 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:01.693 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.271296+0000", 2024-04-24T15:33:01.693 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5769999999999997e-05 2024-04-24T15:33:01.693 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.693 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:01.693 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.693 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.693 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.693 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17346 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 12689408~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:01.693 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.228502+0000", 2024-04-24T15:33:01.693 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.381159231000002, 2024-04-24T15:33:01.693 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042918317999999997, 2024-04-24T15:33:01.693 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:01.693 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:01.693 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:01.694 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:01.694 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:01.694 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17346 2024-04-24T15:33:01.694 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.694 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:01.694 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.694 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:01.694 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228502+0000", 2024-04-24T15:33:01.694 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.694 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.694 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.694 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:01.695 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228502+0000", 2024-04-24T15:33:01.695 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.695 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.695 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.695 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:01.695 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228502+0000", 2024-04-24T15:33:01.695 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:01.695 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.695 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.695 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:01.695 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228504+0000", 2024-04-24T15:33:01.695 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.497e-06 2024-04-24T15:33:01.695 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.695 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.695 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:01.695 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228510+0000", 2024-04-24T15:33:01.696 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.7429999999999999e-06 2024-04-24T15:33:01.696 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.696 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.696 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:01.696 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228514+0000", 2024-04-24T15:33:01.696 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0810000000000004e-06 2024-04-24T15:33:01.696 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.696 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.696 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:01.696 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.271330+0000", 2024-04-24T15:33:01.696 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042816616000000002 2024-04-24T15:33:01.696 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.696 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.697 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:01.697 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.271371+0000", 2024-04-24T15:33:01.697 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0222999999999998e-05 2024-04-24T15:33:01.697 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.697 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.697 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:01.697 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.271420+0000", 2024-04-24T15:33:01.697 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.9539000000000001e-05 2024-04-24T15:33:01.697 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.697 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:01.697 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.697 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.697 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.698 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17347 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 12730368~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:01.698 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.228518+0000", 2024-04-24T15:33:01.698 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.381142702999998, 2024-04-24T15:33:01.698 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043022429000000001, 2024-04-24T15:33:01.698 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:01.698 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:01.698 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:01.698 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:01.698 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:01.698 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17347 2024-04-24T15:33:01.698 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.698 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:01.698 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.699 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:01.699 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228518+0000", 2024-04-24T15:33:01.699 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.699 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.699 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.699 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:01.699 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228518+0000", 2024-04-24T15:33:01.699 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:01.699 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.699 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.699 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:01.699 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228518+0000", 2024-04-24T15:33:01.699 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:01.699 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.700 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.700 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:01.700 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228521+0000", 2024-04-24T15:33:01.700 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.4559999999999999e-06 2024-04-24T15:33:01.700 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.700 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.700 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:01.700 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228526+0000", 2024-04-24T15:33:01.700 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.7259999999999999e-06 2024-04-24T15:33:01.700 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.700 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.700 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:01.700 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228530+0000", 2024-04-24T15:33:01.700 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.658e-06 2024-04-24T15:33:01.700 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.701 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.701 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:01.701 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.271445+0000", 2024-04-24T15:33:01.701 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042915506999999999 2024-04-24T15:33:01.701 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.701 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.701 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:01.701 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.271490+0000", 2024-04-24T15:33:01.701 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4752000000000001e-05 2024-04-24T15:33:01.701 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:01.701 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:01.701 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:01.701 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.271541+0000", 2024-04-24T15:33:01.701 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.0699999999999999e-05 2024-04-24T15:33:01.702 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.702 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:01.702 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.702 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:01.702 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:01.702 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:01.704 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-24T15:33:01.706 INFO:teuthology.orchestra.run.smithi096.stdout:3380 write 0xe8b0090 thru 0xe8b8ba4 (0x8b15 bytes) 2024-04-24T15:33:01.706 INFO:teuthology.orchestra.run.smithi096.stdout:3382 writesame 0x9874633 thru 0x9883192 data_size 0x70(0xeb60 bytes) 2024-04-24T15:33:01.706 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:01.706 INFO:teuthology.orchestra.run.smithi096.stdout:3384 read 0x15b2aac thru 0x15ba5cb (0x7b20 bytes) 2024-04-24T15:33:01.706 INFO:teuthology.orchestra.run.smithi096.stdout:3385 writesame 0xb22dd65 thru 0xb22e30f data_size 0x1(0x5ab bytes) 2024-04-24T15:33:01.706 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:01.706 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:01.706 INFO:teuthology.orchestra.run.smithi096.stdout:3388 writesame 0xe6db303 thru 0xe6e2a43 data_size 0x1(0x7741 bytes) 2024-04-24T15:33:01.707 INFO:teuthology.orchestra.run.smithi096.stdout:3389 write 0x2bb3960 thru 0x2bb3ab9 (0x15a bytes) 2024-04-24T15:33:01.707 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:01.707 INFO:teuthology.orchestra.run.smithi096.stdout:3393 write 0x638c0d6 thru 0x638d078 (0xfa3 bytes) 2024-04-24T15:33:01.707 INFO:teuthology.orchestra.run.smithi096.stdout:3395 writesame 0x2b246d6 thru 0x2b31773 data_size 0x56(0xd09e bytes) 2024-04-24T15:33:01.707 INFO:teuthology.orchestra.run.smithi096.stdout:3397 punch from 0xb5796af to 0xb586f14, (0xd865 bytes) 2024-04-24T15:33:01.707 INFO:teuthology.orchestra.run.smithi096.stdout:3398 read 0x20b3082 thru 0x20b6b28 (0x3aa7 bytes) 2024-04-24T15:33:01.707 INFO:teuthology.orchestra.run.smithi096.stdout:3399 write 0x56afd9 thru 0x57942d (0xe455 bytes) 2024-04-24T15:33:01.707 INFO:teuthology.orchestra.run.smithi096.stdout:3400 writesame 0xefb02f thru 0xf03989 data_size 0x3(0x895b bytes) 2024-04-24T15:33:01.707 INFO:teuthology.orchestra.run.smithi096.stdout:3401 writesame 0xe284b60 thru 0xe28dcf3 data_size 0x4d(0x9194 bytes) 2024-04-24T15:33:01.707 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:01.707 INFO:teuthology.orchestra.run.smithi096.stdout:3408 writesame 0xdb350a3 thru 0xdb42cec data_size 0x75(0xdc4a bytes) 2024-04-24T15:33:01.707 INFO:teuthology.orchestra.run.smithi096.stdout:3410 write 0xa30cb8e thru 0xa30df0f (0x1382 bytes) 2024-04-24T15:33:01.707 INFO:teuthology.orchestra.run.smithi096.stdout:3411 trunc from 0xe8b8ba5 to 0xe23c88f 2024-04-24T15:33:01.708 INFO:teuthology.orchestra.run.smithi096.stdout:3413 read 0xd3c9044 thru 0xd3d4aad (0xba6a bytes) 2024-04-24T15:33:01.708 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:01.708 INFO:teuthology.orchestra.run.smithi096.stdout:3415 read 0xac7ffd thru 0xad4f0e (0xcf12 bytes) 2024-04-24T15:33:01.708 INFO:teuthology.orchestra.run.smithi096.stdout:3416 writesame 0x6e4b81a thru 0x6e59b80 data_size 0x5(0xe367 bytes) 2024-04-24T15:33:01.708 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:33:01.703+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:33:01.714 INFO:teuthology.orchestra.run.smithi096.stdout:3418 trunc from 0xe23c88f to 0x6c0c82a 2024-04-24T15:33:01.744 INFO:teuthology.orchestra.run.smithi096.stdout:3420 write 0xcc4f032 thru 0xcc510fe (0x20cd bytes) 2024-04-24T15:33:01.749 INFO:teuthology.orchestra.run.smithi096.stdout:3423 punch from 0x951bcfd to 0x9522fa7, (0x72aa bytes) 2024-04-24T15:33:01.755 INFO:teuthology.orchestra.run.smithi096.stdout:3424 trunc from 0xcc510ff to 0x75bd019 2024-04-24T15:33:01.762 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-24T15:33:01.808 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:33:01.809+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:33:01.890 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:01.890 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:33:01.890 INFO:teuthology.orchestra.run.smithi151.stdout: "num_ops": 0 2024-04-24T15:33:01.890 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:01.901 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-24T15:33:01.905 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:33:01.905+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:33:02.006 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:33:02.007+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:33:02.024 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:02.025 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:33:02.025 INFO:teuthology.orchestra.run.smithi151.stdout: "complaint_time": 30, 2024-04-24T15:33:02.025 INFO:teuthology.orchestra.run.smithi151.stdout: "num_blocked_ops": 0 2024-04-24T15:33:02.025 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:02.034 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-24T15:33:02.108 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:02.108+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:02.159 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:02.160 INFO:teuthology.orchestra.run.smithi151.stdout: "size": 20, 2024-04-24T15:33:02.160 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 600, 2024-04-24T15:33:02.160 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [ 2024-04-24T15:33:02.160 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.160 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18006 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12419072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:02.160 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.207911+0000", 2024-04-24T15:33:02.160 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.951695710999999, 2024-04-24T15:33:02.160 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042899496000000002, 2024-04-24T15:33:02.160 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:02.160 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:02.160 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:02.160 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:02.160 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:02.160 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18006 2024-04-24T15:33:02.161 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.161 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:02.161 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.161 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:02.161 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207911+0000", 2024-04-24T15:33:02.161 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:02.161 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.161 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.161 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:02.161 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207911+0000", 2024-04-24T15:33:02.161 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:02.161 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.161 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.161 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:02.161 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207910+0000", 2024-04-24T15:33:02.162 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:02.162 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.162 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.162 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:02.162 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207913+0000", 2024-04-24T15:33:02.162 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.2309999999999999e-06 2024-04-24T15:33:02.162 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.162 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.162 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:02.162 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207918+0000", 2024-04-24T15:33:02.162 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.3399999999999997e-06 2024-04-24T15:33:02.162 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.162 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.162 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:02.163 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207922+0000", 2024-04-24T15:33:02.163 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5439999999999999e-06 2024-04-24T15:33:02.163 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.163 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.163 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:02.163 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250726+0000", 2024-04-24T15:33:02.163 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042804129000000003 2024-04-24T15:33:02.163 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.163 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.163 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:02.163 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250765+0000", 2024-04-24T15:33:02.163 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9554000000000001e-05 2024-04-24T15:33:02.163 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.163 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.163 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:02.164 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250810+0000", 2024-04-24T15:33:02.164 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5027000000000001e-05 2024-04-24T15:33:02.164 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:02.164 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:02.164 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:02.164 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.164 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.164 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18007 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12460032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:02.164 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.207961+0000", 2024-04-24T15:33:02.164 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.95164587, 2024-04-24T15:33:02.164 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042965470999999998, 2024-04-24T15:33:02.164 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:02.164 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:02.164 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:02.165 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:02.165 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:02.165 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18007 2024-04-24T15:33:02.165 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.165 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:02.165 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.165 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:02.165 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207961+0000", 2024-04-24T15:33:02.165 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:02.165 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.165 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.165 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:02.165 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207961+0000", 2024-04-24T15:33:02.165 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:02.165 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.166 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.166 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:02.166 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207960+0000", 2024-04-24T15:33:02.166 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:02.166 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.166 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.166 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:02.166 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207963+0000", 2024-04-24T15:33:02.166 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.3759999999999999e-06 2024-04-24T15:33:02.166 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.166 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.166 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:02.166 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207970+0000", 2024-04-24T15:33:02.166 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.0489999999999998e-06 2024-04-24T15:33:02.167 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.167 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.167 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:02.167 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207975+0000", 2024-04-24T15:33:02.167 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.1640000000000003e-06 2024-04-24T15:33:02.167 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.167 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.167 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:02.167 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250834+0000", 2024-04-24T15:33:02.167 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042859060999999997 2024-04-24T15:33:02.167 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.167 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.167 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:02.167 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250874+0000", 2024-04-24T15:33:02.167 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0674999999999999e-05 2024-04-24T15:33:02.168 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.168 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.168 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:02.168 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250926+0000", 2024-04-24T15:33:02.168 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.1693000000000001e-05 2024-04-24T15:33:02.168 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:02.168 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:02.168 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:02.168 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.168 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.168 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18008 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12500992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:02.168 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208030+0000", 2024-04-24T15:33:02.168 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.951576777, 2024-04-24T15:33:02.168 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043003337000000003, 2024-04-24T15:33:02.168 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:02.168 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:02.169 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:02.169 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:02.169 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:02.169 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18008 2024-04-24T15:33:02.169 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.169 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:02.169 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.169 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:02.169 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208030+0000", 2024-04-24T15:33:02.169 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:02.169 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.169 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.169 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:02.169 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208030+0000", 2024-04-24T15:33:02.169 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:02.170 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.170 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.170 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:02.170 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208029+0000", 2024-04-24T15:33:02.170 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:02.170 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.170 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.170 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:02.170 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208032+0000", 2024-04-24T15:33:02.170 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.117e-06 2024-04-24T15:33:02.170 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.170 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.171 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:02.171 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208039+0000", 2024-04-24T15:33:02.171 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.0129999999999996e-06 2024-04-24T15:33:02.171 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.171 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.171 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:02.171 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208043+0000", 2024-04-24T15:33:02.171 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0600000000000001e-06 2024-04-24T15:33:02.171 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.171 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.171 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:02.171 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250948+0000", 2024-04-24T15:33:02.171 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042904577999999999 2024-04-24T15:33:02.171 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.171 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.171 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:02.172 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250989+0000", 2024-04-24T15:33:02.172 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0978e-05 2024-04-24T15:33:02.172 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.172 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.172 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:02.172 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251033+0000", 2024-04-24T15:33:02.172 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4161e-05 2024-04-24T15:33:02.172 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:02.172 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:02.172 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:02.172 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.172 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.172 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18009 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12541952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:02.172 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208093+0000", 2024-04-24T15:33:02.172 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.951513031000001, 2024-04-24T15:33:02.173 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043043451000000003, 2024-04-24T15:33:02.173 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:02.173 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:02.173 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:02.173 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:02.173 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:02.173 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18009 2024-04-24T15:33:02.173 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.173 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:02.173 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.173 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:02.173 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208093+0000", 2024-04-24T15:33:02.174 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:02.174 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.174 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.174 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:02.174 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208093+0000", 2024-04-24T15:33:02.174 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:02.174 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.174 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.174 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:02.174 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208093+0000", 2024-04-24T15:33:02.174 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:02.174 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.175 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.175 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:02.175 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208096+0000", 2024-04-24T15:33:02.175 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.439e-06 2024-04-24T15:33:02.175 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.175 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.175 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:02.175 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208103+0000", 2024-04-24T15:33:02.175 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1019999999999999e-06 2024-04-24T15:33:02.175 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.175 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.175 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:02.175 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208107+0000", 2024-04-24T15:33:02.175 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0910000000000003e-06 2024-04-24T15:33:02.176 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.176 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.176 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:02.176 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251054+0000", 2024-04-24T15:33:02.176 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042946826 2024-04-24T15:33:02.176 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.176 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.176 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:02.176 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251092+0000", 2024-04-24T15:33:02.176 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.7941999999999997e-05 2024-04-24T15:33:02.176 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.176 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.176 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:02.176 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251137+0000", 2024-04-24T15:33:02.176 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4638000000000002e-05 2024-04-24T15:33:02.177 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:02.177 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:02.177 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:02.177 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.177 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.177 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18010 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12615680~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:02.177 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208202+0000", 2024-04-24T15:33:02.177 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.951404199999999, 2024-04-24T15:33:02.177 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043046549000000003, 2024-04-24T15:33:02.177 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:02.177 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:02.177 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:02.177 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:02.177 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:02.177 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18010 2024-04-24T15:33:02.178 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.178 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:02.178 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.178 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:02.178 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208202+0000", 2024-04-24T15:33:02.178 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:02.178 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.178 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.178 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:02.178 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208202+0000", 2024-04-24T15:33:02.178 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:02.178 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.178 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.178 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:02.179 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208202+0000", 2024-04-24T15:33:02.179 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:02.179 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.179 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.179 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:02.179 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208205+0000", 2024-04-24T15:33:02.179 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.2490000000000001e-06 2024-04-24T15:33:02.179 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.179 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.179 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:02.179 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208215+0000", 2024-04-24T15:33:02.179 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.9070000000000002e-06 2024-04-24T15:33:02.179 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.179 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.179 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:02.179 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208220+0000", 2024-04-24T15:33:02.180 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8849999999999998e-06 2024-04-24T15:33:02.180 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.180 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.180 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:02.180 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251158+0000", 2024-04-24T15:33:02.180 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042938416 2024-04-24T15:33:02.180 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.180 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.180 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:02.180 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251197+0000", 2024-04-24T15:33:02.180 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9341000000000002e-05 2024-04-24T15:33:02.180 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.180 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.180 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:02.180 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251249+0000", 2024-04-24T15:33:02.181 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.1391000000000003e-05 2024-04-24T15:33:02.181 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:02.181 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:02.181 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:02.181 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.181 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.181 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18011 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12656640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:02.181 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208258+0000", 2024-04-24T15:33:02.181 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.951348354, 2024-04-24T15:33:02.182 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043102708000000003, 2024-04-24T15:33:02.182 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:02.182 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:02.182 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:02.182 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:02.182 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:02.182 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18011 2024-04-24T15:33:02.182 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.182 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:02.182 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.182 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:02.182 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208258+0000", 2024-04-24T15:33:02.182 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:02.182 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.183 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.183 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:02.183 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208258+0000", 2024-04-24T15:33:02.183 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:02.183 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.183 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.183 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:02.183 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208258+0000", 2024-04-24T15:33:02.183 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:02.183 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.183 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.183 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:02.183 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208260+0000", 2024-04-24T15:33:02.183 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7810000000000001e-06 2024-04-24T15:33:02.184 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.184 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.184 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:02.184 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208267+0000", 2024-04-24T15:33:02.184 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.7610000000000002e-06 2024-04-24T15:33:02.184 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.184 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.184 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:02.184 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208271+0000", 2024-04-24T15:33:02.184 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0890000000000002e-06 2024-04-24T15:33:02.184 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.184 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.184 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:02.184 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251271+0000", 2024-04-24T15:33:02.185 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043000159000000003 2024-04-24T15:33:02.185 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.185 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.185 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:02.185 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251315+0000", 2024-04-24T15:33:02.185 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3446999999999999e-05 2024-04-24T15:33:02.185 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.185 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.185 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:02.185 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251361+0000", 2024-04-24T15:33:02.185 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5952000000000003e-05 2024-04-24T15:33:02.185 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:02.185 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:02.186 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:02.186 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.186 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.186 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18012 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12697600~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:02.186 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208289+0000", 2024-04-24T15:33:02.186 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.95131731, 2024-04-24T15:33:02.186 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043174774999999999, 2024-04-24T15:33:02.186 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:02.186 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:02.186 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:02.186 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:02.186 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:02.186 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18012 2024-04-24T15:33:02.187 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.187 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:02.187 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.187 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:02.187 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208289+0000", 2024-04-24T15:33:02.187 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:02.187 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.187 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.187 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:02.187 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208289+0000", 2024-04-24T15:33:02.187 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:02.187 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.187 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.188 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:02.188 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208289+0000", 2024-04-24T15:33:02.188 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:02.188 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.188 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.188 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:02.188 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208291+0000", 2024-04-24T15:33:02.188 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.542e-06 2024-04-24T15:33:02.188 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.188 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.188 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:02.188 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208298+0000", 2024-04-24T15:33:02.188 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.3899999999999998e-06 2024-04-24T15:33:02.189 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.189 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.189 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:02.189 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208302+0000", 2024-04-24T15:33:02.189 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.019e-06 2024-04-24T15:33:02.189 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.189 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.189 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:02.189 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251382+0000", 2024-04-24T15:33:02.189 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043080790000000001 2024-04-24T15:33:02.189 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.189 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.189 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:02.190 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251421+0000", 2024-04-24T15:33:02.190 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8473000000000002e-05 2024-04-24T15:33:02.190 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.190 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.190 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:02.190 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251464+0000", 2024-04-24T15:33:02.190 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3133999999999997e-05 2024-04-24T15:33:02.190 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:02.190 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:02.190 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:02.190 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.190 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.190 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18013 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12738560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:02.191 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208423+0000", 2024-04-24T15:33:02.191 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.951183201999999, 2024-04-24T15:33:02.191 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043145554000000003, 2024-04-24T15:33:02.191 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:02.191 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:02.191 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:02.191 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:02.191 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:02.191 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18013 2024-04-24T15:33:02.191 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.191 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:02.191 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.191 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:02.191 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208423+0000", 2024-04-24T15:33:02.191 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:02.192 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.192 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.192 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:02.192 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208423+0000", 2024-04-24T15:33:02.192 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:02.192 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.192 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.192 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:02.192 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208423+0000", 2024-04-24T15:33:02.192 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:02.192 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.192 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.192 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:02.192 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208426+0000", 2024-04-24T15:33:02.192 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.0539999999999999e-06 2024-04-24T15:33:02.193 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.193 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.193 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:02.193 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208433+0000", 2024-04-24T15:33:02.193 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.43e-06 2024-04-24T15:33:02.193 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.193 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.193 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:02.193 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208437+0000", 2024-04-24T15:33:02.193 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5590000000000001e-06 2024-04-24T15:33:02.193 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.193 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.193 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:02.193 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251485+0000", 2024-04-24T15:33:02.193 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043048259999999998 2024-04-24T15:33:02.194 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.194 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.194 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:02.194 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251525+0000", 2024-04-24T15:33:02.194 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9916999999999997e-05 2024-04-24T15:33:02.194 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.194 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.194 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:02.194 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251569+0000", 2024-04-24T15:33:02.194 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3939000000000001e-05 2024-04-24T15:33:02.194 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:02.194 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:02.194 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:02.194 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.195 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.195 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18015 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12779520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:02.195 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208630+0000", 2024-04-24T15:33:02.195 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.950976677, 2024-04-24T15:33:02.195 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043047328000000003, 2024-04-24T15:33:02.195 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:02.195 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:02.195 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:02.195 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:02.195 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:02.195 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18015 2024-04-24T15:33:02.195 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.195 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:02.195 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.196 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:02.196 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208630+0000", 2024-04-24T15:33:02.196 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:02.196 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.196 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.196 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:02.196 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208630+0000", 2024-04-24T15:33:02.196 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:02.196 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.196 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.196 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:02.196 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208629+0000", 2024-04-24T15:33:02.196 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999999 2024-04-24T15:33:02.196 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.196 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.197 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:02.197 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208634+0000", 2024-04-24T15:33:02.197 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8960000000000002e-06 2024-04-24T15:33:02.197 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.197 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.197 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:02.197 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208645+0000", 2024-04-24T15:33:02.197 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0939e-05 2024-04-24T15:33:02.197 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.197 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.197 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:02.197 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208651+0000", 2024-04-24T15:33:02.197 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.896e-06 2024-04-24T15:33:02.197 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.197 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.198 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:02.198 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251591+0000", 2024-04-24T15:33:02.198 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042939862000000002 2024-04-24T15:33:02.198 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.198 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.198 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:02.198 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251631+0000", 2024-04-24T15:33:02.198 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0216000000000003e-05 2024-04-24T15:33:02.198 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.198 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.198 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:02.198 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251677+0000", 2024-04-24T15:33:02.198 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6390999999999997e-05 2024-04-24T15:33:02.198 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:02.199 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:02.199 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:02.199 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.199 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.199 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18016 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12820480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:02.199 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208660+0000", 2024-04-24T15:33:02.199 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.950946888000001, 2024-04-24T15:33:02.199 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043127510000000001, 2024-04-24T15:33:02.199 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:02.199 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:02.199 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:02.199 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:02.199 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:02.199 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18016 2024-04-24T15:33:02.199 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.200 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:02.200 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.200 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:02.200 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208660+0000", 2024-04-24T15:33:02.200 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:02.200 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.200 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.200 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:02.200 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208660+0000", 2024-04-24T15:33:02.200 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:02.200 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.200 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.200 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:02.200 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208659+0000", 2024-04-24T15:33:02.200 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:02.200 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.201 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.201 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:02.201 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208662+0000", 2024-04-24T15:33:02.201 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.0979999999999998e-06 2024-04-24T15:33:02.201 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.201 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.201 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:02.201 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208669+0000", 2024-04-24T15:33:02.201 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.4470000000000003e-06 2024-04-24T15:33:02.201 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.201 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.201 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:02.201 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208673+0000", 2024-04-24T15:33:02.201 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2810000000000002e-06 2024-04-24T15:33:02.202 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.202 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.202 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:02.202 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251699+0000", 2024-04-24T15:33:02.202 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043025728999999999 2024-04-24T15:33:02.202 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.202 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.202 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:02.202 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251743+0000", 2024-04-24T15:33:02.202 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4469999999999999e-05 2024-04-24T15:33:02.202 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.202 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.202 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:02.203 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251787+0000", 2024-04-24T15:33:02.203 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3949999999999998e-05 2024-04-24T15:33:02.203 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:02.203 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:02.203 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:02.203 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.203 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.203 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18017 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12861440~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:02.203 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208721+0000", 2024-04-24T15:33:02.203 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.95088544, 2024-04-24T15:33:02.203 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043167749999999998, 2024-04-24T15:33:02.203 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:02.203 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:02.204 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:02.204 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:02.204 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:02.204 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18017 2024-04-24T15:33:02.204 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.204 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:02.204 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.204 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:02.204 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208721+0000", 2024-04-24T15:33:02.204 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:02.204 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.204 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.204 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:02.204 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208721+0000", 2024-04-24T15:33:02.205 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:02.205 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.205 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.205 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:02.205 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208720+0000", 2024-04-24T15:33:02.205 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:02.205 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.205 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.205 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:02.205 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208724+0000", 2024-04-24T15:33:02.205 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.3919999999999999e-06 2024-04-24T15:33:02.205 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.205 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.205 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:02.206 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208729+0000", 2024-04-24T15:33:02.206 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.3480000000000003e-06 2024-04-24T15:33:02.206 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.206 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.206 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:02.206 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208732+0000", 2024-04-24T15:33:02.206 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.9529999999999999e-06 2024-04-24T15:33:02.206 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.206 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.206 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:02.206 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251807+0000", 2024-04-24T15:33:02.206 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043074492999999998 2024-04-24T15:33:02.206 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.207 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.207 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:02.207 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251846+0000", 2024-04-24T15:33:02.207 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8958999999999997e-05 2024-04-24T15:33:02.207 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.207 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.207 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:02.207 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251889+0000", 2024-04-24T15:33:02.207 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3253000000000001e-05 2024-04-24T15:33:02.207 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:02.207 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:02.207 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:02.207 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.207 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.207 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18018 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12902400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:02.208 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208752+0000", 2024-04-24T15:33:02.208 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.950854827000001, 2024-04-24T15:33:02.208 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043241012000000002, 2024-04-24T15:33:02.208 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:02.208 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:02.208 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:02.208 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:02.208 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:02.208 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18018 2024-04-24T15:33:02.208 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.208 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:02.208 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.208 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:02.208 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208752+0000", 2024-04-24T15:33:02.209 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:02.209 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.209 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.209 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:02.209 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208752+0000", 2024-04-24T15:33:02.209 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:02.209 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.209 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.209 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:02.209 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208751+0000", 2024-04-24T15:33:02.209 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:02.209 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.209 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.209 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:02.210 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208753+0000", 2024-04-24T15:33:02.210 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.894e-06 2024-04-24T15:33:02.210 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.210 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.210 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:02.210 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208757+0000", 2024-04-24T15:33:02.210 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.856e-06 2024-04-24T15:33:02.210 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.210 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.210 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:02.210 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208760+0000", 2024-04-24T15:33:02.210 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.6749999999999998e-06 2024-04-24T15:33:02.210 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.210 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.210 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:02.211 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251909+0000", 2024-04-24T15:33:02.211 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043149145999999999 2024-04-24T15:33:02.211 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.211 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.211 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:02.211 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251948+0000", 2024-04-24T15:33:02.211 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9159000000000002e-05 2024-04-24T15:33:02.211 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.211 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.211 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:02.211 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251993+0000", 2024-04-24T15:33:02.211 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4638999999999997e-05 2024-04-24T15:33:02.211 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:02.211 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:02.211 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:02.212 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.212 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.212 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18019 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12943360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:02.212 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208938+0000", 2024-04-24T15:33:02.212 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.950668535999998, 2024-04-24T15:33:02.212 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043162272000000002, 2024-04-24T15:33:02.212 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:02.212 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:02.212 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:02.212 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:02.212 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:02.212 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18019 2024-04-24T15:33:02.212 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.212 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:02.212 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.213 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:02.213 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208938+0000", 2024-04-24T15:33:02.213 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:02.213 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.213 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.213 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:02.213 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208938+0000", 2024-04-24T15:33:02.213 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:02.213 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.213 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.213 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:02.213 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208762+0000", 2024-04-24T15:33:02.213 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999824 2024-04-24T15:33:02.213 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.213 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.213 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:02.214 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208942+0000", 2024-04-24T15:33:02.214 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000180008 2024-04-24T15:33:02.214 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.214 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.214 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:02.214 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208950+0000", 2024-04-24T15:33:02.214 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.1550000000000007e-06 2024-04-24T15:33:02.214 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.214 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.214 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:02.214 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208956+0000", 2024-04-24T15:33:02.214 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.7910000000000001e-06 2024-04-24T15:33:02.214 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.215 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.215 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:02.215 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252014+0000", 2024-04-24T15:33:02.215 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043058427000000003 2024-04-24T15:33:02.215 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.215 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.215 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:02.215 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252054+0000", 2024-04-24T15:33:02.215 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0071999999999999e-05 2024-04-24T15:33:02.215 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.215 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.215 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:02.215 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252100+0000", 2024-04-24T15:33:02.215 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5917000000000001e-05 2024-04-24T15:33:02.216 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:02.216 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:02.216 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:02.216 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.216 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.216 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18020 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12984320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:02.216 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.209092+0000", 2024-04-24T15:33:02.216 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.950514538, 2024-04-24T15:33:02.216 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043111295000000001, 2024-04-24T15:33:02.216 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:02.216 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:02.216 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:02.217 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:02.217 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:02.217 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18020 2024-04-24T15:33:02.217 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.217 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:02.217 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.217 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:02.217 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209092+0000", 2024-04-24T15:33:02.217 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:02.217 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.217 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.217 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:02.218 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209092+0000", 2024-04-24T15:33:02.218 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:02.218 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.218 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.218 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:02.218 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208964+0000", 2024-04-24T15:33:02.218 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998722 2024-04-24T15:33:02.218 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.218 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.218 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:02.218 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209095+0000", 2024-04-24T15:33:02.218 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000130648 2024-04-24T15:33:02.218 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.219 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.219 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:02.219 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209103+0000", 2024-04-24T15:33:02.219 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0320000000000003e-06 2024-04-24T15:33:02.219 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.219 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.219 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:02.219 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209108+0000", 2024-04-24T15:33:02.219 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8690000000000003e-06 2024-04-24T15:33:02.219 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.219 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.219 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:02.219 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252121+0000", 2024-04-24T15:33:02.219 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043013323999999999 2024-04-24T15:33:02.220 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.220 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.220 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:02.220 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252160+0000", 2024-04-24T15:33:02.220 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8376e-05 2024-04-24T15:33:02.220 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.220 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.220 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:02.220 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252203+0000", 2024-04-24T15:33:02.220 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.371e-05 2024-04-24T15:33:02.220 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:02.220 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:02.220 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:02.220 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.221 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.221 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18021 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 13025280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:02.221 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.209132+0000", 2024-04-24T15:33:02.221 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.950474180000001, 2024-04-24T15:33:02.221 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043183903000000003, 2024-04-24T15:33:02.221 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:02.221 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:02.221 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:02.221 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:02.221 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:02.221 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18021 2024-04-24T15:33:02.221 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.221 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:02.221 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.222 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:02.222 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209132+0000", 2024-04-24T15:33:02.222 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:02.222 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.222 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.222 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:02.222 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209132+0000", 2024-04-24T15:33:02.222 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:02.222 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.222 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.222 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:02.223 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209132+0000", 2024-04-24T15:33:02.223 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:02.223 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.223 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.223 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:02.223 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209134+0000", 2024-04-24T15:33:02.223 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.891e-06 2024-04-24T15:33:02.223 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.223 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.223 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:02.223 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209138+0000", 2024-04-24T15:33:02.223 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0659999999999997e-06 2024-04-24T15:33:02.223 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.223 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.223 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:02.224 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209140+0000", 2024-04-24T15:33:02.224 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.5339999999999998e-06 2024-04-24T15:33:02.224 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.224 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.224 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:02.224 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252232+0000", 2024-04-24T15:33:02.224 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043091142999999998 2024-04-24T15:33:02.224 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.224 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.224 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:02.224 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252270+0000", 2024-04-24T15:33:02.224 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8872000000000003e-05 2024-04-24T15:33:02.224 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.224 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.225 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:02.225 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252316+0000", 2024-04-24T15:33:02.225 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5788000000000003e-05 2024-04-24T15:33:02.225 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:02.225 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:02.225 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:02.225 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.225 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.225 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18022 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 13066240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:02.225 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.209411+0000", 2024-04-24T15:33:02.225 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.950195632, 2024-04-24T15:33:02.225 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043016616000000001, 2024-04-24T15:33:02.225 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:02.225 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:02.225 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:02.226 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:02.226 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:02.226 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18022 2024-04-24T15:33:02.226 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.226 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:02.226 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.226 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:02.226 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209411+0000", 2024-04-24T15:33:02.226 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:02.226 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.226 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.226 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:02.226 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209411+0000", 2024-04-24T15:33:02.226 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:02.226 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.227 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.227 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:02.227 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209143+0000", 2024-04-24T15:33:02.227 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999732 2024-04-24T15:33:02.227 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.227 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.227 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:02.227 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209415+0000", 2024-04-24T15:33:02.227 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000271873 2024-04-24T15:33:02.227 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.227 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.227 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:02.227 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209423+0000", 2024-04-24T15:33:02.227 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3080000000000008e-06 2024-04-24T15:33:02.228 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.228 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.228 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:02.228 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209428+0000", 2024-04-24T15:33:02.228 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.5040000000000002e-06 2024-04-24T15:33:02.228 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.228 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.228 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:02.228 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252339+0000", 2024-04-24T15:33:02.228 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042910419999999998 2024-04-24T15:33:02.228 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.228 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.228 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:02.228 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252380+0000", 2024-04-24T15:33:02.228 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0991000000000001e-05 2024-04-24T15:33:02.229 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.229 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.229 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:02.229 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252427+0000", 2024-04-24T15:33:02.229 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.7647e-05 2024-04-24T15:33:02.229 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:02.229 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:02.229 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:02.229 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.229 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.229 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18023 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 13107200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:02.229 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.209594+0000", 2024-04-24T15:33:02.229 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.950012119, 2024-04-24T15:33:02.229 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042938543000000003, 2024-04-24T15:33:02.229 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:02.230 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:02.230 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:02.230 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:02.230 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:02.230 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18023 2024-04-24T15:33:02.230 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.230 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:02.230 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.230 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:02.230 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209594+0000", 2024-04-24T15:33:02.230 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:02.230 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.230 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.231 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:02.231 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209594+0000", 2024-04-24T15:33:02.231 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:02.231 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.231 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.231 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:02.231 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209440+0000", 2024-04-24T15:33:02.231 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999846 2024-04-24T15:33:02.231 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.231 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.231 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:02.231 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209598+0000", 2024-04-24T15:33:02.231 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00015731100000000001 2024-04-24T15:33:02.231 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.231 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.232 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:02.232 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209604+0000", 2024-04-24T15:33:02.232 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.2910000000000004e-06 2024-04-24T15:33:02.232 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.232 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.232 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:02.232 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209607+0000", 2024-04-24T15:33:02.232 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5389999999999999e-06 2024-04-24T15:33:02.232 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.232 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.232 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:02.232 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252449+0000", 2024-04-24T15:33:02.232 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042841833000000003 2024-04-24T15:33:02.232 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.232 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.233 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:02.233 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252489+0000", 2024-04-24T15:33:02.233 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9898000000000003e-05 2024-04-24T15:33:02.233 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.233 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.233 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:02.233 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252533+0000", 2024-04-24T15:33:02.233 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3702999999999998e-05 2024-04-24T15:33:02.233 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:02.233 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:02.233 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:02.233 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.233 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.233 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18025 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 13189120~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:02.233 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.209832+0000", 2024-04-24T15:33:02.233 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.949774617999999, 2024-04-24T15:33:02.234 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042913556999999998, 2024-04-24T15:33:02.234 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:02.234 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:02.234 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:02.234 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:02.234 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:02.234 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18025 2024-04-24T15:33:02.234 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.234 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:02.234 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.234 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:02.234 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209832+0000", 2024-04-24T15:33:02.234 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:02.234 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.235 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.235 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:02.235 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209832+0000", 2024-04-24T15:33:02.235 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:02.235 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.235 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.235 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:02.235 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209774+0000", 2024-04-24T15:33:02.235 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999418 2024-04-24T15:33:02.235 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.235 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.235 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:02.235 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209835+0000", 2024-04-24T15:33:02.235 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.1377000000000005e-05 2024-04-24T15:33:02.235 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.236 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.236 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:02.236 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209843+0000", 2024-04-24T15:33:02.236 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.2330000000000006e-06 2024-04-24T15:33:02.236 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.236 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.236 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:02.236 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209848+0000", 2024-04-24T15:33:02.236 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6149999999999999e-06 2024-04-24T15:33:02.236 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.236 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.236 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:02.236 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252658+0000", 2024-04-24T15:33:02.237 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042810002 2024-04-24T15:33:02.237 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.237 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.237 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:02.237 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252697+0000", 2024-04-24T15:33:02.237 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8757999999999997e-05 2024-04-24T15:33:02.237 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.237 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.237 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:02.237 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252745+0000", 2024-04-24T15:33:02.237 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.863e-05 2024-04-24T15:33:02.237 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:02.237 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:02.237 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:02.237 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.238 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.238 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:24749 4.8 4:137f5701:::rbd_header.114ae15b9a20:head [watch reconnect cookie 140672595753696 gen 11] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-04-24T15:33:02.238 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:37.628703+0000", 2024-04-24T15:33:02.238 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 24.530902998999998, 2024-04-24T15:33:02.238 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0102918700000001, 2024-04-24T15:33:02.238 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:02.238 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:02.238 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:02.238 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:02.238 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:02.238 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 24749 2024-04-24T15:33:02.238 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.238 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:02.238 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.238 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:02.238 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628703+0000", 2024-04-24T15:33:02.239 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:02.239 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.239 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.239 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:02.239 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628703+0000", 2024-04-24T15:33:02.239 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:02.239 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.239 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.239 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:02.239 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628703+0000", 2024-04-24T15:33:02.239 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999999 2024-04-24T15:33:02.239 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.239 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.240 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:02.240 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628708+0000", 2024-04-24T15:33:02.240 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.4589999999999997e-06 2024-04-24T15:33:02.240 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.240 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.240 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:02.240 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628721+0000", 2024-04-24T15:33:02.240 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.2601e-05 2024-04-24T15:33:02.240 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.240 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.240 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:02.240 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628728+0000", 2024-04-24T15:33:02.240 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.9059999999999996e-06 2024-04-24T15:33:02.240 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.241 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.241 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:02.241 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628789+0000", 2024-04-24T15:33:02.241 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.1672000000000002e-05 2024-04-24T15:33:02.241 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.241 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.241 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting_for_map not empty", 2024-04-24T15:33:02.241 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628797+0000", 2024-04-24T15:33:02.241 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.8189999999999992e-06 2024-04-24T15:33:02.241 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.241 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.241 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:02.241 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.631255+0000", 2024-04-24T15:33:02.241 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0024577610000000001 2024-04-24T15:33:02.242 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.242 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.242 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for peered", 2024-04-24T15:33:02.242 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.631263+0000", 2024-04-24T15:33:02.242 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.4109999999999996e-06 2024-04-24T15:33:02.242 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.242 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.242 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:02.242 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:38.637560+0000", 2024-04-24T15:33:02.242 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0062970339999999 2024-04-24T15:33:02.242 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.242 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.242 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:02.242 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:38.637747+0000", 2024-04-24T15:33:02.243 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00018693400000000001 2024-04-24T15:33:02.243 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.243 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.243 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:02.243 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:38.638995+0000", 2024-04-24T15:33:02.243 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0012481300000000001 2024-04-24T15:33:02.243 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:02.243 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:02.243 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:02.243 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.243 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.243 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:24750 4.8 4:137f5701:::rbd_header.114ae15b9a20:head [notify cookie 140672594434416] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-04-24T15:33:02.243 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:37.628736+0000", 2024-04-24T15:33:02.243 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 24.530870739000001, 2024-04-24T15:33:02.243 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0096094309999999, 2024-04-24T15:33:02.244 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:02.244 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:02.244 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:02.244 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:02.244 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:02.244 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 24750 2024-04-24T15:33:02.244 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.244 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:02.244 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.244 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:02.244 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628736+0000", 2024-04-24T15:33:02.244 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:02.244 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.244 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.245 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:02.245 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628736+0000", 2024-04-24T15:33:02.245 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:02.245 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.245 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.245 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:02.245 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628735+0000", 2024-04-24T15:33:02.245 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:02.245 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.245 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.245 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:02.245 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628739+0000", 2024-04-24T15:33:02.245 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5109999999999999e-06 2024-04-24T15:33:02.245 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.245 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.246 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:02.246 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628745+0000", 2024-04-24T15:33:02.246 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.8300000000000001e-06 2024-04-24T15:33:02.246 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.246 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.246 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:02.246 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628749+0000", 2024-04-24T15:33:02.246 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.7110000000000003e-06 2024-04-24T15:33:02.246 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.246 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.246 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:02.246 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628826+0000", 2024-04-24T15:33:02.246 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.6678e-05 2024-04-24T15:33:02.246 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.247 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.247 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting_for_map not empty", 2024-04-24T15:33:02.247 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628834+0000", 2024-04-24T15:33:02.247 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.6190000000000002e-06 2024-04-24T15:33:02.247 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.247 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.247 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:02.247 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.631305+0000", 2024-04-24T15:33:02.247 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0024714400000000001 2024-04-24T15:33:02.247 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.247 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.247 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for peered", 2024-04-24T15:33:02.247 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.631313+0000", 2024-04-24T15:33:02.247 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.6529999999999994e-06 2024-04-24T15:33:02.247 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.247 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.248 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:02.248 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:38.638068+0000", 2024-04-24T15:33:02.248 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0067548369999999 2024-04-24T15:33:02.248 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.248 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.248 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:02.248 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:38.638158+0000", 2024-04-24T15:33:02.248 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.0079000000000003e-05 2024-04-24T15:33:02.248 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:02.248 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:02.248 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:02.248 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:38.638345+0000", 2024-04-24T15:33:02.248 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00018757100000000001 2024-04-24T15:33:02.248 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:02.248 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:02.248 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:02.249 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:02.249 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:02.249 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:02.250 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-24T15:33:02.251 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:02.251+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:02.351 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:02.351+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:02.374 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:02.374 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:33:02.374 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-04-24T15:33:02.374 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:02.384 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-24T15:33:02.469 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:33:02.469+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:33:02.546 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:02.547 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:33:02.547 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-04-24T15:33:02.548 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-04-24T15:33:02.548 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:02.553 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:33:02.554+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:33:02.556 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-24T15:33:02.610 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-24T15:33:02.636 INFO:teuthology.orchestra.run.smithi096.stderr:nodeep-scrub is unset 2024-04-24T15:33:02.654 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:33:02.654+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:33:02.732 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:02.732 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-04-24T15:33:02.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-04-24T15:33:02.732 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-04-24T15:33:02.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.732 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:18 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'8, mlcod=15'5)", 2024-04-24T15:33:02.733 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:00.973932+0000", 2024-04-24T15:33:02.733 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 61.757881054000002, 2024-04-24T15:33:02.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0064286839999999996, 2024-04-24T15:33:02.733 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:02.733 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:02.733 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:02.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:02.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.973932+0000", 2024-04-24T15:33:02.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:02.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:02.734 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.973932+0000", 2024-04-24T15:33:02.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:02.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.734 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.734 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:02.734 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.973931+0000", 2024-04-24T15:33:02.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:33:02.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.734 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.734 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:02.734 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.973938+0000", 2024-04-24T15:33:02.735 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.029e-06 2024-04-24T15:33:02.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.735 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:02.735 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.973948+0000", 2024-04-24T15:33:02.735 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9129999999999998e-06 2024-04-24T15:33:02.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.735 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:02.735 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.973952+0000", 2024-04-24T15:33:02.735 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3069999999999996e-06 2024-04-24T15:33:02.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.736 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.736 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:02.736 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.973986+0000", 2024-04-24T15:33:02.736 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3643000000000003e-05 2024-04-24T15:33:02.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.736 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.736 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:02.736 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.974006+0000", 2024-04-24T15:33:02.736 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9658999999999999e-05 2024-04-24T15:33:02.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.736 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.736 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:02.736 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.980330+0000", 2024-04-24T15:33:02.736 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0063243479999999996 2024-04-24T15:33:02.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:02.737 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.980361+0000", 2024-04-24T15:33:02.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0527000000000001e-05 2024-04-24T15:33:02.737 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:02.737 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:02.737 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:02.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.737 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:21 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'11, mlcod=15'8)", 2024-04-24T15:33:02.737 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:00.986502+0000", 2024-04-24T15:33:02.737 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 61.745311121, 2024-04-24T15:33:02.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0062008819999999996, 2024-04-24T15:33:02.738 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:02.738 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:02.738 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:02.738 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.738 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:02.738 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.986502+0000", 2024-04-24T15:33:02.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:02.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.738 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.738 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:02.738 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.986502+0000", 2024-04-24T15:33:02.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:02.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:02.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.986501+0000", 2024-04-24T15:33:02.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:02.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:02.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.986639+0000", 2024-04-24T15:33:02.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013731400000000001 2024-04-24T15:33:02.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:02.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.986649+0000", 2024-04-24T15:33:02.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.041e-05 2024-04-24T15:33:02.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:02.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.986654+0000", 2024-04-24T15:33:02.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2000000000000002e-06 2024-04-24T15:33:02.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:02.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.986678+0000", 2024-04-24T15:33:02.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3731e-05 2024-04-24T15:33:02.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:02.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.986691+0000", 2024-04-24T15:33:02.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3078000000000001e-05 2024-04-24T15:33:02.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:02.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.992666+0000", 2024-04-24T15:33:02.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0059751650000000002 2024-04-24T15:33:02.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:02.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.992703+0000", 2024-04-24T15:33:02.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6622000000000003e-05 2024-04-24T15:33:02.742 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:02.742 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:02.742 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:02.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.742 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:22 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'12, mlcod=15'8)", 2024-04-24T15:33:02.742 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:00.987102+0000", 2024-04-24T15:33:02.742 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 61.744711275, 2024-04-24T15:33:02.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0056291650000000002, 2024-04-24T15:33:02.742 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:02.742 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:02.743 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:02.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:02.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.987102+0000", 2024-04-24T15:33:02.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:02.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:02.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.987102+0000", 2024-04-24T15:33:02.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:02.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:02.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.987101+0000", 2024-04-24T15:33:02.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:02.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:02.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.987106+0000", 2024-04-24T15:33:02.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0969999999999996e-06 2024-04-24T15:33:02.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:02.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.987113+0000", 2024-04-24T15:33:02.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6789999999999999e-06 2024-04-24T15:33:02.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:02.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.987117+0000", 2024-04-24T15:33:02.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6519999999999999e-06 2024-04-24T15:33:02.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:02.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.987143+0000", 2024-04-24T15:33:02.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6098999999999999e-05 2024-04-24T15:33:02.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:02.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.987157+0000", 2024-04-24T15:33:02.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4100999999999999e-05 2024-04-24T15:33:02.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.746 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:02.746 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.992706+0000", 2024-04-24T15:33:02.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0055495630000000004 2024-04-24T15:33:02.746 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.746 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.746 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:02.746 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.992731+0000", 2024-04-24T15:33:02.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4519e-05 2024-04-24T15:33:02.746 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:02.747 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:02.747 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:02.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.747 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:69 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'43, mlcod=15'39)", 2024-04-24T15:33:02.747 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.058504+0000", 2024-04-24T15:33:02.747 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 61.673308800999997, 2024-04-24T15:33:02.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.006494606, 2024-04-24T15:33:02.747 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:02.747 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:02.747 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:02.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:02.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.058504+0000", 2024-04-24T15:33:02.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:02.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:02.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.058504+0000", 2024-04-24T15:33:02.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:02.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:02.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.058503+0000", 2024-04-24T15:33:02.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:33:02.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:02.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.058588+0000", 2024-04-24T15:33:02.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4087999999999999e-05 2024-04-24T15:33:02.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:02.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.058597+0000", 2024-04-24T15:33:02.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7459999999999995e-06 2024-04-24T15:33:02.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:02.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.058602+0000", 2024-04-24T15:33:02.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6929999999999998e-06 2024-04-24T15:33:02.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:02.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.060983+0000", 2024-04-24T15:33:02.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023812690000000001 2024-04-24T15:33:02.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:02.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.060996+0000", 2024-04-24T15:33:02.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2826999999999999e-05 2024-04-24T15:33:02.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:02.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.064972+0000", 2024-04-24T15:33:02.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0039757890000000004 2024-04-24T15:33:02.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:02.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.064999+0000", 2024-04-24T15:33:02.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6917999999999999e-05 2024-04-24T15:33:02.751 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:02.751 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:02.751 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:02.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.751 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:70 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'44, mlcod=15'39)", 2024-04-24T15:33:02.751 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.059180+0000", 2024-04-24T15:33:02.751 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 61.672633048999998, 2024-04-24T15:33:02.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0058429709999999998, 2024-04-24T15:33:02.752 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:02.752 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:02.752 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:02.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:02.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.059180+0000", 2024-04-24T15:33:02.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:02.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:02.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.059180+0000", 2024-04-24T15:33:02.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:02.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:02.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.059179+0000", 2024-04-24T15:33:02.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:02.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:02.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.059184+0000", 2024-04-24T15:33:02.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.074e-06 2024-04-24T15:33:02.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:02.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.059194+0000", 2024-04-24T15:33:02.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6050000000000004e-06 2024-04-24T15:33:02.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:02.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.059199+0000", 2024-04-24T15:33:02.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2639999999999999e-06 2024-04-24T15:33:02.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:02.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.061424+0000", 2024-04-24T15:33:02.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0022244880000000002 2024-04-24T15:33:02.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:02.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.061444+0000", 2024-04-24T15:33:02.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0633e-05 2024-04-24T15:33:02.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:02.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.065001+0000", 2024-04-24T15:33:02.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035567670000000002 2024-04-24T15:33:02.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:02.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.065023+0000", 2024-04-24T15:33:02.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1724000000000001e-05 2024-04-24T15:33:02.755 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:02.755 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:02.755 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:02.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.755 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:154 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'108, mlcod=15'105)", 2024-04-24T15:33:02.756 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.929033+0000", 2024-04-24T15:33:02.756 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 60.802779667999999, 2024-04-24T15:33:02.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0068887990000000001, 2024-04-24T15:33:02.756 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:02.756 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:02.756 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:02.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:02.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929033+0000", 2024-04-24T15:33:02.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:02.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:02.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929033+0000", 2024-04-24T15:33:02.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:02.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:02.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929033+0000", 2024-04-24T15:33:02.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:02.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:02.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929042+0000", 2024-04-24T15:33:02.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5440000000000004e-06 2024-04-24T15:33:02.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:02.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929051+0000", 2024-04-24T15:33:02.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0580000000000004e-06 2024-04-24T15:33:02.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:02.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929055+0000", 2024-04-24T15:33:02.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.337e-06 2024-04-24T15:33:02.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:02.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929074+0000", 2024-04-24T15:33:02.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9724999999999998e-05 2024-04-24T15:33:02.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:02.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929084+0000", 2024-04-24T15:33:02.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7049999999999994e-06 2024-04-24T15:33:02.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:02.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.935895+0000", 2024-04-24T15:33:02.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0068104359999999996 2024-04-24T15:33:02.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:02.760 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.935922+0000", 2024-04-24T15:33:02.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7538999999999998e-05 2024-04-24T15:33:02.760 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:02.760 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:02.760 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:02.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.760 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:155 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'109, mlcod=15'105)", 2024-04-24T15:33:02.760 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.929398+0000", 2024-04-24T15:33:02.760 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 60.802414687000002, 2024-04-24T15:33:02.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0098754700000000008, 2024-04-24T15:33:02.760 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:02.761 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:02.761 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:02.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:02.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929398+0000", 2024-04-24T15:33:02.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:02.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:02.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929398+0000", 2024-04-24T15:33:02.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:02.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:02.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929398+0000", 2024-04-24T15:33:02.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:02.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:02.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929571+0000", 2024-04-24T15:33:02.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017281999999999999 2024-04-24T15:33:02.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:02.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929578+0000", 2024-04-24T15:33:02.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8929999999999999e-06 2024-04-24T15:33:02.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:02.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929581+0000", 2024-04-24T15:33:02.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1709999999999998e-06 2024-04-24T15:33:02.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:02.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929605+0000", 2024-04-24T15:33:02.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3938e-05 2024-04-24T15:33:02.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:02.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929618+0000", 2024-04-24T15:33:02.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3653e-05 2024-04-24T15:33:02.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:02.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939186+0000", 2024-04-24T15:33:02.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0095674260000000004 2024-04-24T15:33:02.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:02.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939274+0000", 2024-04-24T15:33:02.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7928000000000002e-05 2024-04-24T15:33:02.764 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:02.764 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:02.764 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:02.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.765 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:156 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'110, mlcod=15'105)", 2024-04-24T15:33:02.765 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.930050+0000", 2024-04-24T15:33:02.765 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 60.801763371, 2024-04-24T15:33:02.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0092911650000000005, 2024-04-24T15:33:02.765 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:02.765 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:02.765 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:02.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.765 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:02.765 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930050+0000", 2024-04-24T15:33:02.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:02.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:02.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930050+0000", 2024-04-24T15:33:02.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:02.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:02.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930049+0000", 2024-04-24T15:33:02.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:02.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:02.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930053+0000", 2024-04-24T15:33:02.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.98e-06 2024-04-24T15:33:02.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:02.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930058+0000", 2024-04-24T15:33:02.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3530000000000002e-06 2024-04-24T15:33:02.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:02.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930061+0000", 2024-04-24T15:33:02.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6970000000000002e-06 2024-04-24T15:33:02.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:02.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930082+0000", 2024-04-24T15:33:02.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0815e-05 2024-04-24T15:33:02.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:02.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930093+0000", 2024-04-24T15:33:02.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0699999999999999e-05 2024-04-24T15:33:02.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:02.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939280+0000", 2024-04-24T15:33:02.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0091874880000000006 2024-04-24T15:33:02.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:02.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939341+0000", 2024-04-24T15:33:02.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0608e-05 2024-04-24T15:33:02.769 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:02.769 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:02.769 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:02.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.769 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:157 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'111, mlcod=15'106)", 2024-04-24T15:33:02.770 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.930506+0000", 2024-04-24T15:33:02.770 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 60.801306513, 2024-04-24T15:33:02.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.008889246, 2024-04-24T15:33:02.770 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:02.770 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:02.770 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:02.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:02.770 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930506+0000", 2024-04-24T15:33:02.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:02.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:02.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930506+0000", 2024-04-24T15:33:02.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:02.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:02.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930506+0000", 2024-04-24T15:33:02.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:02.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:02.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930510+0000", 2024-04-24T15:33:02.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1810000000000003e-06 2024-04-24T15:33:02.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:02.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930516+0000", 2024-04-24T15:33:02.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1920000000000002e-06 2024-04-24T15:33:02.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:02.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930520+0000", 2024-04-24T15:33:02.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.501e-06 2024-04-24T15:33:02.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:02.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930539+0000", 2024-04-24T15:33:02.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8929000000000001e-05 2024-04-24T15:33:02.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:02.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930550+0000", 2024-04-24T15:33:02.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1168999999999999e-05 2024-04-24T15:33:02.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:02.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939343+0000", 2024-04-24T15:33:02.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0087932900000000005 2024-04-24T15:33:02.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:02.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939396+0000", 2024-04-24T15:33:02.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2454999999999998e-05 2024-04-24T15:33:02.773 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:02.774 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:02.774 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:02.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.774 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:158 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'112, mlcod=15'106)", 2024-04-24T15:33:02.774 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.930877+0000", 2024-04-24T15:33:02.774 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 60.800935822, 2024-04-24T15:33:02.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.008572741, 2024-04-24T15:33:02.774 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:02.774 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:02.774 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:02.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:02.774 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930877+0000", 2024-04-24T15:33:02.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:02.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:02.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930877+0000", 2024-04-24T15:33:02.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:02.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:02.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930877+0000", 2024-04-24T15:33:02.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:02.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:02.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930968+0000", 2024-04-24T15:33:02.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1638999999999995e-05 2024-04-24T15:33:02.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:02.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930975+0000", 2024-04-24T15:33:02.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2169999999999999e-06 2024-04-24T15:33:02.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:02.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930977+0000", 2024-04-24T15:33:02.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.791e-06 2024-04-24T15:33:02.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:02.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931002+0000", 2024-04-24T15:33:02.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4491e-05 2024-04-24T15:33:02.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:02.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931012+0000", 2024-04-24T15:33:02.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0217000000000001e-05 2024-04-24T15:33:02.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:02.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939398+0000", 2024-04-24T15:33:02.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.008385366 2024-04-24T15:33:02.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:02.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939450+0000", 2024-04-24T15:33:02.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2326e-05 2024-04-24T15:33:02.778 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:02.778 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:02.778 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:02.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.778 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:159 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'113, mlcod=15'106)", 2024-04-24T15:33:02.778 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.931313+0000", 2024-04-24T15:33:02.778 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 60.800500470000003, 2024-04-24T15:33:02.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0081994159999999993, 2024-04-24T15:33:02.779 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:02.779 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:02.779 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:02.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:02.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931313+0000", 2024-04-24T15:33:02.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:02.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:02.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931313+0000", 2024-04-24T15:33:02.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:02.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:02.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931312+0000", 2024-04-24T15:33:02.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:02.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:02.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931316+0000", 2024-04-24T15:33:02.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0589999999999996e-06 2024-04-24T15:33:02.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:02.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931322+0000", 2024-04-24T15:33:02.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2020000000000001e-06 2024-04-24T15:33:02.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:02.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931327+0000", 2024-04-24T15:33:02.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2440000000000004e-06 2024-04-24T15:33:02.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:02.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931348+0000", 2024-04-24T15:33:02.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1464e-05 2024-04-24T15:33:02.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:02.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931359+0000", 2024-04-24T15:33:02.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0699e-05 2024-04-24T15:33:02.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:02.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939452+0000", 2024-04-24T15:33:02.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0080931749999999993 2024-04-24T15:33:02.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:02.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939512+0000", 2024-04-24T15:33:02.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0050000000000003e-05 2024-04-24T15:33:02.782 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:02.783 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:02.783 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:02.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.783 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:2447 4.6 4:668081fa:::rbd_data.114a1f60cb66.00000000000001f0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-24T15:33:02.783 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:23.027189+0000", 2024-04-24T15:33:02.783 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 39.704624445999997, 2024-04-24T15:33:02.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0056741200000000004, 2024-04-24T15:33:02.783 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:02.783 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:02.783 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:02.783 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:02.783 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:02.783 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2447 2024-04-24T15:33:02.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.784 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:02.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:02.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027189+0000", 2024-04-24T15:33:02.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:02.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:02.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027189+0000", 2024-04-24T15:33:02.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:02.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:02.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027188+0000", 2024-04-24T15:33:02.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:33:02.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:02.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027194+0000", 2024-04-24T15:33:02.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0499999999999997e-06 2024-04-24T15:33:02.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:02.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027205+0000", 2024-04-24T15:33:02.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1619e-05 2024-04-24T15:33:02.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:02.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027212+0000", 2024-04-24T15:33:02.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4289999999999998e-06 2024-04-24T15:33:02.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:02.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027243+0000", 2024-04-24T15:33:02.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1510000000000002e-05 2024-04-24T15:33:02.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:02.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027288+0000", 2024-04-24T15:33:02.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4484000000000003e-05 2024-04-24T15:33:02.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-24T15:33:02.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027351+0000", 2024-04-24T15:33:02.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3311000000000004e-05 2024-04-24T15:33:02.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:33:02.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.029874+0000", 2024-04-24T15:33:02.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00252315 2024-04-24T15:33:02.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:33:02.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.032786+0000", 2024-04-24T15:33:02.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0029115370000000001 2024-04-24T15:33:02.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:02.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.032828+0000", 2024-04-24T15:33:02.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2132000000000003e-05 2024-04-24T15:33:02.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:02.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.032863+0000", 2024-04-24T15:33:02.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4912e-05 2024-04-24T15:33:02.788 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:02.788 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:02.788 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:02.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.789 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:2449 4.6 4:61283b7c:::rbd_data.114a1f60cb66.00000000000001f2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-24T15:33:02.789 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:23.027620+0000", 2024-04-24T15:33:02.789 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 39.704193033000003, 2024-04-24T15:33:02.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0053273959999999999, 2024-04-24T15:33:02.789 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:02.789 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:02.789 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:02.789 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:02.789 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:02.789 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2449 2024-04-24T15:33:02.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.789 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:02.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:02.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027620+0000", 2024-04-24T15:33:02.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:02.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:02.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027620+0000", 2024-04-24T15:33:02.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:02.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:02.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027619+0000", 2024-04-24T15:33:02.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:02.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:02.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027623+0000", 2024-04-24T15:33:02.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7100000000000001e-06 2024-04-24T15:33:02.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:02.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027636+0000", 2024-04-24T15:33:02.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2609000000000001e-05 2024-04-24T15:33:02.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:02.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027643+0000", 2024-04-24T15:33:02.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.182e-06 2024-04-24T15:33:02.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:02.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027678+0000", 2024-04-24T15:33:02.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5641e-05 2024-04-24T15:33:02.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:02.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027765+0000", 2024-04-24T15:33:02.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6691000000000001e-05 2024-04-24T15:33:02.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-24T15:33:02.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027864+0000", 2024-04-24T15:33:02.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9257000000000002e-05 2024-04-24T15:33:02.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:33:02.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.032638+0000", 2024-04-24T15:33:02.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004773429 2024-04-24T15:33:02.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:33:02.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.032896+0000", 2024-04-24T15:33:02.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000258352 2024-04-24T15:33:02.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:02.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.032924+0000", 2024-04-24T15:33:02.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8337999999999999e-05 2024-04-24T15:33:02.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:02.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.032947+0000", 2024-04-24T15:33:02.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2849000000000001e-05 2024-04-24T15:33:02.794 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:02.794 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:02.794 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:02.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.795 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4426.0:4119 4.6 e34/33 4:6588fd86:::rbd_data.114a1f60cb66.0000000000000179:head v 34'48, mlcod=34'45)", 2024-04-24T15:33:02.795 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.936723+0000", 2024-04-24T15:33:02.795 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.795090004999999, 2024-04-24T15:33:02.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0058208640000000002, 2024-04-24T15:33:02.795 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:02.795 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:02.795 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:02.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:02.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.936723+0000", 2024-04-24T15:33:02.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:02.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:02.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.936723+0000", 2024-04-24T15:33:02.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:02.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:02.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.936722+0000", 2024-04-24T15:33:02.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:02.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:02.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.936735+0000", 2024-04-24T15:33:02.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2153999999999999e-05 2024-04-24T15:33:02.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:02.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.936743+0000", 2024-04-24T15:33:02.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9360000000000005e-06 2024-04-24T15:33:02.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:02.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.936749+0000", 2024-04-24T15:33:02.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5040000000000002e-06 2024-04-24T15:33:02.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:02.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.936783+0000", 2024-04-24T15:33:02.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3689999999999998e-05 2024-04-24T15:33:02.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:02.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.936803+0000", 2024-04-24T15:33:02.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0047999999999998e-05 2024-04-24T15:33:02.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:02.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.942517+0000", 2024-04-24T15:33:02.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057144220000000003 2024-04-24T15:33:02.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:02.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.942544+0000", 2024-04-24T15:33:02.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6735999999999999e-05 2024-04-24T15:33:02.798 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:02.799 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:02.799 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:02.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.799 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4426.0:24281 4.7 e45/43 4:e1ec41c1:::rbd_data.114ae15b9a20.0000000000000066:head v 45'58, mlcod=0'0)", 2024-04-24T15:33:02.799 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:37.194120+0000", 2024-04-24T15:33:02.799 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 25.537693343000001, 2024-04-24T15:33:02.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0091868560000000002, 2024-04-24T15:33:02.799 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:02.799 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:02.799 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:02.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:02.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.194120+0000", 2024-04-24T15:33:02.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:02.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:02.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.194120+0000", 2024-04-24T15:33:02.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:02.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:02.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.194119+0000", 2024-04-24T15:33:02.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:33:02.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:02.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.194125+0000", 2024-04-24T15:33:02.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5050000000000004e-06 2024-04-24T15:33:02.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:02.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.194144+0000", 2024-04-24T15:33:02.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8454e-05 2024-04-24T15:33:02.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:02.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.194153+0000", 2024-04-24T15:33:02.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3810000000000005e-06 2024-04-24T15:33:02.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:02.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.202288+0000", 2024-04-24T15:33:02.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0081342830000000008 2024-04-24T15:33:02.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.802 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:02.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.202304+0000", 2024-04-24T15:33:02.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6164999999999999e-05 2024-04-24T15:33:02.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.802 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:02.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.203285+0000", 2024-04-24T15:33:02.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00098150400000000001 2024-04-24T15:33:02.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:02.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.203306+0000", 2024-04-24T15:33:02.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1285e-05 2024-04-24T15:33:02.803 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:02.803 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:02.803 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:02.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.803 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4426.0:26385 4.1e e51/33 4:785cf1e5:::rbd_data.114a8e7208a5.0000000000000004:head v 51'141, mlcod=45'139)", 2024-04-24T15:33:02.803 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:43.542226+0000", 2024-04-24T15:33:02.803 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.18958684, 2024-04-24T15:33:02.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0098378319999999995, 2024-04-24T15:33:02.803 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:02.803 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:02.804 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:02.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:02.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:43.542226+0000", 2024-04-24T15:33:02.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:02.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:02.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:43.542226+0000", 2024-04-24T15:33:02.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:02.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:02.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:43.542225+0000", 2024-04-24T15:33:02.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:33:02.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.805 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:02.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:43.549287+0000", 2024-04-24T15:33:02.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0070620470000000001 2024-04-24T15:33:02.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.805 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:02.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:43.549299+0000", 2024-04-24T15:33:02.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1396999999999999e-05 2024-04-24T15:33:02.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.806 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:02.806 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:43.549321+0000", 2024-04-24T15:33:02.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2129000000000001e-05 2024-04-24T15:33:02.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.806 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:02.806 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:43.549361+0000", 2024-04-24T15:33:02.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0561e-05 2024-04-24T15:33:02.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.806 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:02.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:43.549386+0000", 2024-04-24T15:33:02.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4425000000000001e-05 2024-04-24T15:33:02.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:02.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:43.552024+0000", 2024-04-24T15:33:02.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0026388280000000002 2024-04-24T15:33:02.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:02.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:43.552064+0000", 2024-04-24T15:33:02.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9495e-05 2024-04-24T15:33:02.807 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:02.807 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:02.808 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:02.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.808 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4426.0:29864 4.1f e53/46 4:fce51a46:::rbd_data.114a2e5eebc.0000000000000256:head v 53'1252, mlcod=53'1250)", 2024-04-24T15:33:02.808 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:46.445862+0000", 2024-04-24T15:33:02.808 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 16.285950584999998, 2024-04-24T15:33:02.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010959694000000001, 2024-04-24T15:33:02.808 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:02.808 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:02.808 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:02.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:02.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.445862+0000", 2024-04-24T15:33:02.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:02.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:02.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.445862+0000", 2024-04-24T15:33:02.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:02.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:02.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.445861+0000", 2024-04-24T15:33:02.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999986 2024-04-24T15:33:02.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:02.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.446309+0000", 2024-04-24T15:33:02.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00044818000000000001 2024-04-24T15:33:02.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.810 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.810 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:02.810 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.446330+0000", 2024-04-24T15:33:02.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0754000000000002e-05 2024-04-24T15:33:02.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.810 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.810 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:02.810 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.446338+0000", 2024-04-24T15:33:02.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6320000000000008e-06 2024-04-24T15:33:02.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.810 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:02.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.448537+0000", 2024-04-24T15:33:02.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0021991509999999999 2024-04-24T15:33:02.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:02.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.448565+0000", 2024-04-24T15:33:02.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7949e-05 2024-04-24T15:33:02.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:02.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.456773+0000", 2024-04-24T15:33:02.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0082081930000000008 2024-04-24T15:33:02.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:02.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.456822+0000", 2024-04-24T15:33:02.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9270000000000001e-05 2024-04-24T15:33:02.812 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:02.812 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:02.812 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:02.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.812 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4426.0:29863 4.2 e53/43 4:45d67f64:::rbd_data.114a2e5eebc.0000000000000255:head v 53'111, mlcod=53'109)", 2024-04-24T15:33:02.812 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:46.450064+0000", 2024-04-24T15:33:02.812 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 16.281749287, 2024-04-24T15:33:02.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0067851589999999998, 2024-04-24T15:33:02.813 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:02.813 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:02.813 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:02.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:02.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.450064+0000", 2024-04-24T15:33:02.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:02.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:02.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.450064+0000", 2024-04-24T15:33:02.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:02.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.814 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:02.814 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.450062+0000", 2024-04-24T15:33:02.814 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999986 2024-04-24T15:33:02.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.814 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:02.814 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.450080+0000", 2024-04-24T15:33:02.814 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7073000000000001e-05 2024-04-24T15:33:02.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.814 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:02.814 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.450091+0000", 2024-04-24T15:33:02.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1323e-05 2024-04-24T15:33:02.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:02.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.450096+0000", 2024-04-24T15:33:02.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.95e-06 2024-04-24T15:33:02.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:02.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.450146+0000", 2024-04-24T15:33:02.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9849999999999999e-05 2024-04-24T15:33:02.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:02.816 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.450164+0000", 2024-04-24T15:33:02.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8804000000000002e-05 2024-04-24T15:33:02.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:02.816 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.456807+0000", 2024-04-24T15:33:02.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0066423760000000002 2024-04-24T15:33:02.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:02.816 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.456849+0000", 2024-04-24T15:33:02.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1992e-05 2024-04-24T15:33:02.816 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:02.816 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:02.816 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:02.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.817 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4426.0:35373 4.7 e62/54 4:e251aff1:::rbd_data.114a96db60f9.0000000000000017:head v 62'111, mlcod=61'108)", 2024-04-24T15:33:02.817 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:55.741562+0000", 2024-04-24T15:33:02.817 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 6.9902514089999999, 2024-04-24T15:33:02.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0059971139999999996, 2024-04-24T15:33:02.817 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:02.817 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:02.817 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:02.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:02.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:55.741562+0000", 2024-04-24T15:33:02.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:02.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.818 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:02.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:55.741562+0000", 2024-04-24T15:33:02.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:02.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.818 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:02.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:55.741561+0000", 2024-04-24T15:33:02.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:02.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.818 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:02.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:55.741567+0000", 2024-04-24T15:33:02.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4350000000000002e-06 2024-04-24T15:33:02.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.819 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:02.819 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:55.741577+0000", 2024-04-24T15:33:02.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7419999999999992e-06 2024-04-24T15:33:02.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.819 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:02.819 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:55.741583+0000", 2024-04-24T15:33:02.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3770000000000004e-06 2024-04-24T15:33:02.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.819 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:02.820 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:55.742158+0000", 2024-04-24T15:33:02.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00057516800000000003 2024-04-24T15:33:02.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.820 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:02.820 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:55.742177+0000", 2024-04-24T15:33:02.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9250999999999999e-05 2024-04-24T15:33:02.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.820 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:02.820 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:55.747519+0000", 2024-04-24T15:33:02.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0053422519999999996 2024-04-24T15:33:02.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:02.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:55.747559+0000", 2024-04-24T15:33:02.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9484999999999999e-05 2024-04-24T15:33:02.821 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:02.821 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:02.821 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:02.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.821 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4426.0:36199 4.1a e62/41 4:5a86856e:::rbd_data.114ab4f6fc67.000000000000000c:head v 62'269, mlcod=62'267)", 2024-04-24T15:33:02.821 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:56.585759+0000", 2024-04-24T15:33:02.821 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 6.1460543589999999, 2024-04-24T15:33:02.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057717710000000002, 2024-04-24T15:33:02.821 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:02.822 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:02.822 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:02.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.822 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:02.822 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:56.585759+0000", 2024-04-24T15:33:02.822 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:02.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.822 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:02.822 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:56.585759+0000", 2024-04-24T15:33:02.822 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:02.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.822 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:02.823 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:56.585758+0000", 2024-04-24T15:33:02.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:02.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.823 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:02.823 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:56.585763+0000", 2024-04-24T15:33:02.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9439999999999996e-06 2024-04-24T15:33:02.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.823 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:02.823 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:56.585770+0000", 2024-04-24T15:33:02.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8920000000000003e-06 2024-04-24T15:33:02.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.824 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:02.824 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:56.585774+0000", 2024-04-24T15:33:02.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0289999999999999e-06 2024-04-24T15:33:02.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.824 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.824 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:02.824 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:56.585820+0000", 2024-04-24T15:33:02.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6147999999999999e-05 2024-04-24T15:33:02.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.824 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.824 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:02.824 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:56.585839+0000", 2024-04-24T15:33:02.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.863e-05 2024-04-24T15:33:02.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.825 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.825 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:02.825 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:56.591488+0000", 2024-04-24T15:33:02.825 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0056494040000000002 2024-04-24T15:33:02.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:02.825 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:02.825 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:02.825 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:56.591530+0000", 2024-04-24T15:33:02.825 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2199e-05 2024-04-24T15:33:02.825 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:02.825 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:02.825 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:02.826 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:02.826 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:02.826 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:02.827 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-24T15:33:02.828 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-04-24T15:33:02.828 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-04-24T15:33:02.828 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-04-24T15:33:02.828 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:02.828+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:02.837 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-04-24T15:33:02.855 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-04-24T15:33:02.874 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-04-24T15:33:02.893 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-04-24T15:33:02.912 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-04-24T15:33:02.913 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T15:33:02.913 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T15:33:02.913 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T15:33:02.913 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T15:33:02.913 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T15:33:02.913 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T15:33:02.913 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T15:33:02.913 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T15:33:02.928 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:02.928+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:02.962 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:02.962 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:02.972 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-24T15:33:03.056 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:33:03.056+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:33:03.130 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:33:03.130+0000 7f23abcf6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 44021) UID: 0 2024-04-24T15:33:03.135 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:03.135 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:03.145 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-24T15:33:03.230 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:33:03.231+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:33:03.308 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:03.308 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:03.318 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-24T15:33:03.331 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:33:03.332+0000 7f23abcf6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 44021) UID: 0 2024-04-24T15:33:03.432 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:33:03.433+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:33:03.481 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:03.481 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:03.492 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-24T15:33:03.534 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:33:03.535+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:33:03.634 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:03.634+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:03.645 INFO:teuthology.orchestra.run.smithi096.stdout:3426 punch from 0x5eb4d47 to 0x5ebea0a, (0x9cc3 bytes) 2024-04-24T15:33:03.646 INFO:teuthology.orchestra.run.smithi096.stdout:3427 punch from 0x395097d to 0x3954010, (0x3693 bytes) 2024-04-24T15:33:03.646 INFO:teuthology.orchestra.run.smithi096.stdout:3428 writesame 0x2609b65 thru 0x26164aa data_size 0x1(0xc946 bytes) 2024-04-24T15:33:03.659 INFO:teuthology.orchestra.run.smithi096.stdout:3429 read 0x3c318a1 thru 0x3c3a216 (0x8976 bytes) 2024-04-24T15:33:03.659 INFO:teuthology.orchestra.run.smithi096.stdout:3431 read 0x236d186 thru 0x2372f6e (0x5de9 bytes) 2024-04-24T15:33:03.660 INFO:teuthology.orchestra.run.smithi096.stdout:3434 writesame 0x117720b thru 0x1185436 data_size 0x64(0xe22c bytes) 2024-04-24T15:33:03.663 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:03.663 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:03.664 INFO:teuthology.orchestra.run.smithi096.stdout:3436 writesame 0x38a5372 thru 0x38af0d0 data_size 0xd(0x9d5f bytes) 2024-04-24T15:33:03.672 INFO:teuthology.orchestra.run.smithi096.stdout:3437 write 0x51b1a11 thru 0x51b9162 (0x7752 bytes) 2024-04-24T15:33:03.675 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-24T15:33:03.685 INFO:teuthology.orchestra.run.smithi096.stdout:3438 trunc from 0x75bd019 to 0x2209dbe 2024-04-24T15:33:03.711 INFO:teuthology.orchestra.run.smithi096.stdout:3439 write 0x89d3d4d thru 0x89dbb30 (0x7de4 bytes) 2024-04-24T15:33:03.715 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:03.727 INFO:teuthology.orchestra.run.smithi096.stdout:3441 write 0xd434e76 thru 0xd441cfb (0xce86 bytes) 2024-04-24T15:33:03.733 INFO:teuthology.orchestra.run.smithi096.stdout:3442 punch from 0xcc0cb23 to 0xcc11a5a, (0x4f37 bytes) 2024-04-24T15:33:03.733 INFO:teuthology.orchestra.run.smithi096.stdout:3443 write 0x5f18618 thru 0x5f1c4cf (0x3eb8 bytes) 2024-04-24T15:33:03.734 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:03.735+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:03.737 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:03.737 INFO:teuthology.orchestra.run.smithi096.stdout:3445 write 0x81e7aee thru 0x81eaf94 (0x34a7 bytes) 2024-04-24T15:33:03.741 INFO:teuthology.orchestra.run.smithi096.stdout:3446 read 0x92e1802 thru 0x92f0e07 (0xf606 bytes) 2024-04-24T15:33:03.741 INFO:teuthology.orchestra.run.smithi096.stdout:3447 write 0x8f86c81 thru 0x8f8dbfa (0x6f7a bytes) 2024-04-24T15:33:03.749 INFO:teuthology.orchestra.run.smithi096.stdout:3448 write 0xcfe7479 thru 0xcff4c32 (0xd7ba bytes) 2024-04-24T15:33:03.756 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 6, 4] out_osds: [1, 5, 7] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-04-24T15:33:03.756 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-04-24T15:33:03.756 INFO:teuthology.orchestra.run.smithi096.stdout:3451 write 0xc4b9654 thru 0xc4bf7fa (0x61a7 bytes) 2024-04-24T15:33:03.762 INFO:teuthology.orchestra.run.smithi096.stdout:3453 write 0x72bf410 thru 0x72cf0bd (0xfcae bytes) 2024-04-24T15:33:03.789 INFO:teuthology.orchestra.run.smithi096.stdout:3455 trunc from 0xd441cfc to 0x360952e 2024-04-24T15:33:03.824 INFO:teuthology.orchestra.run.smithi096.stdout:3456 writesame 0xc9b6c0e thru 0xc9c651a data_size 0x1(0xf90d bytes) 2024-04-24T15:33:03.824 INFO:teuthology.orchestra.run.smithi096.stdout:3458 read 0x5e9c863 thru 0x5e9c8fc (0x9a bytes) 2024-04-24T15:33:03.835 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:03.835+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:03.837 INFO:teuthology.orchestra.run.smithi096.stdout:3461 trunc from 0xc9c651b to 0x122eca5 2024-04-24T15:33:03.843 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:03.843 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:03.852 INFO:teuthology.orchestra.run.smithi096.stdout:3463 punch from 0xea9b38 to 0xeaba3f, (0x1f07 bytes) 2024-04-24T15:33:03.852 INFO:teuthology.orchestra.run.smithi096.stdout:3464 punch from 0x173c90 to 0x1783ec, (0x475c bytes) 2024-04-24T15:33:03.852 INFO:teuthology.orchestra.run.smithi096.stdout:3466 punch from 0x9f8392 to 0xa07650, (0xf2be bytes) 2024-04-24T15:33:03.852 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:03.857 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-24T15:33:03.864 INFO:teuthology.orchestra.run.smithi096.stdout:3470 trunc from 0x122eca5 to 0xb663b12 2024-04-24T15:33:03.870 INFO:teuthology.orchestra.run.smithi096.stdout:3472 read 0x7439857 thru 0x74434e3 (0x9c8d bytes) 2024-04-24T15:33:03.870 INFO:teuthology.orchestra.run.smithi096.stdout:3473 read 0x3ac47e4 thru 0x3ac85b0 (0x3dcd bytes) 2024-04-24T15:33:03.871 INFO:teuthology.orchestra.run.smithi096.stdout:3474 punch from 0x43a3e67 to 0x43a61e8, (0x2381 bytes) 2024-04-24T15:33:03.871 INFO:teuthology.orchestra.run.smithi096.stdout:3477 write 0x618f5dc thru 0x619363c (0x4061 bytes) 2024-04-24T15:33:03.875 INFO:teuthology.orchestra.run.smithi096.stdout:3479 read 0x670bbfe thru 0x6710acc (0x4ecf bytes) 2024-04-24T15:33:03.875 INFO:teuthology.orchestra.run.smithi096.stdout:3480 writesame 0x824f314 thru 0x82548f8 data_size 0x1(0x55e5 bytes) 2024-04-24T15:33:03.886 INFO:teuthology.orchestra.run.smithi096.stdout:3482 read 0x4206410 thru 0x420d096 (0x6c87 bytes) 2024-04-24T15:33:03.886 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:03.886 INFO:teuthology.orchestra.run.smithi096.stdout:3484 read 0xb5a2a90 thru 0xb5abc8d (0x91fe bytes) 2024-04-24T15:33:03.886 INFO:teuthology.orchestra.run.smithi096.stdout:3485 write 0x2f7b086 thru 0x2f7f09d (0x4018 bytes) 2024-04-24T15:33:03.891 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:03.904 INFO:teuthology.orchestra.run.smithi096.stdout:3487 trunc from 0xb663b12 to 0x4d0aa0 2024-04-24T15:33:03.932 INFO:teuthology.orchestra.run.smithi096.stdout:3488 read 0x8861f thru 0x94f1f (0xc901 bytes) 2024-04-24T15:33:03.936 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:03.937+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:03.944 INFO:teuthology.orchestra.run.smithi096.stdout:3489 write 0x5bcdcbe thru 0x5bd010f (0x2452 bytes) 2024-04-24T15:33:03.949 INFO:teuthology.orchestra.run.smithi096.stdout:3490 punch from 0x5b6a563 to 0x5b75789, (0xb226 bytes) 2024-04-24T15:33:03.949 INFO:teuthology.orchestra.run.smithi096.stdout:3493 read 0x25125ea thru 0x251c35f (0x9d76 bytes) 2024-04-24T15:33:03.949 INFO:teuthology.orchestra.run.smithi096.stdout:3494 read 0x2975b73 thru 0x29820b5 (0xc543 bytes) 2024-04-24T15:33:03.949 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:03.949 INFO:teuthology.orchestra.run.smithi096.stdout:3496 punch from 0x1c5116 to 0x1c5d2f, (0xc19 bytes) 2024-04-24T15:33:03.949 INFO:teuthology.orchestra.run.smithi096.stdout:3498 clone 16 order 24 su 1048576 sc 14 2024-04-24T15:33:04.006 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:04.006 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:04.016 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-24T15:33:04.058 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:04.059+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:04.138 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:04.139+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:04.181 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:04.181 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:04.190 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-24T15:33:04.239 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:04.240+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:04.313 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:04.313 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:04.323 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-24T15:33:04.364 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:04.365+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:04.440 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:04.440+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:04.456 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:04.456 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:04.466 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-24T15:33:04.550 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:33:04.551+0000 7f23abcf6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 44021) UID: 0 2024-04-24T15:33:04.629 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:04.629 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:04.639 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-24T15:33:04.642 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:33:04.643+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:33:04.654 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-24T15:33:04.743 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:33:04.743+0000 7f23abcf6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 44021) UID: 0 2024-04-24T15:33:04.778 INFO:teuthology.orchestra.run.smithi096.stderr:reweighted osd.0 to 1 (10000) 2024-04-24T15:33:04.794 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-04-24T15:33:04.806 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:04.806 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:04.816 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-24T15:33:04.855 INFO:teuthology.orchestra.run.smithi096.stdout:truncating image image_client.0-clone15 from 0x5bd0110 (overlap 0x4d0aa0) to 0x3da219 2024-04-24T15:33:04.857 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:33:04.858+0000 7f23abcf6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 44021) UID: 0 2024-04-24T15:33:04.924 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-24T15:33:04.938 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #14, image image_client.0-clone14 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent15 2024-04-24T15:33:04.939 INFO:teuthology.orchestra.run.smithi096.stdout:3502 read 0x36ed6cd thru 0x36f71c1 (0x9af5 bytes) 2024-04-24T15:33:04.940 INFO:teuthology.orchestra.run.smithi096.stdout:3504 read 0x516a873 thru 0x516d311 (0x2a9f bytes) 2024-04-24T15:33:04.940 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:04.940 INFO:teuthology.orchestra.run.smithi096.stdout:3506 punch from 0x50b6a1f to 0x50bdad4, (0x70b5 bytes) 2024-04-24T15:33:04.944 INFO:teuthology.orchestra.run.smithi096.stdout:3507 punch from 0xbfc13b to 0xc074f3, (0xb3b8 bytes) 2024-04-24T15:33:04.944 INFO:teuthology.orchestra.run.smithi096.stdout:3509 write 0x3740ab thru 0x375c55 (0x1bab bytes) 2024-04-24T15:33:04.945 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:04.945+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:04.961 INFO:teuthology.orchestra.run.smithi096.stdout:3512 write 0x67c858e thru 0x67d6adf (0xe552 bytes) 2024-04-24T15:33:04.965 INFO:teuthology.orchestra.run.smithi096.stdout:3514 write 0x32e407d thru 0x32ea5e3 (0x6567 bytes) 2024-04-24T15:33:04.968 INFO:teuthology.orchestra.run.smithi096.stdout:3517 punch from 0x3a2e5e3 to 0x3a30b1e, (0x253b bytes) 2024-04-24T15:33:04.968 INFO:teuthology.orchestra.run.smithi096.stdout:3518 read 0x500b1bf thru 0x5015c33 (0xaa75 bytes) 2024-04-24T15:33:04.969 INFO:teuthology.orchestra.run.smithi096.stdout:3519 punch from 0x3e97bcb to 0x3ea2ec3, (0xb2f8 bytes) 2024-04-24T15:33:04.969 INFO:teuthology.orchestra.run.smithi096.stdout:3520 punch from 0x31f18e5 to 0x31f4e61, (0x357c bytes) 2024-04-24T15:33:04.983 INFO:teuthology.orchestra.run.smithi096.stdout:3521 write 0xdb24a5f thru 0xdb310b8 (0xc65a bytes) 2024-04-24T15:33:04.987 INFO:teuthology.orchestra.run.smithi096.stdout:3522 read 0x3695550 thru 0x369fae5 (0xa596 bytes) 2024-04-24T15:33:04.987 INFO:teuthology.orchestra.run.smithi096.stdout:3523 write 0xa428ae1 thru 0xa42b8a3 (0x2dc3 bytes) 2024-04-24T15:33:04.989 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:04.989 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:04.990 INFO:teuthology.orchestra.run.smithi096.stdout:3526 writesame 0x76acc87 thru 0x76ba20d data_size 0x39(0xd587 bytes) 2024-04-24T15:33:04.994 INFO:teuthology.orchestra.run.smithi096.stdout:3527 read 0x8366f88 thru 0x8374ac7 (0xdb40 bytes) 2024-04-24T15:33:04.994 INFO:teuthology.orchestra.run.smithi096.stdout:3530 punch from 0x243ac3d to 0x2449bd4, (0xef97 bytes) 2024-04-24T15:33:04.995 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:04.995 INFO:teuthology.orchestra.run.smithi096.stdout:3533 read 0xd4e72dc thru 0xd4e83a0 (0x10c5 bytes) 2024-04-24T15:33:04.995 INFO:teuthology.orchestra.run.smithi096.stdout:3534 read 0x1e15f38 thru 0x1e173e9 (0x14b2 bytes) 2024-04-24T15:33:04.995 INFO:teuthology.orchestra.run.smithi096.stdout:3535 writesame 0xd644b1c thru 0xd6493e8 data_size 0x1(0x48cd bytes) 2024-04-24T15:33:05.000 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-24T15:33:05.006 INFO:teuthology.orchestra.run.smithi096.stdout:3536 read 0x1ba370a thru 0x1bb0ab8 (0xd3af bytes) 2024-04-24T15:33:05.007 INFO:teuthology.orchestra.run.smithi096.stdout:3537 writesame 0xa4d0950 thru 0xa4dd82d data_size 0x1(0xcede bytes) 2024-04-24T15:33:05.035 INFO:teuthology.orchestra.run.smithi096.stdout:3542 write 0xa468905 thru 0xa478222 (0xf91e bytes) 2024-04-24T15:33:05.037 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:05.045 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:05.046+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:05.050 INFO:teuthology.orchestra.run.smithi096.stdout:3544 trunc from 0xdb310b9 to 0x2c067b4 2024-04-24T15:33:05.078 INFO:teuthology.orchestra.run.smithi096.stdout:3545 read 0x2a629d4 thru 0x2a6755c (0x4b89 bytes) 2024-04-24T15:33:05.079 INFO:teuthology.orchestra.run.smithi096.stdout:3546 write 0xa37dd2 thru 0xa3e504 (0x6733 bytes) 2024-04-24T15:33:05.083 INFO:teuthology.orchestra.run.smithi096.stdout:3548 trunc from 0x2c067b4 to 0x5244000 2024-04-24T15:33:05.088 INFO:teuthology.orchestra.run.smithi096.stdout:3549 read 0x21d4506 thru 0x21db059 (0x6b54 bytes) 2024-04-24T15:33:05.089 INFO:teuthology.orchestra.run.smithi096.stdout:3550 read 0x39f275 thru 0x3a7e43 (0x8bcf bytes) 2024-04-24T15:33:05.089 INFO:teuthology.orchestra.run.smithi096.stdout:3551 punch from 0x2c42ae4 to 0x2c46114, (0x3630 bytes) 2024-04-24T15:33:05.090 INFO:teuthology.orchestra.run.smithi096.stdout:3554 punch from 0x239b89f to 0x239d650, (0x1db1 bytes) 2024-04-24T15:33:05.090 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:05.090 INFO:teuthology.orchestra.run.smithi096.stdout:3556 read 0x2ac574d thru 0x2ad3f69 (0xe81d bytes) 2024-04-24T15:33:05.092 INFO:teuthology.orchestra.run.smithi096.stdout:3557 trunc from 0x5244000 to 0x2de28fc 2024-04-24T15:33:05.097 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-04-24T15:33:05.101 INFO:teuthology.orchestra.run.smithi096.stdout:3559 read 0x18dd72f thru 0x18e7199 (0x9a6b bytes) 2024-04-24T15:33:05.102 INFO:teuthology.orchestra.run.smithi096.stdout:3560 writesame 0x1dc0c45 thru 0x1dcb7c2 data_size 0x36(0xab7e bytes) 2024-04-24T15:33:05.116 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-04-24T15:33:05.116 INFO:teuthology.orchestra.run.smithi096.stdout:3563 write 0x80af88b thru 0x80b8512 (0x8c88 bytes) 2024-04-24T15:33:05.118 INFO:teuthology.orchestra.run.smithi096.stdout:3564 write 0x8b45e3 thru 0x8b6559 (0x1f77 bytes) 2024-04-24T15:33:05.119 INFO:teuthology.orchestra.run.smithi096.stdout:3566 read 0x516fab8 thru 0x5170a32 (0xf7b bytes) 2024-04-24T15:33:05.119 INFO:teuthology.orchestra.run.smithi096.stdout:3569 punch from 0x4b7dea to 0x4be40a, (0x6620 bytes) 2024-04-24T15:33:05.119 INFO:teuthology.orchestra.run.smithi096.stdout:3570 punch from 0x25d8525 to 0x25df770, (0x724b bytes) 2024-04-24T15:33:05.120 INFO:teuthology.orchestra.run.smithi096.stdout:3571 read 0x30f65a8 thru 0x30fdc67 (0x76c0 bytes) 2024-04-24T15:33:05.120 INFO:teuthology.orchestra.run.smithi096.stdout:3572 punch from 0x2ea2cf1 to 0x2eb0f8a, (0xe299 bytes) 2024-04-24T15:33:05.124 INFO:teuthology.orchestra.run.smithi096.stdout:3573 trunc from 0x80b8513 to 0x455fa49 2024-04-24T15:33:05.133 INFO:teuthology.orchestra.run.smithi096.stdout:3576 trunc from 0x455fa49 to 0x3d87862 2024-04-24T15:33:05.136 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:05.137 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:05.137 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-04-24T15:33:05.142 INFO:teuthology.orchestra.run.smithi096.stdout:3577 write 0x3a5396c thru 0x3a5c800 (0x8e95 bytes) 2024-04-24T15:33:05.143 INFO:teuthology.orchestra.run.smithi096.stdout:3578 punch from 0x2385fb3 to 0x2390662, (0xa6af bytes) 2024-04-24T15:33:05.143 INFO:teuthology.orchestra.run.smithi096.stdout:3579 write 0x3c90840 thru 0x3c9a004 (0x97c5 bytes) 2024-04-24T15:33:05.145 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:05.147+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:05.146 INFO:teuthology.orchestra.run.smithi096.stdout:3580 trunc from 0x3d87862 to 0x326985e 2024-04-24T15:33:05.147 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-24T15:33:05.154 INFO:teuthology.orchestra.run.smithi096.stdout:3581 read 0x2e890f3 thru 0x2e97f5f (0xee6d bytes) 2024-04-24T15:33:05.160 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-04-24T15:33:05.175 INFO:teuthology.orchestra.run.smithi096.stdout:3582 write 0xe7daf58 thru 0xe7e4118 (0x91c1 bytes) 2024-04-24T15:33:05.178 INFO:teuthology.orchestra.run.smithi096.stdout:3583 read 0x70a515e thru 0x70a6a04 (0x18a7 bytes) 2024-04-24T15:33:05.179 INFO:teuthology.orchestra.run.smithi096.stdout:3585 read 0x30ebf46 thru 0x30f1c60 (0x5d1b bytes) 2024-04-24T15:33:05.179 INFO:teuthology.orchestra.run.smithi096.stdout:3586 read 0x4f0aba8 thru 0x4f1a8a2 (0xfcfb bytes) 2024-04-24T15:33:05.179 INFO:teuthology.orchestra.run.smithi096.stdout:3587 punch from 0x9a75757 to 0x9a7cfbe, (0x7867 bytes) 2024-04-24T15:33:05.179 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:05.179 INFO:teuthology.orchestra.run.smithi096.stdout:3590 read 0x65a2dad thru 0x65ab3fd (0x8651 bytes) 2024-04-24T15:33:05.181 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-04-24T15:33:05.189 INFO:teuthology.orchestra.run.smithi096.stdout:3592 trunc from 0xe7e4119 to 0x6a9153a 2024-04-24T15:33:05.202 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-04-24T15:33:05.205 INFO:teuthology.orchestra.run.smithi096.stdout:3593 read 0x1fb180c thru 0x1fc056c (0xed61 bytes) 2024-04-24T15:33:05.206 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:05.210 INFO:teuthology.orchestra.run.smithi096.stdout:3596 trunc from 0x6a9153a to 0x9e0b682 2024-04-24T15:33:05.219 INFO:teuthology.orchestra.run.smithi096.stdout:3597 trunc from 0x9e0b682 to 0xcccc7b1 2024-04-24T15:33:05.222 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-04-24T15:33:05.224 INFO:teuthology.orchestra.run.smithi096.stdout:3598 punch from 0xca5dda5 to 0xca6165a, (0x38b5 bytes) 2024-04-24T15:33:05.227 INFO:teuthology.orchestra.run.smithi096.stdout:3599 trunc from 0xcccc7b1 to 0xa574909 2024-04-24T15:33:05.243 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-04-24T15:33:05.244 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T15:33:05.245 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T15:33:05.245 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T15:33:05.245 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T15:33:05.245 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T15:33:05.245 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T15:33:05.245 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T15:33:05.245 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T15:33:05.246 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:05.247+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:05.247 INFO:teuthology.orchestra.run.smithi096.stdout:3600 write 0xda5a616 thru 0xda5c5cc (0x1fb7 bytes) 2024-04-24T15:33:05.249 INFO:teuthology.orchestra.run.smithi096.stdout:3601 read 0x49910a5 thru 0x4995715 (0x4671 bytes) 2024-04-24T15:33:05.249 INFO:teuthology.orchestra.run.smithi096.stdout:3602 read 0x465c4c thru 0x471bc8 (0xbf7d bytes) 2024-04-24T15:33:05.250 INFO:teuthology.orchestra.run.smithi096.stdout:3603 writesame 0x809d375 thru 0x80a4440 data_size 0x2(0x70cc bytes) 2024-04-24T15:33:05.258 INFO:teuthology.orchestra.run.smithi096.stdout:3604 writesame 0xa163fd2 thru 0xa164ebc data_size 0x13(0xeeb bytes) 2024-04-24T15:33:05.260 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:05.260 INFO:teuthology.orchestra.run.smithi096.stdout:3606 read 0xb00dd18 thru 0xb01d093 (0xf37c bytes) 2024-04-24T15:33:05.260 INFO:teuthology.orchestra.run.smithi096.stdout:3607 punch from 0xb189973 to 0xb192293, (0x8920 bytes) 2024-04-24T15:33:05.260 INFO:teuthology.orchestra.run.smithi096.stdout:3609 write 0xafe7675 thru 0xafea249 (0x2bd5 bytes) 2024-04-24T15:33:05.261 INFO:teuthology.orchestra.run.smithi096.stdout:3610 punch from 0xc235b71 to 0xc2457e6, (0xfc75 bytes) 2024-04-24T15:33:05.262 INFO:teuthology.orchestra.run.smithi096.stdout:3611 write 0xcd7499e thru 0xcd77c86 (0x32e9 bytes) 2024-04-24T15:33:05.263 INFO:teuthology.orchestra.run.smithi096.stdout:3612 read 0x4de529d thru 0x4de6dd7 (0x1b3b bytes) 2024-04-24T15:33:05.263 INFO:teuthology.orchestra.run.smithi096.stdout:3613 read 0xb6204f thru 0xb6b725 (0x96d7 bytes) 2024-04-24T15:33:05.263 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:05.263 INFO:teuthology.orchestra.run.smithi096.stdout:3615 writesame 0x2dbffca thru 0x2dc48e2 data_size 0x1(0x4919 bytes) 2024-04-24T15:33:05.275 INFO:teuthology.orchestra.run.smithi096.stdout:3616 writesame 0xd0f07ac thru 0xd0f7822 data_size 0x15(0x7077 bytes) 2024-04-24T15:33:05.277 INFO:teuthology.orchestra.run.smithi096.stdout:3619 writesame 0xa3d9cf6 thru 0xa3e7081 data_size 0x6(0xd38c bytes) 2024-04-24T15:33:05.282 INFO:teuthology.orchestra.run.smithi096.stdout:3620 clone 17 order 25 su 8388608 sc 7 2024-04-24T15:33:05.288 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:05.288 INFO:teuthology.orchestra.run.smithi151.stdout: "size": 20, 2024-04-24T15:33:05.289 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 600, 2024-04-24T15:33:05.289 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [ 2024-04-24T15:33:05.289 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.289 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23613 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 14688256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:05.289 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.792522+0000", 2024-04-24T15:33:05.289 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 30.495967718999999, 2024-04-24T15:33:05.289 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040507356000000001, 2024-04-24T15:33:05.289 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:05.289 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:05.289 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:05.289 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:05.289 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:05.289 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23613 2024-04-24T15:33:05.289 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.290 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:05.290 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.290 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:05.290 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792522+0000", 2024-04-24T15:33:05.290 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.290 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.290 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.290 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:05.290 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792522+0000", 2024-04-24T15:33:05.290 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.290 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.290 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.290 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:05.290 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792522+0000", 2024-04-24T15:33:05.291 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:05.291 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.291 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.291 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:05.291 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792523+0000", 2024-04-24T15:33:05.291 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.576e-06 2024-04-24T15:33:05.291 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.291 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.291 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:05.291 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792526+0000", 2024-04-24T15:33:05.291 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7439999999999999e-06 2024-04-24T15:33:05.291 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.291 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.292 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:05.292 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792528+0000", 2024-04-24T15:33:05.292 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.0600000000000002e-06 2024-04-24T15:33:05.292 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.292 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.292 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:05.292 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.832864+0000", 2024-04-24T15:33:05.292 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040335742000000001 2024-04-24T15:33:05.292 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.292 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.292 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:05.292 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.832953+0000", 2024-04-24T15:33:05.292 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.9281000000000001e-05 2024-04-24T15:33:05.293 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.293 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.293 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:05.293 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.833029+0000", 2024-04-24T15:33:05.293 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.6259000000000003e-05 2024-04-24T15:33:05.293 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.293 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:05.293 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.293 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.293 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.293 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23614 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 14729216~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:05.293 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.792530+0000", 2024-04-24T15:33:05.293 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 30.495959455000001, 2024-04-24T15:33:05.293 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040633957999999998, 2024-04-24T15:33:05.294 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:05.294 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:05.294 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:05.294 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:05.294 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:05.294 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23614 2024-04-24T15:33:05.294 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.294 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:05.294 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.294 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:05.294 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792530+0000", 2024-04-24T15:33:05.294 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.294 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.295 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.295 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:05.295 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792530+0000", 2024-04-24T15:33:05.295 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.295 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.295 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.295 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:05.295 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792530+0000", 2024-04-24T15:33:05.295 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:05.295 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.295 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.295 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:05.295 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792531+0000", 2024-04-24T15:33:05.295 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.46e-06 2024-04-24T15:33:05.296 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.296 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.296 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:05.296 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792534+0000", 2024-04-24T15:33:05.296 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8959999999999999e-06 2024-04-24T15:33:05.296 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.296 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.296 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:05.296 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792536+0000", 2024-04-24T15:33:05.296 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.841e-06 2024-04-24T15:33:05.296 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.296 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.296 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:05.296 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.833060+0000", 2024-04-24T15:33:05.297 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040524125000000001 2024-04-24T15:33:05.297 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.297 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.297 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:05.297 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.833108+0000", 2024-04-24T15:33:05.297 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.7547999999999999e-05 2024-04-24T15:33:05.297 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.297 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.297 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:05.297 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.833164+0000", 2024-04-24T15:33:05.297 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6297e-05 2024-04-24T15:33:05.297 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.297 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:05.297 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.298 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.298 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.298 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23623 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15097856~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:05.298 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.793887+0000", 2024-04-24T15:33:05.298 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 30.494602493999999, 2024-04-24T15:33:05.298 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040432784999999999, 2024-04-24T15:33:05.298 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:05.298 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:05.298 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:05.298 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:05.298 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:05.298 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23623 2024-04-24T15:33:05.298 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.299 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:05.299 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.299 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:05.299 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793887+0000", 2024-04-24T15:33:05.299 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.299 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.299 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.299 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:05.299 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793887+0000", 2024-04-24T15:33:05.299 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.299 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.299 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.299 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:05.299 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793887+0000", 2024-04-24T15:33:05.299 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:05.300 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.300 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.300 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:05.300 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793888+0000", 2024-04-24T15:33:05.300 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3659999999999999e-06 2024-04-24T15:33:05.300 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.300 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.300 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:05.300 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793891+0000", 2024-04-24T15:33:05.300 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8389999999999998e-06 2024-04-24T15:33:05.300 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.300 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.300 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:05.300 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793893+0000", 2024-04-24T15:33:05.301 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.9549999999999999e-06 2024-04-24T15:33:05.301 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.301 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.301 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:05.301 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834239+0000", 2024-04-24T15:33:05.301 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040345487999999999 2024-04-24T15:33:05.301 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.301 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.301 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:05.301 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834280+0000", 2024-04-24T15:33:05.301 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.1878000000000001e-05 2024-04-24T15:33:05.301 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.301 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.301 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:05.301 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834320+0000", 2024-04-24T15:33:05.302 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9480000000000001e-05 2024-04-24T15:33:05.302 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.302 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:05.302 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.302 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.302 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.302 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23624 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15138816~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:05.302 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.794053+0000", 2024-04-24T15:33:05.302 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 30.494436545999999, 2024-04-24T15:33:05.302 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040370801999999997, 2024-04-24T15:33:05.302 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:05.302 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:05.303 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:05.303 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:05.303 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:05.303 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23624 2024-04-24T15:33:05.303 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.303 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:05.303 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.303 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:05.303 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794053+0000", 2024-04-24T15:33:05.303 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.303 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.303 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.303 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:05.303 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794053+0000", 2024-04-24T15:33:05.304 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.304 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.304 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.304 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:05.304 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793895+0000", 2024-04-24T15:33:05.304 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998422 2024-04-24T15:33:05.304 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.304 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.304 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:05.304 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794057+0000", 2024-04-24T15:33:05.304 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00016149400000000001 2024-04-24T15:33:05.304 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.304 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.304 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:05.305 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794065+0000", 2024-04-24T15:33:05.305 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.5259999999999993e-06 2024-04-24T15:33:05.305 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.305 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.305 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:05.305 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794071+0000", 2024-04-24T15:33:05.305 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6149999999999996e-06 2024-04-24T15:33:05.305 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.305 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.305 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:05.305 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834341+0000", 2024-04-24T15:33:05.305 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040270366000000002 2024-04-24T15:33:05.305 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.305 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.305 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:05.306 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834381+0000", 2024-04-24T15:33:05.306 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0102999999999999e-05 2024-04-24T15:33:05.306 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.306 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.306 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:05.306 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834424+0000", 2024-04-24T15:33:05.306 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2713000000000003e-05 2024-04-24T15:33:05.306 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.306 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:05.306 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.306 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.306 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.306 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23625 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15179776~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:05.307 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.794160+0000", 2024-04-24T15:33:05.307 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 30.494329663999999, 2024-04-24T15:33:05.307 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040413004000000002, 2024-04-24T15:33:05.307 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:05.307 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:05.307 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:05.307 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:05.307 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:05.307 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23625 2024-04-24T15:33:05.307 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.307 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:05.307 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.307 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:05.307 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794160+0000", 2024-04-24T15:33:05.308 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.308 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.308 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.308 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:05.308 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794160+0000", 2024-04-24T15:33:05.308 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.308 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.308 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.308 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:05.308 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794086+0000", 2024-04-24T15:33:05.308 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999256 2024-04-24T15:33:05.308 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.308 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.308 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:05.309 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794163+0000", 2024-04-24T15:33:05.309 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7896000000000001e-05 2024-04-24T15:33:05.309 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.309 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.309 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:05.309 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794171+0000", 2024-04-24T15:33:05.309 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.9500000000000001e-06 2024-04-24T15:33:05.309 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.309 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.309 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:05.309 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794179+0000", 2024-04-24T15:33:05.309 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7970000000000001e-06 2024-04-24T15:33:05.309 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.309 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.310 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:05.310 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834451+0000", 2024-04-24T15:33:05.310 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040271492999999998 2024-04-24T15:33:05.310 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.310 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.310 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:05.310 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834519+0000", 2024-04-24T15:33:05.310 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.7984000000000005e-05 2024-04-24T15:33:05.310 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.310 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.310 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:05.310 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834573+0000", 2024-04-24T15:33:05.310 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.4339999999999998e-05 2024-04-24T15:33:05.311 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.311 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:05.311 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.311 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.311 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.311 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23629 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15343616~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:05.311 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.794748+0000", 2024-04-24T15:33:05.311 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 30.493741862, 2024-04-24T15:33:05.311 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040419718, 2024-04-24T15:33:05.311 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:05.311 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:05.311 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:05.311 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:05.311 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:05.312 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23629 2024-04-24T15:33:05.312 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.312 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:05.312 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.312 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:05.312 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794748+0000", 2024-04-24T15:33:05.312 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.312 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.312 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.312 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:05.312 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794748+0000", 2024-04-24T15:33:05.312 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.312 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.312 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.313 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:05.313 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794673+0000", 2024-04-24T15:33:05.313 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999251 2024-04-24T15:33:05.313 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.313 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.313 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:05.313 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794751+0000", 2024-04-24T15:33:05.313 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.8480999999999996e-05 2024-04-24T15:33:05.313 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.313 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.313 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:05.313 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794760+0000", 2024-04-24T15:33:05.313 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3429999999999996e-06 2024-04-24T15:33:05.314 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.314 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.314 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:05.314 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794764+0000", 2024-04-24T15:33:05.314 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5600000000000004e-06 2024-04-24T15:33:05.314 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.314 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.314 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:05.314 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835040+0000", 2024-04-24T15:33:05.314 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040275188000000003 2024-04-24T15:33:05.314 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.314 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.314 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:05.314 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835105+0000", 2024-04-24T15:33:05.315 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.5992000000000001e-05 2024-04-24T15:33:05.315 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.315 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.315 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:05.315 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835167+0000", 2024-04-24T15:33:05.315 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.1981000000000002e-05 2024-04-24T15:33:05.315 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.315 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:05.315 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.315 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.315 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.315 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23630 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15384576~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:05.315 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.794839+0000", 2024-04-24T15:33:05.316 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 30.493650793, 2024-04-24T15:33:05.316 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040523894999999997, 2024-04-24T15:33:05.316 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:05.316 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:05.316 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:05.316 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:05.316 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:05.316 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23630 2024-04-24T15:33:05.316 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.316 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:05.316 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.316 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:05.316 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794839+0000", 2024-04-24T15:33:05.316 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.317 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.317 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.317 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:05.317 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794839+0000", 2024-04-24T15:33:05.317 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.317 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.317 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.317 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:05.317 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794768+0000", 2024-04-24T15:33:05.317 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999294 2024-04-24T15:33:05.317 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.317 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.317 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:05.317 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794841+0000", 2024-04-24T15:33:05.318 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2793000000000006e-05 2024-04-24T15:33:05.318 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.318 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.318 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:05.318 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794846+0000", 2024-04-24T15:33:05.318 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.7230000000000004e-06 2024-04-24T15:33:05.318 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.318 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.318 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:05.318 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794848+0000", 2024-04-24T15:33:05.318 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7659999999999999e-06 2024-04-24T15:33:05.318 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.318 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.319 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:05.319 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835223+0000", 2024-04-24T15:33:05.319 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040374275000000001 2024-04-24T15:33:05.319 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.319 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.319 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:05.319 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835295+0000", 2024-04-24T15:33:05.319 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2633999999999996e-05 2024-04-24T15:33:05.319 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.319 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.319 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:05.319 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835363+0000", 2024-04-24T15:33:05.319 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.7355999999999997e-05 2024-04-24T15:33:05.320 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.320 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:05.320 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.320 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.320 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.320 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23631 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15425536~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:05.320 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.794956+0000", 2024-04-24T15:33:05.320 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 30.493533952, 2024-04-24T15:33:05.320 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040562383, 2024-04-24T15:33:05.320 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:05.320 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:05.320 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:05.320 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:05.320 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:05.321 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23631 2024-04-24T15:33:05.321 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.321 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:05.321 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.321 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:05.321 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794956+0000", 2024-04-24T15:33:05.321 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.321 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.321 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.321 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:05.321 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794956+0000", 2024-04-24T15:33:05.321 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.321 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.321 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.321 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:05.322 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794859+0000", 2024-04-24T15:33:05.322 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999032 2024-04-24T15:33:05.322 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.322 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.322 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:05.322 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794959+0000", 2024-04-24T15:33:05.322 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000100207 2024-04-24T15:33:05.322 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.322 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.322 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:05.322 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794967+0000", 2024-04-24T15:33:05.322 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7640000000000005e-06 2024-04-24T15:33:05.323 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.323 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.323 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:05.323 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794971+0000", 2024-04-24T15:33:05.323 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.335e-06 2024-04-24T15:33:05.323 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.323 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.323 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:05.323 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835388+0000", 2024-04-24T15:33:05.323 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040417340000000003 2024-04-24T15:33:05.323 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.323 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.323 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:05.323 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835459+0000", 2024-04-24T15:33:05.324 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.0118999999999997e-05 2024-04-24T15:33:05.324 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.324 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.324 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:05.324 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835518+0000", 2024-04-24T15:33:05.324 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.9438999999999998e-05 2024-04-24T15:33:05.324 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.324 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:05.324 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.324 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.324 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.324 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23632 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15466496~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:05.324 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795060+0000", 2024-04-24T15:33:05.324 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 30.493429242000001, 2024-04-24T15:33:05.324 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040667993, 2024-04-24T15:33:05.325 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:05.325 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:05.325 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:05.325 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:05.325 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:05.325 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23632 2024-04-24T15:33:05.325 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.325 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:05.325 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.325 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:05.325 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795060+0000", 2024-04-24T15:33:05.325 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.325 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.326 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.326 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:05.326 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795060+0000", 2024-04-24T15:33:05.326 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.326 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.326 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.326 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:05.326 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794976+0000", 2024-04-24T15:33:05.326 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999156 2024-04-24T15:33:05.326 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.326 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.326 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:05.326 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795064+0000", 2024-04-24T15:33:05.326 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.7709999999999999e-05 2024-04-24T15:33:05.326 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.327 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.327 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:05.327 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795072+0000", 2024-04-24T15:33:05.327 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3769999999999996e-06 2024-04-24T15:33:05.327 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.327 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.327 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:05.327 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795078+0000", 2024-04-24T15:33:05.327 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.5079999999999996e-06 2024-04-24T15:33:05.327 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.327 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.327 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:05.327 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835541+0000", 2024-04-24T15:33:05.328 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040463440000000003 2024-04-24T15:33:05.328 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.328 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.328 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:05.328 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835657+0000", 2024-04-24T15:33:05.328 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00011572499999999999 2024-04-24T15:33:05.328 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.328 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.328 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:05.328 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835728+0000", 2024-04-24T15:33:05.328 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1402000000000005e-05 2024-04-24T15:33:05.328 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.328 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:05.328 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.329 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.329 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.329 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23633 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15507456~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:05.329 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795134+0000", 2024-04-24T15:33:05.329 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 30.493355587, 2024-04-24T15:33:05.329 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040748591000000001, 2024-04-24T15:33:05.329 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:05.329 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:05.329 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:05.329 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:05.329 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:05.329 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23633 2024-04-24T15:33:05.329 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.329 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:05.330 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.330 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:05.330 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795134+0000", 2024-04-24T15:33:05.330 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.330 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.330 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.330 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:05.330 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795134+0000", 2024-04-24T15:33:05.330 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.330 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.330 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.330 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:05.331 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795083+0000", 2024-04-24T15:33:05.331 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999949 2024-04-24T15:33:05.331 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.331 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.332 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:05.332 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795138+0000", 2024-04-24T15:33:05.332 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.4472000000000003e-05 2024-04-24T15:33:05.332 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.332 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.332 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:05.332 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795145+0000", 2024-04-24T15:33:05.332 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2779999999999996e-06 2024-04-24T15:33:05.332 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.332 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.332 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:05.332 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795149+0000", 2024-04-24T15:33:05.332 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.568e-06 2024-04-24T15:33:05.332 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.333 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.333 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:05.333 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835757+0000", 2024-04-24T15:33:05.333 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040608894999999999 2024-04-24T15:33:05.333 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.333 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.333 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:05.333 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835829+0000", 2024-04-24T15:33:05.333 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1879999999999996e-05 2024-04-24T15:33:05.333 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.333 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.333 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:05.333 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835883+0000", 2024-04-24T15:33:05.333 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.3346000000000001e-05 2024-04-24T15:33:05.333 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.334 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:05.334 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.334 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.334 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.334 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23634 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15548416~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:05.334 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795271+0000", 2024-04-24T15:33:05.334 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 30.493218552999998, 2024-04-24T15:33:05.334 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040755845999999998, 2024-04-24T15:33:05.334 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:05.334 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:05.334 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:05.334 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:05.335 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:05.335 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23634 2024-04-24T15:33:05.335 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.335 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:05.335 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.335 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:05.335 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795271+0000", 2024-04-24T15:33:05.335 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.335 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.335 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.335 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:05.335 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795271+0000", 2024-04-24T15:33:05.335 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.335 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.336 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.336 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:05.336 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795152+0000", 2024-04-24T15:33:05.336 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998813 2024-04-24T15:33:05.336 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.336 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.336 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:05.336 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795276+0000", 2024-04-24T15:33:05.336 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00012401799999999999 2024-04-24T15:33:05.336 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.336 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.336 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:05.336 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795281+0000", 2024-04-24T15:33:05.336 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8860000000000003e-06 2024-04-24T15:33:05.336 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.337 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.337 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:05.337 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795284+0000", 2024-04-24T15:33:05.337 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.0450000000000001e-06 2024-04-24T15:33:05.337 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.337 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.337 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:05.337 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835905+0000", 2024-04-24T15:33:05.337 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040621035999999999 2024-04-24T15:33:05.337 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.337 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.337 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:05.337 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835971+0000", 2024-04-24T15:33:05.338 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.5314000000000005e-05 2024-04-24T15:33:05.338 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.338 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.338 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:05.338 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836027+0000", 2024-04-24T15:33:05.338 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6305999999999999e-05 2024-04-24T15:33:05.338 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.338 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:05.338 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.338 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.338 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.338 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23635 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15589376~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:05.338 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795382+0000", 2024-04-24T15:33:05.339 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 30.493107779999999, 2024-04-24T15:33:05.339 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040805363999999997, 2024-04-24T15:33:05.339 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:05.339 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:05.339 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:05.339 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:05.339 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:05.339 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23635 2024-04-24T15:33:05.339 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.339 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:05.341 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.341 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:05.341 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795382+0000", 2024-04-24T15:33:05.341 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.341 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.341 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.341 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:05.342 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795382+0000", 2024-04-24T15:33:05.342 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.342 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.342 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.342 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:05.342 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795288+0000", 2024-04-24T15:33:05.342 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999056 2024-04-24T15:33:05.342 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.342 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.342 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:05.342 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795385+0000", 2024-04-24T15:33:05.342 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.7288999999999996e-05 2024-04-24T15:33:05.343 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.343 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.343 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:05.343 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795392+0000", 2024-04-24T15:33:05.343 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.452e-06 2024-04-24T15:33:05.343 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.343 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.343 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:05.343 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795397+0000", 2024-04-24T15:33:05.343 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2860000000000001e-06 2024-04-24T15:33:05.343 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.343 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.343 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:05.343 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836054+0000", 2024-04-24T15:33:05.343 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040657581999999998 2024-04-24T15:33:05.344 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.344 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.344 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:05.344 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836124+0000", 2024-04-24T15:33:05.344 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.0020999999999999e-05 2024-04-24T15:33:05.344 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.344 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.344 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:05.344 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836187+0000", 2024-04-24T15:33:05.344 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.3009999999999995e-05 2024-04-24T15:33:05.344 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.344 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:05.344 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.344 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.345 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.345 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23636 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15630336~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:05.345 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795610+0000", 2024-04-24T15:33:05.345 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 30.492879422000001, 2024-04-24T15:33:05.345 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040730942999999999, 2024-04-24T15:33:05.345 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:05.345 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:05.345 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:05.345 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:05.345 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:05.345 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23636 2024-04-24T15:33:05.345 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.345 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:05.346 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.346 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:05.346 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795610+0000", 2024-04-24T15:33:05.346 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.346 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.346 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.346 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:05.346 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795610+0000", 2024-04-24T15:33:05.346 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.346 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.346 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.346 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:05.346 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795401+0000", 2024-04-24T15:33:05.346 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9997911 2024-04-24T15:33:05.347 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.347 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.347 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:05.347 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795614+0000", 2024-04-24T15:33:05.347 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00021228600000000001 2024-04-24T15:33:05.347 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.347 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.347 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:05.347 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795622+0000", 2024-04-24T15:33:05.347 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3920000000000003e-06 2024-04-24T15:33:05.347 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.347 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.347 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:05.348 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795627+0000", 2024-04-24T15:33:05.348 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8199999999999996e-06 2024-04-24T15:33:05.348 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.348 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.348 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:05.348 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836211+0000", 2024-04-24T15:33:05.348 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040583803000000002 2024-04-24T15:33:05.348 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.348 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.348 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:05.348 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836284+0000", 2024-04-24T15:33:05.348 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.3617999999999999e-05 2024-04-24T15:33:05.348 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.348 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.349 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:05.349 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836341+0000", 2024-04-24T15:33:05.349 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6959999999999997e-05 2024-04-24T15:33:05.349 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.349 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:05.349 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.349 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.349 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.349 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23637 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15671296~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:05.349 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795699+0000", 2024-04-24T15:33:05.349 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 30.492790254999999, 2024-04-24T15:33:05.349 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040775556999999997, 2024-04-24T15:33:05.349 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:05.349 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:05.350 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:05.350 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:05.350 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:05.350 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23637 2024-04-24T15:33:05.350 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.350 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:05.350 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.350 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:05.350 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795699+0000", 2024-04-24T15:33:05.350 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.350 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.350 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.350 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:05.351 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795699+0000", 2024-04-24T15:33:05.351 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.351 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.351 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.351 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:05.351 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795632+0000", 2024-04-24T15:33:05.351 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999323 2024-04-24T15:33:05.351 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.351 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.351 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:05.351 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795704+0000", 2024-04-24T15:33:05.351 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2428000000000005e-05 2024-04-24T15:33:05.351 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.351 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.352 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:05.352 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795715+0000", 2024-04-24T15:33:05.352 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.1532000000000001e-05 2024-04-24T15:33:05.352 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.352 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.352 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:05.352 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795721+0000", 2024-04-24T15:33:05.352 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.8200000000000002e-06 2024-04-24T15:33:05.352 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.352 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.352 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:05.352 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836364+0000", 2024-04-24T15:33:05.352 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040642997 2024-04-24T15:33:05.352 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.353 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.353 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:05.353 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836435+0000", 2024-04-24T15:33:05.353 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.0821000000000005e-05 2024-04-24T15:33:05.353 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.353 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.353 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:05.353 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836475+0000", 2024-04-24T15:33:05.353 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9796999999999999e-05 2024-04-24T15:33:05.353 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.353 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:05.353 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.353 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.353 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.354 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23638 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15712256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:05.354 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795932+0000", 2024-04-24T15:33:05.354 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 30.492557224999999, 2024-04-24T15:33:05.354 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040668682999999997, 2024-04-24T15:33:05.354 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:05.354 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:05.354 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:05.354 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:05.354 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:05.354 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23638 2024-04-24T15:33:05.354 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.354 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:05.354 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.354 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:05.355 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795932+0000", 2024-04-24T15:33:05.355 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.355 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.355 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.355 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:05.355 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795932+0000", 2024-04-24T15:33:05.355 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.355 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.355 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.355 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:05.355 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795728+0000", 2024-04-24T15:33:05.355 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9997959 2024-04-24T15:33:05.355 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.356 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.356 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:05.356 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795936+0000", 2024-04-24T15:33:05.356 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000208128 2024-04-24T15:33:05.356 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.356 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.356 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:05.356 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795945+0000", 2024-04-24T15:33:05.356 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.7110000000000001e-06 2024-04-24T15:33:05.356 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.356 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.356 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:05.356 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795957+0000", 2024-04-24T15:33:05.356 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.1800999999999999e-05 2024-04-24T15:33:05.357 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.357 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.357 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:05.357 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836505+0000", 2024-04-24T15:33:05.357 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040547622999999998 2024-04-24T15:33:05.357 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.357 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.357 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:05.357 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836561+0000", 2024-04-24T15:33:05.357 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6598999999999998e-05 2024-04-24T15:33:05.357 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.357 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.357 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:05.357 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836601+0000", 2024-04-24T15:33:05.357 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9907000000000003e-05 2024-04-24T15:33:05.358 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.358 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:05.358 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.358 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.358 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.358 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23639 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15753216~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:05.358 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.796071+0000", 2024-04-24T15:33:05.358 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 30.492418963999999, 2024-04-24T15:33:05.358 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040648199000000003, 2024-04-24T15:33:05.358 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:05.358 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:05.358 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:05.358 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:05.359 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:05.359 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23639 2024-04-24T15:33:05.359 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.359 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:05.359 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.359 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:05.359 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796071+0000", 2024-04-24T15:33:05.359 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.359 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.359 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.359 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:05.359 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796071+0000", 2024-04-24T15:33:05.359 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.359 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.360 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.360 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:05.360 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795967+0000", 2024-04-24T15:33:05.360 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998965 2024-04-24T15:33:05.360 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.360 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.360 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:05.360 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796074+0000", 2024-04-24T15:33:05.360 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000107013 2024-04-24T15:33:05.360 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.360 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.360 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:05.360 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796081+0000", 2024-04-24T15:33:05.360 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2590000000000003e-06 2024-04-24T15:33:05.361 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.361 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.361 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:05.361 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796085+0000", 2024-04-24T15:33:05.361 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5269999999999999e-06 2024-04-24T15:33:05.361 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.361 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.361 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:05.361 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836624+0000", 2024-04-24T15:33:05.361 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040539324000000002 2024-04-24T15:33:05.361 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.361 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.361 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:05.361 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836667+0000", 2024-04-24T15:33:05.362 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2562999999999999e-05 2024-04-24T15:33:05.362 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.362 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.362 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:05.362 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836719+0000", 2024-04-24T15:33:05.362 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.2015999999999998e-05 2024-04-24T15:33:05.362 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.362 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:05.362 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.362 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.362 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.362 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23640 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15794176~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:05.362 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.796492+0000", 2024-04-24T15:33:05.363 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 30.491997206000001, 2024-04-24T15:33:05.363 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040420000999999997, 2024-04-24T15:33:05.363 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:05.363 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:05.363 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:05.363 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:05.363 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:05.363 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23640 2024-04-24T15:33:05.363 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.363 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:05.363 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.363 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:05.363 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796492+0000", 2024-04-24T15:33:05.363 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.364 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.364 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.364 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:05.364 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796492+0000", 2024-04-24T15:33:05.364 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.364 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.364 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.364 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:05.364 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796088+0000", 2024-04-24T15:33:05.364 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9995961 2024-04-24T15:33:05.364 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.364 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.364 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:05.364 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796496+0000", 2024-04-24T15:33:05.364 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00040785700000000002 2024-04-24T15:33:05.365 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.365 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.365 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:05.365 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796503+0000", 2024-04-24T15:33:05.365 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1049999999999997e-06 2024-04-24T15:33:05.365 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.365 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.365 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:05.365 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796508+0000", 2024-04-24T15:33:05.365 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.155e-06 2024-04-24T15:33:05.365 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.365 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.365 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:05.365 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836756+0000", 2024-04-24T15:33:05.365 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040248850000000003 2024-04-24T15:33:05.366 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.366 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.366 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:05.366 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836841+0000", 2024-04-24T15:33:05.366 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.4735999999999996e-05 2024-04-24T15:33:05.366 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.366 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.366 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:05.366 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836912+0000", 2024-04-24T15:33:05.366 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1229999999999994e-05 2024-04-24T15:33:05.366 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.366 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:05.366 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.367 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.367 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.367 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23641 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15835136~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:05.367 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.796658+0000", 2024-04-24T15:33:05.367 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 30.491831130000001, 2024-04-24T15:33:05.367 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040420706000000001, 2024-04-24T15:33:05.367 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:05.367 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:05.367 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:05.367 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:05.367 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:05.367 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23641 2024-04-24T15:33:05.367 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.367 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:05.368 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.368 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:05.368 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796658+0000", 2024-04-24T15:33:05.368 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.368 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.368 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.368 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:05.368 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796658+0000", 2024-04-24T15:33:05.368 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.368 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.368 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.368 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:05.368 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796512+0000", 2024-04-24T15:33:05.368 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998531 2024-04-24T15:33:05.369 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.369 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.369 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:05.369 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796669+0000", 2024-04-24T15:33:05.369 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000157577 2024-04-24T15:33:05.369 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.369 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.369 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:05.369 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796677+0000", 2024-04-24T15:33:05.369 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7800000000000001e-06 2024-04-24T15:33:05.369 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.369 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.369 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:05.369 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796681+0000", 2024-04-24T15:33:05.369 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.721e-06 2024-04-24T15:33:05.370 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.370 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.370 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:05.370 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836941+0000", 2024-04-24T15:33:05.370 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040259907999999997 2024-04-24T15:33:05.370 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.370 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.370 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:05.370 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837017+0000", 2024-04-24T15:33:05.370 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.6071999999999999e-05 2024-04-24T15:33:05.370 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.370 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.370 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:05.370 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837079+0000", 2024-04-24T15:33:05.371 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.2358000000000002e-05 2024-04-24T15:33:05.371 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.371 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:05.371 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.371 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.371 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.371 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23645 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15998976~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:05.371 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.797360+0000", 2024-04-24T15:33:05.371 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 30.491129632, 2024-04-24T15:33:05.371 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040387645999999999, 2024-04-24T15:33:05.371 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:05.371 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:05.371 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:05.371 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:05.372 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:05.372 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23645 2024-04-24T15:33:05.372 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.372 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:05.372 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.372 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:05.372 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797360+0000", 2024-04-24T15:33:05.372 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.372 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.372 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.372 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:05.372 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797360+0000", 2024-04-24T15:33:05.372 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.372 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.372 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.373 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:05.373 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797288+0000", 2024-04-24T15:33:05.373 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999928 2024-04-24T15:33:05.373 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.373 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.373 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:05.373 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797364+0000", 2024-04-24T15:33:05.373 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.5347999999999998e-05 2024-04-24T15:33:05.373 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.373 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.373 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:05.373 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797372+0000", 2024-04-24T15:33:05.373 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.2409999999999995e-06 2024-04-24T15:33:05.373 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.374 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.374 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:05.374 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797376+0000", 2024-04-24T15:33:05.374 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.515e-06 2024-04-24T15:33:05.374 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.374 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.374 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:05.374 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837586+0000", 2024-04-24T15:33:05.374 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040210047999999998 2024-04-24T15:33:05.374 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.374 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.374 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:05.374 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837672+0000", 2024-04-24T15:33:05.375 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.5997999999999999e-05 2024-04-24T15:33:05.375 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.375 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.375 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:05.375 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837748+0000", 2024-04-24T15:33:05.375 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.5328999999999997e-05 2024-04-24T15:33:05.375 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.375 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:05.375 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.375 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.375 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.375 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23646 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 16039936~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:05.375 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.797498+0000", 2024-04-24T15:33:05.376 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 30.490991530999999, 2024-04-24T15:33:05.376 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040433139, 2024-04-24T15:33:05.376 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:05.376 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:05.376 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:05.376 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:05.376 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:05.376 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23646 2024-04-24T15:33:05.376 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.376 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:05.376 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.376 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:05.376 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797498+0000", 2024-04-24T15:33:05.377 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.377 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.377 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.377 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:05.377 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797498+0000", 2024-04-24T15:33:05.377 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.377 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.377 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.377 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:05.377 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797381+0000", 2024-04-24T15:33:05.377 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998832 2024-04-24T15:33:05.377 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.377 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.377 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:05.377 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797501+0000", 2024-04-24T15:33:05.378 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000119759 2024-04-24T15:33:05.378 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.378 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.378 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:05.378 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797510+0000", 2024-04-24T15:33:05.378 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.4770000000000003e-06 2024-04-24T15:33:05.378 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.378 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.378 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:05.378 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797515+0000", 2024-04-24T15:33:05.378 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8330000000000001e-06 2024-04-24T15:33:05.378 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.378 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.379 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:05.379 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837777+0000", 2024-04-24T15:33:05.379 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040262502999999998 2024-04-24T15:33:05.379 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.379 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.379 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:05.379 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837860+0000", 2024-04-24T15:33:05.379 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3002000000000003e-05 2024-04-24T15:33:05.379 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.379 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.379 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:05.379 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837931+0000", 2024-04-24T15:33:05.379 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1222000000000004e-05 2024-04-24T15:33:05.380 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.380 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:05.380 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.380 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.380 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:05.380 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:05.381 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-24T15:33:05.382 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:33:05.382+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:33:05.482 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:33:05.482+0000 7f23abcf6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 44021) UID: 0 2024-04-24T15:33:05.546 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:05.546 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:33:05.546 INFO:teuthology.orchestra.run.smithi151.stdout: "num_ops": 0 2024-04-24T15:33:05.546 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:05.555 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-24T15:33:05.582 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:05.584+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:05.683 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:05.684+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:05.719 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:05.719 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:33:05.719 INFO:teuthology.orchestra.run.smithi151.stdout: "complaint_time": 30, 2024-04-24T15:33:05.720 INFO:teuthology.orchestra.run.smithi151.stdout: "num_blocked_ops": 0 2024-04-24T15:33:05.720 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:05.729 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-24T15:33:05.785 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:33:05.785+0000 7f23abcf6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 44021) UID: 0 2024-04-24T15:33:05.794 INFO:teuthology.orchestra.run.smithi096.stderr:reweighted osd.2 to 1 (10000) 2024-04-24T15:33:05.794 INFO:teuthology.orchestra.run.smithi096.stderr:noscrub is set 2024-04-24T15:33:05.809 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-04-24T15:33:05.858 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:05.858 INFO:teuthology.orchestra.run.smithi151.stdout: "size": 20, 2024-04-24T15:33:05.858 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 600, 2024-04-24T15:33:05.858 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [ 2024-04-24T15:33:05.858 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.858 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17317 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 11509760~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:05.859 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.224408+0000", 2024-04-24T15:33:05.859 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 31.633911771000001, 2024-04-24T15:33:05.859 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042604418999999998, 2024-04-24T15:33:05.859 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:05.859 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:05.859 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:05.859 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:05.859 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:05.859 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17317 2024-04-24T15:33:05.859 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.859 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:05.859 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.859 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:05.859 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224408+0000", 2024-04-24T15:33:05.860 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.860 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.860 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.860 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:05.860 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224408+0000", 2024-04-24T15:33:05.860 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.860 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.860 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.860 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:05.860 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224407+0000", 2024-04-24T15:33:05.860 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:05.860 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.860 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.860 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:05.861 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224409+0000", 2024-04-24T15:33:05.861 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.1059999999999998e-06 2024-04-24T15:33:05.861 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.861 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.861 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:05.861 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224415+0000", 2024-04-24T15:33:05.861 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6219999999999997e-06 2024-04-24T15:33:05.861 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.861 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.861 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:05.861 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224419+0000", 2024-04-24T15:33:05.861 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8340000000000001e-06 2024-04-24T15:33:05.861 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.861 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.861 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:05.862 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.266851+0000", 2024-04-24T15:33:05.862 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042431910000000003 2024-04-24T15:33:05.862 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.862 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.862 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:05.862 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.266931+0000", 2024-04-24T15:33:05.862 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0580000000000004e-05 2024-04-24T15:33:05.862 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.862 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.862 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:05.862 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267012+0000", 2024-04-24T15:33:05.862 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0698999999999994e-05 2024-04-24T15:33:05.862 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.862 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:05.863 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.863 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.863 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.863 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17318 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 11550720~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:05.863 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.224424+0000", 2024-04-24T15:33:05.863 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 31.633895403, 2024-04-24T15:33:05.863 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042771921999999997, 2024-04-24T15:33:05.863 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:05.863 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:05.863 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:05.863 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:05.863 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:05.863 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17318 2024-04-24T15:33:05.864 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.864 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:05.864 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.864 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:05.864 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224424+0000", 2024-04-24T15:33:05.864 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.864 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.864 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.864 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:05.864 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224424+0000", 2024-04-24T15:33:05.864 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.864 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.864 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.864 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:05.864 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224424+0000", 2024-04-24T15:33:05.864 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:05.864 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.865 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.865 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:05.865 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224434+0000", 2024-04-24T15:33:05.865 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.9699999999999994e-06 2024-04-24T15:33:05.865 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.865 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.865 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:05.865 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224443+0000", 2024-04-24T15:33:05.865 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.3689999999999996e-06 2024-04-24T15:33:05.865 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.865 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.865 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:05.866 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224447+0000", 2024-04-24T15:33:05.866 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.783e-06 2024-04-24T15:33:05.866 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.866 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.866 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:05.866 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267027+0000", 2024-04-24T15:33:05.866 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042579847999999997 2024-04-24T15:33:05.866 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.866 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.866 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:05.866 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267108+0000", 2024-04-24T15:33:05.866 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.1420000000000003e-05 2024-04-24T15:33:05.866 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.866 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.867 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:05.867 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267196+0000", 2024-04-24T15:33:05.867 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.7851999999999998e-05 2024-04-24T15:33:05.867 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.867 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:05.867 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.867 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.867 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.867 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17319 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 11591680~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:05.867 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.224453+0000", 2024-04-24T15:33:05.867 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 31.633866477000002, 2024-04-24T15:33:05.867 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042921783999999998, 2024-04-24T15:33:05.867 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:05.867 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:05.867 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:05.868 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:05.868 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:05.868 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17319 2024-04-24T15:33:05.868 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.868 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:05.868 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.868 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:05.868 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224453+0000", 2024-04-24T15:33:05.868 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.868 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.868 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.868 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:05.868 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224453+0000", 2024-04-24T15:33:05.868 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.868 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.868 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.869 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:05.869 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224452+0000", 2024-04-24T15:33:05.869 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:05.869 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.869 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.869 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:05.869 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224455+0000", 2024-04-24T15:33:05.869 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.289e-06 2024-04-24T15:33:05.869 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.869 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.869 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:05.869 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224461+0000", 2024-04-24T15:33:05.869 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.7300000000000002e-06 2024-04-24T15:33:05.869 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.869 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.870 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:05.870 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224464+0000", 2024-04-24T15:33:05.870 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.2890000000000002e-06 2024-04-24T15:33:05.870 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.870 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.870 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:05.870 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267218+0000", 2024-04-24T15:33:05.870 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042753696000000001 2024-04-24T15:33:05.870 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.870 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.870 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:05.870 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267295+0000", 2024-04-24T15:33:05.870 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7762999999999994e-05 2024-04-24T15:33:05.870 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.870 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.871 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:05.871 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267375+0000", 2024-04-24T15:33:05.871 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.9444999999999996e-05 2024-04-24T15:33:05.871 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.871 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:05.871 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.871 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.871 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.871 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17320 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 11632640~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:05.871 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.224468+0000", 2024-04-24T15:33:05.871 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 31.633851443000001, 2024-04-24T15:33:05.871 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043063615, 2024-04-24T15:33:05.871 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:05.871 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:05.871 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:05.871 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:05.871 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:05.872 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17320 2024-04-24T15:33:05.872 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.872 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:05.872 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.872 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:05.872 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224468+0000", 2024-04-24T15:33:05.872 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.872 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.872 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.872 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:05.872 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224468+0000", 2024-04-24T15:33:05.872 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.872 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.872 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.872 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:05.872 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224468+0000", 2024-04-24T15:33:05.873 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:05.873 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.873 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.873 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:05.873 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224470+0000", 2024-04-24T15:33:05.873 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.0600000000000002e-06 2024-04-24T15:33:05.873 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.873 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.873 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:05.873 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224475+0000", 2024-04-24T15:33:05.873 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.1059999999999999e-06 2024-04-24T15:33:05.873 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.873 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.873 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:05.873 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224478+0000", 2024-04-24T15:33:05.874 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.3100000000000001e-06 2024-04-24T15:33:05.874 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.874 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.874 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:05.874 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267386+0000", 2024-04-24T15:33:05.874 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042907529 2024-04-24T15:33:05.874 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.874 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.874 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:05.874 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267447+0000", 2024-04-24T15:33:05.874 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.1173000000000005e-05 2024-04-24T15:33:05.874 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.874 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.874 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:05.875 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267532+0000", 2024-04-24T15:33:05.875 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.4771999999999994e-05 2024-04-24T15:33:05.875 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.875 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:05.875 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.875 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.875 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.875 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17321 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 11673600~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:05.875 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.224499+0000", 2024-04-24T15:33:05.875 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 31.633820554, 2024-04-24T15:33:05.875 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043214447000000003, 2024-04-24T15:33:05.875 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:05.875 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:05.875 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:05.875 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:05.876 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:05.876 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17321 2024-04-24T15:33:05.876 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.876 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:05.876 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.876 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:05.876 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224499+0000", 2024-04-24T15:33:05.876 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.876 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.876 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.876 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:05.876 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224499+0000", 2024-04-24T15:33:05.876 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.876 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.876 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.876 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:05.877 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224499+0000", 2024-04-24T15:33:05.877 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:05.877 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.877 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.877 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:05.877 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224504+0000", 2024-04-24T15:33:05.877 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6869999999999999e-06 2024-04-24T15:33:05.877 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.877 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.877 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:05.877 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224509+0000", 2024-04-24T15:33:05.877 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2259999999999999e-06 2024-04-24T15:33:05.877 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.877 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.877 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:05.877 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224511+0000", 2024-04-24T15:33:05.877 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.418e-06 2024-04-24T15:33:05.878 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.878 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.878 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:05.878 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267552+0000", 2024-04-24T15:33:05.878 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043041455999999999 2024-04-24T15:33:05.878 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.878 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.878 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:05.878 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267629+0000", 2024-04-24T15:33:05.878 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7069000000000004e-05 2024-04-24T15:33:05.878 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.878 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.878 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:05.878 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267713+0000", 2024-04-24T15:33:05.879 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3843000000000003e-05 2024-04-24T15:33:05.879 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.879 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:05.879 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.879 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.879 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.879 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17322 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 11714560~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:05.879 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.224514+0000", 2024-04-24T15:33:05.879 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 31.633805316, 2024-04-24T15:33:05.879 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043339196000000003, 2024-04-24T15:33:05.879 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:05.879 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:05.879 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:05.879 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:05.879 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:05.880 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17322 2024-04-24T15:33:05.880 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.880 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:05.880 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.880 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:05.880 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224514+0000", 2024-04-24T15:33:05.880 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.880 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.880 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.880 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:05.880 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224514+0000", 2024-04-24T15:33:05.880 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.880 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.880 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.880 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:05.880 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224514+0000", 2024-04-24T15:33:05.880 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:05.881 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.881 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.881 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:05.881 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224515+0000", 2024-04-24T15:33:05.881 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.5570000000000001e-06 2024-04-24T15:33:05.881 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.881 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.881 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:05.881 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224518+0000", 2024-04-24T15:33:05.881 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.943e-06 2024-04-24T15:33:05.881 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.881 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.881 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:05.881 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224520+0000", 2024-04-24T15:33:05.881 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.9999999999999999e-06 2024-04-24T15:33:05.881 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.881 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.882 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:05.882 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267730+0000", 2024-04-24T15:33:05.882 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043209466000000002 2024-04-24T15:33:05.882 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.882 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.882 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:05.882 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267772+0000", 2024-04-24T15:33:05.882 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2453000000000002e-05 2024-04-24T15:33:05.882 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.882 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.882 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:05.882 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267853+0000", 2024-04-24T15:33:05.882 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.1094000000000007e-05 2024-04-24T15:33:05.882 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.883 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:05.883 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.883 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.883 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.883 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17323 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 11755520~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:05.883 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.224522+0000", 2024-04-24T15:33:05.883 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 31.633796975999999, 2024-04-24T15:33:05.883 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043506792000000002, 2024-04-24T15:33:05.883 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:05.883 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:05.883 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:05.883 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:05.883 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:05.883 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17323 2024-04-24T15:33:05.883 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.883 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:05.883 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.884 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:05.884 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224522+0000", 2024-04-24T15:33:05.884 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.884 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.884 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.884 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:05.884 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224522+0000", 2024-04-24T15:33:05.884 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.884 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.884 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.884 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:05.884 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224522+0000", 2024-04-24T15:33:05.884 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:05.884 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.884 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.884 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:05.885 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224524+0000", 2024-04-24T15:33:05.885 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3069999999999999e-06 2024-04-24T15:33:05.885 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.885 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.885 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:05.885 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224526+0000", 2024-04-24T15:33:05.885 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.773e-06 2024-04-24T15:33:05.885 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.885 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.885 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:05.885 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.224529+0000", 2024-04-24T15:33:05.885 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.3e-06 2024-04-24T15:33:05.885 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.885 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.885 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:05.885 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267873+0000", 2024-04-24T15:33:05.886 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043344141000000003 2024-04-24T15:33:05.886 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.886 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.886 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:05.886 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.267948+0000", 2024-04-24T15:33:05.886 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.4960000000000001e-05 2024-04-24T15:33:05.886 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.886 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.886 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:05.886 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.268029+0000", 2024-04-24T15:33:05.886 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.1520000000000006e-05 2024-04-24T15:33:05.886 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.886 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:05.886 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.886 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.887 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.887 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17328 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 11960320~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:05.887 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.226273+0000", 2024-04-24T15:33:05.887 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 31.632045990999998, 2024-04-24T15:33:05.887 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042524610999999997, 2024-04-24T15:33:05.887 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:05.887 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:05.887 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:05.887 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:05.887 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:05.887 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17328 2024-04-24T15:33:05.887 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.887 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:05.887 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.887 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:05.888 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226273+0000", 2024-04-24T15:33:05.888 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.888 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.888 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.888 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:05.888 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226273+0000", 2024-04-24T15:33:05.888 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.888 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.888 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.888 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:05.888 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226273+0000", 2024-04-24T15:33:05.888 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:05.888 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.888 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.888 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:05.889 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226275+0000", 2024-04-24T15:33:05.889 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.759e-06 2024-04-24T15:33:05.889 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.889 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.889 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:05.889 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226286+0000", 2024-04-24T15:33:05.889 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0841e-05 2024-04-24T15:33:05.889 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.889 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.889 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:05.889 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226290+0000", 2024-04-24T15:33:05.889 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.794e-06 2024-04-24T15:33:05.889 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.889 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.890 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:05.890 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.268693+0000", 2024-04-24T15:33:05.890 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042403519000000001 2024-04-24T15:33:05.890 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.890 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.890 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:05.890 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.268739+0000", 2024-04-24T15:33:05.890 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6149000000000001e-05 2024-04-24T15:33:05.890 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.890 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.890 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:05.890 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.268798+0000", 2024-04-24T15:33:05.890 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.8752999999999998e-05 2024-04-24T15:33:05.890 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.890 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:05.891 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.891 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.891 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.891 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17329 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 12001280~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:05.891 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.226295+0000", 2024-04-24T15:33:05.891 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 31.632024884, 2024-04-24T15:33:05.891 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042690771000000002, 2024-04-24T15:33:05.891 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:05.891 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:05.891 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:05.891 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:05.891 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:05.891 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17329 2024-04-24T15:33:05.891 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.891 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:05.891 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.892 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:05.892 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226295+0000", 2024-04-24T15:33:05.892 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.892 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.892 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.892 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:05.892 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226295+0000", 2024-04-24T15:33:05.892 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.892 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.892 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.892 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:05.892 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226294+0000", 2024-04-24T15:33:05.892 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:05.892 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.892 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.892 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:05.893 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226297+0000", 2024-04-24T15:33:05.893 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.0189999999999998e-06 2024-04-24T15:33:05.893 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.893 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.893 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:05.893 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226302+0000", 2024-04-24T15:33:05.893 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6260000000000003e-06 2024-04-24T15:33:05.893 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.893 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.893 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:05.893 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226304+0000", 2024-04-24T15:33:05.893 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.2230000000000001e-06 2024-04-24T15:33:05.893 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.893 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.893 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:05.893 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.268831+0000", 2024-04-24T15:33:05.894 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042527224000000002 2024-04-24T15:33:05.894 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.894 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.894 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:05.894 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.268904+0000", 2024-04-24T15:33:05.894 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.3168000000000001e-05 2024-04-24T15:33:05.894 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.894 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.894 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:05.894 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.268985+0000", 2024-04-24T15:33:05.894 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.1009999999999999e-05 2024-04-24T15:33:05.894 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.894 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:05.894 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.895 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.895 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.895 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17330 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 12042240~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:05.895 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.226307+0000", 2024-04-24T15:33:05.895 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 31.632012892999999, 2024-04-24T15:33:05.895 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042812009999999998, 2024-04-24T15:33:05.895 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:05.895 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:05.895 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:05.895 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:05.895 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:05.895 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17330 2024-04-24T15:33:05.895 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.895 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:05.895 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.896 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:05.896 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226307+0000", 2024-04-24T15:33:05.896 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.896 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.896 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.896 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:05.896 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226307+0000", 2024-04-24T15:33:05.896 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.896 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.896 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.896 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:05.896 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226306+0000", 2024-04-24T15:33:05.896 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:05.896 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.897 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.897 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:05.897 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226312+0000", 2024-04-24T15:33:05.897 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6180000000000003e-06 2024-04-24T15:33:05.897 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.897 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.897 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:05.897 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226316+0000", 2024-04-24T15:33:05.897 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2379999999999999e-06 2024-04-24T15:33:05.897 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.897 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.897 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:05.897 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226318+0000", 2024-04-24T15:33:05.897 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.3130000000000001e-06 2024-04-24T15:33:05.897 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.898 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.898 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:05.898 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269015+0000", 2024-04-24T15:33:05.898 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042696698999999998 2024-04-24T15:33:05.898 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.898 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.898 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:05.898 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269061+0000", 2024-04-24T15:33:05.898 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6193000000000003e-05 2024-04-24T15:33:05.898 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.898 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.898 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:05.898 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269119+0000", 2024-04-24T15:33:05.899 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.7169000000000001e-05 2024-04-24T15:33:05.899 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.899 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:05.899 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.899 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.899 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.899 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17331 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 12083200~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:05.899 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.226321+0000", 2024-04-24T15:33:05.899 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 31.631998461999999, 2024-04-24T15:33:05.899 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042976872999999999, 2024-04-24T15:33:05.899 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:05.899 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:05.899 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:05.899 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:05.899 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:05.900 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17331 2024-04-24T15:33:05.900 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.900 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:05.900 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.900 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:05.900 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226321+0000", 2024-04-24T15:33:05.900 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.900 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.900 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.900 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:05.900 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226321+0000", 2024-04-24T15:33:05.900 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.900 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.900 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.901 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:05.901 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226321+0000", 2024-04-24T15:33:05.901 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:05.901 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.901 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.901 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:05.901 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226322+0000", 2024-04-24T15:33:05.901 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.446e-06 2024-04-24T15:33:05.901 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.901 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.901 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:05.901 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226325+0000", 2024-04-24T15:33:05.901 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.004e-06 2024-04-24T15:33:05.901 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.901 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.902 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:05.902 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226327+0000", 2024-04-24T15:33:05.902 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.0990000000000001e-06 2024-04-24T15:33:05.902 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.902 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.902 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:05.902 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269137+0000", 2024-04-24T15:33:05.902 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042809381000000001 2024-04-24T15:33:05.902 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.902 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.902 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:05.902 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269217+0000", 2024-04-24T15:33:05.902 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.085e-05 2024-04-24T15:33:05.902 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.902 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.903 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:05.903 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269298+0000", 2024-04-24T15:33:05.903 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0317999999999999e-05 2024-04-24T15:33:05.903 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.903 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:05.903 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.903 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.903 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.903 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17332 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 12124160~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:05.903 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.226329+0000", 2024-04-24T15:33:05.903 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 31.631989974, 2024-04-24T15:33:05.903 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043084539999999998, 2024-04-24T15:33:05.903 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:05.903 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:05.903 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:05.903 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:05.904 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:05.904 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17332 2024-04-24T15:33:05.904 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.904 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:05.904 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.904 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:05.904 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226329+0000", 2024-04-24T15:33:05.904 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.904 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.904 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.904 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:05.904 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226329+0000", 2024-04-24T15:33:05.904 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.904 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.904 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.905 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:05.905 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226329+0000", 2024-04-24T15:33:05.905 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:05.905 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.905 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.905 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:05.905 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226331+0000", 2024-04-24T15:33:05.905 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.328e-06 2024-04-24T15:33:05.905 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.905 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.905 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:05.905 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226333+0000", 2024-04-24T15:33:05.905 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.9119999999999998e-06 2024-04-24T15:33:05.905 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.905 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.905 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:05.906 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226335+0000", 2024-04-24T15:33:05.906 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.9690000000000001e-06 2024-04-24T15:33:05.906 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.906 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.906 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:05.906 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269313+0000", 2024-04-24T15:33:05.906 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042977518999999999 2024-04-24T15:33:05.906 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.906 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.906 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:05.906 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269357+0000", 2024-04-24T15:33:05.906 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3748999999999997e-05 2024-04-24T15:33:05.906 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.906 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.906 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:05.907 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269414+0000", 2024-04-24T15:33:05.907 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.7265000000000001e-05 2024-04-24T15:33:05.907 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.907 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:05.907 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.907 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.907 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.907 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17333 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 12165120~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:05.907 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.226338+0000", 2024-04-24T15:33:05.907 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 31.631981837000001, 2024-04-24T15:33:05.907 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043264706999999999, 2024-04-24T15:33:05.907 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:05.907 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:05.907 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:05.907 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:05.907 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:05.907 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17333 2024-04-24T15:33:05.908 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.908 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:05.908 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.908 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:05.908 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226338+0000", 2024-04-24T15:33:05.908 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.908 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.908 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.908 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:05.908 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226338+0000", 2024-04-24T15:33:05.908 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.908 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.908 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.908 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:05.908 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226337+0000", 2024-04-24T15:33:05.908 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:05.909 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.909 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.909 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:05.909 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226339+0000", 2024-04-24T15:33:05.909 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.387e-06 2024-04-24T15:33:05.909 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.909 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.909 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:05.909 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226342+0000", 2024-04-24T15:33:05.909 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8059999999999999e-06 2024-04-24T15:33:05.909 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.909 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.909 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:05.909 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226344+0000", 2024-04-24T15:33:05.909 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.1390000000000002e-06 2024-04-24T15:33:05.910 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.910 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.910 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:05.910 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269445+0000", 2024-04-24T15:33:05.910 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043101457000000003 2024-04-24T15:33:05.910 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.910 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.910 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:05.910 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269520+0000", 2024-04-24T15:33:05.910 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.5278000000000007e-05 2024-04-24T15:33:05.910 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.910 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.910 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:05.911 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269602+0000", 2024-04-24T15:33:05.911 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.1847000000000004e-05 2024-04-24T15:33:05.911 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.911 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:05.911 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.911 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.911 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.911 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17334 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 12206080~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:05.911 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.226346+0000", 2024-04-24T15:33:05.911 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 31.631973421000001, 2024-04-24T15:33:05.911 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043468358999999998, 2024-04-24T15:33:05.911 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:05.911 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:05.911 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:05.911 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:05.912 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:05.912 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17334 2024-04-24T15:33:05.912 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.912 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:05.912 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.912 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:05.912 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226346+0000", 2024-04-24T15:33:05.912 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.912 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.912 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.912 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:05.912 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226346+0000", 2024-04-24T15:33:05.912 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.912 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.912 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.913 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:05.913 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226346+0000", 2024-04-24T15:33:05.913 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:05.913 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.913 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.913 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:05.913 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226347+0000", 2024-04-24T15:33:05.913 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.359e-06 2024-04-24T15:33:05.913 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.913 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.913 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:05.913 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226350+0000", 2024-04-24T15:33:05.913 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7800000000000001e-06 2024-04-24T15:33:05.913 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.913 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.913 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:05.914 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.226352+0000", 2024-04-24T15:33:05.914 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.046e-06 2024-04-24T15:33:05.914 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.914 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.914 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:05.914 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269646+0000", 2024-04-24T15:33:05.914 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043294393 2024-04-24T15:33:05.914 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.914 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.914 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:05.914 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269732+0000", 2024-04-24T15:33:05.914 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.5716000000000004e-05 2024-04-24T15:33:05.914 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.914 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.915 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:05.915 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.269814+0000", 2024-04-24T15:33:05.915 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.2270999999999999e-05 2024-04-24T15:33:05.915 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.915 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:05.915 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.915 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.915 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.915 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17342 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 12533760~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:05.915 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.228377+0000", 2024-04-24T15:33:05.915 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 31.629942527000001, 2024-04-24T15:33:05.915 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042573286000000002, 2024-04-24T15:33:05.915 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:05.915 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:05.916 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:05.916 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:05.916 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:05.916 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17342 2024-04-24T15:33:05.916 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.916 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:05.916 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.916 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:05.916 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228377+0000", 2024-04-24T15:33:05.916 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.916 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.916 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.916 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:05.916 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228377+0000", 2024-04-24T15:33:05.916 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.917 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.917 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.917 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:05.917 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228377+0000", 2024-04-24T15:33:05.917 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:05.917 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.917 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.917 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:05.917 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228389+0000", 2024-04-24T15:33:05.917 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.2262999999999999e-05 2024-04-24T15:33:05.917 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.917 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.917 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:05.917 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228398+0000", 2024-04-24T15:33:05.918 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.1079999999999999e-06 2024-04-24T15:33:05.918 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.918 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.918 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:05.918 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228402+0000", 2024-04-24T15:33:05.918 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8920000000000002e-06 2024-04-24T15:33:05.918 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.918 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.918 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:05.918 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.270842+0000", 2024-04-24T15:33:05.918 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042440193000000001 2024-04-24T15:33:05.918 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.918 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.918 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:05.919 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.270887+0000", 2024-04-24T15:33:05.919 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5359999999999999e-05 2024-04-24T15:33:05.919 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.919 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.919 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:05.919 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.270950+0000", 2024-04-24T15:33:05.919 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.2842999999999994e-05 2024-04-24T15:33:05.919 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.919 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:05.919 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.919 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.919 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.919 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17343 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 12574720~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:05.919 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.228407+0000", 2024-04-24T15:33:05.919 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 31.629912654000002, 2024-04-24T15:33:05.920 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042650325000000003, 2024-04-24T15:33:05.920 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:05.920 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:05.920 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:05.920 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:05.920 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:05.920 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17343 2024-04-24T15:33:05.920 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.920 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:05.920 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.920 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:05.920 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228407+0000", 2024-04-24T15:33:05.920 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.921 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.921 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.921 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:05.921 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228407+0000", 2024-04-24T15:33:05.921 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.921 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.921 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.921 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:05.921 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228406+0000", 2024-04-24T15:33:05.921 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:05.921 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.921 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.921 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:05.921 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228409+0000", 2024-04-24T15:33:05.921 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.3980000000000002e-06 2024-04-24T15:33:05.922 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.922 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.922 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:05.922 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228414+0000", 2024-04-24T15:33:05.922 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6590000000000004e-06 2024-04-24T15:33:05.922 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.922 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.922 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:05.922 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228418+0000", 2024-04-24T15:33:05.922 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.625e-06 2024-04-24T15:33:05.922 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.922 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.922 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:05.922 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.270966+0000", 2024-04-24T15:33:05.923 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042547709000000003 2024-04-24T15:33:05.923 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.923 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.923 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:05.923 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.271007+0000", 2024-04-24T15:33:05.923 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.1659000000000002e-05 2024-04-24T15:33:05.923 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.923 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.923 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:05.923 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.271057+0000", 2024-04-24T15:33:05.923 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.9679000000000003e-05 2024-04-24T15:33:05.923 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.923 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:05.923 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.924 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.924 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.924 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17344 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 12607488~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:05.924 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.228461+0000", 2024-04-24T15:33:05.924 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 31.629858666000001, 2024-04-24T15:33:05.924 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042713530999999999, 2024-04-24T15:33:05.924 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:05.924 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:05.924 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:05.924 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:05.924 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:05.924 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17344 2024-04-24T15:33:05.924 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.924 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:05.924 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.925 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:05.925 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228461+0000", 2024-04-24T15:33:05.925 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.925 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.925 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.925 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:05.925 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228461+0000", 2024-04-24T15:33:05.925 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.925 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.925 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.925 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:05.925 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228460+0000", 2024-04-24T15:33:05.925 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:05.925 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.926 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.926 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:05.926 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228464+0000", 2024-04-24T15:33:05.926 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.98e-06 2024-04-24T15:33:05.926 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.926 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.926 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:05.926 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228473+0000", 2024-04-24T15:33:05.926 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.0270000000000001e-06 2024-04-24T15:33:05.926 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.926 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.926 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:05.926 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228479+0000", 2024-04-24T15:33:05.926 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.5910000000000003e-06 2024-04-24T15:33:05.926 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.927 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.927 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:05.927 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.271081+0000", 2024-04-24T15:33:05.927 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042602025000000002 2024-04-24T15:33:05.927 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.927 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.927 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:05.927 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.271126+0000", 2024-04-24T15:33:05.927 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4904999999999998e-05 2024-04-24T15:33:05.927 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.927 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.927 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:05.927 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.271174+0000", 2024-04-24T15:33:05.927 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8603999999999997e-05 2024-04-24T15:33:05.928 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.928 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:05.928 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.928 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.928 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.928 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17345 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 12648448~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:05.928 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.228484+0000", 2024-04-24T15:33:05.928 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 31.62983513, 2024-04-24T15:33:05.928 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042811844000000002, 2024-04-24T15:33:05.928 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:05.928 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:05.928 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:05.928 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:05.928 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:05.929 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17345 2024-04-24T15:33:05.929 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.929 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:05.929 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.929 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:05.929 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228484+0000", 2024-04-24T15:33:05.929 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.929 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.929 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.929 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:05.929 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228484+0000", 2024-04-24T15:33:05.929 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.929 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.929 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.930 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:05.930 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228484+0000", 2024-04-24T15:33:05.930 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:05.930 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.930 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.930 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:05.930 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228487+0000", 2024-04-24T15:33:05.930 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.9950000000000001e-06 2024-04-24T15:33:05.930 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.930 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.930 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:05.930 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228493+0000", 2024-04-24T15:33:05.930 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.2870000000000001e-06 2024-04-24T15:33:05.930 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.930 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.931 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:05.931 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228497+0000", 2024-04-24T15:33:05.931 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.1219999999999997e-06 2024-04-24T15:33:05.931 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.931 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.931 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:05.931 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.271202+0000", 2024-04-24T15:33:05.931 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042704963999999998 2024-04-24T15:33:05.931 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.931 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.931 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:05.931 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.271250+0000", 2024-04-24T15:33:05.931 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8207000000000001e-05 2024-04-24T15:33:05.932 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.932 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.932 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:05.932 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.271296+0000", 2024-04-24T15:33:05.932 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5769999999999997e-05 2024-04-24T15:33:05.932 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.932 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:05.932 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.932 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.932 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.932 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17346 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 12689408~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:05.932 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.228502+0000", 2024-04-24T15:33:05.933 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 31.629817515999999, 2024-04-24T15:33:05.933 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042918317999999997, 2024-04-24T15:33:05.933 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:05.933 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:05.933 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:05.933 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:05.933 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:05.933 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17346 2024-04-24T15:33:05.933 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.933 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:05.934 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.934 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:05.934 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228502+0000", 2024-04-24T15:33:05.934 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.934 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.934 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.934 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:05.934 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228502+0000", 2024-04-24T15:33:05.934 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.934 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.934 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.934 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:05.934 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228502+0000", 2024-04-24T15:33:05.934 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:05.935 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.935 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.935 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:05.935 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228504+0000", 2024-04-24T15:33:05.935 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.497e-06 2024-04-24T15:33:05.935 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.935 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.935 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:05.935 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228510+0000", 2024-04-24T15:33:05.935 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.7429999999999999e-06 2024-04-24T15:33:05.935 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.935 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.935 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:05.935 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228514+0000", 2024-04-24T15:33:05.936 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0810000000000004e-06 2024-04-24T15:33:05.936 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.936 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.936 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:05.936 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.271330+0000", 2024-04-24T15:33:05.936 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042816616000000002 2024-04-24T15:33:05.936 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.936 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.936 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:05.936 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.271371+0000", 2024-04-24T15:33:05.936 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0222999999999998e-05 2024-04-24T15:33:05.936 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.936 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.936 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:05.937 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.271420+0000", 2024-04-24T15:33:05.937 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.9539000000000001e-05 2024-04-24T15:33:05.937 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.937 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:05.937 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.937 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.937 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.937 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:17347 4.12 4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7 [read 12730368~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:05.937 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.228518+0000", 2024-04-24T15:33:05.937 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 31.629800988, 2024-04-24T15:33:05.937 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043022429000000001, 2024-04-24T15:33:05.937 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:05.937 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:05.937 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:05.938 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:05.938 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:05.938 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 17347 2024-04-24T15:33:05.938 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.938 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:05.938 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.938 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:05.938 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228518+0000", 2024-04-24T15:33:05.938 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.938 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.938 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.938 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:05.938 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228518+0000", 2024-04-24T15:33:05.938 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:05.939 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.939 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.939 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:05.939 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228518+0000", 2024-04-24T15:33:05.939 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:05.939 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.939 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.939 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:05.939 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228521+0000", 2024-04-24T15:33:05.939 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.4559999999999999e-06 2024-04-24T15:33:05.939 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.939 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.939 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:05.939 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228526+0000", 2024-04-24T15:33:05.939 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.7259999999999999e-06 2024-04-24T15:33:05.940 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.940 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.940 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:05.940 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.228530+0000", 2024-04-24T15:33:05.940 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.658e-06 2024-04-24T15:33:05.940 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.940 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.940 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:05.940 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.271445+0000", 2024-04-24T15:33:05.940 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042915506999999999 2024-04-24T15:33:05.940 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.940 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.940 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:05.940 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.271490+0000", 2024-04-24T15:33:05.940 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4752000000000001e-05 2024-04-24T15:33:05.940 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:05.941 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:05.941 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:05.941 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.271541+0000", 2024-04-24T15:33:05.941 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.0699999999999999e-05 2024-04-24T15:33:05.941 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.941 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:05.941 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.941 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:05.941 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:05.941 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:05.942 INFO:teuthology.orchestra.run.smithi096.stdout:truncating image image_client.0-clone16 from 0xda5c5cd (overlap 0x2c067b4) to 0x0 2024-04-24T15:33:05.943 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-24T15:33:05.943 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:05.943+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:05.973 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #15, image image_client.0-clone15 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent16 2024-04-24T15:33:05.991 INFO:teuthology.orchestra.run.smithi096.stdout:3621 read 0x7796121 thru 0x77a0e11 (0xacf1 bytes) 2024-04-24T15:33:05.992 INFO:teuthology.orchestra.run.smithi096.stdout:3623 writesame 0x2638791 thru 0x264180c data_size 0xe(0x907c bytes) 2024-04-24T15:33:06.002 INFO:teuthology.orchestra.run.smithi096.stdout:3624 punch from 0xa2a4a20 to 0xa2ab7d3, (0x6db3 bytes) 2024-04-24T15:33:06.003 INFO:teuthology.orchestra.run.smithi096.stdout:3625 punch from 0x447d6de to 0x448bdcc, (0xe6ee bytes) 2024-04-24T15:33:06.003 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:06.003 INFO:teuthology.orchestra.run.smithi096.stdout:3627 read 0xd153799 thru 0xd1568f4 (0x315c bytes) 2024-04-24T15:33:06.003 INFO:teuthology.orchestra.run.smithi096.stdout:3628 writesame 0xba81ae4 thru 0xba91365 data_size 0x2(0xf882 bytes) 2024-04-24T15:33:06.022 INFO:teuthology.orchestra.run.smithi096.stdout:3629 write 0x7757e10 thru 0x775cb11 (0x4d02 bytes) 2024-04-24T15:33:06.040 INFO:teuthology.orchestra.run.smithi096.stdout:3630 write 0xb4f8df8 thru 0xb4fcc29 (0x3e32 bytes) 2024-04-24T15:33:06.043 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:06.044+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:06.051 INFO:teuthology.orchestra.run.smithi096.stdout:3632 trunc from 0xda5c5cd to 0xa0b9a86 2024-04-24T15:33:06.070 INFO:teuthology.orchestra.run.smithi096.stdout:3633 read 0x9644a40 thru 0x964a0fa (0x56bb bytes) 2024-04-24T15:33:06.070 INFO:teuthology.orchestra.run.smithi096.stdout:3634 read 0x28b45d6 thru 0x28c205f (0xda8a bytes) 2024-04-24T15:33:06.075 INFO:teuthology.orchestra.run.smithi096.stdout:3635 write 0xa8a62b8 thru 0xa8ad8c0 (0x7609 bytes) 2024-04-24T15:33:06.077 INFO:teuthology.orchestra.run.smithi096.stdout:3636 writesame 0x5d12de thru 0x5de595 data_size 0x2c(0xd2b8 bytes) 2024-04-24T15:33:06.079 INFO:teuthology.orchestra.run.smithi096.stdout:3637 trunc from 0xa8ad8c1 to 0xaa64053 2024-04-24T15:33:06.083 INFO:teuthology.orchestra.run.smithi096.stdout:3638 punch from 0x50ba2f to 0x513414, (0x79e5 bytes) 2024-04-24T15:33:06.084 INFO:teuthology.orchestra.run.smithi096.stdout:3639 writesame 0x916e83a thru 0x9174744 data_size 0x1(0x5f0b bytes) 2024-04-24T15:33:06.094 INFO:teuthology.orchestra.run.smithi096.stdout:3640 read 0x3adfb36 thru 0x3aec546 (0xca11 bytes) 2024-04-24T15:33:06.107 INFO:teuthology.orchestra.run.smithi096.stdout:3641 trunc from 0xaa64053 to 0x8defc0 2024-04-24T15:33:06.120 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:06.120 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:33:06.120 INFO:teuthology.orchestra.run.smithi151.stdout: "num_ops": 0 2024-04-24T15:33:06.120 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:06.130 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-24T15:33:06.137 INFO:teuthology.orchestra.run.smithi096.stdout:3642 write 0x892573d thru 0x8926a94 (0x1358 bytes) 2024-04-24T15:33:06.140 INFO:teuthology.orchestra.run.smithi096.stdout:3644 trunc from 0x8926a95 to 0x7f6b9cc 2024-04-24T15:33:06.144 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:06.145+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:06.148 INFO:teuthology.orchestra.run.smithi096.stdout:3645 trunc from 0x7f6b9cc to 0x74a46bb 2024-04-24T15:33:06.156 INFO:teuthology.orchestra.run.smithi096.stdout:3647 read 0x2361dc9 thru 0x2361fda (0x212 bytes) 2024-04-24T15:33:06.156 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:06.168 INFO:teuthology.orchestra.run.smithi096.stdout:3650 write 0xdecb2e2 thru 0xdecbf4e (0xc6d bytes) 2024-04-24T15:33:06.171 INFO:teuthology.orchestra.run.smithi096.stdout:3651 write 0x3abbeec thru 0x3ac084d (0x4962 bytes) 2024-04-24T15:33:06.186 INFO:teuthology.orchestra.run.smithi096.stdout:3652 trunc from 0xdecbf4f to 0x21d5613 2024-04-24T15:33:06.196 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:06.196 INFO:teuthology.orchestra.run.smithi096.stdout:3655 read 0x114943f thru 0x114e19f (0x4d61 bytes) 2024-04-24T15:33:06.196 INFO:teuthology.orchestra.run.smithi096.stdout:3658 read 0x1d6d58d thru 0x1d72910 (0x5384 bytes) 2024-04-24T15:33:06.211 INFO:teuthology.orchestra.run.smithi096.stdout:3660 write 0xa898a12 thru 0xa89d20f (0x47fe bytes) 2024-04-24T15:33:06.214 INFO:teuthology.orchestra.run.smithi096.stdout:3661 read 0x4a83d61 thru 0x4a89b63 (0x5e03 bytes) 2024-04-24T15:33:06.214 INFO:teuthology.orchestra.run.smithi096.stdout:3662 punch from 0x58fbdfb to 0x58fd7a8, (0x19ad bytes) 2024-04-24T15:33:06.214 INFO:teuthology.orchestra.run.smithi096.stdout:3663 punch from 0x9c1a6b to 0x9cc4a5, (0xaa3a bytes) 2024-04-24T15:33:06.214 INFO:teuthology.orchestra.run.smithi096.stdout:3664 write 0x88cd3bb thru 0x88d6961 (0x95a7 bytes) 2024-04-24T15:33:06.222 INFO:teuthology.orchestra.run.smithi096.stdout:3666 write 0xc098aea thru 0xc0a4068 (0xb57f bytes) 2024-04-24T15:33:06.223 INFO:teuthology.orchestra.run.smithi096.stdout:3667 read 0xbabc6eb thru 0xbabc733 (0x49 bytes) 2024-04-24T15:33:06.231 INFO:teuthology.orchestra.run.smithi096.stdout:3668 write 0xe6cd89d thru 0xe6d4e8a (0x75ee bytes) 2024-04-24T15:33:06.234 INFO:teuthology.orchestra.run.smithi096.stdout:3669 writesame 0x8dc5397 thru 0x8dd4679 data_size 0x1(0xf2e3 bytes) 2024-04-24T15:33:06.245 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:33:06.246+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:33:06.258 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:06.258 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:33:06.259 INFO:teuthology.orchestra.run.smithi151.stdout: "complaint_time": 30, 2024-04-24T15:33:06.259 INFO:teuthology.orchestra.run.smithi151.stdout: "num_blocked_ops": 0 2024-04-24T15:33:06.259 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:06.263 INFO:teuthology.orchestra.run.smithi096.stdout:3670 write 0x63c665 thru 0x64252d (0x5ec9 bytes) 2024-04-24T15:33:06.264 INFO:teuthology.orchestra.run.smithi096.stdout:3671 write 0x2c97ac5 thru 0x2c97f2e (0x46a bytes) 2024-04-24T15:33:06.267 INFO:teuthology.orchestra.run.smithi096.stdout:3673 write 0x309f94a thru 0x30ac8c3 (0xcf7a bytes) 2024-04-24T15:33:06.269 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-24T15:33:06.270 INFO:teuthology.orchestra.run.smithi096.stdout:3675 read 0x8d200e9 thru 0x8d29d1b (0x9c33 bytes) 2024-04-24T15:33:06.270 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:06.270 INFO:teuthology.orchestra.run.smithi096.stdout:3678 read 0x89475b8 thru 0x8953fe9 (0xca32 bytes) 2024-04-24T15:33:06.270 INFO:teuthology.orchestra.run.smithi096.stdout:3679 writesame 0xca3c809 thru 0xca45e0b data_size 0x11(0x9603 bytes) 2024-04-24T15:33:06.286 INFO:teuthology.orchestra.run.smithi096.stdout:3680 trunc from 0xe6d4e8b to 0x39a1b5c 2024-04-24T15:33:06.301 INFO:teuthology.orchestra.run.smithi096.stdout:3681 read 0x7f60e thru 0x81198 (0x1b8b bytes) 2024-04-24T15:33:06.302 INFO:teuthology.orchestra.run.smithi096.stdout:3682 read 0x1486fe thru 0x1573a4 (0xeca7 bytes) 2024-04-24T15:33:06.317 INFO:teuthology.orchestra.run.smithi096.stdout:3683 write 0xb248f67 thru 0xb257839 (0xe8d3 bytes) 2024-04-24T15:33:06.318 INFO:teuthology.orchestra.run.smithi096.stdout:3684 write 0x523659a thru 0x5241056 (0xaabd bytes) 2024-04-24T15:33:06.321 INFO:teuthology.orchestra.run.smithi096.stdout:3685 read 0x6f9b0b4 thru 0x6fa031e (0x526b bytes) 2024-04-24T15:33:06.321 INFO:teuthology.orchestra.run.smithi096.stdout:3686 write 0x336413c thru 0x336e7fd (0xa6c2 bytes) 2024-04-24T15:33:06.323 INFO:teuthology.orchestra.run.smithi096.stdout:3687 read 0x94f79fa thru 0x95010ce (0x96d5 bytes) 2024-04-24T15:33:06.323 INFO:teuthology.orchestra.run.smithi096.stdout:3690 write 0x74a21c0 thru 0x74a35b7 (0x13f8 bytes) 2024-04-24T15:33:06.325 INFO:teuthology.orchestra.run.smithi096.stdout:3692 writesame 0x9ccc532 thru 0x9cd3130 data_size 0x1(0x6bff bytes) 2024-04-24T15:33:06.339 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:06.339 INFO:teuthology.orchestra.run.smithi096.stdout:3697 read 0x167c633 thru 0x167f350 (0x2d1e bytes) 2024-04-24T15:33:06.340 INFO:teuthology.orchestra.run.smithi096.stdout:3699 punch from 0x135ab16 to 0x135f437, (0x4921 bytes) 2024-04-24T15:33:06.340 INFO:teuthology.orchestra.run.smithi096.stdout:3700 write 0xb16e9b5 thru 0xb179503 (0xab4f bytes) 2024-04-24T15:33:06.342 INFO:teuthology.orchestra.run.smithi096.stdout:3701 read 0x9b5542 thru 0x9be608 (0x90c7 bytes) 2024-04-24T15:33:06.342 INFO:teuthology.orchestra.run.smithi096.stdout:3704 punch from 0x97dbe64 to 0x97e4ab7, (0x8c53 bytes) 2024-04-24T15:33:06.342 INFO:teuthology.orchestra.run.smithi096.stdout:3706 write 0x1416bcc thru 0x141aec0 (0x42f5 bytes) 2024-04-24T15:33:06.345 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:33:06.346+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:33:06.346 INFO:teuthology.orchestra.run.smithi096.stdout:3708 write 0x495ef77 thru 0x49601fd (0x1287 bytes) 2024-04-24T15:33:06.350 INFO:teuthology.orchestra.run.smithi096.stdout:3712 trunc from 0xb25783a to 0x8ccb8cb 2024-04-24T15:33:06.368 INFO:teuthology.orchestra.run.smithi096.stdout:3713 trunc from 0x8ccb8cb to 0x14e841a 2024-04-24T15:33:06.399 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:06.399 INFO:teuthology.orchestra.run.smithi151.stdout: "size": 20, 2024-04-24T15:33:06.399 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 600, 2024-04-24T15:33:06.400 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [ 2024-04-24T15:33:06.400 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.400 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18006 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12419072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:06.400 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.207911+0000", 2024-04-24T15:33:06.400 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 32.191706934999999, 2024-04-24T15:33:06.400 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042899496000000002, 2024-04-24T15:33:06.400 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:06.400 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:06.400 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:06.400 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:06.400 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:06.400 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18006 2024-04-24T15:33:06.400 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.400 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:06.401 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.401 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:06.401 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207911+0000", 2024-04-24T15:33:06.401 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:06.401 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.401 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.401 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:06.401 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207911+0000", 2024-04-24T15:33:06.401 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:06.401 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.401 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.402 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:06.402 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207910+0000", 2024-04-24T15:33:06.402 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:06.402 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.402 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.402 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:06.402 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207913+0000", 2024-04-24T15:33:06.402 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.2309999999999999e-06 2024-04-24T15:33:06.402 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.402 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.403 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:06.403 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207918+0000", 2024-04-24T15:33:06.403 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.3399999999999997e-06 2024-04-24T15:33:06.403 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.403 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.403 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:06.403 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207922+0000", 2024-04-24T15:33:06.403 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5439999999999999e-06 2024-04-24T15:33:06.403 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.403 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.403 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:06.404 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250726+0000", 2024-04-24T15:33:06.404 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042804129000000003 2024-04-24T15:33:06.404 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.404 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.404 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:06.404 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250765+0000", 2024-04-24T15:33:06.404 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9554000000000001e-05 2024-04-24T15:33:06.404 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.404 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.404 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:06.404 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250810+0000", 2024-04-24T15:33:06.404 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5027000000000001e-05 2024-04-24T15:33:06.404 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:06.404 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:06.404 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:06.405 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.405 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.405 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18007 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12460032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:06.405 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.207961+0000", 2024-04-24T15:33:06.405 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 32.191657094, 2024-04-24T15:33:06.405 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042965470999999998, 2024-04-24T15:33:06.405 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:06.405 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:06.405 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:06.405 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:06.405 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:06.405 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18007 2024-04-24T15:33:06.405 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.405 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:06.405 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.406 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:06.406 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207961+0000", 2024-04-24T15:33:06.406 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:06.406 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.406 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.406 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:06.406 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207961+0000", 2024-04-24T15:33:06.406 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:06.406 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.406 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.406 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:06.406 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207960+0000", 2024-04-24T15:33:06.406 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:06.406 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.407 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.407 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:06.407 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207963+0000", 2024-04-24T15:33:06.407 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.3759999999999999e-06 2024-04-24T15:33:06.407 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.407 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.407 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:06.407 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207970+0000", 2024-04-24T15:33:06.407 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.0489999999999998e-06 2024-04-24T15:33:06.407 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.407 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.407 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:06.407 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207975+0000", 2024-04-24T15:33:06.407 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.1640000000000003e-06 2024-04-24T15:33:06.407 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.408 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.408 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:06.408 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250834+0000", 2024-04-24T15:33:06.408 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042859060999999997 2024-04-24T15:33:06.408 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.408 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.408 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:06.408 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250874+0000", 2024-04-24T15:33:06.408 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0674999999999999e-05 2024-04-24T15:33:06.408 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.408 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.408 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:06.409 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250926+0000", 2024-04-24T15:33:06.409 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.1693000000000001e-05 2024-04-24T15:33:06.409 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:06.409 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:06.409 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:06.409 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.409 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.409 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18008 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12500992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:06.409 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208030+0000", 2024-04-24T15:33:06.409 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 32.191588001, 2024-04-24T15:33:06.409 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043003337000000003, 2024-04-24T15:33:06.409 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:06.409 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:06.410 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:06.410 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:06.410 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:06.410 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18008 2024-04-24T15:33:06.410 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.410 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:06.410 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.410 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:06.410 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208030+0000", 2024-04-24T15:33:06.410 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:06.410 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.411 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.411 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:06.411 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208030+0000", 2024-04-24T15:33:06.411 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:06.411 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.411 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.411 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:06.411 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208029+0000", 2024-04-24T15:33:06.411 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:06.411 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.411 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.412 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:06.412 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208032+0000", 2024-04-24T15:33:06.412 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.117e-06 2024-04-24T15:33:06.412 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.412 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.412 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:06.412 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208039+0000", 2024-04-24T15:33:06.413 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.0129999999999996e-06 2024-04-24T15:33:06.413 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.413 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.413 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:06.413 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208043+0000", 2024-04-24T15:33:06.413 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0600000000000001e-06 2024-04-24T15:33:06.413 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.413 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.413 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:06.413 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250948+0000", 2024-04-24T15:33:06.413 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042904577999999999 2024-04-24T15:33:06.413 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.413 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.413 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:06.414 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250989+0000", 2024-04-24T15:33:06.414 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0978e-05 2024-04-24T15:33:06.414 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.414 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.414 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:06.414 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251033+0000", 2024-04-24T15:33:06.414 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4161e-05 2024-04-24T15:33:06.414 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:06.414 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:06.414 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:06.414 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.414 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.414 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18009 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12541952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:06.415 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208093+0000", 2024-04-24T15:33:06.415 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 32.191524254999997, 2024-04-24T15:33:06.415 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043043451000000003, 2024-04-24T15:33:06.415 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:06.415 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:06.415 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:06.415 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:06.415 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:06.415 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18009 2024-04-24T15:33:06.415 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.416 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:06.416 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.416 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:06.416 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208093+0000", 2024-04-24T15:33:06.416 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:06.416 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.416 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.416 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:06.416 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208093+0000", 2024-04-24T15:33:06.416 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:06.416 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.416 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.417 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:06.417 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208093+0000", 2024-04-24T15:33:06.417 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:06.417 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.417 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.417 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:06.417 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208096+0000", 2024-04-24T15:33:06.417 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.439e-06 2024-04-24T15:33:06.417 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.417 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.417 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:06.417 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208103+0000", 2024-04-24T15:33:06.417 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1019999999999999e-06 2024-04-24T15:33:06.417 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.418 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.418 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:06.418 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208107+0000", 2024-04-24T15:33:06.418 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0910000000000003e-06 2024-04-24T15:33:06.418 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.418 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.418 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:06.418 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251054+0000", 2024-04-24T15:33:06.418 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042946826 2024-04-24T15:33:06.418 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.418 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.418 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:06.419 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251092+0000", 2024-04-24T15:33:06.419 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.7941999999999997e-05 2024-04-24T15:33:06.419 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.419 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.419 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:06.419 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251137+0000", 2024-04-24T15:33:06.419 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4638000000000002e-05 2024-04-24T15:33:06.419 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:06.419 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:06.419 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:06.419 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.419 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.420 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18010 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12615680~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:06.420 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208202+0000", 2024-04-24T15:33:06.420 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 32.191415423999999, 2024-04-24T15:33:06.420 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043046549000000003, 2024-04-24T15:33:06.420 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:06.420 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:06.420 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:06.420 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:06.420 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:06.420 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18010 2024-04-24T15:33:06.420 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.420 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:06.420 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.420 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:06.421 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208202+0000", 2024-04-24T15:33:06.421 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:06.421 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.421 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.421 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:06.421 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208202+0000", 2024-04-24T15:33:06.421 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:06.421 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.421 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.421 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:06.421 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208202+0000", 2024-04-24T15:33:06.421 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:06.421 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.421 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.422 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:06.422 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208205+0000", 2024-04-24T15:33:06.422 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.2490000000000001e-06 2024-04-24T15:33:06.422 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.422 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.422 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:06.422 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208215+0000", 2024-04-24T15:33:06.422 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.9070000000000002e-06 2024-04-24T15:33:06.423 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.423 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.423 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:06.423 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208220+0000", 2024-04-24T15:33:06.423 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8849999999999998e-06 2024-04-24T15:33:06.423 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.423 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.423 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:06.423 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251158+0000", 2024-04-24T15:33:06.423 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042938416 2024-04-24T15:33:06.423 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.423 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.423 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:06.424 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251197+0000", 2024-04-24T15:33:06.424 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9341000000000002e-05 2024-04-24T15:33:06.424 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.424 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.424 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:06.424 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251249+0000", 2024-04-24T15:33:06.424 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.1391000000000003e-05 2024-04-24T15:33:06.424 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:06.424 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:06.424 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:06.424 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.424 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.424 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18011 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12656640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:06.424 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208258+0000", 2024-04-24T15:33:06.425 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 32.191359577999997, 2024-04-24T15:33:06.425 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043102708000000003, 2024-04-24T15:33:06.425 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:06.425 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:06.425 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:06.425 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:06.425 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:06.425 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18011 2024-04-24T15:33:06.425 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.425 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:06.425 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.425 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:06.425 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208258+0000", 2024-04-24T15:33:06.425 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:06.425 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.425 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.426 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:06.426 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208258+0000", 2024-04-24T15:33:06.426 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:06.426 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.426 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.426 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:06.426 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208258+0000", 2024-04-24T15:33:06.426 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:06.426 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.426 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.426 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:06.426 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208260+0000", 2024-04-24T15:33:06.426 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7810000000000001e-06 2024-04-24T15:33:06.427 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.427 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.427 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:06.427 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208267+0000", 2024-04-24T15:33:06.427 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.7610000000000002e-06 2024-04-24T15:33:06.427 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.427 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.427 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:06.427 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208271+0000", 2024-04-24T15:33:06.427 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0890000000000002e-06 2024-04-24T15:33:06.427 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.427 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.427 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:06.427 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251271+0000", 2024-04-24T15:33:06.427 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043000159000000003 2024-04-24T15:33:06.428 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.428 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.428 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:06.428 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251315+0000", 2024-04-24T15:33:06.428 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3446999999999999e-05 2024-04-24T15:33:06.428 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.428 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.428 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:06.428 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251361+0000", 2024-04-24T15:33:06.428 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5952000000000003e-05 2024-04-24T15:33:06.428 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:06.428 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:06.428 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:06.428 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.428 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.429 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18012 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12697600~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:06.429 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208289+0000", 2024-04-24T15:33:06.429 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 32.191328534, 2024-04-24T15:33:06.429 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043174774999999999, 2024-04-24T15:33:06.429 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:06.429 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:06.429 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:06.429 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:06.429 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:06.429 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18012 2024-04-24T15:33:06.429 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.429 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:06.429 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.429 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:06.429 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208289+0000", 2024-04-24T15:33:06.430 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:06.430 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.430 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.430 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:06.430 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208289+0000", 2024-04-24T15:33:06.430 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:06.430 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.430 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.430 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:06.430 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208289+0000", 2024-04-24T15:33:06.431 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:06.431 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.431 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.431 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:06.431 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208291+0000", 2024-04-24T15:33:06.431 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.542e-06 2024-04-24T15:33:06.431 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.431 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.431 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:06.431 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208298+0000", 2024-04-24T15:33:06.431 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.3899999999999998e-06 2024-04-24T15:33:06.431 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.431 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.432 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:06.432 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208302+0000", 2024-04-24T15:33:06.432 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.019e-06 2024-04-24T15:33:06.432 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.432 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.432 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:06.432 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251382+0000", 2024-04-24T15:33:06.432 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043080790000000001 2024-04-24T15:33:06.432 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.432 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.432 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:06.432 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251421+0000", 2024-04-24T15:33:06.432 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8473000000000002e-05 2024-04-24T15:33:06.432 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.432 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.433 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:06.433 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251464+0000", 2024-04-24T15:33:06.433 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3133999999999997e-05 2024-04-24T15:33:06.433 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:06.433 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:06.433 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:06.433 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.433 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.434 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18013 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12738560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:06.434 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208423+0000", 2024-04-24T15:33:06.434 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 32.191194426000003, 2024-04-24T15:33:06.434 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043145554000000003, 2024-04-24T15:33:06.434 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:06.434 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:06.434 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:06.434 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:06.434 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:06.434 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18013 2024-04-24T15:33:06.434 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.434 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:06.434 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.435 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:06.435 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208423+0000", 2024-04-24T15:33:06.435 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:06.435 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.435 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.435 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:06.435 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208423+0000", 2024-04-24T15:33:06.435 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:06.435 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.435 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.435 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:06.435 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208423+0000", 2024-04-24T15:33:06.435 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:06.436 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.436 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.436 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:06.436 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208426+0000", 2024-04-24T15:33:06.436 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.0539999999999999e-06 2024-04-24T15:33:06.436 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.436 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.436 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:06.436 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208433+0000", 2024-04-24T15:33:06.436 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.43e-06 2024-04-24T15:33:06.436 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.436 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.436 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:06.436 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208437+0000", 2024-04-24T15:33:06.437 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5590000000000001e-06 2024-04-24T15:33:06.437 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.437 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.437 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:06.437 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251485+0000", 2024-04-24T15:33:06.437 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043048259999999998 2024-04-24T15:33:06.437 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.437 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.437 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:06.437 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251525+0000", 2024-04-24T15:33:06.437 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9916999999999997e-05 2024-04-24T15:33:06.437 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.437 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.437 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:06.438 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251569+0000", 2024-04-24T15:33:06.438 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3939000000000001e-05 2024-04-24T15:33:06.438 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:06.438 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:06.438 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:06.438 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.438 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.438 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18015 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12779520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:06.438 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208630+0000", 2024-04-24T15:33:06.438 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 32.190987901, 2024-04-24T15:33:06.439 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043047328000000003, 2024-04-24T15:33:06.439 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:06.439 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:06.439 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:06.439 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:06.439 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:06.439 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18015 2024-04-24T15:33:06.439 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.439 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:06.439 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.439 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:06.439 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208630+0000", 2024-04-24T15:33:06.439 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:06.440 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.440 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.440 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:06.440 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208630+0000", 2024-04-24T15:33:06.440 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:06.440 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.440 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.440 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:06.440 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208629+0000", 2024-04-24T15:33:06.440 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999999 2024-04-24T15:33:06.440 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.440 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.440 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:06.440 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208634+0000", 2024-04-24T15:33:06.440 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8960000000000002e-06 2024-04-24T15:33:06.441 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.441 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.441 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:06.441 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208645+0000", 2024-04-24T15:33:06.441 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0939e-05 2024-04-24T15:33:06.441 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.441 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.441 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:06.441 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208651+0000", 2024-04-24T15:33:06.441 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.896e-06 2024-04-24T15:33:06.441 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.441 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.441 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:06.442 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251591+0000", 2024-04-24T15:33:06.442 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042939862000000002 2024-04-24T15:33:06.442 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.442 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.442 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:06.442 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251631+0000", 2024-04-24T15:33:06.442 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0216000000000003e-05 2024-04-24T15:33:06.442 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.442 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.442 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:06.442 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251677+0000", 2024-04-24T15:33:06.442 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6390999999999997e-05 2024-04-24T15:33:06.442 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:06.443 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:06.443 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:06.443 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.443 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.443 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18016 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12820480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:06.443 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208660+0000", 2024-04-24T15:33:06.443 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 32.190958111999997, 2024-04-24T15:33:06.443 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043127510000000001, 2024-04-24T15:33:06.443 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:06.443 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:06.443 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:06.443 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:06.444 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:06.444 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18016 2024-04-24T15:33:06.444 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.444 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:06.444 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.444 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:06.444 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208660+0000", 2024-04-24T15:33:06.444 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:06.444 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.444 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.444 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:06.444 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208660+0000", 2024-04-24T15:33:06.444 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:06.445 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.445 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.445 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:06.445 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208659+0000", 2024-04-24T15:33:06.445 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:06.445 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.445 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.445 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:06.445 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208662+0000", 2024-04-24T15:33:06.445 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.0979999999999998e-06 2024-04-24T15:33:06.445 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.445 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.445 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:06.446 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208669+0000", 2024-04-24T15:33:06.446 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.4470000000000003e-06 2024-04-24T15:33:06.446 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.446 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.446 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:06.446 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208673+0000", 2024-04-24T15:33:06.446 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2810000000000002e-06 2024-04-24T15:33:06.446 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.446 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.446 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:06.446 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251699+0000", 2024-04-24T15:33:06.446 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043025728999999999 2024-04-24T15:33:06.446 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.447 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.447 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:06.447 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251743+0000", 2024-04-24T15:33:06.447 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4469999999999999e-05 2024-04-24T15:33:06.447 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.447 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.447 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:06.447 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251787+0000", 2024-04-24T15:33:06.447 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3949999999999998e-05 2024-04-24T15:33:06.447 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:06.447 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:06.447 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:06.447 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.447 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.448 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18017 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12861440~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:06.448 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208721+0000", 2024-04-24T15:33:06.448 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 32.190896664, 2024-04-24T15:33:06.448 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043167749999999998, 2024-04-24T15:33:06.448 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:06.448 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:06.448 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:06.448 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:06.448 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:06.448 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18017 2024-04-24T15:33:06.448 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.448 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:06.449 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.449 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:06.449 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208721+0000", 2024-04-24T15:33:06.449 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:06.449 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.449 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.449 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:06.449 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208721+0000", 2024-04-24T15:33:06.449 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:06.449 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.449 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.449 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:06.449 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208720+0000", 2024-04-24T15:33:06.449 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:06.450 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.450 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.450 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:06.450 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208724+0000", 2024-04-24T15:33:06.450 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.3919999999999999e-06 2024-04-24T15:33:06.450 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.450 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.450 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:06.450 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208729+0000", 2024-04-24T15:33:06.450 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.3480000000000003e-06 2024-04-24T15:33:06.450 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.450 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.450 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:06.451 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208732+0000", 2024-04-24T15:33:06.451 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.9529999999999999e-06 2024-04-24T15:33:06.451 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.451 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.451 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:06.451 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251807+0000", 2024-04-24T15:33:06.451 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043074492999999998 2024-04-24T15:33:06.451 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.451 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.451 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:06.451 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251846+0000", 2024-04-24T15:33:06.451 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8958999999999997e-05 2024-04-24T15:33:06.452 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.452 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.452 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:06.452 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251889+0000", 2024-04-24T15:33:06.452 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3253000000000001e-05 2024-04-24T15:33:06.452 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:06.452 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:06.452 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:06.452 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.452 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.452 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18018 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12902400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:06.452 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208752+0000", 2024-04-24T15:33:06.452 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 32.190866051, 2024-04-24T15:33:06.452 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043241012000000002, 2024-04-24T15:33:06.452 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:06.452 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:06.453 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:06.453 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:06.453 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:06.453 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18018 2024-04-24T15:33:06.453 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.453 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:06.453 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.453 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:06.453 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208752+0000", 2024-04-24T15:33:06.453 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:06.453 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.453 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.453 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:06.453 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208752+0000", 2024-04-24T15:33:06.454 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:06.454 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.454 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.454 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:06.454 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208751+0000", 2024-04-24T15:33:06.454 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:06.454 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.454 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.454 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:06.454 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208753+0000", 2024-04-24T15:33:06.455 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.894e-06 2024-04-24T15:33:06.455 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.455 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.455 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:06.455 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208757+0000", 2024-04-24T15:33:06.455 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.856e-06 2024-04-24T15:33:06.455 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.455 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.455 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:06.455 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208760+0000", 2024-04-24T15:33:06.455 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.6749999999999998e-06 2024-04-24T15:33:06.455 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.455 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.455 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:06.455 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251909+0000", 2024-04-24T15:33:06.455 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043149145999999999 2024-04-24T15:33:06.456 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.456 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.456 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:06.456 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251948+0000", 2024-04-24T15:33:06.456 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9159000000000002e-05 2024-04-24T15:33:06.456 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.456 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.456 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:06.456 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251993+0000", 2024-04-24T15:33:06.456 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4638999999999997e-05 2024-04-24T15:33:06.456 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:06.456 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:06.456 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:06.456 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.456 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.456 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18019 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12943360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:06.457 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208938+0000", 2024-04-24T15:33:06.457 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 32.190679760000002, 2024-04-24T15:33:06.457 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043162272000000002, 2024-04-24T15:33:06.457 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:06.457 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:06.457 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:06.457 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:06.457 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:06.457 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18019 2024-04-24T15:33:06.457 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.457 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:06.457 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.457 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:06.457 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208938+0000", 2024-04-24T15:33:06.457 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:06.457 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.458 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.458 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:06.458 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208938+0000", 2024-04-24T15:33:06.458 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:06.458 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.458 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.458 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:06.458 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208762+0000", 2024-04-24T15:33:06.458 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999824 2024-04-24T15:33:06.458 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.458 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.458 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:06.458 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208942+0000", 2024-04-24T15:33:06.459 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000180008 2024-04-24T15:33:06.459 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.459 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.459 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:06.459 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208950+0000", 2024-04-24T15:33:06.459 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.1550000000000007e-06 2024-04-24T15:33:06.459 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.459 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.459 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:06.459 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208956+0000", 2024-04-24T15:33:06.459 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.7910000000000001e-06 2024-04-24T15:33:06.459 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.459 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.459 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:06.459 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252014+0000", 2024-04-24T15:33:06.459 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043058427000000003 2024-04-24T15:33:06.460 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.460 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.460 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:06.460 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252054+0000", 2024-04-24T15:33:06.460 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0071999999999999e-05 2024-04-24T15:33:06.460 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.460 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.460 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:06.460 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252100+0000", 2024-04-24T15:33:06.460 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5917000000000001e-05 2024-04-24T15:33:06.460 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:06.460 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:06.460 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:06.460 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.460 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.460 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18020 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12984320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:06.461 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.209092+0000", 2024-04-24T15:33:06.461 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 32.190525762, 2024-04-24T15:33:06.461 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043111295000000001, 2024-04-24T15:33:06.461 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:06.461 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:06.461 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:06.461 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:06.461 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:06.461 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18020 2024-04-24T15:33:06.461 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.461 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:06.461 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.461 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:06.461 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209092+0000", 2024-04-24T15:33:06.461 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:06.462 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.462 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.462 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:06.462 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209092+0000", 2024-04-24T15:33:06.462 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:06.462 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.462 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.462 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:06.462 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208964+0000", 2024-04-24T15:33:06.462 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998722 2024-04-24T15:33:06.462 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.462 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.462 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:06.463 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209095+0000", 2024-04-24T15:33:06.463 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000130648 2024-04-24T15:33:06.463 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.463 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.463 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:06.463 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209103+0000", 2024-04-24T15:33:06.463 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0320000000000003e-06 2024-04-24T15:33:06.463 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.463 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.463 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:06.463 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209108+0000", 2024-04-24T15:33:06.463 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8690000000000003e-06 2024-04-24T15:33:06.464 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.464 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.464 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:06.464 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252121+0000", 2024-04-24T15:33:06.464 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043013323999999999 2024-04-24T15:33:06.464 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.464 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.464 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:06.464 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252160+0000", 2024-04-24T15:33:06.464 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8376e-05 2024-04-24T15:33:06.464 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.464 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.464 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:06.465 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252203+0000", 2024-04-24T15:33:06.465 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.371e-05 2024-04-24T15:33:06.465 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:06.465 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:06.465 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:06.465 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.465 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.465 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18021 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 13025280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:06.465 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.209132+0000", 2024-04-24T15:33:06.465 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 32.190485404, 2024-04-24T15:33:06.465 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043183903000000003, 2024-04-24T15:33:06.465 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:06.465 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:06.465 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:06.465 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:06.466 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:06.466 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18021 2024-04-24T15:33:06.466 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.466 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:06.466 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.466 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:06.466 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209132+0000", 2024-04-24T15:33:06.466 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:06.466 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.466 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.466 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:06.466 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209132+0000", 2024-04-24T15:33:06.467 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:06.467 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.467 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.467 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:06.467 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209132+0000", 2024-04-24T15:33:06.467 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:06.467 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.467 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.467 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:06.467 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209134+0000", 2024-04-24T15:33:06.467 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.891e-06 2024-04-24T15:33:06.468 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.468 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.468 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:06.468 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209138+0000", 2024-04-24T15:33:06.468 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0659999999999997e-06 2024-04-24T15:33:06.468 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.468 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.468 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:06.468 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209140+0000", 2024-04-24T15:33:06.468 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.5339999999999998e-06 2024-04-24T15:33:06.468 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.468 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.468 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:06.468 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252232+0000", 2024-04-24T15:33:06.468 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043091142999999998 2024-04-24T15:33:06.469 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.469 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.469 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:06.469 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252270+0000", 2024-04-24T15:33:06.469 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8872000000000003e-05 2024-04-24T15:33:06.469 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.469 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.469 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:06.469 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252316+0000", 2024-04-24T15:33:06.469 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5788000000000003e-05 2024-04-24T15:33:06.469 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:06.469 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:06.469 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:06.469 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.469 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.470 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18022 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 13066240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:06.470 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.209411+0000", 2024-04-24T15:33:06.470 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 32.190206856000003, 2024-04-24T15:33:06.470 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043016616000000001, 2024-04-24T15:33:06.470 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:06.470 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:06.470 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:06.470 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:06.470 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:06.470 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18022 2024-04-24T15:33:06.470 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.470 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:06.470 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.471 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:06.471 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209411+0000", 2024-04-24T15:33:06.471 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:06.471 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.471 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.471 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:06.471 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209411+0000", 2024-04-24T15:33:06.471 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:06.471 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.471 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.471 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:06.471 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209143+0000", 2024-04-24T15:33:06.471 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999732 2024-04-24T15:33:06.471 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.471 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.471 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:06.472 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209415+0000", 2024-04-24T15:33:06.472 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000271873 2024-04-24T15:33:06.472 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.472 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.472 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:06.472 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209423+0000", 2024-04-24T15:33:06.472 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3080000000000008e-06 2024-04-24T15:33:06.472 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.472 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.472 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:06.472 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209428+0000", 2024-04-24T15:33:06.472 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.5040000000000002e-06 2024-04-24T15:33:06.472 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.472 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.472 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:06.473 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252339+0000", 2024-04-24T15:33:06.473 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042910419999999998 2024-04-24T15:33:06.473 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.473 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.473 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:06.473 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252380+0000", 2024-04-24T15:33:06.473 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0991000000000001e-05 2024-04-24T15:33:06.473 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.473 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.473 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:06.473 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252427+0000", 2024-04-24T15:33:06.473 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.7647e-05 2024-04-24T15:33:06.474 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:06.474 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:06.474 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:06.474 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.474 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.474 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18023 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 13107200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:06.474 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.209594+0000", 2024-04-24T15:33:06.474 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 32.190023343, 2024-04-24T15:33:06.474 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042938543000000003, 2024-04-24T15:33:06.474 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:06.474 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:06.475 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:06.475 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:06.475 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:06.475 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18023 2024-04-24T15:33:06.475 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.475 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:06.475 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.475 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:06.475 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209594+0000", 2024-04-24T15:33:06.475 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:06.475 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.476 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.476 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:06.476 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209594+0000", 2024-04-24T15:33:06.476 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:06.476 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.476 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.476 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:06.476 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209440+0000", 2024-04-24T15:33:06.476 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999846 2024-04-24T15:33:06.476 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.476 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.476 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:06.476 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209598+0000", 2024-04-24T15:33:06.476 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00015731100000000001 2024-04-24T15:33:06.476 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.477 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.477 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:06.477 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209604+0000", 2024-04-24T15:33:06.477 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.2910000000000004e-06 2024-04-24T15:33:06.477 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.477 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.477 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:06.477 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209607+0000", 2024-04-24T15:33:06.477 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5389999999999999e-06 2024-04-24T15:33:06.477 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.477 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.477 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:06.477 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252449+0000", 2024-04-24T15:33:06.477 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042841833000000003 2024-04-24T15:33:06.478 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.478 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.478 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:06.478 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252489+0000", 2024-04-24T15:33:06.478 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9898000000000003e-05 2024-04-24T15:33:06.478 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.478 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.478 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:06.478 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252533+0000", 2024-04-24T15:33:06.478 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3702999999999998e-05 2024-04-24T15:33:06.478 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:06.478 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:06.478 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:06.479 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.479 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.479 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18025 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 13189120~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:06.479 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.209832+0000", 2024-04-24T15:33:06.479 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 32.189785841999999, 2024-04-24T15:33:06.479 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042913556999999998, 2024-04-24T15:33:06.479 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:06.479 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:06.479 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:06.479 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:06.479 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:06.479 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18025 2024-04-24T15:33:06.480 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.480 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:06.480 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.480 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:06.480 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209832+0000", 2024-04-24T15:33:06.480 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:06.480 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.480 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.480 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:06.480 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209832+0000", 2024-04-24T15:33:06.480 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:06.480 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.480 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.481 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:06.481 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209774+0000", 2024-04-24T15:33:06.481 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999418 2024-04-24T15:33:06.481 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.481 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.481 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:06.481 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209835+0000", 2024-04-24T15:33:06.481 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.1377000000000005e-05 2024-04-24T15:33:06.481 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.481 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.481 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:06.481 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209843+0000", 2024-04-24T15:33:06.481 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.2330000000000006e-06 2024-04-24T15:33:06.482 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.482 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.482 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:06.482 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209848+0000", 2024-04-24T15:33:06.482 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6149999999999999e-06 2024-04-24T15:33:06.482 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.482 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.482 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:06.482 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252658+0000", 2024-04-24T15:33:06.482 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042810002 2024-04-24T15:33:06.482 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.482 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.482 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:06.483 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252697+0000", 2024-04-24T15:33:06.483 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8757999999999997e-05 2024-04-24T15:33:06.483 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.483 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.483 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:06.483 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252745+0000", 2024-04-24T15:33:06.483 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.863e-05 2024-04-24T15:33:06.483 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:06.483 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:06.483 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:06.483 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.483 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.483 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:24749 4.8 4:137f5701:::rbd_header.114ae15b9a20:head [watch reconnect cookie 140672595753696 gen 11] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-04-24T15:33:06.483 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:37.628703+0000", 2024-04-24T15:33:06.484 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 28.770914222999998, 2024-04-24T15:33:06.484 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0102918700000001, 2024-04-24T15:33:06.484 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:06.484 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:06.484 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:06.484 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:06.484 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:06.484 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 24749 2024-04-24T15:33:06.484 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.484 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:06.484 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.484 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:06.484 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628703+0000", 2024-04-24T15:33:06.484 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:06.484 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.485 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.485 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:06.485 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628703+0000", 2024-04-24T15:33:06.485 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:06.485 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.485 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.485 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:06.485 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628703+0000", 2024-04-24T15:33:06.485 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999999 2024-04-24T15:33:06.485 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.486 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.486 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:06.486 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628708+0000", 2024-04-24T15:33:06.486 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.4589999999999997e-06 2024-04-24T15:33:06.486 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.486 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.486 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:06.486 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628721+0000", 2024-04-24T15:33:06.486 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.2601e-05 2024-04-24T15:33:06.486 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.486 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.487 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:06.487 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628728+0000", 2024-04-24T15:33:06.487 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.9059999999999996e-06 2024-04-24T15:33:06.487 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.487 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.487 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:06.487 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628789+0000", 2024-04-24T15:33:06.487 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.1672000000000002e-05 2024-04-24T15:33:06.487 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.487 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.487 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting_for_map not empty", 2024-04-24T15:33:06.487 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628797+0000", 2024-04-24T15:33:06.487 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.8189999999999992e-06 2024-04-24T15:33:06.487 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.488 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.488 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:06.488 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.631255+0000", 2024-04-24T15:33:06.488 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0024577610000000001 2024-04-24T15:33:06.488 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.488 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.488 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for peered", 2024-04-24T15:33:06.488 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.631263+0000", 2024-04-24T15:33:06.488 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.4109999999999996e-06 2024-04-24T15:33:06.488 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.488 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.488 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:06.488 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:38.637560+0000", 2024-04-24T15:33:06.488 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0062970339999999 2024-04-24T15:33:06.489 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.489 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.489 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:06.489 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:38.637747+0000", 2024-04-24T15:33:06.489 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00018693400000000001 2024-04-24T15:33:06.489 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.489 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.489 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:06.489 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:38.638995+0000", 2024-04-24T15:33:06.489 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0012481300000000001 2024-04-24T15:33:06.489 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:06.489 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:06.489 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:06.489 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.490 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.490 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:24750 4.8 4:137f5701:::rbd_header.114ae15b9a20:head [notify cookie 140672594434416] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-04-24T15:33:06.490 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:37.628736+0000", 2024-04-24T15:33:06.490 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 28.770881963000001, 2024-04-24T15:33:06.490 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0096094309999999, 2024-04-24T15:33:06.490 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:06.490 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:06.490 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:06.490 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:06.490 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:06.490 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 24750 2024-04-24T15:33:06.490 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.491 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:06.491 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.491 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:06.491 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628736+0000", 2024-04-24T15:33:06.491 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:06.491 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.491 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.491 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:06.491 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628736+0000", 2024-04-24T15:33:06.491 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:06.491 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.491 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.491 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:06.492 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628735+0000", 2024-04-24T15:33:06.492 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:06.492 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.492 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.492 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:06.492 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628739+0000", 2024-04-24T15:33:06.492 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5109999999999999e-06 2024-04-24T15:33:06.492 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.492 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.492 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:06.492 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628745+0000", 2024-04-24T15:33:06.492 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.8300000000000001e-06 2024-04-24T15:33:06.492 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.492 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.493 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:06.493 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628749+0000", 2024-04-24T15:33:06.493 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.7110000000000003e-06 2024-04-24T15:33:06.493 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.493 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.493 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:06.493 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628826+0000", 2024-04-24T15:33:06.493 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.6678e-05 2024-04-24T15:33:06.493 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.493 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.494 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting_for_map not empty", 2024-04-24T15:33:06.494 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628834+0000", 2024-04-24T15:33:06.494 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.6190000000000002e-06 2024-04-24T15:33:06.494 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.494 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.494 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:06.494 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.631305+0000", 2024-04-24T15:33:06.494 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0024714400000000001 2024-04-24T15:33:06.494 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.494 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.495 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for peered", 2024-04-24T15:33:06.495 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.631313+0000", 2024-04-24T15:33:06.495 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.6529999999999994e-06 2024-04-24T15:33:06.495 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.495 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.495 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:06.495 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:38.638068+0000", 2024-04-24T15:33:06.495 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0067548369999999 2024-04-24T15:33:06.495 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.495 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.496 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:06.496 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:38.638158+0000", 2024-04-24T15:33:06.496 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.0079000000000003e-05 2024-04-24T15:33:06.496 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:06.496 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:06.496 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:06.496 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:38.638345+0000", 2024-04-24T15:33:06.496 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00018757100000000001 2024-04-24T15:33:06.496 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:06.496 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:06.497 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:06.497 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:06.497 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:06.497 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:06.500 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:06.499+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:06.500 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-24T15:33:06.501 INFO:teuthology.orchestra.run.smithi096.stdout:3715 trunc from 0x14e841a to 0xcb168b2 2024-04-24T15:33:06.501 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:06.501 INFO:teuthology.orchestra.run.smithi096.stdout:3718 writesame 0x70c72ed thru 0x70c793f data_size 0x1(0x653 bytes) 2024-04-24T15:33:06.502 INFO:teuthology.orchestra.run.smithi096.stdout:3720 write 0xeb8351f thru 0xeb8d598 (0xa07a bytes) 2024-04-24T15:33:06.502 INFO:teuthology.orchestra.run.smithi096.stdout:3721 writesame 0xd8a015b thru 0xd8ad02a data_size 0x18(0xced0 bytes) 2024-04-24T15:33:06.502 INFO:teuthology.orchestra.run.smithi096.stdout:3722 trunc from 0xeb8d599 to 0x8de7522 2024-04-24T15:33:06.502 INFO:teuthology.orchestra.run.smithi096.stdout:3723 read 0x1b93ac1 thru 0x1b99e54 (0x6394 bytes) 2024-04-24T15:33:06.502 INFO:teuthology.orchestra.run.smithi096.stdout:3724 write 0x1f6206 thru 0x1f68b8 (0x6b3 bytes) 2024-04-24T15:33:06.502 INFO:teuthology.orchestra.run.smithi096.stdout:3725 write 0x4c95482 thru 0x4ca26a2 (0xd221 bytes) 2024-04-24T15:33:06.502 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:06.502 INFO:teuthology.orchestra.run.smithi096.stdout:3728 writesame 0x6766bc2 thru 0x6772860 data_size 0x1(0xbc9f bytes) 2024-04-24T15:33:06.502 INFO:teuthology.orchestra.run.smithi096.stdout:3729 read 0x154cd71 thru 0x1559baf (0xce3f bytes) 2024-04-24T15:33:06.502 INFO:teuthology.orchestra.run.smithi096.stdout:3733 write 0xdfa8190 thru 0xdfb4610 (0xc481 bytes) 2024-04-24T15:33:06.502 INFO:teuthology.orchestra.run.smithi096.stdout:3734 write 0x85eb139 thru 0x85f5b69 (0xaa31 bytes) 2024-04-24T15:33:06.502 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:06.502 INFO:teuthology.orchestra.run.smithi096.stdout:3737 write 0x5803582 thru 0x5812716 (0xf195 bytes) 2024-04-24T15:33:06.503 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:06.503 INFO:teuthology.orchestra.run.smithi096.stdout:3741 trunc from 0xdfb4611 to 0x6262b47 2024-04-24T15:33:06.516 INFO:teuthology.orchestra.run.smithi096.stdout:3743 write 0xa4707c5 thru 0xa47676f (0x5fab bytes) 2024-04-24T15:33:06.518 INFO:teuthology.orchestra.run.smithi096.stdout:3744 writesame 0x5447655 thru 0x54554a4 data_size 0x2(0xde50 bytes) 2024-04-24T15:33:06.533 INFO:teuthology.orchestra.run.smithi096.stdout:3746 trunc from 0xa476770 to 0x85b8d19 2024-04-24T15:33:06.541 INFO:teuthology.orchestra.run.smithi096.stdout:3747 write 0xc9236 thru 0xd16db (0x84a6 bytes) 2024-04-24T15:33:06.542 INFO:teuthology.orchestra.run.smithi096.stdout:3748 read 0x8097892 thru 0x80a73ab (0xfb1a bytes) 2024-04-24T15:33:06.547 INFO:teuthology.orchestra.run.smithi096.stdout:3749 trunc from 0x85b8d19 to 0xbceb2b9 2024-04-24T15:33:06.552 INFO:teuthology.orchestra.run.smithi096.stdout:3750 read 0x2cac030 thru 0x2cae799 (0x276a bytes) 2024-04-24T15:33:06.552 INFO:teuthology.orchestra.run.smithi096.stdout:3751 read 0x3913ea8 thru 0x391a815 (0x696e bytes) 2024-04-24T15:33:06.553 INFO:teuthology.orchestra.run.smithi096.stdout:3755 read 0xb933480 thru 0xb93719e (0x3d1f bytes) 2024-04-24T15:33:06.560 INFO:teuthology.orchestra.run.smithi096.stdout:3756 trunc from 0xbceb2b9 to 0x5d65479 2024-04-24T15:33:06.570 INFO:teuthology.orchestra.run.smithi096.stdout:3757 read 0x15b281a thru 0x15b3dd1 (0x15b8 bytes) 2024-04-24T15:33:06.570 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:06.570 INFO:teuthology.orchestra.run.smithi096.stdout:3760 read 0x5a646a6 thru 0x5a70128 (0xba83 bytes) 2024-04-24T15:33:06.570 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:06.570 INFO:teuthology.orchestra.run.smithi096.stdout:3765 punch from 0x66b98d to 0x670dec, (0x545f bytes) 2024-04-24T15:33:06.570 INFO:teuthology.orchestra.run.smithi096.stdout:3769 punch from 0xb18e3 to 0xbe0fc, (0xc819 bytes) 2024-04-24T15:33:06.570 INFO:teuthology.orchestra.run.smithi096.stdout:3770 clone 18 order 22 su 32768 sc 13 2024-04-24T15:33:06.598 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:06.600+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:06.634 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:06.634 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:33:06.634 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-04-24T15:33:06.635 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:06.644 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-24T15:33:06.699 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:33:06.700+0000 7f23abcf6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 44021) UID: 0 2024-04-24T15:33:06.795 INFO:teuthology.orchestra.run.smithi096.stderr:reweighted osd.3 to 1 (10000) 2024-04-24T15:33:06.800 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:33:06.801+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:33:06.808 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-04-24T15:33:06.814 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:06.814 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:33:06.814 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-04-24T15:33:06.814 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-04-24T15:33:06.814 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:06.827 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-24T15:33:06.881 INFO:teuthology.orchestra.run.smithi096.stdout:truncating image image_client.0-clone17 from 0x5d65479 (overlap 0x8defc0) to 0x718c99 2024-04-24T15:33:06.900 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:33:06.902+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:33:06.973 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #16, image image_client.0-clone16 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent17 2024-04-24T15:33:06.985 INFO:teuthology.orchestra.run.smithi096.stdout:3774 write 0x80a4749 thru 0x80a53f8 (0xcb0 bytes) 2024-04-24T15:33:06.988 INFO:teuthology.orchestra.run.smithi096.stdout:3775 read 0x4bbae22 thru 0x4bbce2d (0x200c bytes) 2024-04-24T15:33:06.989 INFO:teuthology.orchestra.run.smithi096.stdout:3776 read 0x4c41403 thru 0x4c51195 (0xfd93 bytes) 2024-04-24T15:33:07.001 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:07.002+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:07.002 INFO:teuthology.orchestra.run.smithi096.stdout:3777 write 0xdbb5895 thru 0xdbc52c2 (0xfa2e bytes) 2024-04-24T15:33:07.009 INFO:teuthology.orchestra.run.smithi096.stdout:3778 write 0xd0d3da7 thru 0xd0d8ade (0x4d38 bytes) 2024-04-24T15:33:07.012 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:07.012 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-04-24T15:33:07.012 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-04-24T15:33:07.012 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-04-24T15:33:07.013 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.013 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:18 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'8, mlcod=15'5)", 2024-04-24T15:33:07.013 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:00.973932+0000", 2024-04-24T15:33:07.013 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 66.038632845999999, 2024-04-24T15:33:07.013 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0064286839999999996, 2024-04-24T15:33:07.013 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:07.013 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:07.013 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:07.013 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.013 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:07.013 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.973932+0000", 2024-04-24T15:33:07.013 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:07.014 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.014 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.014 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:07.014 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.973932+0000", 2024-04-24T15:33:07.014 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:07.014 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.014 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.014 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:07.014 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.973931+0000", 2024-04-24T15:33:07.014 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:33:07.014 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.014 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.014 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:07.014 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.973938+0000", 2024-04-24T15:33:07.015 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.029e-06 2024-04-24T15:33:07.015 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.015 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.015 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:07.015 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.973948+0000", 2024-04-24T15:33:07.015 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9129999999999998e-06 2024-04-24T15:33:07.015 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.015 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.015 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:07.015 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.973952+0000", 2024-04-24T15:33:07.015 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3069999999999996e-06 2024-04-24T15:33:07.015 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.015 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.015 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:07.015 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.973986+0000", 2024-04-24T15:33:07.016 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3643000000000003e-05 2024-04-24T15:33:07.016 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.016 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.016 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:07.016 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.974006+0000", 2024-04-24T15:33:07.016 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9658999999999999e-05 2024-04-24T15:33:07.016 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.016 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.016 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:07.016 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.980330+0000", 2024-04-24T15:33:07.016 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0063243479999999996 2024-04-24T15:33:07.016 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.016 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.017 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:07.017 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.980361+0000", 2024-04-24T15:33:07.017 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0527000000000001e-05 2024-04-24T15:33:07.017 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:07.017 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:07.017 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:07.017 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.017 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.017 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:21 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'11, mlcod=15'8)", 2024-04-24T15:33:07.017 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:00.986502+0000", 2024-04-24T15:33:07.017 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 66.026062913000004, 2024-04-24T15:33:07.017 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0062008819999999996, 2024-04-24T15:33:07.017 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:07.017 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:07.017 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:07.017 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.018 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:07.018 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.986502+0000", 2024-04-24T15:33:07.018 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:07.018 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.018 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.018 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:07.018 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.986502+0000", 2024-04-24T15:33:07.018 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:07.018 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.018 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.018 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:07.018 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.986501+0000", 2024-04-24T15:33:07.018 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:07.019 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.019 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.019 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:07.019 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.986639+0000", 2024-04-24T15:33:07.019 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013731400000000001 2024-04-24T15:33:07.019 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.019 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.019 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:07.019 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.986649+0000", 2024-04-24T15:33:07.019 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.041e-05 2024-04-24T15:33:07.019 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.019 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.020 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:07.020 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.986654+0000", 2024-04-24T15:33:07.020 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2000000000000002e-06 2024-04-24T15:33:07.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.020 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.020 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:07.020 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.986678+0000", 2024-04-24T15:33:07.020 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3731e-05 2024-04-24T15:33:07.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.020 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.020 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:07.020 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.986691+0000", 2024-04-24T15:33:07.020 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3078000000000001e-05 2024-04-24T15:33:07.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.021 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.021 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:07.021 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.992666+0000", 2024-04-24T15:33:07.021 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0059751650000000002 2024-04-24T15:33:07.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.021 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.021 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:07.021 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.992703+0000", 2024-04-24T15:33:07.021 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6622000000000003e-05 2024-04-24T15:33:07.021 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:07.021 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:07.021 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:07.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.021 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.022 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:22 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'12, mlcod=15'8)", 2024-04-24T15:33:07.022 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:00.987102+0000", 2024-04-24T15:33:07.022 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 66.025463067000004, 2024-04-24T15:33:07.022 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0056291650000000002, 2024-04-24T15:33:07.022 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:07.022 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:07.022 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:07.022 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.022 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:07.022 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.987102+0000", 2024-04-24T15:33:07.022 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:07.022 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.022 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.022 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:07.023 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.987102+0000", 2024-04-24T15:33:07.023 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:07.023 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.023 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.023 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:07.023 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.987101+0000", 2024-04-24T15:33:07.023 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:07.023 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.023 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.023 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:07.024 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.987106+0000", 2024-04-24T15:33:07.024 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0969999999999996e-06 2024-04-24T15:33:07.024 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.024 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.024 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:07.024 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.987113+0000", 2024-04-24T15:33:07.024 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6789999999999999e-06 2024-04-24T15:33:07.024 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.024 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.024 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:07.024 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.987117+0000", 2024-04-24T15:33:07.024 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6519999999999999e-06 2024-04-24T15:33:07.024 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.024 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.024 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:07.024 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.987143+0000", 2024-04-24T15:33:07.025 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6098999999999999e-05 2024-04-24T15:33:07.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.025 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.025 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:07.025 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.987157+0000", 2024-04-24T15:33:07.025 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4100999999999999e-05 2024-04-24T15:33:07.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.025 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.025 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:07.025 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.992706+0000", 2024-04-24T15:33:07.025 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0055495630000000004 2024-04-24T15:33:07.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.025 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.025 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:07.025 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.992731+0000", 2024-04-24T15:33:07.025 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4519e-05 2024-04-24T15:33:07.026 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:07.026 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:07.026 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:07.026 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.026 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.026 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:69 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'43, mlcod=15'39)", 2024-04-24T15:33:07.026 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.058504+0000", 2024-04-24T15:33:07.026 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 65.954060592999994, 2024-04-24T15:33:07.026 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.006494606, 2024-04-24T15:33:07.026 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:07.026 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:07.026 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:07.026 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.027 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:07.027 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.058504+0000", 2024-04-24T15:33:07.027 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:07.027 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.027 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.027 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:07.027 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.058504+0000", 2024-04-24T15:33:07.027 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:07.027 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.027 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.027 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:07.027 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.058503+0000", 2024-04-24T15:33:07.027 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:33:07.027 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.028 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.028 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:07.028 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.058588+0000", 2024-04-24T15:33:07.028 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4087999999999999e-05 2024-04-24T15:33:07.028 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.028 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.028 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:07.028 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.058597+0000", 2024-04-24T15:33:07.028 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7459999999999995e-06 2024-04-24T15:33:07.028 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.028 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.028 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:07.028 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.058602+0000", 2024-04-24T15:33:07.028 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6929999999999998e-06 2024-04-24T15:33:07.028 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.029 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.029 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:07.029 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.060983+0000", 2024-04-24T15:33:07.029 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023812690000000001 2024-04-24T15:33:07.029 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.029 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.029 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:07.029 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.060996+0000", 2024-04-24T15:33:07.029 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2826999999999999e-05 2024-04-24T15:33:07.029 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.029 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.029 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:07.029 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.064972+0000", 2024-04-24T15:33:07.029 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0039757890000000004 2024-04-24T15:33:07.029 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.030 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.030 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:07.030 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.064999+0000", 2024-04-24T15:33:07.030 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6917999999999999e-05 2024-04-24T15:33:07.030 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:07.030 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:07.030 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:07.030 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.030 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.030 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:70 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'44, mlcod=15'39)", 2024-04-24T15:33:07.030 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.059180+0000", 2024-04-24T15:33:07.030 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 65.953384841000002, 2024-04-24T15:33:07.030 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0058429709999999998, 2024-04-24T15:33:07.030 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:07.030 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:07.031 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:07.031 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.031 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:07.031 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.059180+0000", 2024-04-24T15:33:07.031 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:07.031 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.031 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.031 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:07.031 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.059180+0000", 2024-04-24T15:33:07.031 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:07.031 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.031 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.031 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:07.031 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.059179+0000", 2024-04-24T15:33:07.031 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:07.032 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.032 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.032 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:07.032 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.059184+0000", 2024-04-24T15:33:07.032 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.074e-06 2024-04-24T15:33:07.032 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.032 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.032 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:07.032 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.059194+0000", 2024-04-24T15:33:07.032 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6050000000000004e-06 2024-04-24T15:33:07.032 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.032 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.032 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:07.032 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.059199+0000", 2024-04-24T15:33:07.032 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2639999999999999e-06 2024-04-24T15:33:07.032 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.033 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.033 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:07.033 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.061424+0000", 2024-04-24T15:33:07.033 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0022244880000000002 2024-04-24T15:33:07.033 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.033 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.033 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:07.033 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.061444+0000", 2024-04-24T15:33:07.033 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0633e-05 2024-04-24T15:33:07.033 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.033 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.033 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:07.033 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.065001+0000", 2024-04-24T15:33:07.033 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035567670000000002 2024-04-24T15:33:07.033 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.034 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.034 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:07.034 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.065023+0000", 2024-04-24T15:33:07.034 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1724000000000001e-05 2024-04-24T15:33:07.034 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:07.034 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:07.034 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:07.034 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.034 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.034 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:154 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'108, mlcod=15'105)", 2024-04-24T15:33:07.034 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.929033+0000", 2024-04-24T15:33:07.034 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 65.083531460000003, 2024-04-24T15:33:07.035 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0068887990000000001, 2024-04-24T15:33:07.035 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:07.035 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:07.035 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:07.035 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.035 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:07.035 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929033+0000", 2024-04-24T15:33:07.035 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:07.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.035 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.035 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:07.035 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929033+0000", 2024-04-24T15:33:07.035 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:07.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.035 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.035 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:07.036 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929033+0000", 2024-04-24T15:33:07.036 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:07.036 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.036 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.036 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:07.036 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929042+0000", 2024-04-24T15:33:07.036 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5440000000000004e-06 2024-04-24T15:33:07.036 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.036 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.036 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:07.036 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929051+0000", 2024-04-24T15:33:07.036 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0580000000000004e-06 2024-04-24T15:33:07.036 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.036 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.036 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:07.036 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929055+0000", 2024-04-24T15:33:07.037 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.337e-06 2024-04-24T15:33:07.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.037 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:07.037 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929074+0000", 2024-04-24T15:33:07.037 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9724999999999998e-05 2024-04-24T15:33:07.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.037 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:07.037 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929084+0000", 2024-04-24T15:33:07.037 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7049999999999994e-06 2024-04-24T15:33:07.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.038 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.038 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:07.038 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.935895+0000", 2024-04-24T15:33:07.038 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0068104359999999996 2024-04-24T15:33:07.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.038 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.038 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:07.038 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.935922+0000", 2024-04-24T15:33:07.038 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7538999999999998e-05 2024-04-24T15:33:07.038 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:07.038 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:07.038 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:07.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.039 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:155 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'109, mlcod=15'105)", 2024-04-24T15:33:07.039 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.929398+0000", 2024-04-24T15:33:07.039 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 65.083166478999999, 2024-04-24T15:33:07.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0098754700000000008, 2024-04-24T15:33:07.039 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:07.039 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:07.039 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:07.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:07.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929398+0000", 2024-04-24T15:33:07.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:07.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:07.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929398+0000", 2024-04-24T15:33:07.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:07.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.040 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:07.040 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929398+0000", 2024-04-24T15:33:07.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:07.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.040 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:07.040 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929571+0000", 2024-04-24T15:33:07.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017281999999999999 2024-04-24T15:33:07.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.041 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:07.041 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929578+0000", 2024-04-24T15:33:07.041 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8929999999999999e-06 2024-04-24T15:33:07.041 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.041 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:07.041 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929581+0000", 2024-04-24T15:33:07.041 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1709999999999998e-06 2024-04-24T15:33:07.041 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.041 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:07.041 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929605+0000", 2024-04-24T15:33:07.041 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3938e-05 2024-04-24T15:33:07.042 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.042 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.042 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:07.042 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929618+0000", 2024-04-24T15:33:07.042 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3653e-05 2024-04-24T15:33:07.042 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.042 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.042 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:07.042 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939186+0000", 2024-04-24T15:33:07.042 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0095674260000000004 2024-04-24T15:33:07.042 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.042 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.042 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:07.042 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939274+0000", 2024-04-24T15:33:07.042 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7928000000000002e-05 2024-04-24T15:33:07.043 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:07.043 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:07.043 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:07.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.043 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.043 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:156 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'110, mlcod=15'105)", 2024-04-24T15:33:07.043 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.930050+0000", 2024-04-24T15:33:07.043 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 65.082515162999997, 2024-04-24T15:33:07.043 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0092911650000000005, 2024-04-24T15:33:07.043 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:07.043 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:07.043 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:07.043 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.043 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:07.043 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930050+0000", 2024-04-24T15:33:07.044 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:07.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:07.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930050+0000", 2024-04-24T15:33:07.044 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:07.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:07.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930049+0000", 2024-04-24T15:33:07.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:07.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.045 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.045 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:07.045 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930053+0000", 2024-04-24T15:33:07.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.98e-06 2024-04-24T15:33:07.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.045 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.045 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:07.045 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930058+0000", 2024-04-24T15:33:07.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3530000000000002e-06 2024-04-24T15:33:07.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.045 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.045 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:07.046 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930061+0000", 2024-04-24T15:33:07.046 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6970000000000002e-06 2024-04-24T15:33:07.046 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.046 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.046 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:07.046 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930082+0000", 2024-04-24T15:33:07.046 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0815e-05 2024-04-24T15:33:07.046 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.046 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.046 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:07.046 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930093+0000", 2024-04-24T15:33:07.046 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0699999999999999e-05 2024-04-24T15:33:07.046 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.046 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.047 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:07.047 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939280+0000", 2024-04-24T15:33:07.047 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0091874880000000006 2024-04-24T15:33:07.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.047 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.047 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:07.047 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939341+0000", 2024-04-24T15:33:07.047 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0608e-05 2024-04-24T15:33:07.047 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:07.047 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:07.047 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:07.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.047 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.047 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:157 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'111, mlcod=15'106)", 2024-04-24T15:33:07.047 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.930506+0000", 2024-04-24T15:33:07.047 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 65.082058305000004, 2024-04-24T15:33:07.047 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.008889246, 2024-04-24T15:33:07.048 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:07.048 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:07.048 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:07.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:07.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930506+0000", 2024-04-24T15:33:07.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:07.048 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:07.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930506+0000", 2024-04-24T15:33:07.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:07.048 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:07.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930506+0000", 2024-04-24T15:33:07.049 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:07.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.049 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:07.049 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930510+0000", 2024-04-24T15:33:07.049 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1810000000000003e-06 2024-04-24T15:33:07.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.049 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:07.049 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930516+0000", 2024-04-24T15:33:07.050 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1920000000000002e-06 2024-04-24T15:33:07.050 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.050 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.050 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:07.050 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930520+0000", 2024-04-24T15:33:07.050 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.501e-06 2024-04-24T15:33:07.050 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.050 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.050 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:07.051 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930539+0000", 2024-04-24T15:33:07.051 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8929000000000001e-05 2024-04-24T15:33:07.051 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.051 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.051 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:07.051 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930550+0000", 2024-04-24T15:33:07.051 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1168999999999999e-05 2024-04-24T15:33:07.051 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.051 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.051 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:07.052 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939343+0000", 2024-04-24T15:33:07.052 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0087932900000000005 2024-04-24T15:33:07.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.052 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:07.052 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939396+0000", 2024-04-24T15:33:07.052 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2454999999999998e-05 2024-04-24T15:33:07.052 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:07.052 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:07.052 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:07.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.052 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:158 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'112, mlcod=15'106)", 2024-04-24T15:33:07.052 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.930877+0000", 2024-04-24T15:33:07.052 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 65.081687614000003, 2024-04-24T15:33:07.052 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.008572741, 2024-04-24T15:33:07.053 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:07.053 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:07.053 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:07.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.053 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:07.053 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930877+0000", 2024-04-24T15:33:07.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:07.053 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.053 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:07.053 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930877+0000", 2024-04-24T15:33:07.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:07.053 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.053 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:07.053 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930877+0000", 2024-04-24T15:33:07.054 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:07.054 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.054 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.054 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:07.054 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930968+0000", 2024-04-24T15:33:07.054 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1638999999999995e-05 2024-04-24T15:33:07.054 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.054 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.054 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:07.054 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930975+0000", 2024-04-24T15:33:07.054 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2169999999999999e-06 2024-04-24T15:33:07.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.055 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.055 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:07.055 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930977+0000", 2024-04-24T15:33:07.055 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.791e-06 2024-04-24T15:33:07.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.055 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.055 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:07.055 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931002+0000", 2024-04-24T15:33:07.055 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4491e-05 2024-04-24T15:33:07.056 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.056 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.056 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:07.056 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931012+0000", 2024-04-24T15:33:07.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0217000000000001e-05 2024-04-24T15:33:07.056 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.056 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.056 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:07.056 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939398+0000", 2024-04-24T15:33:07.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.008385366 2024-04-24T15:33:07.056 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.056 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.056 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:07.056 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939450+0000", 2024-04-24T15:33:07.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2326e-05 2024-04-24T15:33:07.056 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:07.057 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:07.057 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:07.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.057 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:159 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'113, mlcod=15'106)", 2024-04-24T15:33:07.057 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.931313+0000", 2024-04-24T15:33:07.057 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 65.081252262000007, 2024-04-24T15:33:07.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0081994159999999993, 2024-04-24T15:33:07.057 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:07.057 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:07.057 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:07.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:07.057 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931313+0000", 2024-04-24T15:33:07.058 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:07.058 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.058 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.058 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:07.058 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931313+0000", 2024-04-24T15:33:07.058 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:07.058 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.058 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.058 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:07.058 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931312+0000", 2024-04-24T15:33:07.058 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:07.058 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.058 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.058 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:07.059 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931316+0000", 2024-04-24T15:33:07.059 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0589999999999996e-06 2024-04-24T15:33:07.059 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.059 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.059 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:07.059 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931322+0000", 2024-04-24T15:33:07.059 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2020000000000001e-06 2024-04-24T15:33:07.059 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.059 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.059 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:07.059 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931327+0000", 2024-04-24T15:33:07.059 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2440000000000004e-06 2024-04-24T15:33:07.059 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.059 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.060 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:07.060 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931348+0000", 2024-04-24T15:33:07.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1464e-05 2024-04-24T15:33:07.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.060 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.060 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:07.060 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931359+0000", 2024-04-24T15:33:07.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0699e-05 2024-04-24T15:33:07.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.060 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.060 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:07.060 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939452+0000", 2024-04-24T15:33:07.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0080931749999999993 2024-04-24T15:33:07.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.060 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.060 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:07.061 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939512+0000", 2024-04-24T15:33:07.061 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0050000000000003e-05 2024-04-24T15:33:07.061 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:07.061 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:07.061 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:07.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.061 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:2447 4.6 4:668081fa:::rbd_data.114a1f60cb66.00000000000001f0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-24T15:33:07.061 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:23.027189+0000", 2024-04-24T15:33:07.061 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 43.985376238000001, 2024-04-24T15:33:07.061 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0056741200000000004, 2024-04-24T15:33:07.061 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:07.062 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:07.062 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:07.062 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:07.062 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:07.062 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2447 2024-04-24T15:33:07.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.062 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:07.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.062 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:07.062 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027189+0000", 2024-04-24T15:33:07.062 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:07.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.062 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:07.062 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027189+0000", 2024-04-24T15:33:07.062 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:07.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.063 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.063 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:07.063 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027188+0000", 2024-04-24T15:33:07.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:33:07.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.063 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.063 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:07.063 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027194+0000", 2024-04-24T15:33:07.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0499999999999997e-06 2024-04-24T15:33:07.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.063 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.063 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:07.063 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027205+0000", 2024-04-24T15:33:07.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1619e-05 2024-04-24T15:33:07.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:07.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027212+0000", 2024-04-24T15:33:07.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4289999999999998e-06 2024-04-24T15:33:07.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:07.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027243+0000", 2024-04-24T15:33:07.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1510000000000002e-05 2024-04-24T15:33:07.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:07.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027288+0000", 2024-04-24T15:33:07.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4484000000000003e-05 2024-04-24T15:33:07.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-24T15:33:07.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027351+0000", 2024-04-24T15:33:07.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3311000000000004e-05 2024-04-24T15:33:07.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:33:07.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.029874+0000", 2024-04-24T15:33:07.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00252315 2024-04-24T15:33:07.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:33:07.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.032786+0000", 2024-04-24T15:33:07.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0029115370000000001 2024-04-24T15:33:07.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.066 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:07.066 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.032828+0000", 2024-04-24T15:33:07.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2132000000000003e-05 2024-04-24T15:33:07.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.066 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:07.066 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.032863+0000", 2024-04-24T15:33:07.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4912e-05 2024-04-24T15:33:07.066 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:07.066 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:07.067 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:07.067 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.067 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.067 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4426.0:4119 4.6 e34/33 4:6588fd86:::rbd_data.114a1f60cb66.0000000000000179:head v 34'48, mlcod=34'45)", 2024-04-24T15:33:07.067 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.936723+0000", 2024-04-24T15:33:07.067 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 42.075841797000002, 2024-04-24T15:33:07.067 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0058208640000000002, 2024-04-24T15:33:07.067 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:07.067 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:07.067 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:07.067 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.067 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:07.067 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.936723+0000", 2024-04-24T15:33:07.067 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:07.067 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.067 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.068 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:07.068 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.936723+0000", 2024-04-24T15:33:07.068 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:07.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.068 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:07.068 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.936722+0000", 2024-04-24T15:33:07.068 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:07.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.068 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:07.068 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.936735+0000", 2024-04-24T15:33:07.068 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2153999999999999e-05 2024-04-24T15:33:07.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.069 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:07.069 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.936743+0000", 2024-04-24T15:33:07.069 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9360000000000005e-06 2024-04-24T15:33:07.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.069 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:07.069 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.936749+0000", 2024-04-24T15:33:07.069 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5040000000000002e-06 2024-04-24T15:33:07.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.069 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:07.069 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.936783+0000", 2024-04-24T15:33:07.069 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3689999999999998e-05 2024-04-24T15:33:07.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.070 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.070 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:07.070 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.936803+0000", 2024-04-24T15:33:07.070 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0047999999999998e-05 2024-04-24T15:33:07.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.070 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.070 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:07.070 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.942517+0000", 2024-04-24T15:33:07.070 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057144220000000003 2024-04-24T15:33:07.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.070 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.070 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:07.071 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.942544+0000", 2024-04-24T15:33:07.071 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6735999999999999e-05 2024-04-24T15:33:07.071 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:07.071 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:07.071 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:07.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.071 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.071 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4426.0:24281 4.7 e45/43 4:e1ec41c1:::rbd_data.114ae15b9a20.0000000000000066:head v 45'58, mlcod=0'0)", 2024-04-24T15:33:07.071 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:37.194120+0000", 2024-04-24T15:33:07.071 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.818445135000001, 2024-04-24T15:33:07.071 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0091868560000000002, 2024-04-24T15:33:07.071 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:07.071 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:07.071 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:07.072 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.072 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:07.072 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.194120+0000", 2024-04-24T15:33:07.072 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:07.072 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.072 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.072 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:07.072 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.194120+0000", 2024-04-24T15:33:07.072 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:07.072 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.072 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.072 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:07.072 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.194119+0000", 2024-04-24T15:33:07.072 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:33:07.072 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.073 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.073 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:07.073 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.194125+0000", 2024-04-24T15:33:07.073 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5050000000000004e-06 2024-04-24T15:33:07.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.073 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.073 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:07.073 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.194144+0000", 2024-04-24T15:33:07.073 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8454e-05 2024-04-24T15:33:07.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.073 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.073 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:07.073 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.194153+0000", 2024-04-24T15:33:07.074 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3810000000000005e-06 2024-04-24T15:33:07.074 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.074 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.074 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:07.074 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.202288+0000", 2024-04-24T15:33:07.074 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0081342830000000008 2024-04-24T15:33:07.074 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.074 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.074 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:07.074 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.202304+0000", 2024-04-24T15:33:07.074 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6164999999999999e-05 2024-04-24T15:33:07.074 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.074 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.075 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:07.075 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.203285+0000", 2024-04-24T15:33:07.075 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00098150400000000001 2024-04-24T15:33:07.075 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.075 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.075 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:07.075 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.203306+0000", 2024-04-24T15:33:07.075 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1285e-05 2024-04-24T15:33:07.075 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:07.075 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:07.075 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:07.075 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.076 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.076 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4426.0:26385 4.1e e51/33 4:785cf1e5:::rbd_data.114a8e7208a5.0000000000000004:head v 51'141, mlcod=45'139)", 2024-04-24T15:33:07.076 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:43.542226+0000", 2024-04-24T15:33:07.076 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.470338632000001, 2024-04-24T15:33:07.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0098378319999999995, 2024-04-24T15:33:07.076 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:07.076 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:07.076 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:07.076 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.077 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:07.077 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:43.542226+0000", 2024-04-24T15:33:07.077 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:07.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.077 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:07.077 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:43.542226+0000", 2024-04-24T15:33:07.077 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:07.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.077 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:07.077 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:43.542225+0000", 2024-04-24T15:33:07.077 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:33:07.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.077 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:07.078 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:43.549287+0000", 2024-04-24T15:33:07.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0070620470000000001 2024-04-24T15:33:07.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.078 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.078 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:07.078 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:43.549299+0000", 2024-04-24T15:33:07.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1396999999999999e-05 2024-04-24T15:33:07.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.078 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.078 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:07.078 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:43.549321+0000", 2024-04-24T15:33:07.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2129000000000001e-05 2024-04-24T15:33:07.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.078 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.079 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:07.079 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:43.549361+0000", 2024-04-24T15:33:07.079 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0561e-05 2024-04-24T15:33:07.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.079 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:07.079 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:43.549386+0000", 2024-04-24T15:33:07.079 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4425000000000001e-05 2024-04-24T15:33:07.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.079 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:07.079 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:43.552024+0000", 2024-04-24T15:33:07.079 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0026388280000000002 2024-04-24T15:33:07.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.079 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:07.080 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:43.552064+0000", 2024-04-24T15:33:07.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9495e-05 2024-04-24T15:33:07.080 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:07.080 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:07.080 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:07.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.080 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.080 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4426.0:29864 4.1f e53/46 4:fce51a46:::rbd_data.114a2e5eebc.0000000000000256:head v 53'1252, mlcod=53'1250)", 2024-04-24T15:33:07.080 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:46.445862+0000", 2024-04-24T15:33:07.080 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.566702376999999, 2024-04-24T15:33:07.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010959694000000001, 2024-04-24T15:33:07.080 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:07.080 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:07.080 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:07.080 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.080 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:07.081 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.445862+0000", 2024-04-24T15:33:07.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:07.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:07.081 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.445862+0000", 2024-04-24T15:33:07.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:07.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:07.081 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.445861+0000", 2024-04-24T15:33:07.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999986 2024-04-24T15:33:07.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:07.081 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.446309+0000", 2024-04-24T15:33:07.082 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00044818000000000001 2024-04-24T15:33:07.082 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.082 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.082 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:07.082 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.446330+0000", 2024-04-24T15:33:07.082 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0754000000000002e-05 2024-04-24T15:33:07.082 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.082 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.082 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:07.082 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.446338+0000", 2024-04-24T15:33:07.082 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6320000000000008e-06 2024-04-24T15:33:07.082 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.082 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.082 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:07.083 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.448537+0000", 2024-04-24T15:33:07.083 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0021991509999999999 2024-04-24T15:33:07.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.083 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.083 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:07.083 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.448565+0000", 2024-04-24T15:33:07.083 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7949e-05 2024-04-24T15:33:07.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.083 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.083 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:07.083 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.456773+0000", 2024-04-24T15:33:07.083 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0082081930000000008 2024-04-24T15:33:07.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.083 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.083 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:07.083 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.456822+0000", 2024-04-24T15:33:07.084 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9270000000000001e-05 2024-04-24T15:33:07.084 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:07.084 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:07.084 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:07.084 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.084 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.084 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4426.0:29863 4.2 e53/43 4:45d67f64:::rbd_data.114a2e5eebc.0000000000000255:head v 53'111, mlcod=53'109)", 2024-04-24T15:33:07.084 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:46.450064+0000", 2024-04-24T15:33:07.084 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.562501079, 2024-04-24T15:33:07.084 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0067851589999999998, 2024-04-24T15:33:07.084 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:07.084 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:07.084 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:07.084 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.084 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:07.084 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.450064+0000", 2024-04-24T15:33:07.085 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:07.085 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.085 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.085 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:07.085 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.450064+0000", 2024-04-24T15:33:07.085 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:07.085 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.085 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.085 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:07.085 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.450062+0000", 2024-04-24T15:33:07.085 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999986 2024-04-24T15:33:07.085 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.085 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.085 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:07.085 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.450080+0000", 2024-04-24T15:33:07.085 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7073000000000001e-05 2024-04-24T15:33:07.086 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.086 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.086 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:07.086 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.450091+0000", 2024-04-24T15:33:07.086 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1323e-05 2024-04-24T15:33:07.086 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.086 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.086 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:07.086 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.450096+0000", 2024-04-24T15:33:07.086 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.95e-06 2024-04-24T15:33:07.087 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.087 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.087 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:07.087 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.450146+0000", 2024-04-24T15:33:07.087 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9849999999999999e-05 2024-04-24T15:33:07.087 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.087 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.087 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:07.087 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.450164+0000", 2024-04-24T15:33:07.087 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8804000000000002e-05 2024-04-24T15:33:07.087 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.087 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.088 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:07.088 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.456807+0000", 2024-04-24T15:33:07.088 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0066423760000000002 2024-04-24T15:33:07.088 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.088 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.088 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:07.088 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.456849+0000", 2024-04-24T15:33:07.088 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1992e-05 2024-04-24T15:33:07.088 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:07.088 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:07.088 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:07.088 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.088 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.089 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4426.0:35373 4.7 e62/54 4:e251aff1:::rbd_data.114a96db60f9.0000000000000017:head v 62'111, mlcod=61'108)", 2024-04-24T15:33:07.089 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:55.741562+0000", 2024-04-24T15:33:07.089 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 11.271003200999999, 2024-04-24T15:33:07.089 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0059971139999999996, 2024-04-24T15:33:07.089 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:07.089 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:07.089 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:07.089 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.089 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:07.089 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:55.741562+0000", 2024-04-24T15:33:07.089 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:07.089 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.089 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.089 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:07.089 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:55.741562+0000", 2024-04-24T15:33:07.090 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:07.090 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.090 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.090 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:07.090 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:55.741561+0000", 2024-04-24T15:33:07.090 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:07.090 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.091 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.091 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:07.091 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:55.741567+0000", 2024-04-24T15:33:07.091 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4350000000000002e-06 2024-04-24T15:33:07.091 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.092 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.092 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:07.092 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:55.741577+0000", 2024-04-24T15:33:07.092 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7419999999999992e-06 2024-04-24T15:33:07.092 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.092 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.092 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:07.092 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:55.741583+0000", 2024-04-24T15:33:07.092 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3770000000000004e-06 2024-04-24T15:33:07.092 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.092 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.093 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:07.093 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:55.742158+0000", 2024-04-24T15:33:07.093 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00057516800000000003 2024-04-24T15:33:07.093 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.093 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.093 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:07.093 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:55.742177+0000", 2024-04-24T15:33:07.094 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9250999999999999e-05 2024-04-24T15:33:07.094 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.094 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.094 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:07.094 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:55.747519+0000", 2024-04-24T15:33:07.094 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0053422519999999996 2024-04-24T15:33:07.094 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.094 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.094 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:07.094 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:55.747559+0000", 2024-04-24T15:33:07.095 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9484999999999999e-05 2024-04-24T15:33:07.095 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:07.095 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:07.095 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:07.095 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.095 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.095 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4426.0:36199 4.1a e62/41 4:5a86856e:::rbd_data.114ab4f6fc67.000000000000000c:head v 62'269, mlcod=62'267)", 2024-04-24T15:33:07.095 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:56.585759+0000", 2024-04-24T15:33:07.095 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 10.426806150999999, 2024-04-24T15:33:07.095 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057717710000000002, 2024-04-24T15:33:07.095 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:07.095 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:07.095 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:07.095 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.095 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:07.096 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:56.585759+0000", 2024-04-24T15:33:07.096 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:07.096 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.096 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.096 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:07.096 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:56.585759+0000", 2024-04-24T15:33:07.096 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:07.096 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.096 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.096 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:07.096 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:56.585758+0000", 2024-04-24T15:33:07.096 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:07.096 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.096 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.097 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:07.097 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:56.585763+0000", 2024-04-24T15:33:07.097 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9439999999999996e-06 2024-04-24T15:33:07.097 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.097 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.097 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:07.097 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:56.585770+0000", 2024-04-24T15:33:07.097 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8920000000000003e-06 2024-04-24T15:33:07.097 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.098 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.098 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:07.098 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:56.585774+0000", 2024-04-24T15:33:07.098 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0289999999999999e-06 2024-04-24T15:33:07.098 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.098 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.098 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:07.098 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:56.585820+0000", 2024-04-24T15:33:07.098 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6147999999999999e-05 2024-04-24T15:33:07.098 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.098 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.098 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:07.098 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:56.585839+0000", 2024-04-24T15:33:07.098 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.863e-05 2024-04-24T15:33:07.099 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.099 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.099 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:07.099 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:56.591488+0000", 2024-04-24T15:33:07.099 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0056494040000000002 2024-04-24T15:33:07.099 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.099 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.099 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:07.099 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:56.591530+0000", 2024-04-24T15:33:07.099 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2199e-05 2024-04-24T15:33:07.099 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:07.099 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:07.099 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:07.099 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.100 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.100 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4426.0:40766 4.a e72/57 4:514762fe:::rbd_data.114af25c4d4b.0000000000000000:head v 72'243, mlcod=72'241)", 2024-04-24T15:33:07.100 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:06.035222+0000", 2024-04-24T15:33:07.100 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.97734299400000002, 2024-04-24T15:33:07.100 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0063806660000000001, 2024-04-24T15:33:07.100 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:07.100 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:07.100 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:07.100 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.100 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:07.100 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:06.035222+0000", 2024-04-24T15:33:07.100 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:07.100 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.100 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.101 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:07.101 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:06.035222+0000", 2024-04-24T15:33:07.101 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:07.101 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.101 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.101 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:07.101 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:06.035221+0000", 2024-04-24T15:33:07.101 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:33:07.101 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.101 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.101 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:07.101 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:06.038346+0000", 2024-04-24T15:33:07.101 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0031252699999999999 2024-04-24T15:33:07.101 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.101 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.102 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:07.102 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:06.038362+0000", 2024-04-24T15:33:07.102 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6079e-05 2024-04-24T15:33:07.102 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.102 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.102 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:07.102 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:06.038369+0000", 2024-04-24T15:33:07.102 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.17e-06 2024-04-24T15:33:07.102 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.102 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.102 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:07.102 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:06.038407+0000", 2024-04-24T15:33:07.102 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7526e-05 2024-04-24T15:33:07.103 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.103 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.103 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:07.103 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:06.038424+0000", 2024-04-24T15:33:07.103 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7314000000000001e-05 2024-04-24T15:33:07.103 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.103 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.103 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:07.103 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:06.041571+0000", 2024-04-24T15:33:07.103 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0031464029999999999 2024-04-24T15:33:07.103 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:07.103 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:07.103 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:07.103 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:06.041602+0000", 2024-04-24T15:33:07.103 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1714000000000002e-05 2024-04-24T15:33:07.104 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:07.104 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:07.104 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:07.104 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:07.104 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:07.104 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:07.105 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:07.105 INFO:teuthology.orchestra.run.smithi096.stdout:3780 write 0x812a1b7 thru 0x812a2c8 (0x112 bytes) 2024-04-24T15:33:07.105 INFO:teuthology.orchestra.run.smithi096.stdout:3781 write 0x21a0b56 thru 0x21a4ed5 (0x4380 bytes) 2024-04-24T15:33:07.106 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-24T15:33:07.107 INFO:teuthology.orchestra.run.smithi096.stdout:3782 read 0x462bd64 thru 0x4637ad6 (0xbd73 bytes) 2024-04-24T15:33:07.107 INFO:teuthology.orchestra.run.smithi096.stdout:3783 read 0x2a92c4d thru 0x2a9a709 (0x7abd bytes) 2024-04-24T15:33:07.107 INFO:teuthology.orchestra.run.smithi096.stdout:3785 read 0x864a74a thru 0x8657549 (0xce00 bytes) 2024-04-24T15:33:07.107 INFO:teuthology.orchestra.run.smithi096.stdout:3787 trunc from 0xdbc52c3 to 0x6d3211 2024-04-24T15:33:07.107 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:33:07.106+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:33:07.206 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:07.207+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:07.207 INFO:teuthology.orchestra.run.smithi096.stdout:3788 write 0x352b8e9 thru 0x3533578 (0x7c90 bytes) 2024-04-24T15:33:07.213 INFO:teuthology.orchestra.run.smithi096.stdout:3791 read 0x2d8aa6b thru 0x2d94318 (0x98ae bytes) 2024-04-24T15:33:07.221 INFO:teuthology.orchestra.run.smithi096.stdout:3792 trunc from 0x3533579 to 0x9269526 2024-04-24T15:33:07.227 INFO:teuthology.orchestra.run.smithi096.stdout:3795 read 0x767758e thru 0x76818e2 (0xa355 bytes) 2024-04-24T15:33:07.235 INFO:teuthology.orchestra.run.smithi096.stdout:3796 write 0xb842e28 thru 0xb84b282 (0x845b bytes) 2024-04-24T15:33:07.239 INFO:teuthology.orchestra.run.smithi096.stdout:3797 read 0x9748d00 thru 0x974cdc6 (0x40c7 bytes) 2024-04-24T15:33:07.239 INFO:teuthology.orchestra.run.smithi096.stdout:3798 punch from 0x97c1c8c to 0x97c9cb4, (0x8028 bytes) 2024-04-24T15:33:07.248 INFO:teuthology.orchestra.run.smithi096.stdout:3799 write 0xe543f3e thru 0xe5518d9 (0xd99c bytes) 2024-04-24T15:33:07.256 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:07.256 INFO:teuthology.orchestra.run.smithi096.stdout:3802 read 0x9f57ad2 thru 0x9f597a2 (0x1cd1 bytes) 2024-04-24T15:33:07.257 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-24T15:33:07.268 INFO:teuthology.orchestra.run.smithi096.stdout:3803 trunc from 0xe5518da to 0x436f8e6 2024-04-24T15:33:07.298 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:07.298 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:33:07.298 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-04-24T15:33:07.298 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:07.300 INFO:teuthology.orchestra.run.smithi096.stdout:3804 write 0x9f3f01e thru 0x9f456c1 (0x66a4 bytes) 2024-04-24T15:33:07.304 INFO:teuthology.orchestra.run.smithi096.stdout:3805 write 0x9578511 thru 0x957d48b (0x4f7b bytes) 2024-04-24T15:33:07.307 INFO:teuthology.orchestra.run.smithi096.stdout:3806 writesame 0x58478f7 thru 0x585426b data_size 0x1(0xc975 bytes) 2024-04-24T15:33:07.307 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:33:07.307+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:33:07.309 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-24T15:33:07.326 INFO:teuthology.orchestra.run.smithi096.stdout:3807 punch from 0x75130c0 to 0x751374d, (0x68d bytes) 2024-04-24T15:33:07.326 INFO:teuthology.orchestra.run.smithi096.stdout:3808 read 0x2aea90d thru 0x2aedf87 (0x367b bytes) 2024-04-24T15:33:07.327 INFO:teuthology.orchestra.run.smithi096.stdout:3809 write 0x8ded6d6 thru 0x8dee35a (0xc85 bytes) 2024-04-24T15:33:07.330 INFO:teuthology.orchestra.run.smithi096.stdout:3810 writesame 0x62113e2 thru 0x6217959 data_size 0x44(0x6578 bytes) 2024-04-24T15:33:07.331 INFO:teuthology.orchestra.run.smithi096.stdout:3811 read 0xabb8e4 thru 0xac9df5 (0xe512 bytes) 2024-04-24T15:33:07.332 INFO:teuthology.orchestra.run.smithi096.stdout:3813 read 0x23e1005 thru 0x23e8f62 (0x7f5e bytes) 2024-04-24T15:33:07.345 INFO:teuthology.orchestra.run.smithi096.stdout:3814 write 0xe973001 thru 0xe976268 (0x3268 bytes) 2024-04-24T15:33:07.348 INFO:teuthology.orchestra.run.smithi096.stdout:3815 read 0xd14e29c thru 0xd15a065 (0xbdca bytes) 2024-04-24T15:33:07.348 INFO:teuthology.orchestra.run.smithi096.stdout:3816 read 0x7789ab1 thru 0x7797cf9 (0xe249 bytes) 2024-04-24T15:33:07.349 INFO:teuthology.orchestra.run.smithi096.stdout:3818 read 0x439f519 thru 0x43a7c88 (0x8770 bytes) 2024-04-24T15:33:07.349 INFO:teuthology.orchestra.run.smithi096.stdout:3819 read 0x8dba598 thru 0x8dc75b0 (0xd019 bytes) 2024-04-24T15:33:07.349 INFO:teuthology.orchestra.run.smithi096.stdout:3820 writesame 0xe4c1b29 thru 0xe4c4f0b data_size 0x1(0x33e3 bytes) 2024-04-24T15:33:07.357 INFO:teuthology.orchestra.run.smithi096.stdout:3821 read 0xa8c398a thru 0xa8c7eb3 (0x452a bytes) 2024-04-24T15:33:07.358 INFO:teuthology.orchestra.run.smithi096.stdout:3824 read 0x4368dc0 thru 0x4376f46 (0xe187 bytes) 2024-04-24T15:33:07.358 INFO:teuthology.orchestra.run.smithi096.stdout:3825 write 0x2f4b8c0 thru 0x2f5002f (0x4770 bytes) 2024-04-24T15:33:07.360 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:07.360 INFO:teuthology.orchestra.run.smithi096.stdout:3828 write 0xcb76b6b thru 0xcb81bb5 (0xb04b bytes) 2024-04-24T15:33:07.363 INFO:teuthology.orchestra.run.smithi096.stdout:3829 write 0xacef351 thru 0xacf3a97 (0x4747 bytes) 2024-04-24T15:33:07.369 INFO:teuthology.orchestra.run.smithi096.stdout:3830 read 0x8dadf90 thru 0x8dae7cf (0x840 bytes) 2024-04-24T15:33:07.369 INFO:teuthology.orchestra.run.smithi096.stdout:3833 write 0x9661634 thru 0x9671230 (0xfbfd bytes) 2024-04-24T15:33:07.373 INFO:teuthology.orchestra.run.smithi096.stdout:3835 writesame 0x80b3abb thru 0x80bed93 data_size 0x1(0xb2d9 bytes) 2024-04-24T15:33:07.394 INFO:teuthology.orchestra.run.smithi096.stdout:3836 write 0xab93715 thru 0xab93f2c (0x818 bytes) 2024-04-24T15:33:07.398 INFO:teuthology.orchestra.run.smithi096.stdout:3837 write 0x6607423 thru 0x6610072 (0x8c50 bytes) 2024-04-24T15:33:07.404 INFO:teuthology.orchestra.run.smithi096.stdout:3838 read 0x11c0e84 thru 0x11c457c (0x36f9 bytes) 2024-04-24T15:33:07.405 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:07.405 INFO:teuthology.orchestra.run.smithi096.stdout:3841 clone 19 order 22 su 131072 sc 9 2024-04-24T15:33:07.408 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:33:07.409+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:33:07.425 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-04-24T15:33:07.443 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-04-24T15:33:07.461 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:07.461 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:07.461 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-04-24T15:33:07.471 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-24T15:33:07.480 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-04-24T15:33:07.501 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-04-24T15:33:07.508 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:33:07.509+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:33:07.520 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-04-24T15:33:07.539 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-04-24T15:33:07.558 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-04-24T15:33:07.559 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T15:33:07.559 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T15:33:07.559 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T15:33:07.560 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T15:33:07.560 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T15:33:07.560 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T15:33:07.560 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T15:33:07.560 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T15:33:07.606 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:07.606 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:07.610 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:07.611+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:07.616 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-24T15:33:07.711 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:33:07.711+0000 7f23abcf6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 44021) UID: 0 2024-04-24T15:33:07.780 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:07.780 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:07.790 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-24T15:33:07.798 INFO:teuthology.orchestra.run.smithi096.stderr:reweighted osd.6 to 1 (10000) 2024-04-24T15:33:07.811 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-24T15:33:07.811 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:07.812+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:07.814 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-04-24T15:33:07.899 INFO:teuthology.orchestra.run.smithi096.stdout:flattening image image_client.0-clone18 2024-04-24T15:33:07.913 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:07.913+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:07.936 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:07.936 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:07.947 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-24T15:33:08.013 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:08.015+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:08.100 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:08.100 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:08.110 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-24T15:33:08.140 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #17, image image_client.0-clone17 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent18 2024-04-24T15:33:08.141 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:08.142+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:08.157 INFO:teuthology.orchestra.run.smithi096.stdout:3842 read 0xc408603 thru 0xc40f715 (0x7113 bytes) 2024-04-24T15:33:08.157 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:08.172 INFO:teuthology.orchestra.run.smithi096.stdout:3844 trunc from 0xe976269 to 0x32424f7 2024-04-24T15:33:08.216 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:33:08.218+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:33:08.247 INFO:teuthology.orchestra.run.smithi096.stdout:3848 writesame 0x21af0d2 thru 0x21b7f99 data_size 0x4(0x8ec8 bytes) 2024-04-24T15:33:08.261 INFO:teuthology.orchestra.run.smithi096.stdout:3849 read 0x8bf0ad thru 0x8cd3a5 (0xe2f9 bytes) 2024-04-24T15:33:08.261 INFO:teuthology.orchestra.run.smithi096.stdout:3850 read 0x6dca53 thru 0x6e8a6c (0xc01a bytes) 2024-04-24T15:33:08.271 INFO:teuthology.orchestra.run.smithi096.stdout:3851 write 0x7082889 thru 0x7082d40 (0x4b8 bytes) 2024-04-24T15:33:08.290 INFO:teuthology.orchestra.run.smithi096.stdout:3852 write 0xe439cda thru 0xe43cf27 (0x324e bytes) 2024-04-24T15:33:08.294 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:08.294 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:08.294 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:08.294 INFO:teuthology.orchestra.run.smithi096.stdout:3860 write 0xc7c722c thru 0xc7cba36 (0x480b bytes) 2024-04-24T15:33:08.296 INFO:teuthology.orchestra.run.smithi096.stdout:3861 punch from 0xdcbcca1 to 0xdcc483d, (0x7b9c bytes) 2024-04-24T15:33:08.297 INFO:teuthology.orchestra.run.smithi096.stdout:3862 write 0x86488c2 thru 0x8654404 (0xbb43 bytes) 2024-04-24T15:33:08.299 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:08.300 INFO:teuthology.orchestra.run.smithi096.stdout:3865 writesame 0x5aa9eb7 thru 0x5ab94a7 data_size 0x1f(0xf5f1 bytes) 2024-04-24T15:33:08.304 INFO:teuthology.orchestra.run.smithi096.stdout:3867 read 0x23c1f40 thru 0x23c36e6 (0x17a7 bytes) 2024-04-24T15:33:08.304 INFO:teuthology.orchestra.run.smithi096.stdout:3868 writesame 0x1106648 thru 0x1108e39 data_size 0x1(0x27f2 bytes) 2024-04-24T15:33:08.308 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:08.308 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:08.315 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:08.317+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:08.318 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-24T15:33:08.319 INFO:teuthology.orchestra.run.smithi096.stdout:3869 write 0x38a37eb thru 0x38afa88 (0xc29e bytes) 2024-04-24T15:33:08.321 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:08.321 INFO:teuthology.orchestra.run.smithi096.stdout:3871 read 0xd5c84d3 thru 0xd5d8154 (0xfc82 bytes) 2024-04-24T15:33:08.321 INFO:teuthology.orchestra.run.smithi096.stdout:3872 read 0xb69e0c9 thru 0xb6a3f86 (0x5ebe bytes) 2024-04-24T15:33:08.321 INFO:teuthology.orchestra.run.smithi096.stdout:3874 writesame 0x44ff485 thru 0x450cd3d data_size 0x1(0xd8b9 bytes) 2024-04-24T15:33:08.348 INFO:teuthology.orchestra.run.smithi096.stdout:3875 write 0x612b5df thru 0x612f7f0 (0x4212 bytes) 2024-04-24T15:33:08.352 INFO:teuthology.orchestra.run.smithi096.stdout:3876 read 0x79599bd thru 0x795c25b (0x289f bytes) 2024-04-24T15:33:08.352 INFO:teuthology.orchestra.run.smithi096.stdout:3877 read 0x5aea86b thru 0x5aeb18e (0x924 bytes) 2024-04-24T15:33:08.352 INFO:teuthology.orchestra.run.smithi096.stdout:3879 punch from 0x3ef6160 to 0x3efcb65, (0x6a05 bytes) 2024-04-24T15:33:08.352 INFO:teuthology.orchestra.run.smithi096.stdout:3880 writesame 0xe155dd2 thru 0xe165972 data_size 0x1(0xfba1 bytes) 2024-04-24T15:33:08.379 INFO:teuthology.orchestra.run.smithi096.stdout:3882 write 0x301260c thru 0x301a973 (0x8368 bytes) 2024-04-24T15:33:08.395 INFO:teuthology.orchestra.run.smithi096.stdout:3883 trunc from 0xe43cf28 to 0x1cb003f 2024-04-24T15:33:08.417 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:33:08.417+0000 7f23abcf6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 44021) UID: 0 2024-04-24T15:33:08.456 INFO:teuthology.orchestra.run.smithi096.stdout:3884 read 0xa4d320 thru 0xa554a0 (0x8181 bytes) 2024-04-24T15:33:08.465 INFO:teuthology.orchestra.run.smithi096.stdout:3886 write 0x4623fe3 thru 0x462a893 (0x68b1 bytes) 2024-04-24T15:33:08.467 INFO:teuthology.orchestra.run.smithi096.stdout:3887 punch from 0x1a4f6da to 0x1a5abaf, (0xb4d5 bytes) 2024-04-24T15:33:08.467 INFO:teuthology.orchestra.run.smithi096.stdout:3888 punch from 0x3eb83d0 to 0x3ec295f, (0xa58f bytes) 2024-04-24T15:33:08.471 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:08.471 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:08.480 INFO:teuthology.orchestra.run.smithi096.stdout:3890 write 0xab47032 thru 0xab54773 (0xd742 bytes) 2024-04-24T15:33:08.484 INFO:teuthology.orchestra.run.smithi096.stdout:3891 clone 20 order 21 su 16384 sc 4 2024-04-24T15:33:08.485 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-24T15:33:08.526 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:33:08.527+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:33:08.618 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:33:08.620+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:33:08.647 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:08.647 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:08.658 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-24T15:33:08.720 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:08.721+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:08.781 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:08.782 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:08.791 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-24T15:33:08.801 INFO:teuthology.orchestra.run.smithi096.stderr:nodeep-scrub is set 2024-04-24T15:33:08.802 INFO:teuthology.orchestra.run.smithi096.stderr:reweighted osd.4 to 1 (10000) 2024-04-24T15:33:08.821 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-04-24T15:33:08.821 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T15:33:08.821 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:08.821+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:08.854 INFO:teuthology.orchestra.run.smithi096.stdout:truncating image image_client.0-clone19 from 0xab54774 (overlap 0x1cb003f) to 0x19d19d2 2024-04-24T15:33:08.921 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #18, image image_client.0-clone18 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent19 2024-04-24T15:33:08.921 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:08.922+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:08.924 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:08.924 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:08.934 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-24T15:33:09.023 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:09.023+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:09.099 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:09.099 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:09.109 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-24T15:33:09.124 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:33:09.125+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:33:09.191 INFO:teuthology.orchestra.run.smithi096.stdout:3892 write 0x2530328 thru 0x253a1e5 (0x9ebe bytes) 2024-04-24T15:33:09.202 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:33:09.210 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-04-24T15:33:09.211 INFO:teuthology.orchestra.run.smithi096.stdout:3893 trunc from 0xab54774 to 0x97dbb31 2024-04-24T15:33:09.218 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":103,"stamp":"2024-04-24T15:33:08.797529+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":464198514,"num_objects":564,"num_object_clones":142,"num_object_copies":1128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":562,"num_whiteouts":172,"num_read":57251,"num_read_kb":210981,"num_write":13020,"num_write_kb":116724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1803,"num_bytes_recovered":56550993,"num_keys_recovered":162,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10248,"ondisk_log_size":10248,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":101,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":471859200,"kb_used":1545628,"kb_used_data":21976,"kb_used_omap":8,"kb_used_meta":1523575,"kb_avail":470313572,"statfs":{"total":483183820800,"available":481601097728,"internally_reserved":0,"allocated":22503424,"data_stored":149066929,"data_compressed":6032005,"data_compressed_allocated":13586432,"data_compressed_original":144353402,"omap_allocated":8520,"internal_metadata":1560141496},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.023115"},"pg_stats":[{"pgid":"4.1f","version":"70'1310","reported_seq":1516,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-24T15:33:03.887644+0000","last_change":"2024-04-24T15:32:38.636177+0000","last_active":"2024-04-24T15:33:03.887644+0000","last_peered":"2024-04-24T15:33:03.887644+0000","last_clean":"2024-04-24T15:33:03.887644+0000","last_became_active":"2024-04-24T15:32:38.635375+0000","last_became_peered":"2024-04-24T15:32:38.635375+0000","last_unstale":"2024-04-24T15:33:03.887644+0000","last_undegraded":"2024-04-24T15:33:03.887644+0000","last_fullsized":"2024-04-24T15:33:03.887644+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":1310,"log_dups_size":0,"ondisk_log_size":1310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:43:14.421657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000483979,"stat_sum":{"num_bytes":8725432,"num_objects":15,"num_object_clones":4,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":4,"num_read":6875,"num_read_kb":10318,"num_write":2391,"num_write_kb":9598,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":3834120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"67'275","reported_seq":4816,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T15:33:02.635747+0000","last_change":"2024-04-24T15:32:50.655547+0000","last_active":"2024-04-24T15:33:02.635747+0000","last_peered":"2024-04-24T15:33:02.635747+0000","last_clean":"2024-04-24T15:33:02.635747+0000","last_became_active":"2024-04-24T15:32:50.654760+0000","last_became_peered":"2024-04-24T15:32:50.654760+0000","last_unstale":"2024-04-24T15:33:02.635747+0000","last_undegraded":"2024-04-24T15:33:02.635747+0000","last_fullsized":"2024-04-24T15:33:02.635747+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":275,"log_dups_size":0,"ondisk_log_size":275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:50:53.605739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047491999999999999,"stat_sum":{"num_bytes":28802790,"num_objects":15,"num_object_clones":4,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":5,"num_read":4821,"num_read_kb":26680,"num_write":321,"num_write_kb":4173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":2247966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"70'249","reported_seq":1036,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-24T15:33:03.923754+0000","last_change":"2024-04-24T15:33:03.638857+0000","last_active":"2024-04-24T15:33:03.923754+0000","last_peered":"2024-04-24T15:33:03.923754+0000","last_clean":"2024-04-24T15:33:03.923754+0000","last_became_active":"2024-04-24T15:33:03.637986+0000","last_became_peered":"2024-04-24T15:33:03.637986+0000","last_unstale":"2024-04-24T15:33:03.923754+0000","last_undegraded":"2024-04-24T15:33:03.923754+0000","last_fullsized":"2024-04-24T15:33:03.923754+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:32:29.354937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050849800000000002,"stat_sum":{"num_bytes":4939555,"num_objects":13,"num_object_clones":2,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":4,"num_read":1382,"num_read_kb":2866,"num_write":243,"num_write_kb":1281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":580671,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"70'322","reported_seq":1266,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-24T15:33:03.796381+0000","last_change":"2024-04-24T15:32:42.208543+0000","last_active":"2024-04-24T15:33:03.796381+0000","last_peered":"2024-04-24T15:33:03.796381+0000","last_clean":"2024-04-24T15:33:03.796381+0000","last_became_active":"2024-04-24T15:32:42.207673+0000","last_became_peered":"2024-04-24T15:32:42.207673+0000","last_unstale":"2024-04-24T15:33:03.796381+0000","last_undegraded":"2024-04-24T15:33:03.796381+0000","last_fullsized":"2024-04-24T15:33:03.796381+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":322,"log_dups_size":0,"ondisk_log_size":322,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:35:35.261035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034583299999999998,"stat_sum":{"num_bytes":8579374,"num_objects":19,"num_object_clones":2,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":3,"num_read":2253,"num_read_kb":3360,"num_write":333,"num_write_kb":2362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1b","version":"70'176","reported_seq":856,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-24T15:33:03.923720+0000","last_change":"2024-04-24T15:33:02.639380+0000","last_active":"2024-04-24T15:33:03.923720+0000","last_peered":"2024-04-24T15:33:03.923720+0000","last_clean":"2024-04-24T15:33:03.923720+0000","last_became_active":"2024-04-24T15:33:02.638431+0000","last_became_peered":"2024-04-24T15:33:02.638431+0000","last_unstale":"2024-04-24T15:33:03.923720+0000","last_undegraded":"2024-04-24T15:33:03.923720+0000","last_fullsized":"2024-04-24T15:33:03.923720+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:44:40.561360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056539999999999997,"stat_sum":{"num_bytes":14909878,"num_objects":20,"num_object_clones":7,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":6,"num_read":874,"num_read_kb":3220,"num_write":173,"num_write_kb":2853,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":79,"num_bytes_recovered":2143880,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"70'299","reported_seq":4591,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-24T15:33:03.871716+0000","last_change":"2024-04-24T15:33:02.639319+0000","last_active":"2024-04-24T15:33:03.871716+0000","last_peered":"2024-04-24T15:33:03.871716+0000","last_clean":"2024-04-24T15:33:03.871716+0000","last_became_active":"2024-04-24T15:33:02.638273+0000","last_became_peered":"2024-04-24T15:33:02.638273+0000","last_unstale":"2024-04-24T15:33:03.871716+0000","last_undegraded":"2024-04-24T15:33:03.871716+0000","last_fullsized":"2024-04-24T15:33:03.871716+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":299,"log_dups_size":0,"ondisk_log_size":299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:35:38.376100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087547500000000002,"stat_sum":{"num_bytes":16721395,"num_objects":15,"num_object_clones":4,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":5,"num_read":4563,"num_read_kb":9140,"num_write":373,"num_write_kb":2822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":89,"num_bytes_recovered":1137673,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"70'314","reported_seq":1044,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-24T15:33:03.911502+0000","last_change":"2024-04-24T15:32:19.051001+0000","last_active":"2024-04-24T15:33:03.911502+0000","last_peered":"2024-04-24T15:33:03.911502+0000","last_clean":"2024-04-24T15:33:03.911502+0000","last_became_active":"2024-04-24T15:32:17.047842+0000","last_became_peered":"2024-04-24T15:32:17.047842+0000","last_unstale":"2024-04-24T15:33:03.911502+0000","last_undegraded":"2024-04-24T15:33:03.911502+0000","last_fullsized":"2024-04-24T15:33:03.911502+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:09:52.065772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048963000000000001,"stat_sum":{"num_bytes":16795920,"num_objects":19,"num_object_clones":4,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":5,"num_read":1584,"num_read_kb":3576,"num_write":301,"num_write_kb":4331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"70'334","reported_seq":711,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-24T15:33:03.796974+0000","last_change":"2024-04-24T15:33:02.636428+0000","last_active":"2024-04-24T15:33:03.796974+0000","last_peered":"2024-04-24T15:33:03.796974+0000","last_clean":"2024-04-24T15:33:03.796974+0000","last_became_active":"2024-04-24T15:33:02.635377+0000","last_became_peered":"2024-04-24T15:33:02.635377+0000","last_unstale":"2024-04-24T15:33:03.796974+0000","last_undegraded":"2024-04-24T15:33:03.796974+0000","last_fullsized":"2024-04-24T15:33:03.796974+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":334,"log_dups_size":0,"ondisk_log_size":334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:14:40.654607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033230299999999999,"stat_sum":{"num_bytes":8344973,"num_objects":25,"num_object_clones":6,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":9,"num_read":951,"num_read_kb":1942,"num_write":438,"num_write_kb":2143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":92,"num_bytes_recovered":1436100,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"70'235","reported_seq":871,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-24T15:33:03.801059+0000","last_change":"2024-04-24T15:32:50.658710+0000","last_active":"2024-04-24T15:33:03.801059+0000","last_peered":"2024-04-24T15:33:03.801059+0000","last_clean":"2024-04-24T15:33:03.801059+0000","last_became_active":"2024-04-24T15:32:50.657240+0000","last_became_peered":"2024-04-24T15:32:50.657240+0000","last_unstale":"2024-04-24T15:33:03.801059+0000","last_undegraded":"2024-04-24T15:33:03.801059+0000","last_fullsized":"2024-04-24T15:33:03.801059+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":235,"log_dups_size":0,"ondisk_log_size":235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:48:26.843070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090326899999999995,"stat_sum":{"num_bytes":3552117,"num_objects":15,"num_object_clones":2,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":4,"num_read":1126,"num_read_kb":4496,"num_write":228,"num_write_kb":1038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":45056,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"67'207","reported_seq":738,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T15:33:02.637754+0000","last_change":"2024-04-24T15:33:02.637003+0000","last_active":"2024-04-24T15:33:02.637754+0000","last_peered":"2024-04-24T15:33:02.637754+0000","last_clean":"2024-04-24T15:33:02.637754+0000","last_became_active":"2024-04-24T15:33:02.635290+0000","last_became_peered":"2024-04-24T15:33:02.635290+0000","last_unstale":"2024-04-24T15:33:02.637754+0000","last_undegraded":"2024-04-24T15:33:02.637754+0000","last_fullsized":"2024-04-24T15:33:02.637754+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:34:41.643440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000468851,"stat_sum":{"num_bytes":10632051,"num_objects":16,"num_object_clones":3,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":5,"num_read":283,"num_read_kb":5365,"num_write":207,"num_write_kb":7383,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":6125156,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":70,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T15:33:02.633097+0000","last_change":"2024-04-24T15:32:47.635776+0000","last_active":"2024-04-24T15:33:02.633097+0000","last_peered":"2024-04-24T15:33:02.633097+0000","last_clean":"2024-04-24T15:33:02.633097+0000","last_became_active":"2024-04-24T15:32:47.635578+0000","last_became_peered":"2024-04-24T15:32:47.635578+0000","last_unstale":"2024-04-24T15:33:02.633097+0000","last_undegraded":"2024-04-24T15:33:02.633097+0000","last_fullsized":"2024-04-24T15:33:02.633097+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:51:24.165624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"70'546","reported_seq":1098,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-24T15:33:03.912434+0000","last_change":"2024-04-24T15:32:50.655414+0000","last_active":"2024-04-24T15:33:03.912434+0000","last_peered":"2024-04-24T15:33:03.912434+0000","last_clean":"2024-04-24T15:33:03.912434+0000","last_became_active":"2024-04-24T15:32:50.654697+0000","last_became_peered":"2024-04-24T15:32:50.654697+0000","last_unstale":"2024-04-24T15:33:03.912434+0000","last_undegraded":"2024-04-24T15:33:03.912434+0000","last_fullsized":"2024-04-24T15:33:03.912434+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":546,"log_dups_size":0,"ondisk_log_size":546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:15:13.259423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064568399999999999,"stat_sum":{"num_bytes":24891440,"num_objects":35,"num_object_clones":9,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":13,"num_read":2061,"num_read_kb":5800,"num_write":777,"num_write_kb":5029,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":2375192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":72,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T15:33:02.633042+0000","last_change":"2024-04-24T15:32:59.745415+0000","last_active":"2024-04-24T15:33:02.633042+0000","last_peered":"2024-04-24T15:33:02.633042+0000","last_clean":"2024-04-24T15:33:02.633042+0000","last_became_active":"2024-04-24T15:32:59.745082+0000","last_became_peered":"2024-04-24T15:32:59.745082+0000","last_unstale":"2024-04-24T15:33:02.633042+0000","last_undegraded":"2024-04-24T15:33:02.633042+0000","last_fullsized":"2024-04-24T15:33:02.633042+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:13:21.504682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T15:33:02.631885+0000","last_change":"2024-04-24T15:32:59.750351+0000","last_active":"2024-04-24T15:33:02.631885+0000","last_peered":"2024-04-24T15:33:02.631885+0000","last_clean":"2024-04-24T15:33:02.631885+0000","last_became_active":"2024-04-24T15:32:59.750060+0000","last_became_peered":"2024-04-24T15:32:59.750060+0000","last_unstale":"2024-04-24T15:33:02.631885+0000","last_undegraded":"2024-04-24T15:33:02.631885+0000","last_fullsized":"2024-04-24T15:33:02.631885+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:30:23.191594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"70'247","reported_seq":3618,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-24T15:33:03.692441+0000","last_change":"2024-04-24T15:32:50.653117+0000","last_active":"2024-04-24T15:33:03.692441+0000","last_peered":"2024-04-24T15:33:03.692441+0000","last_clean":"2024-04-24T15:33:03.692441+0000","last_became_active":"2024-04-24T15:32:50.651725+0000","last_became_peered":"2024-04-24T15:32:50.651725+0000","last_unstale":"2024-04-24T15:33:03.692441+0000","last_undegraded":"2024-04-24T15:33:03.692441+0000","last_fullsized":"2024-04-24T15:33:03.692441+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":247,"log_dups_size":0,"ondisk_log_size":247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:34:39.675295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057421700000000002,"stat_sum":{"num_bytes":2990383,"num_objects":6,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":2213,"num_read_kb":2978,"num_write":239,"num_write_kb":1793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":11799,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"60'1","reported_seq":17,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T15:33:02.630869+0000","last_change":"2024-04-24T15:32:59.752573+0000","last_active":"2024-04-24T15:33:02.630869+0000","last_peered":"2024-04-24T15:33:02.630869+0000","last_clean":"2024-04-24T15:33:02.630869+0000","last_became_active":"2024-04-24T15:32:59.750157+0000","last_became_peered":"2024-04-24T15:32:59.750157+0000","last_unstale":"2024-04-24T15:33:02.630869+0000","last_undegraded":"2024-04-24T15:33:02.630869+0000","last_fullsized":"2024-04-24T15:33:02.630869+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_clean_scrub_stamp":"2024-04-24T15:32:06.972928+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:30:52.191826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":24,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T15:33:02.635639+0000","last_change":"2024-04-24T15:32:47.635254+0000","last_active":"2024-04-24T15:33:02.635639+0000","last_peered":"2024-04-24T15:33:02.635639+0000","last_clean":"2024-04-24T15:33:02.635639+0000","last_became_active":"2024-04-24T15:32:47.634970+0000","last_became_peered":"2024-04-24T15:32:47.634970+0000","last_unstale":"2024-04-24T15:33:02.635639+0000","last_undegraded":"2024-04-24T15:33:02.635639+0000","last_fullsized":"2024-04-24T15:33:02.635639+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:23:36.303998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"70'222","reported_seq":547,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-24T15:33:03.922337+0000","last_change":"2024-04-24T15:32:24.074434+0000","last_active":"2024-04-24T15:33:03.922337+0000","last_peered":"2024-04-24T15:33:03.922337+0000","last_clean":"2024-04-24T15:33:03.922337+0000","last_became_active":"2024-04-24T15:32:24.073408+0000","last_became_peered":"2024-04-24T15:32:24.073408+0000","last_unstale":"2024-04-24T15:33:03.922337+0000","last_undegraded":"2024-04-24T15:33:03.922337+0000","last_fullsized":"2024-04-24T15:33:03.922337+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":222,"log_dups_size":0,"ondisk_log_size":222,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:04:54.522860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058732700000000001,"stat_sum":{"num_bytes":28694181,"num_objects":14,"num_object_clones":4,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":3,"num_read":310,"num_read_kb":5511,"num_write":219,"num_write_kb":3897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"60'1","reported_seq":68,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T15:33:02.636374+0000","last_change":"2024-04-24T15:32:36.177281+0000","last_active":"2024-04-24T15:33:02.636374+0000","last_peered":"2024-04-24T15:33:02.636374+0000","last_clean":"2024-04-24T15:33:02.636374+0000","last_became_active":"2024-04-24T15:32:36.176816+0000","last_became_peered":"2024-04-24T15:32:36.176816+0000","last_unstale":"2024-04-24T15:33:02.636374+0000","last_undegraded":"2024-04-24T15:33:02.636374+0000","last_fullsized":"2024-04-24T15:33:02.636374+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_clean_scrub_stamp":"2024-04-24T15:32:06.972928+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:47:52.036808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":186,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T15:33:02.633556+0000","last_change":"2024-04-24T15:32:50.652024+0000","last_active":"2024-04-24T15:33:02.633556+0000","last_peered":"2024-04-24T15:33:02.633556+0000","last_clean":"2024-04-24T15:33:02.633556+0000","last_became_active":"2024-04-24T15:32:50.651718+0000","last_became_peered":"2024-04-24T15:32:50.651718+0000","last_unstale":"2024-04-24T15:33:02.633556+0000","last_undegraded":"2024-04-24T15:33:02.633556+0000","last_fullsized":"2024-04-24T15:33:02.633556+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:31:59.935287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:31:59.935287+0000","last_clean_scrub_stamp":"2024-04-24T15:31:59.935287+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:28:49.270861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":47,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T15:33:02.634609+0000","last_change":"2024-04-24T15:32:47.639623+0000","last_active":"2024-04-24T15:33:02.634609+0000","last_peered":"2024-04-24T15:33:02.634609+0000","last_clean":"2024-04-24T15:33:02.634609+0000","last_became_active":"2024-04-24T15:32:47.639356+0000","last_became_peered":"2024-04-24T15:32:47.639356+0000","last_unstale":"2024-04-24T15:33:02.634609+0000","last_undegraded":"2024-04-24T15:33:02.634609+0000","last_fullsized":"2024-04-24T15:33:02.634609+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:10:46.402999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"70'392","reported_seq":1008,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-24T15:33:03.665779+0000","last_change":"2024-04-24T15:33:02.639498+0000","last_active":"2024-04-24T15:33:03.665779+0000","last_peered":"2024-04-24T15:33:03.665779+0000","last_clean":"2024-04-24T15:33:03.665779+0000","last_became_active":"2024-04-24T15:33:02.638740+0000","last_became_peered":"2024-04-24T15:33:02.638740+0000","last_unstale":"2024-04-24T15:33:03.665779+0000","last_undegraded":"2024-04-24T15:33:03.665779+0000","last_fullsized":"2024-04-24T15:33:03.665779+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":392,"log_dups_size":0,"ondisk_log_size":392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:24:14.533112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032205999999999999,"stat_sum":{"num_bytes":15598208,"num_objects":20,"num_object_clones":4,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":4,"num_read":1657,"num_read_kb":8767,"num_write":593,"num_write_kb":4452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":6723939,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":36,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T15:33:02.633709+0000","last_change":"2024-04-24T15:32:42.209959+0000","last_active":"2024-04-24T15:33:02.633709+0000","last_peered":"2024-04-24T15:33:02.633709+0000","last_clean":"2024-04-24T15:33:02.633709+0000","last_became_active":"2024-04-24T15:32:42.209603+0000","last_became_peered":"2024-04-24T15:32:42.209603+0000","last_unstale":"2024-04-24T15:33:02.633709+0000","last_undegraded":"2024-04-24T15:33:02.633709+0000","last_fullsized":"2024-04-24T15:33:02.633709+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_clean_scrub_stamp":"2024-04-24T15:32:06.972928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:12:44.842597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":32,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T15:33:02.635835+0000","last_change":"2024-04-24T15:32:42.207668+0000","last_active":"2024-04-24T15:33:02.635835+0000","last_peered":"2024-04-24T15:33:02.635835+0000","last_clean":"2024-04-24T15:33:02.635835+0000","last_became_active":"2024-04-24T15:32:42.207408+0000","last_became_peered":"2024-04-24T15:32:42.207408+0000","last_unstale":"2024-04-24T15:33:02.635835+0000","last_undegraded":"2024-04-24T15:33:02.635835+0000","last_fullsized":"2024-04-24T15:33:02.635835+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:03:09.182470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"70'147","reported_seq":239,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-24T15:33:03.924115+0000","last_change":"2024-04-24T15:32:47.672581+0000","last_active":"2024-04-24T15:33:03.924115+0000","last_peered":"2024-04-24T15:33:03.924115+0000","last_clean":"2024-04-24T15:33:03.924115+0000","last_became_active":"2024-04-24T15:32:47.635898+0000","last_became_peered":"2024-04-24T15:32:47.635898+0000","last_unstale":"2024-04-24T15:33:03.924115+0000","last_undegraded":"2024-04-24T15:33:03.924115+0000","last_fullsized":"2024-04-24T15:33:03.924115+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":147,"log_dups_size":0,"ondisk_log_size":147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:23:43.925824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098765599999999995,"stat_sum":{"num_bytes":6908106,"num_objects":15,"num_object_clones":5,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":7,"num_read":738,"num_read_kb":5479,"num_write":138,"num_write_kb":1165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":67213,"num_keys_recovered":18,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T15:33:02.632346+0000","last_change":"2024-04-24T15:32:59.750425+0000","last_active":"2024-04-24T15:33:02.632346+0000","last_peered":"2024-04-24T15:33:02.632346+0000","last_clean":"2024-04-24T15:33:02.632346+0000","last_became_active":"2024-04-24T15:32:59.750038+0000","last_became_peered":"2024-04-24T15:32:59.750038+0000","last_unstale":"2024-04-24T15:33:02.632346+0000","last_undegraded":"2024-04-24T15:33:02.632346+0000","last_fullsized":"2024-04-24T15:33:02.632346+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_clean_scrub_stamp":"2024-04-24T15:32:06.972928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:48:12.195815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":56,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T15:33:02.632888+0000","last_change":"2024-04-24T15:32:36.178533+0000","last_active":"2024-04-24T15:33:02.632888+0000","last_peered":"2024-04-24T15:33:02.632888+0000","last_clean":"2024-04-24T15:33:02.632888+0000","last_became_active":"2024-04-24T15:32:36.178234+0000","last_became_peered":"2024-04-24T15:32:36.178234+0000","last_unstale":"2024-04-24T15:33:02.632888+0000","last_undegraded":"2024-04-24T15:33:02.632888+0000","last_fullsized":"2024-04-24T15:33:02.632888+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:00:02.842510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.4","version":"70'233","reported_seq":625,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-24T15:33:03.915211+0000","last_change":"2024-04-24T15:32:19.045228+0000","last_active":"2024-04-24T15:33:03.915211+0000","last_peered":"2024-04-24T15:33:03.915211+0000","last_clean":"2024-04-24T15:33:03.915211+0000","last_became_active":"2024-04-24T15:32:18.044036+0000","last_became_peered":"2024-04-24T15:32:18.044036+0000","last_unstale":"2024-04-24T15:33:03.915211+0000","last_undegraded":"2024-04-24T15:33:03.915211+0000","last_fullsized":"2024-04-24T15:33:03.915211+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":233,"log_dups_size":0,"ondisk_log_size":233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:48:33.464105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050914600000000003,"stat_sum":{"num_bytes":29421873,"num_objects":30,"num_object_clones":10,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":10,"num_read":607,"num_read_kb":7161,"num_write":230,"num_write_kb":5438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"70'174","reported_seq":357,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-24T15:33:03.922568+0000","last_change":"2024-04-24T15:33:02.635068+0000","last_active":"2024-04-24T15:33:03.922568+0000","last_peered":"2024-04-24T15:33:03.922568+0000","last_clean":"2024-04-24T15:33:03.922568+0000","last_became_active":"2024-04-24T15:33:02.634103+0000","last_became_peered":"2024-04-24T15:33:02.634103+0000","last_unstale":"2024-04-24T15:33:03.922568+0000","last_undegraded":"2024-04-24T15:33:03.922568+0000","last_fullsized":"2024-04-24T15:33:03.922568+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:17:55.155562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039442500000000001,"stat_sum":{"num_bytes":17175671,"num_objects":22,"num_object_clones":7,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":7,"num_read":144,"num_read_kb":184,"num_write":180,"num_write_kb":1470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":578400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":42,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T15:33:02.633589+0000","last_change":"2024-04-24T15:32:47.636966+0000","last_active":"2024-04-24T15:33:02.633589+0000","last_peered":"2024-04-24T15:33:02.633589+0000","last_clean":"2024-04-24T15:33:02.633589+0000","last_became_active":"2024-04-24T15:32:47.636739+0000","last_became_peered":"2024-04-24T15:32:47.636739+0000","last_unstale":"2024-04-24T15:33:02.633589+0000","last_undegraded":"2024-04-24T15:33:02.633589+0000","last_fullsized":"2024-04-24T15:33:02.633589+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:09:56.338464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"70'275","reported_seq":1175,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-24T15:33:03.921178+0000","last_change":"2024-04-24T15:33:02.639625+0000","last_active":"2024-04-24T15:33:03.921178+0000","last_peered":"2024-04-24T15:33:03.921178+0000","last_clean":"2024-04-24T15:33:03.921178+0000","last_became_active":"2024-04-24T15:33:02.638684+0000","last_became_peered":"2024-04-24T15:33:02.638684+0000","last_unstale":"2024-04-24T15:33:03.921178+0000","last_undegraded":"2024-04-24T15:33:03.921178+0000","last_fullsized":"2024-04-24T15:33:03.921178+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":275,"log_dups_size":0,"ondisk_log_size":275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:10:45.684566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032331100000000001,"stat_sum":{"num_bytes":9853111,"num_objects":21,"num_object_clones":6,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":7,"num_read":1373,"num_read_kb":3207,"num_write":281,"num_write_kb":2851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":128,"num_bytes_recovered":3330138,"num_keys_recovered":32,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"70'186","reported_seq":595,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-24T15:33:03.843846+0000","last_change":"2024-04-24T15:33:02.639148+0000","last_active":"2024-04-24T15:33:03.843846+0000","last_peered":"2024-04-24T15:33:03.843846+0000","last_clean":"2024-04-24T15:33:03.843846+0000","last_became_active":"2024-04-24T15:33:02.638235+0000","last_became_peered":"2024-04-24T15:33:02.638235+0000","last_unstale":"2024-04-24T15:33:03.843846+0000","last_undegraded":"2024-04-24T15:33:03.843846+0000","last_fullsized":"2024-04-24T15:33:03.843846+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:44:57.206364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034825399999999999,"stat_sum":{"num_bytes":10619079,"num_objects":20,"num_object_clones":4,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":8,"num_read":270,"num_read_kb":10440,"num_write":196,"num_write_kb":6379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":5713612,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"70'193","reported_seq":729,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-24T15:33:03.892641+0000","last_change":"2024-04-24T15:32:50.652552+0000","last_active":"2024-04-24T15:33:03.892641+0000","last_peered":"2024-04-24T15:33:03.892641+0000","last_clean":"2024-04-24T15:33:03.892641+0000","last_became_active":"2024-04-24T15:32:50.651889+0000","last_became_peered":"2024-04-24T15:32:50.651889+0000","last_unstale":"2024-04-24T15:33:03.892641+0000","last_undegraded":"2024-04-24T15:33:03.892641+0000","last_fullsized":"2024-04-24T15:33:03.892641+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":193,"log_dups_size":0,"ondisk_log_size":193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:15:10.390810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032027999999999999,"stat_sum":{"num_bytes":10290255,"num_objects":17,"num_object_clones":4,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":4,"num_read":693,"num_read_kb":4312,"num_write":187,"num_write_kb":3980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":1560706,"num_keys_recovered":19,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"70'388","reported_seq":591,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-24T15:33:03.799808+0000","last_change":"2024-04-24T15:33:03.636716+0000","last_active":"2024-04-24T15:33:03.799808+0000","last_peered":"2024-04-24T15:33:03.799808+0000","last_clean":"2024-04-24T15:33:03.799808+0000","last_became_active":"2024-04-24T15:33:03.635751+0000","last_became_peered":"2024-04-24T15:33:03.635751+0000","last_unstale":"2024-04-24T15:33:03.799808+0000","last_undegraded":"2024-04-24T15:33:03.799808+0000","last_fullsized":"2024-04-24T15:33:03.799808+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":388,"log_dups_size":0,"ondisk_log_size":388,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:30:51.876165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041177,"stat_sum":{"num_bytes":4001934,"num_objects":11,"num_object_clones":2,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":2,"num_read":1500,"num_read_kb":2683,"num_write":573,"num_write_kb":2719,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":1490474,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.c","version":"70'886","reported_seq":1333,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-24T15:33:03.927472+0000","last_change":"2024-04-24T15:33:03.640858+0000","last_active":"2024-04-24T15:33:03.927472+0000","last_peered":"2024-04-24T15:33:03.927472+0000","last_clean":"2024-04-24T15:33:03.927472+0000","last_became_active":"2024-04-24T15:33:03.638316+0000","last_became_peered":"2024-04-24T15:33:03.638316+0000","last_unstale":"2024-04-24T15:33:03.927472+0000","last_undegraded":"2024-04-24T15:33:03.927472+0000","last_fullsized":"2024-04-24T15:33:03.927472+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":886,"log_dups_size":0,"ondisk_log_size":886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:00:53.637603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066318200000000001,"stat_sum":{"num_bytes":19874272,"num_objects":14,"num_object_clones":3,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":6,"num_read":4927,"num_read_kb":5572,"num_write":1513,"num_write_kb":3296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":54,"num_bytes_recovered":1219753,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.d","version":"67'125","reported_seq":327,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T15:33:02.638053+0000","last_change":"2024-04-24T15:33:02.637311+0000","last_active":"2024-04-24T15:33:02.638053+0000","last_peered":"2024-04-24T15:33:02.638053+0000","last_clean":"2024-04-24T15:33:02.638053+0000","last_became_active":"2024-04-24T15:33:02.636600+0000","last_became_peered":"2024-04-24T15:33:02.636600+0000","last_unstale":"2024-04-24T15:33:02.638053+0000","last_undegraded":"2024-04-24T15:33:02.638053+0000","last_fullsized":"2024-04-24T15:33:02.638053+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":125,"log_dups_size":0,"ondisk_log_size":125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:59:07.207018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041245600000000001,"stat_sum":{"num_bytes":7160656,"num_objects":8,"num_object_clones":2,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":3,"num_read":33,"num_read_kb":8288,"num_write":126,"num_write_kb":4859,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":4246835,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"70'218","reported_seq":967,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-24T15:33:03.876273+0000","last_change":"2024-04-24T15:33:02.636525+0000","last_active":"2024-04-24T15:33:03.876273+0000","last_peered":"2024-04-24T15:33:03.876273+0000","last_clean":"2024-04-24T15:33:03.876273+0000","last_became_active":"2024-04-24T15:33:02.635376+0000","last_became_peered":"2024-04-24T15:33:02.635376+0000","last_unstale":"2024-04-24T15:33:03.876273+0000","last_undegraded":"2024-04-24T15:33:03.876273+0000","last_fullsized":"2024-04-24T15:33:03.876273+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":218,"log_dups_size":0,"ondisk_log_size":218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:24:51.899239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076249599999999999,"stat_sum":{"num_bytes":8744429,"num_objects":15,"num_object_clones":3,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":3,"num_read":993,"num_read_kb":2903,"num_write":215,"num_write_kb":1154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":93,"num_bytes_recovered":117055,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"70'597","reported_seq":1774,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-24T15:33:03.691511+0000","last_change":"2024-04-24T15:33:03.638821+0000","last_active":"2024-04-24T15:33:03.691511+0000","last_peered":"2024-04-24T15:33:03.691511+0000","last_clean":"2024-04-24T15:33:03.691511+0000","last_became_active":"2024-04-24T15:33:03.638156+0000","last_became_peered":"2024-04-24T15:33:03.638156+0000","last_unstale":"2024-04-24T15:33:03.691511+0000","last_undegraded":"2024-04-24T15:33:03.691511+0000","last_fullsized":"2024-04-24T15:33:03.691511+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":597,"log_dups_size":0,"ondisk_log_size":597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:07:21.801310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000652694,"stat_sum":{"num_bytes":9428197,"num_objects":18,"num_object_clones":3,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":3,"num_read":3410,"num_read_kb":4493,"num_write":735,"num_write_kb":2574,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":126,"num_bytes_recovered":630687,"num_keys_recovered":35,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"70'136","reported_seq":207,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-24T15:33:03.910373+0000","last_change":"2024-04-24T15:32:19.050772+0000","last_active":"2024-04-24T15:33:03.910373+0000","last_peered":"2024-04-24T15:33:03.910373+0000","last_clean":"2024-04-24T15:33:03.910373+0000","last_became_active":"2024-04-24T15:32:18.055440+0000","last_became_peered":"2024-04-24T15:32:18.055440+0000","last_unstale":"2024-04-24T15:33:03.910373+0000","last_undegraded":"2024-04-24T15:33:03.910373+0000","last_fullsized":"2024-04-24T15:33:03.910373+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:20:47.837497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000458918,"stat_sum":{"num_bytes":8386079,"num_objects":12,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":4,"num_read":40,"num_read_kb":244,"num_write":132,"num_write_kb":833,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.11","version":"70'308","reported_seq":536,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-24T15:33:03.921546+0000","last_change":"2024-04-24T15:32:50.658419+0000","last_active":"2024-04-24T15:33:03.921546+0000","last_peered":"2024-04-24T15:33:03.921546+0000","last_clean":"2024-04-24T15:33:03.921546+0000","last_became_active":"2024-04-24T15:32:50.657072+0000","last_became_peered":"2024-04-24T15:32:50.657072+0000","last_unstale":"2024-04-24T15:33:03.921546+0000","last_undegraded":"2024-04-24T15:33:03.921546+0000","last_fullsized":"2024-04-24T15:33:03.921546+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":308,"log_dups_size":0,"ondisk_log_size":308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:32:00.412926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057133599999999998,"stat_sum":{"num_bytes":21743524,"num_objects":19,"num_object_clones":6,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":7,"num_read":707,"num_read_kb":5558,"num_write":387,"num_write_kb":3544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":2255518,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"70'254","reported_seq":4723,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-24T15:33:03.921532+0000","last_change":"2024-04-24T15:32:19.040444+0000","last_active":"2024-04-24T15:33:03.921532+0000","last_peered":"2024-04-24T15:33:03.921532+0000","last_clean":"2024-04-24T15:33:03.921532+0000","last_became_active":"2024-04-24T15:32:18.055189+0000","last_became_peered":"2024-04-24T15:32:18.055189+0000","last_unstale":"2024-04-24T15:33:03.921532+0000","last_undegraded":"2024-04-24T15:33:03.921532+0000","last_fullsized":"2024-04-24T15:33:03.921532+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":254,"log_dups_size":0,"ondisk_log_size":254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:08:21.721301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041921100000000001,"stat_sum":{"num_bytes":39501882,"num_objects":17,"num_object_clones":6,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":6,"num_read":4506,"num_read_kb":36291,"num_write":263,"num_write_kb":7642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"70'150","reported_seq":555,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-24T15:33:03.691064+0000","last_change":"2024-04-24T15:33:02.636946+0000","last_active":"2024-04-24T15:33:03.691064+0000","last_peered":"2024-04-24T15:33:03.691064+0000","last_clean":"2024-04-24T15:33:03.691064+0000","last_became_active":"2024-04-24T15:33:02.635902+0000","last_became_peered":"2024-04-24T15:33:02.635902+0000","last_unstale":"2024-04-24T15:33:03.691064+0000","last_undegraded":"2024-04-24T15:33:03.691064+0000","last_fullsized":"2024-04-24T15:33:03.691064+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:42:19.840204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091721200000000004,"stat_sum":{"num_bytes":4247427,"num_objects":11,"num_object_clones":1,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":2,"num_read":154,"num_read_kb":1263,"num_write":151,"num_write_kb":2190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":1268965,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"70'265","reported_seq":3401,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-24T15:33:03.921539+0000","last_change":"2024-04-24T15:32:50.657999+0000","last_active":"2024-04-24T15:33:03.921539+0000","last_peered":"2024-04-24T15:33:03.921539+0000","last_clean":"2024-04-24T15:33:03.921539+0000","last_became_active":"2024-04-24T15:32:50.657302+0000","last_became_peered":"2024-04-24T15:32:50.657302+0000","last_unstale":"2024-04-24T15:33:03.921539+0000","last_undegraded":"2024-04-24T15:33:03.921539+0000","last_fullsized":"2024-04-24T15:33:03.921539+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":265,"log_dups_size":0,"ondisk_log_size":265,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:30:15.118724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060281400000000002,"stat_sum":{"num_bytes":20925052,"num_objects":22,"num_object_clones":6,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":6,"num_read":4195,"num_read_kb":6108,"num_write":262,"num_write_kb":3470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":2229438,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"70'208","reported_seq":757,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-24T15:33:03.798187+0000","last_change":"2024-04-24T15:32:50.656447+0000","last_active":"2024-04-24T15:33:03.798187+0000","last_peered":"2024-04-24T15:33:03.798187+0000","last_clean":"2024-04-24T15:33:03.798187+0000","last_became_active":"2024-04-24T15:32:50.655746+0000","last_became_peered":"2024-04-24T15:32:50.655746+0000","last_unstale":"2024-04-24T15:33:03.798187+0000","last_undegraded":"2024-04-24T15:33:03.798187+0000","last_fullsized":"2024-04-24T15:33:03.798187+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:31:51.573617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001011678,"stat_sum":{"num_bytes":28630953,"num_objects":20,"num_object_clones":7,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":6,"num_read":367,"num_read_kb":7697,"num_write":208,"num_write_kb":5868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":4590279,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"70'258","reported_seq":1072,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-24T15:33:03.922173+0000","last_change":"2024-04-24T15:32:24.075856+0000","last_active":"2024-04-24T15:33:03.922173+0000","last_peered":"2024-04-24T15:33:03.922173+0000","last_clean":"2024-04-24T15:33:03.922173+0000","last_became_active":"2024-04-24T15:32:24.075461+0000","last_became_peered":"2024-04-24T15:32:24.075461+0000","last_unstale":"2024-04-24T15:33:03.922173+0000","last_undegraded":"2024-04-24T15:33:03.922173+0000","last_fullsized":"2024-04-24T15:33:03.922173+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":258,"log_dups_size":0,"ondisk_log_size":258,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:07:28.709798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078032999999999998,"stat_sum":{"num_bytes":12517949,"num_objects":21,"num_object_clones":7,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":8,"num_read":1525,"num_read_kb":4979,"num_write":248,"num_write_kb":1721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":463608146,"num_objects":560,"num_object_clones":142,"num_object_copies":1120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":560,"num_whiteouts":170,"num_read":57135,"num_read_kb":210881,"num_write":12861,"num_write_kb":114308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1800,"num_bytes_recovered":55960625,"num_keys_recovered":162,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":19685376,"data_stored":167552349,"data_compressed":6258892,"data_compressed_allocated":14512128,"data_compressed_original":164613322,"omap_allocated":0,"internal_metadata":0},"log_size":10129,"ondisk_log_size":10129,"up":64,"acting":64,"num_store_stats":7},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":8,"acting":8,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":1771104,"data_compressed":15361,"data_compressed_allocated":110592,"data_compressed_original":1736704,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":0,"seq":55834574864,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":6,"up_from":13,"seq":55834574864,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302172,"kb_used_data":4812,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94069668,"statfs":{"total":96636764160,"available":96327340032,"internally_reserved":0,"allocated":4927488,"data_stored":36241593,"data_compressed":1415182,"data_compressed_allocated":3059712,"data_compressed_original":35242101,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 15:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.61299999999999999}]},{"osd":1,"last update":"Wed Apr 24 15:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.60499999999999998}]},{"osd":2,"last update":"Wed Apr 24 15:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.622},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.59099999999999997}]},{"osd":3,"last update":"Wed Apr 24 15:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.48099999999999998}]},{"osd":4,"last update":"Wed Apr 24 15:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.35099999999999998}]},{"osd":5,"last update":"Wed Apr 24 15:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.36299999999999999}]},{"osd":7,"last update":"Wed Apr 24 15:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.312},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.39200000000000002}]}]},{"osd":1,"up_from":0,"seq":55834574863,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":13,"seq":55834574864,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312852,"kb_used_data":3204,"kb_used_omap":1,"kb_used_meta":309630,"kb_avail":94058988,"statfs":{"total":96636764160,"available":96316403712,"internally_reserved":0,"allocated":3280896,"data_stored":19126632,"data_compressed":815747,"data_compressed_allocated":1802240,"data_compressed_original":18367241,"omap_allocated":1704,"internal_metadata":317061464},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Apr 24 15:33:00 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":3.7890000000000001,"5min":3.7890000000000001,"15min":3.7890000000000001},"last":0.251},{"interface":"front","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":2.7890000000000001,"5min":2.7890000000000001,"15min":2.7890000000000001},"last":0.224}]},{"osd":2,"last update":"Wed Apr 24 15:33:00 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":3.9950000000000001,"5min":3.9950000000000001,"15min":3.9950000000000001},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.443,"5min":0.443,"15min":0.443},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":1.8500000000000001,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":0.27600000000000002}]},{"osd":3,"last update":"Wed Apr 24 15:33:00 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":2.097,"5min":2.097,"15min":2.097},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":3.2109999999999999,"5min":3.2109999999999999,"15min":3.2109999999999999},"last":0.45400000000000001}]},{"osd":4,"last update":"Wed Apr 24 15:33:00 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":3.282,"5min":3.282,"15min":3.282},"last":0.52400000000000002}]},{"osd":5,"last update":"Wed Apr 24 15:33:00 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":2.2229999999999999,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":2.1699999999999999,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":0.44}]},{"osd":6,"last update":"Wed Apr 24 15:33:00 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":3.2509999999999999,"5min":3.2509999999999999,"15min":3.2509999999999999},"last":0.499},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":2.1909999999999998,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":0.36399999999999999}]},{"osd":7,"last update":"Wed Apr 24 15:33:00 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":2.1349999999999998,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":2.206,"5min":2.206,"15min":2.206},"last":0.48799999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309440,"kb_used_data":3892,"kb_used_omap":1,"kb_used_meta":305534,"kb_avail":94062400,"statfs":{"total":96636764160,"available":96319897600,"internally_reserved":0,"allocated":3985408,"data_stored":23676097,"data_compressed":1117040,"data_compressed_allocated":2379776,"data_compressed_original":22835570,"omap_allocated":1704,"internal_metadata":312867160},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 15:33:03 2024","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.317,"15min":0.317},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"last":0.248}]},{"osd":1,"last update":"Wed Apr 24 15:33:03 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.222},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":3.1110000000000002,"5min":3.1110000000000002,"15min":3.1110000000000002},"last":0.49399999999999999}]},{"osd":3,"last update":"Wed Apr 24 15:33:03 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":2.056,"5min":2.056,"15min":2.056},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.315}]},{"osd":4,"last update":"Wed Apr 24 15:33:03 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.42399999999999999}]},{"osd":5,"last update":"Wed Apr 24 15:33:03 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":2.1190000000000002,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":0.60699999999999998}]},{"osd":6,"last update":"Wed Apr 24 15:33:03 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.1000000000000001,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.376}]},{"osd":7,"last update":"Wed Apr 24 15:33:03 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":2.1360000000000001,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":0.56000000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574864,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307048,"kb_used_data":5600,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94064792,"statfs":{"total":96636764160,"available":96322347008,"internally_reserved":0,"allocated":5734400,"data_stored":40659981,"data_compressed":1577827,"data_compressed_allocated":3530752,"data_compressed_original":39475792,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 15:33:00 2024","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.36599999999999999}]},{"osd":1,"last update":"Wed Apr 24 15:33:00 2024","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"min":{"1min":0.187,"5min":0.187,"15min":0.187},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.40400000000000003}]},{"osd":2,"last update":"Wed Apr 24 15:33:00 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.433},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.42699999999999999}]},{"osd":4,"last update":"Wed Apr 24 15:33:00 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.51000000000000001}]},{"osd":5,"last update":"Wed Apr 24 15:33:00 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.6020000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.66000000000000003}]},{"osd":6,"last update":"Wed Apr 24 15:33:00 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.59499999999999997}]},{"osd":7,"last update":"Wed Apr 24 15:33:00 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.635,"5min":1.635,"15min":1.635},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.63200000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":314116,"kb_used_data":4468,"kb_used_omap":1,"kb_used_meta":309630,"kb_avail":94057724,"statfs":{"total":96636764160,"available":96315109376,"internally_reserved":0,"allocated":4575232,"data_stored":29362626,"data_compressed":1106209,"data_compressed_allocated":2813952,"data_compressed_original":28432698,"omap_allocated":1704,"internal_metadata":317061464},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 15:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.309},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.40000000000000002}]},{"osd":1,"last update":"Wed Apr 24 15:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.42999999999999999}]},{"osd":2,"last update":"Wed Apr 24 15:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.38800000000000001}]},{"osd":3,"last update":"Wed Apr 24 15:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":2.04}]},{"osd":5,"last update":"Wed Apr 24 15:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.373}]},{"osd":6,"last update":"Wed Apr 24 15:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.47599999999999998}]},{"osd":7,"last update":"Wed Apr 24 15:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.56000000000000005}]}]},{"osd":5,"up_from":0,"seq":55834574864,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4842,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":5677,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4842,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2293760,"data_stored":18012661,"data_compressed":764570,"data_compressed_allocated":1708032,"data_compressed_original":17703689,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2932736,"data_stored":22555216,"data_compressed":1035677,"data_compressed_allocated":2248704,"data_compressed_original":22167922,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4558848,"data_stored":39781544,"data_compressed":1423010,"data_compressed_allocated":3309568,"data_compressed_original":39078480,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":3641344,"data_stored":28765884,"data_compressed":1034014,"data_compressed_allocated":2707456,"data_compressed_original":28236090,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":204800,"data_stored":181730,"data_compressed":8376,"data_compressed_allocated":32768,"data_compressed_original":77824,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":3710976,"data_stored":33381943,"data_compressed":1255366,"data_compressed_allocated":2748416,"data_compressed_original":32821365,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2342912,"data_stored":24873371,"data_compressed":737879,"data_compressed_allocated":1757184,"data_compressed_original":24527952,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T15:33:09.221 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T15:33:09.225 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:09.225+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:09.226 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:09.227 INFO:teuthology.orchestra.run.smithi096.stdout:3897 read 0x185106f thru 0x1860d82 (0xfd14 bytes) 2024-04-24T15:33:09.237 INFO:teuthology.orchestra.run.smithi096.stdout:3898 write 0xcfc47b5 thru 0xcfce31f (0x9b6b bytes) 2024-04-24T15:33:09.245 INFO:teuthology.orchestra.run.smithi096.stdout:3899 write 0xc076617 thru 0xc07ed27 (0x8711 bytes) 2024-04-24T15:33:09.254 INFO:teuthology.orchestra.run.smithi096.stdout:3900 punch from 0x645bbeb to 0x6461cd5, (0x60ea bytes) 2024-04-24T15:33:09.262 INFO:teuthology.orchestra.run.smithi096.stdout:3901 writesame 0xee3276d thru 0xee3f597 data_size 0x49(0xce2b bytes) 2024-04-24T15:33:09.262 INFO:teuthology.orchestra.run.smithi096.stdout:3902 read 0x1e2df55 thru 0x1e3968a (0xb736 bytes) 2024-04-24T15:33:09.263 INFO:teuthology.orchestra.run.smithi096.stdout:3903 write 0x99f8786 thru 0x9a0254d (0x9dc8 bytes) 2024-04-24T15:33:09.272 INFO:teuthology.orchestra.run.smithi096.stdout:3904 write 0x99344c3 thru 0x993a633 (0x6171 bytes) 2024-04-24T15:33:09.279 INFO:teuthology.orchestra.run.smithi096.stdout:3905 punch from 0xb33da52 to 0xb33f1a8, (0x1756 bytes) 2024-04-24T15:33:09.279 INFO:teuthology.orchestra.run.smithi096.stdout:3906 punch from 0x402f149 to 0x403833b, (0x91f2 bytes) 2024-04-24T15:33:09.279 INFO:teuthology.orchestra.run.smithi096.stdout:3907 write 0xd166585 thru 0xd175c71 (0xf6ed bytes) 2024-04-24T15:33:09.280 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:09.280 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:09.291 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-24T15:33:09.291 INFO:teuthology.orchestra.run.smithi096.stdout:3908 write 0xcccfb8b thru 0xccd5ab8 (0x5f2e bytes) 2024-04-24T15:33:09.295 INFO:teuthology.orchestra.run.smithi096.stdout:3909 punch from 0x585bf6d to 0x586b107, (0xf19a bytes) 2024-04-24T15:33:09.295 INFO:teuthology.orchestra.run.smithi096.stdout:3910 read 0x3157d11 thru 0x3166514 (0xe804 bytes) 2024-04-24T15:33:09.295 INFO:teuthology.orchestra.run.smithi096.stdout:3911 read 0x2d01f9b thru 0x2d0b98b (0x99f1 bytes) 2024-04-24T15:33:09.299 INFO:teuthology.orchestra.run.smithi096.stdout:3912 trunc from 0xee3f598 to 0xb0c49a5 2024-04-24T15:33:09.316 INFO:teuthology.orchestra.run.smithi096.stdout:3913 punch from 0x25136a9 to 0x251899d, (0x52f4 bytes) 2024-04-24T15:33:09.316 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:09.317 INFO:teuthology.orchestra.run.smithi096.stdout:3917 read 0x48918dc thru 0x489dc93 (0xc3b8 bytes) 2024-04-24T15:33:09.317 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:09.317 INFO:teuthology.orchestra.run.smithi096.stdout:3919 write 0x5338df1 thru 0x533c75a (0x396a bytes) 2024-04-24T15:33:09.326 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:09.327+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:09.329 INFO:teuthology.orchestra.run.smithi096.stdout:3920 write 0xccb8658 thru 0xccbdd14 (0x56bd bytes) 2024-04-24T15:33:09.343 INFO:teuthology.orchestra.run.smithi096.stdout:3922 writesame 0xe583404 thru 0xe58bce6 data_size 0x1(0x88e3 bytes) 2024-04-24T15:33:09.344 INFO:teuthology.orchestra.run.smithi096.stdout:3923 write 0x93d87a6 thru 0x93e472a (0xbf85 bytes) 2024-04-24T15:33:09.348 INFO:teuthology.orchestra.run.smithi096.stdout:3925 read 0x7085c65 thru 0x7094d50 (0xf0ec bytes) 2024-04-24T15:33:09.348 INFO:teuthology.orchestra.run.smithi096.stdout:3926 punch from 0xdbbddea to 0xdbc7aa8, (0x9cbe bytes) 2024-04-24T15:33:09.348 INFO:teuthology.orchestra.run.smithi096.stdout:3928 read 0x5ca9e98 thru 0x5cacb58 (0x2cc1 bytes) 2024-04-24T15:33:09.348 INFO:teuthology.orchestra.run.smithi096.stdout:3930 writesame 0xe32a50e thru 0xe334901 data_size 0xe(0xa3f4 bytes) 2024-04-24T15:33:09.357 INFO:teuthology.orchestra.run.smithi096.stdout:3931 writesame 0xa2f1ad2 thru 0xa2f5f07 data_size 0x1(0x4436 bytes) 2024-04-24T15:33:09.367 INFO:teuthology.orchestra.run.smithi096.stdout:3932 read 0x29a5450 thru 0x29b1c94 (0xc845 bytes) 2024-04-24T15:33:09.367 INFO:teuthology.orchestra.run.smithi096.stdout:3933 punch from 0x6a78234 to 0x6a85b97, (0xd963 bytes) 2024-04-24T15:33:09.367 INFO:teuthology.orchestra.run.smithi096.stdout:3934 writesame 0x694dc13 thru 0x694f686 data_size 0x2(0x1a74 bytes) 2024-04-24T15:33:09.372 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:09.372 INFO:teuthology.orchestra.run.smithi096.stdout:3941 punch from 0xb2aeced to 0xb2b4cad, (0x5fc0 bytes) 2024-04-24T15:33:09.372 INFO:teuthology.orchestra.run.smithi096.stdout:3942 write 0x42439b6 thru 0x4244db7 (0x1402 bytes) 2024-04-24T15:33:09.380 INFO:teuthology.orchestra.run.smithi096.stdout:3943 writesame 0x1e03cac thru 0x1e11d9c data_size 0x37(0xe0f1 bytes) 2024-04-24T15:33:09.427 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:09.428+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:09.451 INFO:teuthology.orchestra.run.smithi096.stdout:3944 write 0x549e83c thru 0x54a9178 (0xa93d bytes) 2024-04-24T15:33:09.467 INFO:teuthology.orchestra.run.smithi096.stdout:3946 trunc from 0xe58bce7 to 0x4d9d5ba 2024-04-24T15:33:09.485 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:09.485 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:09.488 INFO:teuthology.orchestra.run.smithi096.stdout:3947 writesame 0x366d6b9 thru 0x3673a82 data_size 0x35(0x63ca bytes) 2024-04-24T15:33:09.496 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-24T15:33:09.496 INFO:teuthology.orchestra.run.smithi096.stdout:3948 punch from 0x4d79079 to 0x4d7b628, (0x25af bytes) 2024-04-24T15:33:09.496 INFO:teuthology.orchestra.run.smithi096.stdout:3949 read 0xe038c4 thru 0xe1291d (0xf05a bytes) 2024-04-24T15:33:09.497 INFO:teuthology.orchestra.run.smithi096.stdout:3950 punch from 0x3139221 to 0x313e9a3, (0x5782 bytes) 2024-04-24T15:33:09.497 INFO:teuthology.orchestra.run.smithi096.stdout:3951 read 0x4c51301 thru 0x4c52348 (0x1048 bytes) 2024-04-24T15:33:09.497 INFO:teuthology.orchestra.run.smithi096.stdout:3952 punch from 0x14b3362 to 0x14bb16f, (0x7e0d bytes) 2024-04-24T15:33:09.497 INFO:teuthology.orchestra.run.smithi096.stdout:3953 punch from 0x11f674f to 0x11fe67d, (0x7f2e bytes) 2024-04-24T15:33:09.498 INFO:teuthology.orchestra.run.smithi096.stdout:3954 read 0x436a487 thru 0x436e4a9 (0x4023 bytes) 2024-04-24T15:33:09.498 INFO:teuthology.orchestra.run.smithi096.stdout:3955 read 0x249ec thru 0x2552d (0xb42 bytes) 2024-04-24T15:33:09.498 INFO:teuthology.orchestra.run.smithi096.stdout:3956 punch from 0x1cdfc93 to 0x1cedfb8, (0xe325 bytes) 2024-04-24T15:33:09.508 INFO:teuthology.orchestra.run.smithi096.stdout:3957 write 0x92b17c4 thru 0x92bb258 (0x9a95 bytes) 2024-04-24T15:33:09.518 INFO:teuthology.orchestra.run.smithi096.stdout:3958 punch from 0xb4d2e0 to 0xb52f6b, (0x5c8b bytes) 2024-04-24T15:33:09.518 INFO:teuthology.orchestra.run.smithi096.stdout:3959 read 0x145e367 thru 0x1468b19 (0xa7b3 bytes) 2024-04-24T15:33:09.519 INFO:teuthology.orchestra.run.smithi096.stdout:3960 read 0x904798 thru 0x91207d (0xd8e6 bytes) 2024-04-24T15:33:09.519 INFO:teuthology.orchestra.run.smithi096.stdout:3961 punch from 0x50a0477 to 0x50ada6d, (0xd5f6 bytes) 2024-04-24T15:33:09.526 INFO:teuthology.orchestra.run.smithi096.stdout:3962 trunc from 0x92bb259 to 0xe4d0cd5 2024-04-24T15:33:09.528 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:33:09.529+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:33:09.531 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:09.531 INFO:teuthology.orchestra.run.smithi096.stdout:3968 write 0x8059d83 thru 0x8060fed (0x726b bytes) 2024-04-24T15:33:09.544 INFO:teuthology.orchestra.run.smithi096.stdout:3969 write 0xe664a48 thru 0xe66ef67 (0xa520 bytes) 2024-04-24T15:33:09.550 INFO:teuthology.orchestra.run.smithi096.stdout:3971 writesame 0x81276cc thru 0x8127d64 data_size 0x1(0x699 bytes) 2024-04-24T15:33:09.553 INFO:teuthology.orchestra.run.smithi096.stdout:3973 read 0x30dad77 thru 0x30e48e7 (0x9b71 bytes) 2024-04-24T15:33:09.554 INFO:teuthology.orchestra.run.smithi096.stdout:3974 read 0x6cbeb25 thru 0x6cc9d26 (0xb202 bytes) 2024-04-24T15:33:09.554 INFO:teuthology.orchestra.run.smithi096.stdout:3975 punch from 0xdf284f4 to 0xdf2c9b3, (0x44bf bytes) 2024-04-24T15:33:09.554 INFO:teuthology.orchestra.run.smithi096.stdout:3976 write 0x5678443 thru 0x567858b (0x149 bytes) 2024-04-24T15:33:09.558 INFO:teuthology.orchestra.run.smithi096.stdout:3977 writesame 0x2f2e5dd thru 0x2f3c1be data_size 0x41(0xdbe2 bytes) 2024-04-24T15:33:09.571 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-24T15:33:09.573 INFO:teuthology.orchestra.run.smithi096.stdout:3979 read 0x680913d thru 0x68135ce (0xa492 bytes) 2024-04-24T15:33:09.573 INFO:teuthology.orchestra.run.smithi096.stdout:3980 punch from 0xc45c615 to 0xc4614e1, (0x4ecc bytes) 2024-04-24T15:33:09.574 INFO:teuthology.orchestra.run.smithi096.stdout:3982 read 0x674664e thru 0x6752ffa (0xc9ad bytes) 2024-04-24T15:33:09.574 INFO:teuthology.orchestra.run.smithi096.stdout:3983 clone 21 order 18 su 32768 sc 6 2024-04-24T15:33:09.605 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:33:09.606 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-04-24T15:33:09.618 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":104,"stamp":"2024-04-24T15:33:09.563220+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":813002617,"num_objects":697,"num_object_clones":183,"num_object_copies":1394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":695,"num_whiteouts":196,"num_read":68140,"num_read_kb":318398,"num_write":14007,"num_write_kb":145000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1803,"num_bytes_recovered":56550993,"num_keys_recovered":162,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11090,"ondisk_log_size":11090,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":471859200,"kb_used":1557516,"kb_used_data":25672,"kb_used_omap":8,"kb_used_meta":1531767,"kb_avail":470301684,"statfs":{"total":483183820800,"available":481588924416,"internally_reserved":0,"allocated":26288128,"data_stored":185389135,"data_compressed":7777192,"data_compressed_allocated":16683008,"data_compressed_original":180346680,"omap_allocated":8520,"internal_metadata":1568530104},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":348804103,"num_objects":133,"num_object_clones":41,"num_object_copies":266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":133,"num_whiteouts":24,"num_read":10889,"num_read_kb":107417,"num_write":987,"num_write_kb":28276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000561"},"pg_stats":[{"pgid":"4.1f","version":"75'1317","reported_seq":1550,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-24T15:33:08.902694+0000","last_change":"2024-04-24T15:32:38.636177+0000","last_active":"2024-04-24T15:33:08.902694+0000","last_peered":"2024-04-24T15:33:08.902694+0000","last_clean":"2024-04-24T15:33:08.902694+0000","last_became_active":"2024-04-24T15:32:38.635375+0000","last_became_peered":"2024-04-24T15:32:38.635375+0000","last_unstale":"2024-04-24T15:33:08.902694+0000","last_undegraded":"2024-04-24T15:33:08.902694+0000","last_fullsized":"2024-04-24T15:33:08.902694+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":1317,"log_dups_size":0,"ondisk_log_size":1317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:43:14.421657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000483979,"stat_sum":{"num_bytes":12493752,"num_objects":17,"num_object_clones":4,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":4,"num_read":6942,"num_read_kb":10898,"num_write":2396,"num_write_kb":9634,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":3834120,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"75'338","reported_seq":4889,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-24T15:33:08.874712+0000","last_change":"2024-04-24T15:32:50.655547+0000","last_active":"2024-04-24T15:33:08.874712+0000","last_peered":"2024-04-24T15:33:08.874712+0000","last_clean":"2024-04-24T15:33:08.874712+0000","last_became_active":"2024-04-24T15:32:50.654760+0000","last_became_peered":"2024-04-24T15:32:50.654760+0000","last_unstale":"2024-04-24T15:33:08.874712+0000","last_undegraded":"2024-04-24T15:33:08.874712+0000","last_fullsized":"2024-04-24T15:33:08.874712+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":338,"log_dups_size":0,"ondisk_log_size":338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:50:53.605739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047491999999999999,"stat_sum":{"num_bytes":43502548,"num_objects":21,"num_object_clones":4,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":5,"num_read":5103,"num_read_kb":26937,"num_write":425,"num_write_kb":4382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":2247966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"75'266","reported_seq":1553,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-24T15:33:08.831187+0000","last_change":"2024-04-24T15:33:03.638857+0000","last_active":"2024-04-24T15:33:08.831187+0000","last_peered":"2024-04-24T15:33:08.831187+0000","last_clean":"2024-04-24T15:33:08.831187+0000","last_became_active":"2024-04-24T15:33:03.637986+0000","last_became_peered":"2024-04-24T15:33:03.637986+0000","last_unstale":"2024-04-24T15:33:08.831187+0000","last_undegraded":"2024-04-24T15:33:08.831187+0000","last_fullsized":"2024-04-24T15:33:08.831187+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":266,"log_dups_size":0,"ondisk_log_size":266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:32:29.354937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050849800000000002,"stat_sum":{"num_bytes":13885265,"num_objects":17,"num_object_clones":3,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":5,"num_read":1883,"num_read_kb":12288,"num_write":260,"num_write_kb":1943,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":580671,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"75'351","reported_seq":1581,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-24T15:33:08.908067+0000","last_change":"2024-04-24T15:32:42.208543+0000","last_active":"2024-04-24T15:33:08.908067+0000","last_peered":"2024-04-24T15:33:08.908067+0000","last_clean":"2024-04-24T15:33:08.908067+0000","last_became_active":"2024-04-24T15:32:42.207673+0000","last_became_peered":"2024-04-24T15:32:42.207673+0000","last_unstale":"2024-04-24T15:33:08.908067+0000","last_undegraded":"2024-04-24T15:33:08.908067+0000","last_fullsized":"2024-04-24T15:33:08.908067+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:35:35.261035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034583299999999998,"stat_sum":{"num_bytes":12313240,"num_objects":22,"num_object_clones":2,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":3,"num_read":2768,"num_read_kb":7402,"num_write":360,"num_write_kb":2428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1b","version":"73'222","reported_seq":1540,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-24T15:33:08.447330+0000","last_change":"2024-04-24T15:33:02.639380+0000","last_active":"2024-04-24T15:33:08.447330+0000","last_peered":"2024-04-24T15:33:08.447330+0000","last_clean":"2024-04-24T15:33:08.447330+0000","last_became_active":"2024-04-24T15:33:02.638431+0000","last_became_peered":"2024-04-24T15:33:02.638431+0000","last_unstale":"2024-04-24T15:33:08.447330+0000","last_undegraded":"2024-04-24T15:33:08.447330+0000","last_fullsized":"2024-04-24T15:33:08.447330+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":222,"log_dups_size":0,"ondisk_log_size":222,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:44:40.561360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056539999999999997,"stat_sum":{"num_bytes":49567279,"num_objects":27,"num_object_clones":10,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":9,"num_read":1525,"num_read_kb":22318,"num_write":213,"num_write_kb":4381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":79,"num_bytes_recovered":2143880,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"73'303","reported_seq":4771,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-24T15:33:08.912853+0000","last_change":"2024-04-24T15:33:02.639319+0000","last_active":"2024-04-24T15:33:08.912853+0000","last_peered":"2024-04-24T15:33:08.912853+0000","last_clean":"2024-04-24T15:33:08.912853+0000","last_became_active":"2024-04-24T15:33:02.638273+0000","last_became_peered":"2024-04-24T15:33:02.638273+0000","last_unstale":"2024-04-24T15:33:08.912853+0000","last_undegraded":"2024-04-24T15:33:08.912853+0000","last_fullsized":"2024-04-24T15:33:08.912853+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:35:38.376100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087547500000000002,"stat_sum":{"num_bytes":16721395,"num_objects":15,"num_object_clones":4,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":5,"num_read":4743,"num_read_kb":9311,"num_write":377,"num_write_kb":2860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":89,"num_bytes_recovered":1137673,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"74'345","reported_seq":1486,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-24T15:33:08.407715+0000","last_change":"2024-04-24T15:32:19.051001+0000","last_active":"2024-04-24T15:33:08.407715+0000","last_peered":"2024-04-24T15:33:08.407715+0000","last_clean":"2024-04-24T15:33:08.407715+0000","last_became_active":"2024-04-24T15:32:17.047842+0000","last_became_peered":"2024-04-24T15:32:17.047842+0000","last_unstale":"2024-04-24T15:33:08.407715+0000","last_undegraded":"2024-04-24T15:33:08.407715+0000","last_fullsized":"2024-04-24T15:33:08.407715+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":345,"log_dups_size":0,"ondisk_log_size":345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:09:52.065772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048963000000000001,"stat_sum":{"num_bytes":33573168,"num_objects":23,"num_object_clones":5,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":6,"num_read":2321,"num_read_kb":12329,"num_write":331,"num_write_kb":4525,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"75'343","reported_seq":878,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-24T15:33:08.898085+0000","last_change":"2024-04-24T15:33:02.636428+0000","last_active":"2024-04-24T15:33:08.898085+0000","last_peered":"2024-04-24T15:33:08.898085+0000","last_clean":"2024-04-24T15:33:08.898085+0000","last_became_active":"2024-04-24T15:33:02.635377+0000","last_became_peered":"2024-04-24T15:33:02.635377+0000","last_unstale":"2024-04-24T15:33:08.898085+0000","last_undegraded":"2024-04-24T15:33:08.898085+0000","last_fullsized":"2024-04-24T15:33:08.898085+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:14:40.654607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033230299999999999,"stat_sum":{"num_bytes":12277149,"num_objects":29,"num_object_clones":7,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":9,"num_read":1112,"num_read_kb":2616,"num_write":447,"num_write_kb":2740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":92,"num_bytes_recovered":1436100,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"75'271","reported_seq":1366,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-24T15:33:08.921929+0000","last_change":"2024-04-24T15:32:50.658710+0000","last_active":"2024-04-24T15:33:08.921929+0000","last_peered":"2024-04-24T15:33:08.921929+0000","last_clean":"2024-04-24T15:33:08.921929+0000","last_became_active":"2024-04-24T15:32:50.657240+0000","last_became_peered":"2024-04-24T15:32:50.657240+0000","last_unstale":"2024-04-24T15:33:08.921929+0000","last_undegraded":"2024-04-24T15:33:08.921929+0000","last_fullsized":"2024-04-24T15:33:08.921929+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:48:26.843070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090326899999999995,"stat_sum":{"num_bytes":10346853,"num_objects":22,"num_object_clones":3,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":4,"num_read":1933,"num_read_kb":8951,"num_write":260,"num_write_kb":1670,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":45056,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"75'212","reported_seq":797,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-24T15:33:08.805803+0000","last_change":"2024-04-24T15:33:02.637003+0000","last_active":"2024-04-24T15:33:08.805803+0000","last_peered":"2024-04-24T15:33:08.805803+0000","last_clean":"2024-04-24T15:33:08.805803+0000","last_became_active":"2024-04-24T15:33:02.635290+0000","last_became_peered":"2024-04-24T15:33:02.635290+0000","last_unstale":"2024-04-24T15:33:08.805803+0000","last_undegraded":"2024-04-24T15:33:08.805803+0000","last_fullsized":"2024-04-24T15:33:08.805803+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:34:41.643440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000468851,"stat_sum":{"num_bytes":17972083,"num_objects":19,"num_object_clones":4,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":5,"num_read":336,"num_read_kb":6965,"num_write":222,"num_write_kb":8702,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":6125156,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":75,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-24T15:33:07.804311+0000","last_change":"2024-04-24T15:32:47.635776+0000","last_active":"2024-04-24T15:33:07.804311+0000","last_peered":"2024-04-24T15:33:07.804311+0000","last_clean":"2024-04-24T15:33:07.804311+0000","last_became_active":"2024-04-24T15:32:47.635578+0000","last_became_peered":"2024-04-24T15:32:47.635578+0000","last_unstale":"2024-04-24T15:33:07.804311+0000","last_undegraded":"2024-04-24T15:33:07.804311+0000","last_fullsized":"2024-04-24T15:33:07.804311+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:51:24.165624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"74'550","reported_seq":1116,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-24T15:33:08.405125+0000","last_change":"2024-04-24T15:32:50.655414+0000","last_active":"2024-04-24T15:33:08.405125+0000","last_peered":"2024-04-24T15:33:08.405125+0000","last_clean":"2024-04-24T15:33:08.405125+0000","last_became_active":"2024-04-24T15:32:50.654697+0000","last_became_peered":"2024-04-24T15:32:50.654697+0000","last_unstale":"2024-04-24T15:33:08.405125+0000","last_undegraded":"2024-04-24T15:33:08.405125+0000","last_fullsized":"2024-04-24T15:33:08.405125+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":550,"log_dups_size":0,"ondisk_log_size":550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:15:13.259423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064568399999999999,"stat_sum":{"num_bytes":24891440,"num_objects":35,"num_object_clones":9,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":13,"num_read":2080,"num_read_kb":5809,"num_write":786,"num_write_kb":5256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":2375192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":77,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-24T15:33:07.804341+0000","last_change":"2024-04-24T15:32:59.745415+0000","last_active":"2024-04-24T15:33:07.804341+0000","last_peered":"2024-04-24T15:33:07.804341+0000","last_clean":"2024-04-24T15:33:07.804341+0000","last_became_active":"2024-04-24T15:32:59.745082+0000","last_became_peered":"2024-04-24T15:32:59.745082+0000","last_unstale":"2024-04-24T15:33:07.804341+0000","last_undegraded":"2024-04-24T15:33:07.804341+0000","last_fullsized":"2024-04-24T15:33:07.804341+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:13:21.504682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":18,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-24T15:33:07.795345+0000","last_change":"2024-04-24T15:32:59.750351+0000","last_active":"2024-04-24T15:33:07.795345+0000","last_peered":"2024-04-24T15:33:07.795345+0000","last_clean":"2024-04-24T15:33:07.795345+0000","last_became_active":"2024-04-24T15:32:59.750060+0000","last_became_peered":"2024-04-24T15:32:59.750060+0000","last_unstale":"2024-04-24T15:33:07.795345+0000","last_undegraded":"2024-04-24T15:33:07.795345+0000","last_fullsized":"2024-04-24T15:33:07.795345+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:30:23.191594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"75'252","reported_seq":3757,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-24T15:33:08.815041+0000","last_change":"2024-04-24T15:32:50.653117+0000","last_active":"2024-04-24T15:33:08.815041+0000","last_peered":"2024-04-24T15:33:08.815041+0000","last_clean":"2024-04-24T15:33:08.815041+0000","last_became_active":"2024-04-24T15:32:50.651725+0000","last_became_peered":"2024-04-24T15:32:50.651725+0000","last_unstale":"2024-04-24T15:33:08.815041+0000","last_undegraded":"2024-04-24T15:33:08.815041+0000","last_fullsized":"2024-04-24T15:33:08.815041+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":252,"log_dups_size":0,"ondisk_log_size":252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:34:39.675295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057421700000000002,"stat_sum":{"num_bytes":6804288,"num_objects":9,"num_object_clones":2,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":1,"num_read":2347,"num_read_kb":3524,"num_write":245,"num_write_kb":5890,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":11799,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"60'1","reported_seq":22,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-24T15:33:07.795063+0000","last_change":"2024-04-24T15:32:59.752573+0000","last_active":"2024-04-24T15:33:07.795063+0000","last_peered":"2024-04-24T15:33:07.795063+0000","last_clean":"2024-04-24T15:33:07.795063+0000","last_became_active":"2024-04-24T15:32:59.750157+0000","last_became_peered":"2024-04-24T15:32:59.750157+0000","last_unstale":"2024-04-24T15:33:07.795063+0000","last_undegraded":"2024-04-24T15:33:07.795063+0000","last_fullsized":"2024-04-24T15:33:07.795063+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_clean_scrub_stamp":"2024-04-24T15:32:06.972928+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:30:52.191826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":29,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-24T15:33:07.798200+0000","last_change":"2024-04-24T15:32:47.635254+0000","last_active":"2024-04-24T15:33:07.798200+0000","last_peered":"2024-04-24T15:33:07.798200+0000","last_clean":"2024-04-24T15:33:07.798200+0000","last_became_active":"2024-04-24T15:32:47.634970+0000","last_became_peered":"2024-04-24T15:32:47.634970+0000","last_unstale":"2024-04-24T15:33:07.798200+0000","last_undegraded":"2024-04-24T15:33:07.798200+0000","last_fullsized":"2024-04-24T15:33:07.798200+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:23:36.303998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"74'314","reported_seq":736,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-24T15:33:08.830891+0000","last_change":"2024-04-24T15:32:24.074434+0000","last_active":"2024-04-24T15:33:08.830891+0000","last_peered":"2024-04-24T15:33:08.830891+0000","last_clean":"2024-04-24T15:33:08.830891+0000","last_became_active":"2024-04-24T15:32:24.073408+0000","last_became_peered":"2024-04-24T15:32:24.073408+0000","last_unstale":"2024-04-24T15:33:08.830891+0000","last_undegraded":"2024-04-24T15:33:08.830891+0000","last_fullsized":"2024-04-24T15:33:08.830891+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:04:54.522860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058732700000000001,"stat_sum":{"num_bytes":31643387,"num_objects":18,"num_object_clones":5,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":3,"num_read":830,"num_read_kb":8022,"num_write":366,"num_write_kb":4087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"60'1","reported_seq":73,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-24T15:33:07.798400+0000","last_change":"2024-04-24T15:32:36.177281+0000","last_active":"2024-04-24T15:33:07.798400+0000","last_peered":"2024-04-24T15:33:07.798400+0000","last_clean":"2024-04-24T15:33:07.798400+0000","last_became_active":"2024-04-24T15:32:36.176816+0000","last_became_peered":"2024-04-24T15:32:36.176816+0000","last_unstale":"2024-04-24T15:33:07.798400+0000","last_undegraded":"2024-04-24T15:33:07.798400+0000","last_fullsized":"2024-04-24T15:33:07.798400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_clean_scrub_stamp":"2024-04-24T15:32:06.972928+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:47:52.036808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":191,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-24T15:33:07.804038+0000","last_change":"2024-04-24T15:32:50.652024+0000","last_active":"2024-04-24T15:33:07.804038+0000","last_peered":"2024-04-24T15:33:07.804038+0000","last_clean":"2024-04-24T15:33:07.804038+0000","last_became_active":"2024-04-24T15:32:50.651718+0000","last_became_peered":"2024-04-24T15:32:50.651718+0000","last_unstale":"2024-04-24T15:33:07.804038+0000","last_undegraded":"2024-04-24T15:33:07.804038+0000","last_fullsized":"2024-04-24T15:33:07.804038+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:31:59.935287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:31:59.935287+0000","last_clean_scrub_stamp":"2024-04-24T15:31:59.935287+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:28:49.270861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":52,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-24T15:33:07.796339+0000","last_change":"2024-04-24T15:32:47.639623+0000","last_active":"2024-04-24T15:33:07.796339+0000","last_peered":"2024-04-24T15:33:07.796339+0000","last_clean":"2024-04-24T15:33:07.796339+0000","last_became_active":"2024-04-24T15:32:47.639356+0000","last_became_peered":"2024-04-24T15:32:47.639356+0000","last_unstale":"2024-04-24T15:33:07.796339+0000","last_undegraded":"2024-04-24T15:33:07.796339+0000","last_fullsized":"2024-04-24T15:33:07.796339+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:10:46.402999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"75'398","reported_seq":1149,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-24T15:33:08.798713+0000","last_change":"2024-04-24T15:33:02.639498+0000","last_active":"2024-04-24T15:33:08.798713+0000","last_peered":"2024-04-24T15:33:08.798713+0000","last_clean":"2024-04-24T15:33:08.798713+0000","last_became_active":"2024-04-24T15:33:02.638740+0000","last_became_peered":"2024-04-24T15:33:02.638740+0000","last_unstale":"2024-04-24T15:33:08.798713+0000","last_undegraded":"2024-04-24T15:33:08.798713+0000","last_fullsized":"2024-04-24T15:33:08.798713+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":398,"log_dups_size":0,"ondisk_log_size":398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:24:14.533112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032205999999999999,"stat_sum":{"num_bytes":22960498,"num_objects":24,"num_object_clones":5,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":4,"num_read":1792,"num_read_kb":12516,"num_write":607,"num_write_kb":6203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":6723939,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":41,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-24T15:33:07.804204+0000","last_change":"2024-04-24T15:32:42.209959+0000","last_active":"2024-04-24T15:33:07.804204+0000","last_peered":"2024-04-24T15:33:07.804204+0000","last_clean":"2024-04-24T15:33:07.804204+0000","last_became_active":"2024-04-24T15:32:42.209603+0000","last_became_peered":"2024-04-24T15:32:42.209603+0000","last_unstale":"2024-04-24T15:33:07.804204+0000","last_undegraded":"2024-04-24T15:33:07.804204+0000","last_fullsized":"2024-04-24T15:33:07.804204+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_clean_scrub_stamp":"2024-04-24T15:32:06.972928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:12:44.842597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":37,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-24T15:33:07.798435+0000","last_change":"2024-04-24T15:32:42.207668+0000","last_active":"2024-04-24T15:33:07.798435+0000","last_peered":"2024-04-24T15:33:07.798435+0000","last_clean":"2024-04-24T15:33:07.798435+0000","last_became_active":"2024-04-24T15:32:42.207408+0000","last_became_peered":"2024-04-24T15:32:42.207408+0000","last_unstale":"2024-04-24T15:33:07.798435+0000","last_undegraded":"2024-04-24T15:33:07.798435+0000","last_fullsized":"2024-04-24T15:33:07.798435+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:03:09.182470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"74'172","reported_seq":377,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-24T15:33:08.407523+0000","last_change":"2024-04-24T15:32:47.672581+0000","last_active":"2024-04-24T15:33:08.407523+0000","last_peered":"2024-04-24T15:33:08.407523+0000","last_clean":"2024-04-24T15:33:08.407523+0000","last_became_active":"2024-04-24T15:32:47.635898+0000","last_became_peered":"2024-04-24T15:32:47.635898+0000","last_unstale":"2024-04-24T15:33:08.407523+0000","last_undegraded":"2024-04-24T15:33:08.407523+0000","last_fullsized":"2024-04-24T15:33:08.407523+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:23:43.925824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098765599999999995,"stat_sum":{"num_bytes":22636746,"num_objects":19,"num_object_clones":7,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":9,"num_read":869,"num_read_kb":6787,"num_write":151,"num_write_kb":1399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":67213,"num_keys_recovered":18,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-24T15:33:07.795314+0000","last_change":"2024-04-24T15:32:59.750425+0000","last_active":"2024-04-24T15:33:07.795314+0000","last_peered":"2024-04-24T15:33:07.795314+0000","last_clean":"2024-04-24T15:33:07.795314+0000","last_became_active":"2024-04-24T15:32:59.750038+0000","last_became_peered":"2024-04-24T15:32:59.750038+0000","last_unstale":"2024-04-24T15:33:07.795314+0000","last_undegraded":"2024-04-24T15:33:07.795314+0000","last_fullsized":"2024-04-24T15:33:07.795314+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_clean_scrub_stamp":"2024-04-24T15:32:06.972928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:48:12.195815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":61,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-24T15:33:07.804118+0000","last_change":"2024-04-24T15:32:36.178533+0000","last_active":"2024-04-24T15:33:07.804118+0000","last_peered":"2024-04-24T15:33:07.804118+0000","last_clean":"2024-04-24T15:33:07.804118+0000","last_became_active":"2024-04-24T15:32:36.178234+0000","last_became_peered":"2024-04-24T15:32:36.178234+0000","last_unstale":"2024-04-24T15:33:07.804118+0000","last_undegraded":"2024-04-24T15:33:07.804118+0000","last_fullsized":"2024-04-24T15:33:07.804118+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:00:02.842510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.4","version":"73'254","reported_seq":1109,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-24T15:33:08.425405+0000","last_change":"2024-04-24T15:32:19.045228+0000","last_active":"2024-04-24T15:33:08.425405+0000","last_peered":"2024-04-24T15:33:08.425405+0000","last_clean":"2024-04-24T15:33:08.425405+0000","last_became_active":"2024-04-24T15:32:18.044036+0000","last_became_peered":"2024-04-24T15:32:18.044036+0000","last_unstale":"2024-04-24T15:33:08.425405+0000","last_undegraded":"2024-04-24T15:33:08.425405+0000","last_fullsized":"2024-04-24T15:33:08.425405+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":254,"log_dups_size":0,"ondisk_log_size":254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:48:33.464105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050914600000000003,"stat_sum":{"num_bytes":42911228,"num_objects":34,"num_object_clones":11,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":11,"num_read":1084,"num_read_kb":17753,"num_write":249,"num_write_kb":6305,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"73'189","reported_seq":399,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-24T15:33:07.795288+0000","last_change":"2024-04-24T15:33:02.635068+0000","last_active":"2024-04-24T15:33:07.795288+0000","last_peered":"2024-04-24T15:33:07.795288+0000","last_clean":"2024-04-24T15:33:07.795288+0000","last_became_active":"2024-04-24T15:33:02.634103+0000","last_became_peered":"2024-04-24T15:33:02.634103+0000","last_unstale":"2024-04-24T15:33:07.795288+0000","last_undegraded":"2024-04-24T15:33:07.795288+0000","last_fullsized":"2024-04-24T15:33:07.795288+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:17:55.155562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039442500000000001,"stat_sum":{"num_bytes":34926339,"num_objects":27,"num_object_clones":10,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":8,"num_read":181,"num_read_kb":294,"num_write":186,"num_write_kb":1512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":578400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":47,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-24T15:33:07.802430+0000","last_change":"2024-04-24T15:32:47.636966+0000","last_active":"2024-04-24T15:33:07.802430+0000","last_peered":"2024-04-24T15:33:07.802430+0000","last_clean":"2024-04-24T15:33:07.802430+0000","last_became_active":"2024-04-24T15:32:47.636739+0000","last_became_peered":"2024-04-24T15:32:47.636739+0000","last_unstale":"2024-04-24T15:33:07.802430+0000","last_undegraded":"2024-04-24T15:33:07.802430+0000","last_fullsized":"2024-04-24T15:33:07.802430+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:09:56.338464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"73'279","reported_seq":1227,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-24T15:33:08.918300+0000","last_change":"2024-04-24T15:33:02.639625+0000","last_active":"2024-04-24T15:33:08.918300+0000","last_peered":"2024-04-24T15:33:08.918300+0000","last_clean":"2024-04-24T15:33:08.918300+0000","last_became_active":"2024-04-24T15:33:02.638684+0000","last_became_peered":"2024-04-24T15:33:02.638684+0000","last_unstale":"2024-04-24T15:33:08.918300+0000","last_undegraded":"2024-04-24T15:33:08.918300+0000","last_fullsized":"2024-04-24T15:33:08.918300+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":279,"log_dups_size":0,"ondisk_log_size":279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:10:45.684566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032331100000000001,"stat_sum":{"num_bytes":10049735,"num_objects":24,"num_object_clones":7,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":7,"num_read":1424,"num_read_kb":3802,"num_write":284,"num_write_kb":2852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":128,"num_bytes_recovered":3330138,"num_keys_recovered":32,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"74'231","reported_seq":938,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-24T15:33:08.405064+0000","last_change":"2024-04-24T15:33:02.639148+0000","last_active":"2024-04-24T15:33:08.405064+0000","last_peered":"2024-04-24T15:33:08.405064+0000","last_clean":"2024-04-24T15:33:08.405064+0000","last_became_active":"2024-04-24T15:33:02.638235+0000","last_became_peered":"2024-04-24T15:33:02.638235+0000","last_unstale":"2024-04-24T15:33:08.405064+0000","last_undegraded":"2024-04-24T15:33:08.405064+0000","last_fullsized":"2024-04-24T15:33:08.405064+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":231,"log_dups_size":0,"ondisk_log_size":231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:44:57.206364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034825399999999999,"stat_sum":{"num_bytes":14649658,"num_objects":22,"num_object_clones":4,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":8,"num_read":1034,"num_read_kb":12624,"num_write":238,"num_write_kb":6480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":5713612,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"75'274","reported_seq":1454,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-24T15:33:08.848979+0000","last_change":"2024-04-24T15:32:50.652552+0000","last_active":"2024-04-24T15:33:08.848979+0000","last_peered":"2024-04-24T15:33:08.848979+0000","last_clean":"2024-04-24T15:33:08.848979+0000","last_became_active":"2024-04-24T15:32:50.651889+0000","last_became_peered":"2024-04-24T15:32:50.651889+0000","last_unstale":"2024-04-24T15:33:08.848979+0000","last_undegraded":"2024-04-24T15:33:08.848979+0000","last_fullsized":"2024-04-24T15:33:08.848979+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":274,"log_dups_size":0,"ondisk_log_size":274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:15:10.390810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032027999999999999,"stat_sum":{"num_bytes":52831897,"num_objects":27,"num_object_clones":8,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":6,"num_read":1821,"num_read_kb":16525,"num_write":266,"num_write_kb":8586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":1560706,"num_keys_recovered":19,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"73'412","reported_seq":913,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-24T15:33:07.795202+0000","last_change":"2024-04-24T15:33:03.636716+0000","last_active":"2024-04-24T15:33:07.795202+0000","last_peered":"2024-04-24T15:33:07.795202+0000","last_clean":"2024-04-24T15:33:07.795202+0000","last_became_active":"2024-04-24T15:33:03.635751+0000","last_became_peered":"2024-04-24T15:33:03.635751+0000","last_unstale":"2024-04-24T15:33:07.795202+0000","last_undegraded":"2024-04-24T15:33:07.795202+0000","last_fullsized":"2024-04-24T15:33:07.795202+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":412,"log_dups_size":0,"ondisk_log_size":412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:30:51.876165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041177,"stat_sum":{"num_bytes":29167758,"num_objects":15,"num_object_clones":4,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":4,"num_read":1806,"num_read_kb":11530,"num_write":591,"num_write_kb":2986,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":1490474,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.c","version":"71'902","reported_seq":1445,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-24T15:33:08.138226+0000","last_change":"2024-04-24T15:33:03.640858+0000","last_active":"2024-04-24T15:33:08.138226+0000","last_peered":"2024-04-24T15:33:08.138226+0000","last_clean":"2024-04-24T15:33:08.138226+0000","last_became_active":"2024-04-24T15:33:03.638316+0000","last_became_peered":"2024-04-24T15:33:03.638316+0000","last_unstale":"2024-04-24T15:33:08.138226+0000","last_undegraded":"2024-04-24T15:33:08.138226+0000","last_fullsized":"2024-04-24T15:33:08.138226+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":902,"log_dups_size":0,"ondisk_log_size":902,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:00:53.637603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066318200000000001,"stat_sum":{"num_bytes":19874094,"num_objects":14,"num_object_clones":3,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":6,"num_read":5322,"num_read_kb":5907,"num_write":1540,"num_write_kb":3322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":54,"num_bytes_recovered":1219753,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.d","version":"75'142","reported_seq":368,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-24T15:33:08.867570+0000","last_change":"2024-04-24T15:33:02.637311+0000","last_active":"2024-04-24T15:33:08.867570+0000","last_peered":"2024-04-24T15:33:08.867570+0000","last_clean":"2024-04-24T15:33:08.867570+0000","last_became_active":"2024-04-24T15:33:02.636600+0000","last_became_peered":"2024-04-24T15:33:02.636600+0000","last_unstale":"2024-04-24T15:33:08.867570+0000","last_undegraded":"2024-04-24T15:33:08.867570+0000","last_fullsized":"2024-04-24T15:33:08.867570+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:59:07.207018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041245600000000001,"stat_sum":{"num_bytes":22299535,"num_objects":14,"num_object_clones":5,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":4,"num_read":68,"num_read_kb":10139,"num_write":154,"num_write_kb":8230,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":4246835,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"75'292","reported_seq":1204,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-24T15:33:08.806094+0000","last_change":"2024-04-24T15:33:02.636525+0000","last_active":"2024-04-24T15:33:08.806094+0000","last_peered":"2024-04-24T15:33:08.806094+0000","last_clean":"2024-04-24T15:33:08.806094+0000","last_became_active":"2024-04-24T15:33:02.635376+0000","last_became_peered":"2024-04-24T15:33:02.635376+0000","last_unstale":"2024-04-24T15:33:08.806094+0000","last_undegraded":"2024-04-24T15:33:08.806094+0000","last_fullsized":"2024-04-24T15:33:08.806094+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":292,"log_dups_size":0,"ondisk_log_size":292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:24:51.899239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076249599999999999,"stat_sum":{"num_bytes":14167373,"num_objects":22,"num_object_clones":5,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":3,"num_read":1502,"num_read_kb":4000,"num_write":350,"num_write_kb":3105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":93,"num_bytes_recovered":117055,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"74'638","reported_seq":1881,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-24T15:33:08.404811+0000","last_change":"2024-04-24T15:33:03.638821+0000","last_active":"2024-04-24T15:33:08.404811+0000","last_peered":"2024-04-24T15:33:08.404811+0000","last_clean":"2024-04-24T15:33:08.404811+0000","last_became_active":"2024-04-24T15:33:03.638156+0000","last_became_peered":"2024-04-24T15:33:03.638156+0000","last_unstale":"2024-04-24T15:33:08.404811+0000","last_undegraded":"2024-04-24T15:33:08.404811+0000","last_fullsized":"2024-04-24T15:33:08.404811+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":638,"log_dups_size":0,"ondisk_log_size":638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:07:21.801310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000652694,"stat_sum":{"num_bytes":52902910,"num_objects":28,"num_object_clones":6,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":6,"num_read":3512,"num_read_kb":5740,"num_write":763,"num_write_kb":3812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":126,"num_bytes_recovered":630687,"num_keys_recovered":35,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"74'150","reported_seq":244,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-24T15:33:08.401602+0000","last_change":"2024-04-24T15:32:19.050772+0000","last_active":"2024-04-24T15:33:08.401602+0000","last_peered":"2024-04-24T15:33:08.401602+0000","last_clean":"2024-04-24T15:33:08.401602+0000","last_became_active":"2024-04-24T15:32:18.055440+0000","last_became_peered":"2024-04-24T15:32:18.055440+0000","last_unstale":"2024-04-24T15:33:08.401602+0000","last_undegraded":"2024-04-24T15:33:08.401602+0000","last_fullsized":"2024-04-24T15:33:08.401602+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:20:47.837497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000458918,"stat_sum":{"num_bytes":15841071,"num_objects":16,"num_object_clones":6,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":6,"num_read":69,"num_read_kb":284,"num_write":138,"num_write_kb":834,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.11","version":"74'366","reported_seq":764,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-24T15:33:08.407582+0000","last_change":"2024-04-24T15:32:50.658419+0000","last_active":"2024-04-24T15:33:08.407582+0000","last_peered":"2024-04-24T15:33:08.407582+0000","last_clean":"2024-04-24T15:33:08.407582+0000","last_became_active":"2024-04-24T15:32:50.657072+0000","last_became_peered":"2024-04-24T15:32:50.657072+0000","last_unstale":"2024-04-24T15:33:08.407582+0000","last_undegraded":"2024-04-24T15:33:08.407582+0000","last_fullsized":"2024-04-24T15:33:08.407582+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":366,"log_dups_size":0,"ondisk_log_size":366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:32:00.412926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057133599999999998,"stat_sum":{"num_bytes":32020174,"num_objects":24,"num_object_clones":7,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":8,"num_read":1052,"num_read_kb":9066,"num_write":457,"num_write_kb":3678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":2255518,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"75'274","reported_seq":4926,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-24T15:33:08.798772+0000","last_change":"2024-04-24T15:32:19.040444+0000","last_active":"2024-04-24T15:33:08.798772+0000","last_peered":"2024-04-24T15:33:08.798772+0000","last_clean":"2024-04-24T15:33:08.798772+0000","last_became_active":"2024-04-24T15:32:18.055189+0000","last_became_peered":"2024-04-24T15:32:18.055189+0000","last_unstale":"2024-04-24T15:33:08.798772+0000","last_undegraded":"2024-04-24T15:33:08.798772+0000","last_fullsized":"2024-04-24T15:33:08.798772+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":274,"log_dups_size":0,"ondisk_log_size":274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:08:21.721301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041921100000000001,"stat_sum":{"num_bytes":58426421,"num_objects":22,"num_object_clones":8,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":8,"num_read":4698,"num_read_kb":37429,"num_write":277,"num_write_kb":8388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"75'159","reported_seq":846,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-24T15:33:08.805896+0000","last_change":"2024-04-24T15:33:02.636946+0000","last_active":"2024-04-24T15:33:08.805896+0000","last_peered":"2024-04-24T15:33:08.805896+0000","last_clean":"2024-04-24T15:33:08.805896+0000","last_became_active":"2024-04-24T15:33:02.635902+0000","last_became_peered":"2024-04-24T15:33:02.635902+0000","last_unstale":"2024-04-24T15:33:08.805896+0000","last_undegraded":"2024-04-24T15:33:08.805896+0000","last_fullsized":"2024-04-24T15:33:08.805896+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:42:19.840204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091721200000000004,"stat_sum":{"num_bytes":12577593,"num_objects":16,"num_object_clones":2,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":2,"num_read":436,"num_read_kb":3103,"num_write":171,"num_write_kb":4599,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":1268965,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"74'272","reported_seq":3718,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-24T15:33:08.402432+0000","last_change":"2024-04-24T15:32:50.657999+0000","last_active":"2024-04-24T15:33:08.402432+0000","last_peered":"2024-04-24T15:33:08.402432+0000","last_clean":"2024-04-24T15:33:08.402432+0000","last_became_active":"2024-04-24T15:32:50.657302+0000","last_became_peered":"2024-04-24T15:32:50.657302+0000","last_unstale":"2024-04-24T15:33:08.402432+0000","last_undegraded":"2024-04-24T15:33:08.402432+0000","last_fullsized":"2024-04-24T15:33:08.402432+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":272,"log_dups_size":0,"ondisk_log_size":272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:30:15.118724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060281400000000002,"stat_sum":{"num_bytes":22299490,"num_objects":25,"num_object_clones":7,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":7,"num_read":4943,"num_read_kb":9243,"num_write":268,"num_write_kb":3493,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":2229438,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"72'213","reported_seq":770,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-24T15:33:07.798479+0000","last_change":"2024-04-24T15:32:50.656447+0000","last_active":"2024-04-24T15:33:07.798479+0000","last_peered":"2024-04-24T15:33:07.798479+0000","last_clean":"2024-04-24T15:33:07.798479+0000","last_became_active":"2024-04-24T15:32:50.655746+0000","last_became_peered":"2024-04-24T15:32:50.655746+0000","last_unstale":"2024-04-24T15:33:07.798479+0000","last_undegraded":"2024-04-24T15:33:07.798479+0000","last_fullsized":"2024-04-24T15:33:07.798479+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:31:51.573617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001011678,"stat_sum":{"num_bytes":28827561,"num_objects":22,"num_object_clones":8,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":6,"num_read":373,"num_read_kb":7745,"num_write":212,"num_write_kb":5909,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":4590279,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"75'270","reported_seq":1433,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-24T15:33:08.800015+0000","last_change":"2024-04-24T15:32:24.075856+0000","last_active":"2024-04-24T15:33:08.800015+0000","last_peered":"2024-04-24T15:33:08.800015+0000","last_clean":"2024-04-24T15:33:08.800015+0000","last_became_active":"2024-04-24T15:32:24.075461+0000","last_became_peered":"2024-04-24T15:32:24.075461+0000","last_unstale":"2024-04-24T15:33:08.800015+0000","last_undegraded":"2024-04-24T15:33:08.800015+0000","last_fullsized":"2024-04-24T15:33:08.800015+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":270,"log_dups_size":0,"ondisk_log_size":270,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:07:28.709798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078032999999999998,"stat_sum":{"num_bytes":17050321,"num_objects":24,"num_object_clones":8,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":9,"num_read":2115,"num_read_kb":6441,"num_write":258,"num_write_kb":2393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":812412249,"num_objects":693,"num_object_clones":183,"num_object_copies":1386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":693,"num_whiteouts":194,"num_read":68024,"num_read_kb":318298,"num_write":13848,"num_write_kb":142584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1800,"num_bytes_recovered":55960625,"num_keys_recovered":162,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":25264128,"data_stored":232870375,"data_compressed":9135582,"data_compressed_allocated":19214336,"data_compressed_original":229450632,"omap_allocated":0,"internal_metadata":0},"log_size":10971,"ondisk_log_size":10971,"up":64,"acting":64,"num_store_stats":7},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":8,"acting":8,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":1771104,"data_compressed":15361,"data_compressed_allocated":110592,"data_compressed_original":1736704,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":0,"seq":55834574865,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303312,"kb_used_data":5944,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94068528,"statfs":{"total":96636764160,"available":96326172672,"internally_reserved":0,"allocated":6086656,"data_stored":49896719,"data_compressed":1994958,"data_compressed_allocated":4091904,"data_compressed_original":48810435,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 15:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.5},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.44500000000000001}]},{"osd":1,"last update":"Wed Apr 24 15:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.46700000000000003}]},{"osd":2,"last update":"Wed Apr 24 15:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.436},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.51000000000000001}]},{"osd":3,"last update":"Wed Apr 24 15:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.45800000000000002}]},{"osd":4,"last update":"Wed Apr 24 15:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.377},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.47699999999999998}]},{"osd":5,"last update":"Wed Apr 24 15:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.52900000000000003}]},{"osd":7,"last update":"Wed Apr 24 15:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.51800000000000002}]}]},{"osd":1,"up_from":0,"seq":55834574864,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":313668,"kb_used_data":4024,"kb_used_omap":1,"kb_used_meta":309630,"kb_avail":94058172,"statfs":{"total":96636764160,"available":96315568128,"internally_reserved":0,"allocated":4120576,"data_stored":27677609,"data_compressed":1364351,"data_compressed_allocated":2473984,"data_compressed_original":26824296,"omap_allocated":1704,"internal_metadata":317061464},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Apr 24 15:33:00 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":3.7890000000000001,"5min":3.7890000000000001,"15min":3.7890000000000001},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":2.7890000000000001,"5min":2.7890000000000001,"15min":2.7890000000000001},"last":0.59899999999999998}]},{"osd":2,"last update":"Wed Apr 24 15:33:00 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":3.9950000000000001,"5min":3.9950000000000001,"15min":3.9950000000000001},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.443,"5min":0.443,"15min":0.443},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":1.8500000000000001,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":0.27500000000000002}]},{"osd":3,"last update":"Wed Apr 24 15:33:00 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":2.097,"5min":2.097,"15min":2.097},"last":0.318},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":3.2109999999999999,"5min":3.2109999999999999,"15min":3.2109999999999999},"last":0.79500000000000004}]},{"osd":4,"last update":"Wed Apr 24 15:33:00 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":3.282,"5min":3.282,"15min":3.282},"last":0.85599999999999998}]},{"osd":5,"last update":"Wed Apr 24 15:33:00 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":2.2229999999999999,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":2.1699999999999999,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":0.628}]},{"osd":6,"last update":"Wed Apr 24 15:33:00 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":3.2509999999999999,"5min":3.2509999999999999,"15min":3.2509999999999999},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":2.1909999999999998,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":0.67200000000000004}]},{"osd":7,"last update":"Wed Apr 24 15:33:00 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":2.1349999999999998,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":0.753},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":2.206,"5min":2.206,"15min":2.206},"last":0.72799999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":313760,"kb_used_data":4116,"kb_used_omap":1,"kb_used_meta":309630,"kb_avail":94058080,"statfs":{"total":96636764160,"available":96315473920,"internally_reserved":0,"allocated":4214784,"data_stored":26550086,"data_compressed":1134854,"data_compressed_allocated":2560000,"data_compressed_original":25699203,"omap_allocated":1704,"internal_metadata":317061464},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 15:33:03 2024","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.317,"15min":0.317},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"last":0.248}]},{"osd":1,"last update":"Wed Apr 24 15:33:03 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.222},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":3.1110000000000002,"5min":3.1110000000000002,"15min":3.1110000000000002},"last":0.49399999999999999}]},{"osd":3,"last update":"Wed Apr 24 15:33:03 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":2.056,"5min":2.056,"15min":2.056},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.315}]},{"osd":4,"last update":"Wed Apr 24 15:33:03 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.42399999999999999}]},{"osd":5,"last update":"Wed Apr 24 15:33:03 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":2.1190000000000002,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":0.60699999999999998}]},{"osd":6,"last update":"Wed Apr 24 15:33:03 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.1000000000000001,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.376}]},{"osd":7,"last update":"Wed Apr 24 15:33:03 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":2.1360000000000001,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":0.56000000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311760,"kb_used_data":6216,"kb_used_omap":1,"kb_used_meta":305534,"kb_avail":94060080,"statfs":{"total":96636764160,"available":96317521920,"internally_reserved":0,"allocated":6365184,"data_stored":45145565,"data_compressed":1820539,"data_compressed_allocated":4018176,"data_compressed_original":43924048,"omap_allocated":1704,"internal_metadata":312867160},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 15:33:00 2024","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.24099999999999999}]},{"osd":1,"last update":"Wed Apr 24 15:33:00 2024","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"min":{"1min":0.187,"5min":0.187,"15min":0.187},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.33900000000000002}]},{"osd":2,"last update":"Wed Apr 24 15:33:00 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.371}]},{"osd":4,"last update":"Wed Apr 24 15:33:00 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.49099999999999999}]},{"osd":5,"last update":"Wed Apr 24 15:33:00 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.6020000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.46600000000000003}]},{"osd":6,"last update":"Wed Apr 24 15:33:00 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.52200000000000002}]},{"osd":7,"last update":"Wed Apr 24 15:33:00 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.635,"5min":1.635,"15min":1.635},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.502}]}]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":315016,"kb_used_data":5372,"kb_used_omap":1,"kb_used_meta":309630,"kb_avail":94056824,"statfs":{"total":96636764160,"available":96314187776,"internally_reserved":0,"allocated":5500928,"data_stored":36119156,"data_compressed":1462490,"data_compressed_allocated":3538944,"data_compressed_original":35088698,"omap_allocated":1704,"internal_metadata":317061464},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 15:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.44},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.26900000000000002}]},{"osd":1,"last update":"Wed Apr 24 15:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.35499999999999998}]},{"osd":2,"last update":"Wed Apr 24 15:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.29399999999999998}]},{"osd":3,"last update":"Wed Apr 24 15:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.374}]},{"osd":5,"last update":"Wed Apr 24 15:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.40300000000000002}]},{"osd":6,"last update":"Wed Apr 24 15:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.38700000000000001}]},{"osd":7,"last update":"Wed Apr 24 15:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.47099999999999997}]}]},{"osd":5,"up_from":0,"seq":55834574865,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4842,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":5677,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4842,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3182592,"data_stored":27775948,"data_compressed":1406249,"data_compressed_allocated":2437120,"data_compressed_original":27364968,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3543040,"data_stored":30807624,"data_compressed":1175783,"data_compressed_allocated":2740224,"data_compressed_original":30368643,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":5505024,"data_stored":44726790,"data_compressed":1820539,"data_compressed_allocated":4018176,"data_compressed_original":43924048,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":5292032,"data_stored":49804864,"data_compressed":1801641,"data_compressed_allocated":4169728,"data_compressed_original":49170746,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":204800,"data_stored":181730,"data_compressed":8376,"data_compressed_allocated":32768,"data_compressed_original":77824,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":5193728,"data_stored":54700048,"data_compressed":2185115,"data_compressed_allocated":4059136,"data_compressed_original":54016451,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2342912,"data_stored":24873371,"data_compressed":737879,"data_compressed_allocated":1757184,"data_compressed_original":24527952,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T15:33:09.620 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-04-24T15:33:09.621 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-24T15:33:09.621 INFO:tasks.thrashosds.thrasher:Killing osd 6, live_osds are [0, 1, 2, 3, 4, 5, 6, 7] 2024-04-24T15:33:09.621 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-24T15:33:09.621 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T15:33:09.628 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:09.629+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:09.661 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:09.661 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:09.666 INFO:tasks.ceph.osd.6:Stopped 2024-04-24T15:33:09.666 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.6 2024-04-24T15:33:09.666 INFO:tasks.thrashosds.thrasher:remote for osd 6 is ubuntu@smithi151.front.sepia.ceph.com 2024-04-24T15:33:09.666 DEBUG:teuthology.orchestra.run.smithi151:> type ceph-objectstore-tool 2024-04-24T15:33:09.672 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-24T15:33:09.714 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-04-24T15:33:09.730 INFO:teuthology.orchestra.run.smithi151.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-04-24T15:33:09.730 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-6 --journal-path /var/lib/ceph/osd/ceph-6/journal --op list-pgs 2024-04-24T15:33:09.731 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:09.730+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:09.732 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-04-24T15:33:09.749 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-04-24T15:33:09.767 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-04-24T15:33:09.786 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-04-24T15:33:09.807 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-04-24T15:33:09.830 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:33:09.831+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:33:09.851 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:09.852 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:09.862 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-24T15:33:09.931 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:09.932+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:09.987 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:09.987 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:09.998 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-24T15:33:10.041 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:10.042+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:10.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-24T15:33:10.067+0000 7f72a5200340 -1 bluestore::NCB::__restore_allocator::No Valid allocation info on disk (empty file) 2024-04-24T15:33:10.123 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:10.123 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:10.133 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:33:10.134+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:33:10.134 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-24T15:33:10.233 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:10.234+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:10.258 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:10.258 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-04-24T15:33:10.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.258 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:47384 4.1f 4.33f435bf (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-04-24T15:33:10.258 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:09.802333+0000", 2024-04-24T15:33:10.258 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.45728355500000001, 2024-04-24T15:33:10.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.457305825, 2024-04-24T15:33:10.258 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:10.259 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for peered", 2024-04-24T15:33:10.259 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:10.259 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:10.259 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:10.259 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 47384 2024-04-24T15:33:10.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.259 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:10.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.259 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:10.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.802333+0000", 2024-04-24T15:33:10.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:10.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.259 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:10.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.802333+0000", 2024-04-24T15:33:10.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:10.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:10.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.802332+0000", 2024-04-24T15:33:10.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999981 2024-04-24T15:33:10.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:10.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.802340+0000", 2024-04-24T15:33:10.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7700000000000007e-06 2024-04-24T15:33:10.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:10.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.802360+0000", 2024-04-24T15:33:10.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9959999999999999e-05 2024-04-24T15:33:10.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:10.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.802371+0000", 2024-04-24T15:33:10.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0473e-05 2024-04-24T15:33:10.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:10.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.802447+0000", 2024-04-24T15:33:10.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6089999999999998e-05 2024-04-24T15:33:10.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op must wait for map", 2024-04-24T15:33:10.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.802456+0000", 2024-04-24T15:33:10.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0599999999999997e-06 2024-04-24T15:33:10.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:10.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.808378+0000", 2024-04-24T15:33:10.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0059224910000000002 2024-04-24T15:33:10.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-04-24T15:33:10.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.808384+0000", 2024-04-24T15:33:10.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7740000000000001e-06 2024-04-24T15:33:10.263 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:10.263 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:10.263 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:10.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.263 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:47385 4.1f 4.33f435bf (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-04-24T15:33:10.263 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:09.803953+0000", 2024-04-24T15:33:10.263 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.45566364100000001, 2024-04-24T15:33:10.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.45573911299999997, 2024-04-24T15:33:10.263 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:10.263 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for peered", 2024-04-24T15:33:10.263 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:10.263 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:10.263 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:10.264 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 47385 2024-04-24T15:33:10.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.264 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:10.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:10.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.803953+0000", 2024-04-24T15:33:10.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:10.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:10.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.803953+0000", 2024-04-24T15:33:10.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:10.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:10.265 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.803952+0000", 2024-04-24T15:33:10.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999981 2024-04-24T15:33:10.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.265 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:10.265 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.803961+0000", 2024-04-24T15:33:10.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9549999999999998e-06 2024-04-24T15:33:10.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.265 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:10.265 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.803977+0000", 2024-04-24T15:33:10.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6498999999999999e-05 2024-04-24T15:33:10.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:10.266 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.803985+0000", 2024-04-24T15:33:10.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5000000000000002e-06 2024-04-24T15:33:10.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:10.266 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.804032+0000", 2024-04-24T15:33:10.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7657999999999997e-05 2024-04-24T15:33:10.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2024-04-24T15:33:10.266 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.804039+0000", 2024-04-24T15:33:10.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5719999999999999e-06 2024-04-24T15:33:10.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:10.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.808397+0000", 2024-04-24T15:33:10.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0043579860000000003 2024-04-24T15:33:10.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-04-24T15:33:10.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.808401+0000", 2024-04-24T15:33:10.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3089999999999997e-06 2024-04-24T15:33:10.267 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:10.267 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:10.268 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:10.268 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:10.268 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-24T15:33:10.268 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 2 2024-04-24T15:33:10.268 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:10.269 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-24T15:33:10.334 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:10.336+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:10.392 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:10.392 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:33:10.392 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-04-24T15:33:10.392 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-04-24T15:33:10.392 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:10.402 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-24T15:33:10.435 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:33:10.436+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:33:10.536 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:10.537+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:10.568 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:10.568 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-04-24T15:33:10.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-04-24T15:33:10.568 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-04-24T15:33:10.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.568 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:18 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'8, mlcod=15'5)", 2024-04-24T15:33:10.568 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:00.973932+0000", 2024-04-24T15:33:10.568 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 69.593918144, 2024-04-24T15:33:10.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0064286839999999996, 2024-04-24T15:33:10.568 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:10.569 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:10.569 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:10.569 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.569 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:10.569 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.973932+0000", 2024-04-24T15:33:10.569 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:10.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.569 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.569 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:10.569 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.973932+0000", 2024-04-24T15:33:10.569 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:10.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.569 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.569 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:10.569 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.973931+0000", 2024-04-24T15:33:10.570 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:33:10.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.570 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.570 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:10.570 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.973938+0000", 2024-04-24T15:33:10.570 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.029e-06 2024-04-24T15:33:10.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.570 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.570 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:10.570 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.973948+0000", 2024-04-24T15:33:10.570 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9129999999999998e-06 2024-04-24T15:33:10.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.570 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.571 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:10.571 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.973952+0000", 2024-04-24T15:33:10.571 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3069999999999996e-06 2024-04-24T15:33:10.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.571 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.571 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:10.571 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.973986+0000", 2024-04-24T15:33:10.571 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3643000000000003e-05 2024-04-24T15:33:10.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.571 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.571 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:10.571 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.974006+0000", 2024-04-24T15:33:10.571 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9658999999999999e-05 2024-04-24T15:33:10.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.571 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.572 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:10.572 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.980330+0000", 2024-04-24T15:33:10.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0063243479999999996 2024-04-24T15:33:10.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.572 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:10.572 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.980361+0000", 2024-04-24T15:33:10.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0527000000000001e-05 2024-04-24T15:33:10.572 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:10.572 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:10.572 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:10.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.572 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:21 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'11, mlcod=15'8)", 2024-04-24T15:33:10.572 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:00.986502+0000", 2024-04-24T15:33:10.573 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 69.581348211000005, 2024-04-24T15:33:10.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0062008819999999996, 2024-04-24T15:33:10.573 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:10.573 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:10.573 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:10.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.573 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:10.573 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.986502+0000", 2024-04-24T15:33:10.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:10.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.573 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:10.573 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.986502+0000", 2024-04-24T15:33:10.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:10.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.574 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:10.574 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.986501+0000", 2024-04-24T15:33:10.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:10.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.574 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:10.574 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.986639+0000", 2024-04-24T15:33:10.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013731400000000001 2024-04-24T15:33:10.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:10.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.986649+0000", 2024-04-24T15:33:10.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.041e-05 2024-04-24T15:33:10.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:10.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.986654+0000", 2024-04-24T15:33:10.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2000000000000002e-06 2024-04-24T15:33:10.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:10.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.986678+0000", 2024-04-24T15:33:10.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3731e-05 2024-04-24T15:33:10.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:10.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.986691+0000", 2024-04-24T15:33:10.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3078000000000001e-05 2024-04-24T15:33:10.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:10.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.992666+0000", 2024-04-24T15:33:10.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0059751650000000002 2024-04-24T15:33:10.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:10.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:00.992703+0000", 2024-04-24T15:33:10.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6622000000000003e-05 2024-04-24T15:33:10.578 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:10.578 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:10.578 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:10.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.578 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.578 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:69 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'43, mlcod=15'39)", 2024-04-24T15:33:10.578 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.058504+0000", 2024-04-24T15:33:10.578 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 69.509345890999995, 2024-04-24T15:33:10.579 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.006494606, 2024-04-24T15:33:10.579 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:10.579 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:10.579 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:10.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.579 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:10.579 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.058504+0000", 2024-04-24T15:33:10.579 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:10.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:10.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.058504+0000", 2024-04-24T15:33:10.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:10.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:10.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.058503+0000", 2024-04-24T15:33:10.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:33:10.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:10.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.058588+0000", 2024-04-24T15:33:10.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4087999999999999e-05 2024-04-24T15:33:10.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:10.582 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.058597+0000", 2024-04-24T15:33:10.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7459999999999995e-06 2024-04-24T15:33:10.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.582 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:10.582 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.058602+0000", 2024-04-24T15:33:10.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6929999999999998e-06 2024-04-24T15:33:10.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.582 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:10.582 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.060983+0000", 2024-04-24T15:33:10.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023812690000000001 2024-04-24T15:33:10.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:10.583 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.060996+0000", 2024-04-24T15:33:10.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2826999999999999e-05 2024-04-24T15:33:10.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:10.583 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.064972+0000", 2024-04-24T15:33:10.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0039757890000000004 2024-04-24T15:33:10.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:10.583 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.064999+0000", 2024-04-24T15:33:10.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6917999999999999e-05 2024-04-24T15:33:10.584 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:10.584 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:10.584 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:10.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.584 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:70 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'44, mlcod=15'39)", 2024-04-24T15:33:10.584 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.059180+0000", 2024-04-24T15:33:10.584 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 69.508670139000003, 2024-04-24T15:33:10.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0058429709999999998, 2024-04-24T15:33:10.584 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:10.584 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:10.584 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:10.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:10.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.059180+0000", 2024-04-24T15:33:10.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:10.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:10.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.059180+0000", 2024-04-24T15:33:10.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:10.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:10.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.059179+0000", 2024-04-24T15:33:10.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:10.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:10.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.059184+0000", 2024-04-24T15:33:10.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.074e-06 2024-04-24T15:33:10.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:10.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.059194+0000", 2024-04-24T15:33:10.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6050000000000004e-06 2024-04-24T15:33:10.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:10.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.059199+0000", 2024-04-24T15:33:10.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2639999999999999e-06 2024-04-24T15:33:10.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.587 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:10.587 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.061424+0000", 2024-04-24T15:33:10.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0022244880000000002 2024-04-24T15:33:10.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.587 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:10.587 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.061444+0000", 2024-04-24T15:33:10.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0633e-05 2024-04-24T15:33:10.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:10.588 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.065001+0000", 2024-04-24T15:33:10.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035567670000000002 2024-04-24T15:33:10.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:10.588 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.065023+0000", 2024-04-24T15:33:10.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1724000000000001e-05 2024-04-24T15:33:10.588 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:10.588 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:10.588 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:10.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.589 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:154 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'108, mlcod=15'105)", 2024-04-24T15:33:10.589 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.929033+0000", 2024-04-24T15:33:10.589 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 68.638816758000004, 2024-04-24T15:33:10.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0068887990000000001, 2024-04-24T15:33:10.589 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:10.589 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:10.589 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:10.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.589 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:10.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929033+0000", 2024-04-24T15:33:10.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:10.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.589 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:10.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929033+0000", 2024-04-24T15:33:10.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:10.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.590 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:10.590 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929033+0000", 2024-04-24T15:33:10.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:10.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.590 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:10.590 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929042+0000", 2024-04-24T15:33:10.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5440000000000004e-06 2024-04-24T15:33:10.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.590 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:10.590 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929051+0000", 2024-04-24T15:33:10.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0580000000000004e-06 2024-04-24T15:33:10.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.591 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:10.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929055+0000", 2024-04-24T15:33:10.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.337e-06 2024-04-24T15:33:10.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.591 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:10.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929074+0000", 2024-04-24T15:33:10.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9724999999999998e-05 2024-04-24T15:33:10.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.591 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:10.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929084+0000", 2024-04-24T15:33:10.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7049999999999994e-06 2024-04-24T15:33:10.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:10.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.935895+0000", 2024-04-24T15:33:10.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0068104359999999996 2024-04-24T15:33:10.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:10.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.935922+0000", 2024-04-24T15:33:10.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7538999999999998e-05 2024-04-24T15:33:10.592 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:10.592 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:10.593 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:10.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.593 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:155 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'109, mlcod=15'105)", 2024-04-24T15:33:10.593 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.929398+0000", 2024-04-24T15:33:10.593 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 68.638451777, 2024-04-24T15:33:10.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0098754700000000008, 2024-04-24T15:33:10.593 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:10.593 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:10.593 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:10.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:10.594 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929398+0000", 2024-04-24T15:33:10.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:10.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.594 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.594 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:10.594 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929398+0000", 2024-04-24T15:33:10.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:10.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.594 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.594 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:10.594 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929398+0000", 2024-04-24T15:33:10.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:10.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.595 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:10.595 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929571+0000", 2024-04-24T15:33:10.595 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017281999999999999 2024-04-24T15:33:10.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.595 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:10.595 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929578+0000", 2024-04-24T15:33:10.595 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8929999999999999e-06 2024-04-24T15:33:10.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.595 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:10.595 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929581+0000", 2024-04-24T15:33:10.595 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1709999999999998e-06 2024-04-24T15:33:10.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:10.596 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929605+0000", 2024-04-24T15:33:10.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3938e-05 2024-04-24T15:33:10.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:10.596 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.929618+0000", 2024-04-24T15:33:10.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3653e-05 2024-04-24T15:33:10.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:10.596 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939186+0000", 2024-04-24T15:33:10.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0095674260000000004 2024-04-24T15:33:10.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:10.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939274+0000", 2024-04-24T15:33:10.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7928000000000002e-05 2024-04-24T15:33:10.597 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:10.597 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:10.597 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:10.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.597 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:156 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'110, mlcod=15'105)", 2024-04-24T15:33:10.597 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.930050+0000", 2024-04-24T15:33:10.597 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 68.637800460999998, 2024-04-24T15:33:10.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0092911650000000005, 2024-04-24T15:33:10.597 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:10.597 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:10.598 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:10.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.598 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:10.598 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930050+0000", 2024-04-24T15:33:10.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:10.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.598 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:10.598 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930050+0000", 2024-04-24T15:33:10.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:10.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:10.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930049+0000", 2024-04-24T15:33:10.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:10.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:10.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930053+0000", 2024-04-24T15:33:10.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.98e-06 2024-04-24T15:33:10.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:10.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930058+0000", 2024-04-24T15:33:10.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3530000000000002e-06 2024-04-24T15:33:10.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:10.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930061+0000", 2024-04-24T15:33:10.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6970000000000002e-06 2024-04-24T15:33:10.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:10.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930082+0000", 2024-04-24T15:33:10.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0815e-05 2024-04-24T15:33:10.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:10.602 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930093+0000", 2024-04-24T15:33:10.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0699999999999999e-05 2024-04-24T15:33:10.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:10.602 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939280+0000", 2024-04-24T15:33:10.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0091874880000000006 2024-04-24T15:33:10.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:10.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939341+0000", 2024-04-24T15:33:10.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0608e-05 2024-04-24T15:33:10.603 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:10.603 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:10.603 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:10.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.603 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:157 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'111, mlcod=15'106)", 2024-04-24T15:33:10.603 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.930506+0000", 2024-04-24T15:33:10.603 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 68.637343603000005, 2024-04-24T15:33:10.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.008889246, 2024-04-24T15:33:10.603 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:10.603 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:10.603 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:10.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:10.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930506+0000", 2024-04-24T15:33:10.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:10.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:10.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930506+0000", 2024-04-24T15:33:10.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:10.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:10.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930506+0000", 2024-04-24T15:33:10.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:10.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:10.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930510+0000", 2024-04-24T15:33:10.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1810000000000003e-06 2024-04-24T15:33:10.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:10.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930516+0000", 2024-04-24T15:33:10.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1920000000000002e-06 2024-04-24T15:33:10.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:10.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930520+0000", 2024-04-24T15:33:10.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.501e-06 2024-04-24T15:33:10.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:10.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930539+0000", 2024-04-24T15:33:10.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8929000000000001e-05 2024-04-24T15:33:10.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:10.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930550+0000", 2024-04-24T15:33:10.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1168999999999999e-05 2024-04-24T15:33:10.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:10.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939343+0000", 2024-04-24T15:33:10.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0087932900000000005 2024-04-24T15:33:10.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:10.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939396+0000", 2024-04-24T15:33:10.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2454999999999998e-05 2024-04-24T15:33:10.607 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:10.607 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:10.607 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:10.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.607 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:158 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'112, mlcod=15'106)", 2024-04-24T15:33:10.607 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.930877+0000", 2024-04-24T15:33:10.607 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 68.636972912000004, 2024-04-24T15:33:10.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.008572741, 2024-04-24T15:33:10.607 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:10.607 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:10.608 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:10.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:10.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930877+0000", 2024-04-24T15:33:10.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:10.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:10.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930877+0000", 2024-04-24T15:33:10.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:10.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:10.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930877+0000", 2024-04-24T15:33:10.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:10.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:10.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930968+0000", 2024-04-24T15:33:10.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1638999999999995e-05 2024-04-24T15:33:10.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:10.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930975+0000", 2024-04-24T15:33:10.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2169999999999999e-06 2024-04-24T15:33:10.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:10.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.930977+0000", 2024-04-24T15:33:10.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.791e-06 2024-04-24T15:33:10.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:10.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931002+0000", 2024-04-24T15:33:10.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4491e-05 2024-04-24T15:33:10.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:10.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931012+0000", 2024-04-24T15:33:10.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0217000000000001e-05 2024-04-24T15:33:10.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:10.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939398+0000", 2024-04-24T15:33:10.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.008385366 2024-04-24T15:33:10.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:10.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939450+0000", 2024-04-24T15:33:10.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2326e-05 2024-04-24T15:33:10.611 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:10.611 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:10.611 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:10.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.611 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4184.0:159 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'113, mlcod=15'106)", 2024-04-24T15:33:10.611 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:01.931313+0000", 2024-04-24T15:33:10.611 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 68.636537559999994, 2024-04-24T15:33:10.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0081994159999999993, 2024-04-24T15:33:10.612 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:10.612 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:10.612 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:10.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:10.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931313+0000", 2024-04-24T15:33:10.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:10.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:10.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931313+0000", 2024-04-24T15:33:10.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:10.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:10.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931312+0000", 2024-04-24T15:33:10.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:10.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:10.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931316+0000", 2024-04-24T15:33:10.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0589999999999996e-06 2024-04-24T15:33:10.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:10.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931322+0000", 2024-04-24T15:33:10.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2020000000000001e-06 2024-04-24T15:33:10.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:10.614 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931327+0000", 2024-04-24T15:33:10.614 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2440000000000004e-06 2024-04-24T15:33:10.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.614 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:10.614 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931348+0000", 2024-04-24T15:33:10.614 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1464e-05 2024-04-24T15:33:10.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.614 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:10.614 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.931359+0000", 2024-04-24T15:33:10.614 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0699e-05 2024-04-24T15:33:10.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:10.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939452+0000", 2024-04-24T15:33:10.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0080931749999999993 2024-04-24T15:33:10.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:10.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:01.939512+0000", 2024-04-24T15:33:10.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0050000000000003e-05 2024-04-24T15:33:10.615 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:10.615 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:10.615 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:10.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.615 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:2447 4.6 4:668081fa:::rbd_data.114a1f60cb66.00000000000001f0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-24T15:33:10.615 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:23.027189+0000", 2024-04-24T15:33:10.616 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 47.540661536000002, 2024-04-24T15:33:10.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0056741200000000004, 2024-04-24T15:33:10.616 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:10.616 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:10.616 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:10.616 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:10.616 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:10.616 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2447 2024-04-24T15:33:10.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.616 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:10.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:10.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027189+0000", 2024-04-24T15:33:10.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:10.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:10.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027189+0000", 2024-04-24T15:33:10.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:10.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:10.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027188+0000", 2024-04-24T15:33:10.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:33:10.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:10.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027194+0000", 2024-04-24T15:33:10.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0499999999999997e-06 2024-04-24T15:33:10.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.618 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:10.618 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027205+0000", 2024-04-24T15:33:10.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1619e-05 2024-04-24T15:33:10.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.618 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:10.618 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027212+0000", 2024-04-24T15:33:10.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4289999999999998e-06 2024-04-24T15:33:10.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.618 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:10.618 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027243+0000", 2024-04-24T15:33:10.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1510000000000002e-05 2024-04-24T15:33:10.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:10.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027288+0000", 2024-04-24T15:33:10.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4484000000000003e-05 2024-04-24T15:33:10.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-24T15:33:10.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.027351+0000", 2024-04-24T15:33:10.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3311000000000004e-05 2024-04-24T15:33:10.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:33:10.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.029874+0000", 2024-04-24T15:33:10.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00252315 2024-04-24T15:33:10.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:33:10.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.032786+0000", 2024-04-24T15:33:10.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0029115370000000001 2024-04-24T15:33:10.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:10.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.032828+0000", 2024-04-24T15:33:10.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2132000000000003e-05 2024-04-24T15:33:10.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:10.621 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.032863+0000", 2024-04-24T15:33:10.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4912e-05 2024-04-24T15:33:10.621 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:10.621 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:10.621 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:10.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.621 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4426.0:4119 4.6 e34/33 4:6588fd86:::rbd_data.114a1f60cb66.0000000000000179:head v 34'48, mlcod=34'45)", 2024-04-24T15:33:10.621 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:24.936723+0000", 2024-04-24T15:33:10.622 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 45.631127094999997, 2024-04-24T15:33:10.622 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0058208640000000002, 2024-04-24T15:33:10.622 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:10.622 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:10.622 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:10.622 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.622 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:10.622 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.936723+0000", 2024-04-24T15:33:10.623 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:10.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.623 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:10.623 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.936723+0000", 2024-04-24T15:33:10.623 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:10.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.623 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:10.624 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.936722+0000", 2024-04-24T15:33:10.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:10.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.624 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:10.624 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.936735+0000", 2024-04-24T15:33:10.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2153999999999999e-05 2024-04-24T15:33:10.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.625 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:10.625 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.936743+0000", 2024-04-24T15:33:10.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9360000000000005e-06 2024-04-24T15:33:10.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.625 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:10.625 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.936749+0000", 2024-04-24T15:33:10.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5040000000000002e-06 2024-04-24T15:33:10.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.625 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:10.626 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.936783+0000", 2024-04-24T15:33:10.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3689999999999998e-05 2024-04-24T15:33:10.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.626 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:10.626 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.936803+0000", 2024-04-24T15:33:10.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0047999999999998e-05 2024-04-24T15:33:10.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.626 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:10.626 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.942517+0000", 2024-04-24T15:33:10.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057144220000000003 2024-04-24T15:33:10.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.626 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:10.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.942544+0000", 2024-04-24T15:33:10.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6735999999999999e-05 2024-04-24T15:33:10.627 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:10.627 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:10.627 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:10.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.627 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4426.0:24281 4.7 e45/43 4:e1ec41c1:::rbd_data.114ae15b9a20.0000000000000066:head v 45'58, mlcod=0'0)", 2024-04-24T15:33:10.627 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:37.194120+0000", 2024-04-24T15:33:10.627 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.373730432999999, 2024-04-24T15:33:10.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0091868560000000002, 2024-04-24T15:33:10.627 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:10.627 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:10.627 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:10.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:10.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.194120+0000", 2024-04-24T15:33:10.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:10.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:10.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.194120+0000", 2024-04-24T15:33:10.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:10.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:10.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.194119+0000", 2024-04-24T15:33:10.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:33:10.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:10.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.194125+0000", 2024-04-24T15:33:10.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5050000000000004e-06 2024-04-24T15:33:10.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:10.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.194144+0000", 2024-04-24T15:33:10.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8454e-05 2024-04-24T15:33:10.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:10.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.194153+0000", 2024-04-24T15:33:10.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3810000000000005e-06 2024-04-24T15:33:10.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:10.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.202288+0000", 2024-04-24T15:33:10.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0081342830000000008 2024-04-24T15:33:10.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:10.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.202304+0000", 2024-04-24T15:33:10.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6164999999999999e-05 2024-04-24T15:33:10.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:10.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.203285+0000", 2024-04-24T15:33:10.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00098150400000000001 2024-04-24T15:33:10.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:10.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:37.203306+0000", 2024-04-24T15:33:10.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1285e-05 2024-04-24T15:33:10.631 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:10.631 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:10.631 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:10.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.631 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4426.0:26385 4.1e e51/33 4:785cf1e5:::rbd_data.114a8e7208a5.0000000000000004:head v 51'141, mlcod=45'139)", 2024-04-24T15:33:10.631 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:43.542226+0000", 2024-04-24T15:33:10.631 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.025623929999998, 2024-04-24T15:33:10.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0098378319999999995, 2024-04-24T15:33:10.632 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:10.632 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:10.632 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:10.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:10.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:43.542226+0000", 2024-04-24T15:33:10.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:10.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:10.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:43.542226+0000", 2024-04-24T15:33:10.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:10.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:10.633 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:43.542225+0000", 2024-04-24T15:33:10.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:33:10.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.633 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:10.633 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:43.549287+0000", 2024-04-24T15:33:10.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0070620470000000001 2024-04-24T15:33:10.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.633 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:10.633 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:43.549299+0000", 2024-04-24T15:33:10.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1396999999999999e-05 2024-04-24T15:33:10.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.633 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:10.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:43.549321+0000", 2024-04-24T15:33:10.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2129000000000001e-05 2024-04-24T15:33:10.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:10.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:43.549361+0000", 2024-04-24T15:33:10.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0561e-05 2024-04-24T15:33:10.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:10.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:43.549386+0000", 2024-04-24T15:33:10.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4425000000000001e-05 2024-04-24T15:33:10.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:10.635 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:43.552024+0000", 2024-04-24T15:33:10.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0026388280000000002 2024-04-24T15:33:10.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:10.635 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:43.552064+0000", 2024-04-24T15:33:10.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9495e-05 2024-04-24T15:33:10.635 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:10.635 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:10.635 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:10.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.635 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4426.0:29864 4.1f e53/46 4:fce51a46:::rbd_data.114a2e5eebc.0000000000000256:head v 53'1252, mlcod=53'1250)", 2024-04-24T15:33:10.635 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:46.445862+0000", 2024-04-24T15:33:10.635 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 24.121987675, 2024-04-24T15:33:10.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010959694000000001, 2024-04-24T15:33:10.636 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:10.636 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:10.636 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:10.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:10.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.445862+0000", 2024-04-24T15:33:10.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:10.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:10.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.445862+0000", 2024-04-24T15:33:10.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:10.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.637 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:10.637 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.445861+0000", 2024-04-24T15:33:10.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999986 2024-04-24T15:33:10.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.637 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:10.637 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.446309+0000", 2024-04-24T15:33:10.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00044818000000000001 2024-04-24T15:33:10.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.637 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:10.637 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.446330+0000", 2024-04-24T15:33:10.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0754000000000002e-05 2024-04-24T15:33:10.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.638 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:10.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.446338+0000", 2024-04-24T15:33:10.638 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6320000000000008e-06 2024-04-24T15:33:10.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.638 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:10.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.448537+0000", 2024-04-24T15:33:10.638 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0021991509999999999 2024-04-24T15:33:10.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.638 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:10.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.448565+0000", 2024-04-24T15:33:10.638 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7949e-05 2024-04-24T15:33:10.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.639 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:10.639 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.456773+0000", 2024-04-24T15:33:10.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0082081930000000008 2024-04-24T15:33:10.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.639 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:10.639 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.456822+0000", 2024-04-24T15:33:10.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9270000000000001e-05 2024-04-24T15:33:10.639 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:10.639 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:10.639 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:10.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.639 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4426.0:29863 4.2 e53/43 4:45d67f64:::rbd_data.114a2e5eebc.0000000000000255:head v 53'111, mlcod=53'109)", 2024-04-24T15:33:10.640 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:46.450064+0000", 2024-04-24T15:33:10.640 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 24.117786377000002, 2024-04-24T15:33:10.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0067851589999999998, 2024-04-24T15:33:10.640 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:10.640 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:10.640 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:10.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:10.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.450064+0000", 2024-04-24T15:33:10.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:10.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:10.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.450064+0000", 2024-04-24T15:33:10.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:10.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.641 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:10.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.450062+0000", 2024-04-24T15:33:10.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999986 2024-04-24T15:33:10.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.641 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:10.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.450080+0000", 2024-04-24T15:33:10.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7073000000000001e-05 2024-04-24T15:33:10.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.642 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:10.642 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.450091+0000", 2024-04-24T15:33:10.642 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1323e-05 2024-04-24T15:33:10.642 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.642 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.642 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:10.642 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.450096+0000", 2024-04-24T15:33:10.642 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.95e-06 2024-04-24T15:33:10.642 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.643 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:10.643 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.450146+0000", 2024-04-24T15:33:10.643 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9849999999999999e-05 2024-04-24T15:33:10.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.643 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:10.643 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.450164+0000", 2024-04-24T15:33:10.643 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8804000000000002e-05 2024-04-24T15:33:10.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:10.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.456807+0000", 2024-04-24T15:33:10.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0066423760000000002 2024-04-24T15:33:10.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:10.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:46.456849+0000", 2024-04-24T15:33:10.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1992e-05 2024-04-24T15:33:10.645 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:10.645 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:10.645 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:10.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.645 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4426.0:35373 4.7 e62/54 4:e251aff1:::rbd_data.114a96db60f9.0000000000000017:head v 62'111, mlcod=61'108)", 2024-04-24T15:33:10.645 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:55.741562+0000", 2024-04-24T15:33:10.645 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.826288499, 2024-04-24T15:33:10.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0059971139999999996, 2024-04-24T15:33:10.645 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:10.646 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:10.646 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:10.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.646 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:10.646 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:55.741562+0000", 2024-04-24T15:33:10.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:10.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.646 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:10.646 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:55.741562+0000", 2024-04-24T15:33:10.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:10.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.647 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:10.647 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:55.741561+0000", 2024-04-24T15:33:10.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:10.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.647 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:10.647 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:55.741567+0000", 2024-04-24T15:33:10.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4350000000000002e-06 2024-04-24T15:33:10.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.647 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:10.647 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:55.741577+0000", 2024-04-24T15:33:10.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7419999999999992e-06 2024-04-24T15:33:10.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.648 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:10.648 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:55.741583+0000", 2024-04-24T15:33:10.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3770000000000004e-06 2024-04-24T15:33:10.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.648 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:10.648 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:55.742158+0000", 2024-04-24T15:33:10.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00057516800000000003 2024-04-24T15:33:10.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.648 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:10.648 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:55.742177+0000", 2024-04-24T15:33:10.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9250999999999999e-05 2024-04-24T15:33:10.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:10.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:55.747519+0000", 2024-04-24T15:33:10.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0053422519999999996 2024-04-24T15:33:10.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:10.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:55.747559+0000", 2024-04-24T15:33:10.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9484999999999999e-05 2024-04-24T15:33:10.649 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:10.649 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:10.649 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:10.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.650 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4426.0:36199 4.1a e62/41 4:5a86856e:::rbd_data.114ab4f6fc67.000000000000000c:head v 62'269, mlcod=62'267)", 2024-04-24T15:33:10.650 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:56.585759+0000", 2024-04-24T15:33:10.650 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.982091449, 2024-04-24T15:33:10.650 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057717710000000002, 2024-04-24T15:33:10.650 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:10.650 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:10.650 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:10.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.650 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:10.650 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:56.585759+0000", 2024-04-24T15:33:10.650 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:10.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.650 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:10.650 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:56.585759+0000", 2024-04-24T15:33:10.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:10.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.651 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:10.651 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:56.585758+0000", 2024-04-24T15:33:10.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:10.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.651 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:10.651 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:56.585763+0000", 2024-04-24T15:33:10.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9439999999999996e-06 2024-04-24T15:33:10.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.651 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:10.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:56.585770+0000", 2024-04-24T15:33:10.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8920000000000003e-06 2024-04-24T15:33:10.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:10.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:56.585774+0000", 2024-04-24T15:33:10.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0289999999999999e-06 2024-04-24T15:33:10.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:10.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:56.585820+0000", 2024-04-24T15:33:10.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6147999999999999e-05 2024-04-24T15:33:10.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:10.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:56.585839+0000", 2024-04-24T15:33:10.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.863e-05 2024-04-24T15:33:10.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:10.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:56.591488+0000", 2024-04-24T15:33:10.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0056494040000000002 2024-04-24T15:33:10.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:10.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:56.591530+0000", 2024-04-24T15:33:10.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2199e-05 2024-04-24T15:33:10.653 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:10.653 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:10.653 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:10.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.654 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4426.0:40766 4.a e72/57 4:514762fe:::rbd_data.114af25c4d4b.0000000000000000:head v 72'243, mlcod=72'241)", 2024-04-24T15:33:10.654 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:06.035222+0000", 2024-04-24T15:33:10.654 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.5326282920000001, 2024-04-24T15:33:10.654 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0063806660000000001, 2024-04-24T15:33:10.654 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:10.654 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:10.654 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:10.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.654 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:10.654 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:06.035222+0000", 2024-04-24T15:33:10.654 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:10.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.655 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:10.655 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:06.035222+0000", 2024-04-24T15:33:10.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:10.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.655 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:10.655 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:06.035221+0000", 2024-04-24T15:33:10.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:33:10.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.655 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:10.655 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:06.038346+0000", 2024-04-24T15:33:10.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0031252699999999999 2024-04-24T15:33:10.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.656 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:10.656 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:06.038362+0000", 2024-04-24T15:33:10.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6079e-05 2024-04-24T15:33:10.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.656 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:10.656 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:06.038369+0000", 2024-04-24T15:33:10.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.17e-06 2024-04-24T15:33:10.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.656 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:10.656 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:06.038407+0000", 2024-04-24T15:33:10.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7526e-05 2024-04-24T15:33:10.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.656 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:10.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:06.038424+0000", 2024-04-24T15:33:10.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7314000000000001e-05 2024-04-24T15:33:10.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:10.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:06.041571+0000", 2024-04-24T15:33:10.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0031464029999999999 2024-04-24T15:33:10.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:10.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:06.041602+0000", 2024-04-24T15:33:10.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1714000000000002e-05 2024-04-24T15:33:10.657 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:10.657 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:10.657 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:10.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.658 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4426.0:41752 4.3 e73/57 4:c2048668:::rbd_data.114ad3a18a61.000000000000000a:head v 73'248, mlcod=70'246)", 2024-04-24T15:33:10.658 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.036404+0000", 2024-04-24T15:33:10.658 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.5314462610000001, 2024-04-24T15:33:10.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.005983492, 2024-04-24T15:33:10.658 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:10.658 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:10.658 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:10.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:10.658 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.036404+0000", 2024-04-24T15:33:10.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:10.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:10.658 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.036404+0000", 2024-04-24T15:33:10.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:10.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.659 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:10.659 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.036402+0000", 2024-04-24T15:33:10.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999981 2024-04-24T15:33:10.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.659 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:10.659 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.038886+0000", 2024-04-24T15:33:10.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024842919999999999 2024-04-24T15:33:10.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.659 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:10.659 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.038910+0000", 2024-04-24T15:33:10.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3688000000000001e-05 2024-04-24T15:33:10.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.660 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:10.660 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.038921+0000", 2024-04-24T15:33:10.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0827e-05 2024-04-24T15:33:10.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.660 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:10.660 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.038983+0000", 2024-04-24T15:33:10.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2281999999999998e-05 2024-04-24T15:33:10.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.660 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:10.660 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.039003+0000", 2024-04-24T15:33:10.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0339e-05 2024-04-24T15:33:10.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:10.661 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.042340+0000", 2024-04-24T15:33:10.661 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0033363669999999998 2024-04-24T15:33:10.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:10.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:10.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:10.661 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.042387+0000", 2024-04-24T15:33:10.661 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7691999999999997e-05 2024-04-24T15:33:10.661 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:10.661 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:10.661 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:10.661 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:10.661 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:10.661 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:10.663 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-24T15:33:10.664 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:10.663+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:10.764 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:33:10.765+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:33:10.787 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:10.787 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:33:10.787 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-04-24T15:33:10.787 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:10.797 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-24T15:33:10.822 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-24T15:33:10.871 INFO:teuthology.orchestra.run.smithi096.stdout:flattening image image_client.0-clone20 2024-04-24T15:33:10.871 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:10.872+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:10.966 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:10.966+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:10.994 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:10.994 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:33:10.994 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-04-24T15:33:10.994 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-04-24T15:33:10.995 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:11.011 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-24T15:33:11.066 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:11.067+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:11.148 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #19, image image_client.0-clone19 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent20 2024-04-24T15:33:11.167 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:33:11.169+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:33:11.186 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:11.187 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-04-24T15:33:11.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-04-24T15:33:11.187 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [] 2024-04-24T15:33:11.187 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:11.198 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-24T15:33:11.199 INFO:teuthology.orchestra.run.smithi096.stdout:3985 read 0x52deff1 thru 0x52ee03a (0xf04a bytes) 2024-04-24T15:33:11.199 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:11.200 INFO:teuthology.orchestra.run.smithi096.stdout:3989 punch from 0x4bf4160 to 0x4bf7782, (0x3622 bytes) 2024-04-24T15:33:11.218 INFO:teuthology.orchestra.run.smithi096.stdout:3990 trunc from 0xe66ef68 to 0xe16c06 2024-04-24T15:33:11.231 INFO:teuthology.orchestra.run.smithi096.stdout:3991 read 0x70989d thru 0x7157d9 (0xbf3d bytes) 2024-04-24T15:33:11.232 INFO:teuthology.orchestra.run.smithi096.stdout:3992 write 0x7e00d4 thru 0x7ef8d9 (0xf806 bytes) 2024-04-24T15:33:11.240 INFO:teuthology.orchestra.run.smithi096.stdout:3993 punch from 0x81fc45 to 0x8232a8, (0x3663 bytes) 2024-04-24T15:33:11.241 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:11.260 INFO:teuthology.orchestra.run.smithi096.stdout:3997 writesame 0xc003a7d thru 0xc0122ed data_size 0x5(0xe871 bytes) 2024-04-24T15:33:11.260 INFO:teuthology.orchestra.run.smithi096.stdout:3999 punch from 0x81eadad to 0x81f04c8, (0x571b bytes) 2024-04-24T15:33:11.260 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:11.260 INFO:teuthology.orchestra.run.smithi096.stdout:4001 punch from 0xbb5ce8e to 0xbb67a6a, (0xabdc bytes) 2024-04-24T15:33:11.261 INFO:teuthology.orchestra.run.smithi096.stdout:4002 write 0x5a61a97 thru 0x5a6989f (0x7e09 bytes) 2024-04-24T15:33:11.265 INFO:teuthology.orchestra.run.smithi096.stdout:4003 read 0x3534 thru 0x7666 (0x4133 bytes) 2024-04-24T15:33:11.266 INFO:teuthology.orchestra.run.smithi096.stdout:4005 write 0xb0227ce thru 0xb02354f (0xd82 bytes) 2024-04-24T15:33:11.269 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:11.269+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:11.272 INFO:teuthology.orchestra.run.smithi096.stdout:4006 trunc from 0xc0122ee to 0x9f785a7 2024-04-24T15:33:11.283 INFO:teuthology.orchestra.run.smithi096.stdout:4007 read 0x4770f81 thru 0x4779ee5 (0x8f65 bytes) 2024-04-24T15:33:11.283 INFO:teuthology.orchestra.run.smithi096.stdout:4008 read 0x139ab30 thru 0x13a517d (0xa64e bytes) 2024-04-24T15:33:11.284 INFO:teuthology.orchestra.run.smithi096.stdout:4009 write 0x9e1729d thru 0x9e1f10c (0x7e70 bytes) 2024-04-24T15:33:11.291 INFO:teuthology.orchestra.run.smithi096.stdout:4010 trunc from 0x9f785a7 to 0xbddb9b5 2024-04-24T15:33:11.298 INFO:teuthology.orchestra.run.smithi096.stdout:4013 read 0x3a85924 thru 0x3a8ab47 (0x5224 bytes) 2024-04-24T15:33:11.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-24T15:33:11.300+0000 7f72a5200340 -1 bluestore::NCB::__restore_allocator::No Valid allocation info on disk (empty file) 2024-04-24T15:33:11.305 INFO:teuthology.orchestra.run.smithi096.stdout:4014 write 0xd7823f0 thru 0xd784a6e (0x267f bytes) 2024-04-24T15:33:11.309 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:11.309 INFO:teuthology.orchestra.run.smithi096.stdout:4018 read 0x240b22e thru 0x240dead (0x2c80 bytes) 2024-04-24T15:33:11.309 INFO:teuthology.orchestra.run.smithi096.stdout:4020 read 0x7dd7a7d thru 0x7ddba0a (0x3f8e bytes) 2024-04-24T15:33:11.311 INFO:teuthology.orchestra.run.smithi096.stdout:4021 trunc from 0xd784a6f to 0xbfc6659 2024-04-24T15:33:11.322 INFO:teuthology.orchestra.run.smithi096.stdout:4022 write 0x7e7bbe9 thru 0x7e7e927 (0x2d3f bytes) 2024-04-24T15:33:11.326 INFO:teuthology.orchestra.run.smithi096.stdout:4023 writesame 0x8393889 thru 0x839b0ef data_size 0x1(0x7867 bytes) 2024-04-24T15:33:11.338 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:11.338 INFO:teuthology.orchestra.run.smithi096.stdout:4028 clone 22 order 22 su 524288 sc 11 2024-04-24T15:33:11.344 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:11.344 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:33:11.344 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-04-24T15:33:11.345 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:11.355 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-24T15:33:11.369 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:11.370+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:11.470 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:33:11.471+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:33:11.518 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:11.518 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:33:11.518 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-04-24T15:33:11.518 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-04-24T15:33:11.518 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:11.528 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-24T15:33:11.572 INFO:teuthology.orchestra.run.smithi151.stdout:4.a 2024-04-24T15:33:11.572 INFO:teuthology.orchestra.run.smithi151.stdout:4.1f 2024-04-24T15:33:11.572 INFO:teuthology.orchestra.run.smithi151.stdout:4.5 2024-04-24T15:33:11.572 INFO:teuthology.orchestra.run.smithi151.stdout:4.1 2024-04-24T15:33:11.572 INFO:teuthology.orchestra.run.smithi151.stdout:4.18 2024-04-24T15:33:11.572 INFO:teuthology.orchestra.run.smithi151.stdout:4.15 2024-04-24T15:33:11.572 INFO:teuthology.orchestra.run.smithi151.stdout:4.e 2024-04-24T15:33:11.572 INFO:teuthology.orchestra.run.smithi151.stdout:4.13 2024-04-24T15:33:11.572 INFO:teuthology.orchestra.run.smithi151.stdout:4.12 2024-04-24T15:33:11.572 INFO:teuthology.orchestra.run.smithi151.stdout:4.11 2024-04-24T15:33:11.572 INFO:teuthology.orchestra.run.smithi151.stdout:4.10 2024-04-24T15:33:11.572 INFO:teuthology.orchestra.run.smithi151.stdout:3.2 2024-04-24T15:33:11.573 INFO:teuthology.orchestra.run.smithi151.stdout:3.0 2024-04-24T15:33:11.573 INFO:teuthology.orchestra.run.smithi151.stdout:2.7 2024-04-24T15:33:11.573 INFO:teuthology.orchestra.run.smithi151.stdout:4.8 2024-04-24T15:33:11.573 INFO:teuthology.orchestra.run.smithi151.stdout:2.6 2024-04-24T15:33:11.573 INFO:teuthology.orchestra.run.smithi151.stdout:2.2 2024-04-24T15:33:11.573 INFO:teuthology.orchestra.run.smithi151.stdout:1.0 2024-04-24T15:33:11.611 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:11.613+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:11.672 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:33:11.673+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:33:11.694 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:11.694 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-04-24T15:33:11.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-04-24T15:33:11.694 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-04-24T15:33:11.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.694 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:2522 4.6 4:60ceedf0:::rbd_data.114a1f60cb66.000000000000011d:head [set-alloc-hint object_size 262144 write_size 262144,write 110592~2048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-24T15:33:11.694 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:23.088240+0000", 2024-04-24T15:33:11.695 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 48.605737941000001, 2024-04-24T15:33:11.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.988232797, 2024-04-24T15:33:11.695 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:11.695 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:11.695 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:11.695 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:11.695 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:11.695 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2522 2024-04-24T15:33:11.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.695 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:11.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:11.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088240+0000", 2024-04-24T15:33:11.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:11.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.696 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:11.696 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088240+0000", 2024-04-24T15:33:11.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:11.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.696 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:11.696 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088238+0000", 2024-04-24T15:33:11.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999981 2024-04-24T15:33:11.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.696 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:11.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088254+0000", 2024-04-24T15:33:11.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5562000000000001e-05 2024-04-24T15:33:11.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.697 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:11.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088275+0000", 2024-04-24T15:33:11.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1256000000000001e-05 2024-04-24T15:33:11.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.697 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:11.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088283+0000", 2024-04-24T15:33:11.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3219999999999993e-06 2024-04-24T15:33:11.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.698 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:11.698 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088351+0000", 2024-04-24T15:33:11.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7595000000000006e-05 2024-04-24T15:33:11.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.698 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-04-24T15:33:11.698 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088362+0000", 2024-04-24T15:33:11.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1764e-05 2024-04-24T15:33:11.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.698 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:11.698 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.073748+0000", 2024-04-24T15:33:11.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98538541899999998 2024-04-24T15:33:11.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:11.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.073925+0000", 2024-04-24T15:33:11.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017747600000000001 2024-04-24T15:33:11.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-24T15:33:11.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.074092+0000", 2024-04-24T15:33:11.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000166622 2024-04-24T15:33:11.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:33:11.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.074774+0000", 2024-04-24T15:33:11.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00068225000000000002 2024-04-24T15:33:11.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:33:11.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.076412+0000", 2024-04-24T15:33:11.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001638189 2024-04-24T15:33:11.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:11.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.076446+0000", 2024-04-24T15:33:11.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3587000000000002e-05 2024-04-24T15:33:11.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:11.701 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.076473+0000", 2024-04-24T15:33:11.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6886999999999999e-05 2024-04-24T15:33:11.701 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:11.701 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:11.701 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:11.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.701 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43108 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 327680~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:11.701 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.093966+0000", 2024-04-24T15:33:11.701 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.600011673, 2024-04-24T15:33:11.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.030814699000000001, 2024-04-24T15:33:11.701 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:11.701 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:11.702 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:11.702 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:11.702 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:11.702 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43108 2024-04-24T15:33:11.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.702 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:11.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.702 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:11.702 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.093966+0000", 2024-04-24T15:33:11.702 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:11.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.702 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:11.703 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.093966+0000", 2024-04-24T15:33:11.703 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:11.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.703 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:11.703 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.093966+0000", 2024-04-24T15:33:11.703 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:11.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.703 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:11.703 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.093968+0000", 2024-04-24T15:33:11.703 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2120000000000002e-06 2024-04-24T15:33:11.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.703 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:11.704 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.093981+0000", 2024-04-24T15:33:11.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2797e-05 2024-04-24T15:33:11.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.704 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.704 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:11.704 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.093985+0000", 2024-04-24T15:33:11.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0539999999999996e-06 2024-04-24T15:33:11.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.704 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.704 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:11.704 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.124662+0000", 2024-04-24T15:33:11.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.030676717999999999 2024-04-24T15:33:11.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.704 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:11.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.124712+0000", 2024-04-24T15:33:11.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0523999999999999e-05 2024-04-24T15:33:11.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:11.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.124781+0000", 2024-04-24T15:33:11.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8753999999999999e-05 2024-04-24T15:33:11.705 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:11.705 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:11.705 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:11.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.705 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43109 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 753664~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:11.705 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.093990+0000", 2024-04-24T15:33:11.706 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.5999881760000001, 2024-04-24T15:33:11.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.031011216000000001, 2024-04-24T15:33:11.706 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:11.706 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:11.706 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:11.706 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:11.706 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:11.706 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43109 2024-04-24T15:33:11.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.706 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:11.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.706 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:11.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.093990+0000", 2024-04-24T15:33:11.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:11.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.707 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:11.707 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.093990+0000", 2024-04-24T15:33:11.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:11.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.707 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:11.707 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.093990+0000", 2024-04-24T15:33:11.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:11.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.707 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:11.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.093992+0000", 2024-04-24T15:33:11.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8990000000000001e-06 2024-04-24T15:33:11.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:11.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.093998+0000", 2024-04-24T15:33:11.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7019999999999998e-06 2024-04-24T15:33:11.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:11.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094002+0000", 2024-04-24T15:33:11.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5829999999999998e-06 2024-04-24T15:33:11.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:11.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.124822+0000", 2024-04-24T15:33:11.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.030820276000000001 2024-04-24T15:33:11.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:11.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.124909+0000", 2024-04-24T15:33:11.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6762999999999996e-05 2024-04-24T15:33:11.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:11.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.125001+0000", 2024-04-24T15:33:11.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2356999999999997e-05 2024-04-24T15:33:11.709 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:11.709 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:11.710 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:11.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.710 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43110 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 1179648~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:11.710 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094006+0000", 2024-04-24T15:33:11.710 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.5999719539999999, 2024-04-24T15:33:11.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.031113297000000002, 2024-04-24T15:33:11.710 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:11.710 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:11.710 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:11.710 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:11.710 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:11.711 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43110 2024-04-24T15:33:11.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.711 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:11.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:11.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094006+0000", 2024-04-24T15:33:11.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:11.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:11.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094006+0000", 2024-04-24T15:33:11.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:11.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:11.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094006+0000", 2024-04-24T15:33:11.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:11.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:11.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094008+0000", 2024-04-24T15:33:11.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6089999999999999e-06 2024-04-24T15:33:11.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:11.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094014+0000", 2024-04-24T15:33:11.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3940000000000004e-06 2024-04-24T15:33:11.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:11.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094017+0000", 2024-04-24T15:33:11.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4709999999999999e-06 2024-04-24T15:33:11.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.713 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:11.713 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.125013+0000", 2024-04-24T15:33:11.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.030995742 2024-04-24T15:33:11.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.713 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:11.713 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.125060+0000", 2024-04-24T15:33:11.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6894999999999997e-05 2024-04-24T15:33:11.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.713 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:11.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.125119+0000", 2024-04-24T15:33:11.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9521000000000002e-05 2024-04-24T15:33:11.714 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:11.714 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:11.714 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:11.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.714 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43111 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 1605632~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:11.714 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094022+0000", 2024-04-24T15:33:11.714 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.5999562530000002, 2024-04-24T15:33:11.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.031292357999999999, 2024-04-24T15:33:11.714 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:11.714 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:11.715 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:11.715 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:11.715 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:11.715 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43111 2024-04-24T15:33:11.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.715 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:11.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:11.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094022+0000", 2024-04-24T15:33:11.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:11.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:11.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094022+0000", 2024-04-24T15:33:11.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:11.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.716 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:11.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094021+0000", 2024-04-24T15:33:11.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:11.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.716 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:11.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094024+0000", 2024-04-24T15:33:11.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2749999999999998e-06 2024-04-24T15:33:11.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.716 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:11.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094029+0000", 2024-04-24T15:33:11.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7270000000000004e-06 2024-04-24T15:33:11.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:11.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094033+0000", 2024-04-24T15:33:11.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3239999999999999e-06 2024-04-24T15:33:11.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:11.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.125138+0000", 2024-04-24T15:33:11.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.031105738000000001 2024-04-24T15:33:11.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:11.718 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.125216+0000", 2024-04-24T15:33:11.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7824000000000006e-05 2024-04-24T15:33:11.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:11.718 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.125314+0000", 2024-04-24T15:33:11.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7853e-05 2024-04-24T15:33:11.718 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:11.718 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:11.718 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:11.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.718 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43112 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 2031616~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:11.718 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094037+0000", 2024-04-24T15:33:11.719 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.5999412719999997, 2024-04-24T15:33:11.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.031503482999999999, 2024-04-24T15:33:11.719 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:11.719 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:11.719 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:11.719 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:11.719 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:11.719 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43112 2024-04-24T15:33:11.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.719 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:11.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:11.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094037+0000", 2024-04-24T15:33:11.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:11.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:11.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094037+0000", 2024-04-24T15:33:11.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:11.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:11.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094036+0000", 2024-04-24T15:33:11.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:11.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:11.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094039+0000", 2024-04-24T15:33:11.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0590000000000001e-06 2024-04-24T15:33:11.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.721 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:11.721 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094044+0000", 2024-04-24T15:33:11.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7710000000000003e-06 2024-04-24T15:33:11.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.721 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:11.721 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094048+0000", 2024-04-24T15:33:11.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9419999999999997e-06 2024-04-24T15:33:11.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.721 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:11.721 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.125352+0000", 2024-04-24T15:33:11.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.031304037999999999 2024-04-24T15:33:11.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.722 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.722 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:11.722 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.125438+0000", 2024-04-24T15:33:11.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5681999999999997e-05 2024-04-24T15:33:11.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.722 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.722 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:11.722 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.125540+0000", 2024-04-24T15:33:11.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010229399999999999 2024-04-24T15:33:11.722 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:11.722 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:11.722 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:11.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.723 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43113 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 2457600~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:11.723 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094056+0000", 2024-04-24T15:33:11.723 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.599921664, 2024-04-24T15:33:11.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.031639255999999998, 2024-04-24T15:33:11.723 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:11.723 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:11.723 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:11.723 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:11.723 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:11.723 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43113 2024-04-24T15:33:11.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.724 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:11.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.724 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:11.724 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094056+0000", 2024-04-24T15:33:11.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:11.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.724 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:11.724 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094056+0000", 2024-04-24T15:33:11.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:11.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.724 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:11.724 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094056+0000", 2024-04-24T15:33:11.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:11.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.725 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:11.725 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094058+0000", 2024-04-24T15:33:11.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2510000000000001e-06 2024-04-24T15:33:11.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.725 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:11.725 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094064+0000", 2024-04-24T15:33:11.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0070000000000003e-06 2024-04-24T15:33:11.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.725 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:11.725 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094068+0000", 2024-04-24T15:33:11.726 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.912e-06 2024-04-24T15:33:11.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.726 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.726 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:11.726 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.125568+0000", 2024-04-24T15:33:11.726 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.031499857999999999 2024-04-24T15:33:11.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.726 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.726 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:11.726 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.125629+0000", 2024-04-24T15:33:11.726 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0877e-05 2024-04-24T15:33:11.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.726 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.726 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:11.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.125696+0000", 2024-04-24T15:33:11.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6680999999999994e-05 2024-04-24T15:33:11.727 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:11.727 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:11.727 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:11.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.727 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43114 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 2883584~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:11.727 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094073+0000", 2024-04-24T15:33:11.727 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.5999053840000004, 2024-04-24T15:33:11.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.031834029999999999, 2024-04-24T15:33:11.727 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:11.727 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:11.727 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:11.728 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:11.728 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:11.728 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43114 2024-04-24T15:33:11.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.728 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:11.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:11.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094073+0000", 2024-04-24T15:33:11.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:11.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:11.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094073+0000", 2024-04-24T15:33:11.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:11.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:11.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094072+0000", 2024-04-24T15:33:11.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:11.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:11.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094074+0000", 2024-04-24T15:33:11.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0899999999999999e-06 2024-04-24T15:33:11.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:11.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094080+0000", 2024-04-24T15:33:11.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4949999999999999e-06 2024-04-24T15:33:11.730 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.730 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.730 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:11.730 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094084+0000", 2024-04-24T15:33:11.730 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2570000000000001e-06 2024-04-24T15:33:11.730 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.730 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.730 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:11.730 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.125730+0000", 2024-04-24T15:33:11.730 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.031646035000000003 2024-04-24T15:33:11.730 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.730 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.730 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:11.731 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.125815+0000", 2024-04-24T15:33:11.731 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4628000000000004e-05 2024-04-24T15:33:11.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.731 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.731 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:11.731 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.125907+0000", 2024-04-24T15:33:11.731 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1848000000000006e-05 2024-04-24T15:33:11.731 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:11.731 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:11.731 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:11.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.731 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.731 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43115 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 3309568~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:11.732 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094088+0000", 2024-04-24T15:33:11.732 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.5998897239999996, 2024-04-24T15:33:11.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.031951305999999999, 2024-04-24T15:33:11.732 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:11.732 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:11.732 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:11.732 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:11.732 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:11.733 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43115 2024-04-24T15:33:11.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.733 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:11.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:11.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094088+0000", 2024-04-24T15:33:11.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:11.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.734 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.734 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:11.734 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094088+0000", 2024-04-24T15:33:11.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:11.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.734 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.735 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:11.735 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094088+0000", 2024-04-24T15:33:11.735 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:11.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.735 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:11.736 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094090+0000", 2024-04-24T15:33:11.736 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2189999999999998e-06 2024-04-24T15:33:11.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.736 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.736 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:11.736 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094096+0000", 2024-04-24T15:33:11.736 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9560000000000002e-06 2024-04-24T15:33:11.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:11.737 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094100+0000", 2024-04-24T15:33:11.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5710000000000002e-06 2024-04-24T15:33:11.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.738 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:11.738 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.125924+0000", 2024-04-24T15:33:11.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.031823885000000003 2024-04-24T15:33:11.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.738 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.738 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:11.738 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.125986+0000", 2024-04-24T15:33:11.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2558999999999995e-05 2024-04-24T15:33:11.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:11.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.126040+0000", 2024-04-24T15:33:11.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3439000000000001e-05 2024-04-24T15:33:11.740 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:11.740 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:11.740 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:11.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.740 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43116 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 3735552~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:11.741 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094104+0000", 2024-04-24T15:33:11.741 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.5998741240000003, 2024-04-24T15:33:11.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032126963000000001, 2024-04-24T15:33:11.741 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:11.741 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:11.741 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:11.741 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:11.741 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:11.742 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43116 2024-04-24T15:33:11.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.742 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:11.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.742 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:11.742 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094104+0000", 2024-04-24T15:33:11.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:11.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:11.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094104+0000", 2024-04-24T15:33:11.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:11.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:11.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094104+0000", 2024-04-24T15:33:11.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:11.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:11.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094106+0000", 2024-04-24T15:33:11.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.99e-06 2024-04-24T15:33:11.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:11.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094111+0000", 2024-04-24T15:33:11.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2739999999999998e-06 2024-04-24T15:33:11.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:11.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094114+0000", 2024-04-24T15:33:11.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2729999999999998e-06 2024-04-24T15:33:11.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:11.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.126069+0000", 2024-04-24T15:33:11.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.031954713000000003 2024-04-24T15:33:11.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:11.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.126147+0000", 2024-04-24T15:33:11.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7706000000000004e-05 2024-04-24T15:33:11.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:11.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.126231+0000", 2024-04-24T15:33:11.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4331000000000003e-05 2024-04-24T15:33:11.745 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:11.745 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:11.745 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:11.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.746 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.746 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43117 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 4161536~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:11.746 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094118+0000", 2024-04-24T15:33:11.746 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.5998599520000001, 2024-04-24T15:33:11.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032239986999999998, 2024-04-24T15:33:11.746 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:11.746 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:11.746 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:11.746 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:11.746 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:11.746 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43117 2024-04-24T15:33:11.746 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.747 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:11.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:11.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094118+0000", 2024-04-24T15:33:11.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:11.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:11.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094118+0000", 2024-04-24T15:33:11.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:11.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:11.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094118+0000", 2024-04-24T15:33:11.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:11.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:11.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094120+0000", 2024-04-24T15:33:11.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1500000000000002e-06 2024-04-24T15:33:11.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:11.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094126+0000", 2024-04-24T15:33:11.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9490000000000001e-06 2024-04-24T15:33:11.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:11.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094129+0000", 2024-04-24T15:33:11.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6009999999999999e-06 2024-04-24T15:33:11.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:11.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.126248+0000", 2024-04-24T15:33:11.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032118531999999998 2024-04-24T15:33:11.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:11.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.126306+0000", 2024-04-24T15:33:11.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7843999999999997e-05 2024-04-24T15:33:11.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:11.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.126358+0000", 2024-04-24T15:33:11.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2240999999999997e-05 2024-04-24T15:33:11.750 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:11.750 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:11.750 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:11.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.750 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43118 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 4587520~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:11.750 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094134+0000", 2024-04-24T15:33:11.750 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.5998445390000002, 2024-04-24T15:33:11.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032417003, 2024-04-24T15:33:11.750 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:11.750 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:11.750 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:11.750 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:11.751 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:11.751 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43118 2024-04-24T15:33:11.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.751 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:11.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:11.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094134+0000", 2024-04-24T15:33:11.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:11.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:11.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094134+0000", 2024-04-24T15:33:11.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:11.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:11.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094133+0000", 2024-04-24T15:33:11.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:11.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:11.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094135+0000", 2024-04-24T15:33:11.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0590000000000001e-06 2024-04-24T15:33:11.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:11.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094141+0000", 2024-04-24T15:33:11.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3190000000000002e-06 2024-04-24T15:33:11.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:11.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094144+0000", 2024-04-24T15:33:11.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2430000000000001e-06 2024-04-24T15:33:11.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:11.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.126387+0000", 2024-04-24T15:33:11.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032243124999999997 2024-04-24T15:33:11.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:11.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.126464+0000", 2024-04-24T15:33:11.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6607999999999996e-05 2024-04-24T15:33:11.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:11.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.126551+0000", 2024-04-24T15:33:11.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6953000000000006e-05 2024-04-24T15:33:11.754 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:11.754 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:11.754 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:11.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.754 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43119 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 5013504~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:11.754 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094148+0000", 2024-04-24T15:33:11.754 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.5998302979999997, 2024-04-24T15:33:11.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032531142999999998, 2024-04-24T15:33:11.754 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:11.755 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:11.755 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:11.755 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:11.755 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:11.755 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43119 2024-04-24T15:33:11.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.755 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:11.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:11.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094148+0000", 2024-04-24T15:33:11.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:11.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:11.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094148+0000", 2024-04-24T15:33:11.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:11.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:11.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094147+0000", 2024-04-24T15:33:11.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:11.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:11.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094150+0000", 2024-04-24T15:33:11.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1100000000000001e-06 2024-04-24T15:33:11.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:11.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094155+0000", 2024-04-24T15:33:11.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4480000000000002e-06 2024-04-24T15:33:11.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:11.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094159+0000", 2024-04-24T15:33:11.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5499999999999999e-06 2024-04-24T15:33:11.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:11.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.126567+0000", 2024-04-24T15:33:11.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032407997000000001 2024-04-24T15:33:11.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:11.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.126622+0000", 2024-04-24T15:33:11.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5847000000000002e-05 2024-04-24T15:33:11.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:11.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.126679+0000", 2024-04-24T15:33:11.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6508999999999997e-05 2024-04-24T15:33:11.759 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:11.759 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:11.759 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:11.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.759 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43120 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 5439488~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:11.759 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094163+0000", 2024-04-24T15:33:11.759 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.5998151890000001, 2024-04-24T15:33:11.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032680305999999999, 2024-04-24T15:33:11.760 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:11.760 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:11.760 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:11.760 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:11.760 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:11.760 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43120 2024-04-24T15:33:11.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.760 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:11.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:11.760 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094163+0000", 2024-04-24T15:33:11.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:11.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:11.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094163+0000", 2024-04-24T15:33:11.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:11.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:11.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094163+0000", 2024-04-24T15:33:11.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:11.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:11.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094165+0000", 2024-04-24T15:33:11.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.181e-06 2024-04-24T15:33:11.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:11.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094170+0000", 2024-04-24T15:33:11.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6019999999999999e-06 2024-04-24T15:33:11.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:11.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094174+0000", 2024-04-24T15:33:11.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4410000000000002e-06 2024-04-24T15:33:11.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:11.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.126731+0000", 2024-04-24T15:33:11.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032557599999999999 2024-04-24T15:33:11.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:11.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.126783+0000", 2024-04-24T15:33:11.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1378000000000002e-05 2024-04-24T15:33:11.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:11.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.126843+0000", 2024-04-24T15:33:11.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0433000000000002e-05 2024-04-24T15:33:11.764 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:11.764 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:11.764 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:11.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.764 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43121 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 5865472~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:11.764 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094178+0000", 2024-04-24T15:33:11.764 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.5998002580000001, 2024-04-24T15:33:11.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032860598999999997, 2024-04-24T15:33:11.764 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:11.764 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:11.764 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:11.765 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:11.765 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:11.765 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43121 2024-04-24T15:33:11.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.765 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:11.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.765 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:11.765 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094178+0000", 2024-04-24T15:33:11.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:11.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.765 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:11.765 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094178+0000", 2024-04-24T15:33:11.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:11.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:11.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094178+0000", 2024-04-24T15:33:11.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:11.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:11.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094180+0000", 2024-04-24T15:33:11.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9989999999999998e-06 2024-04-24T15:33:11.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:11.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094185+0000", 2024-04-24T15:33:11.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.541e-06 2024-04-24T15:33:11.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:11.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094189+0000", 2024-04-24T15:33:11.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0509999999999998e-06 2024-04-24T15:33:11.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:11.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.126877+0000", 2024-04-24T15:33:11.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032687473000000002 2024-04-24T15:33:11.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:11.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.126952+0000", 2024-04-24T15:33:11.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5924e-05 2024-04-24T15:33:11.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:11.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127038+0000", 2024-04-24T15:33:11.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5908999999999994e-05 2024-04-24T15:33:11.769 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:11.769 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:11.769 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:11.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.769 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43122 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 6291456~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:11.769 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094193+0000", 2024-04-24T15:33:11.769 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.5997850140000001, 2024-04-24T15:33:11.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032978383999999999, 2024-04-24T15:33:11.769 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:11.769 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:11.769 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:11.770 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:11.770 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:11.770 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43122 2024-04-24T15:33:11.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.770 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:11.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:11.770 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094193+0000", 2024-04-24T15:33:11.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:11.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:11.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094193+0000", 2024-04-24T15:33:11.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:11.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:11.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094193+0000", 2024-04-24T15:33:11.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:11.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:11.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094195+0000", 2024-04-24T15:33:11.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3269999999999999e-06 2024-04-24T15:33:11.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:11.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094201+0000", 2024-04-24T15:33:11.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9850000000000003e-06 2024-04-24T15:33:11.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:11.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094205+0000", 2024-04-24T15:33:11.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8430000000000003e-06 2024-04-24T15:33:11.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:11.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127068+0000", 2024-04-24T15:33:11.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032863305000000002 2024-04-24T15:33:11.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:11.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127113+0000", 2024-04-24T15:33:11.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4412000000000001e-05 2024-04-24T15:33:11.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:11.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127171+0000", 2024-04-24T15:33:11.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8807999999999997e-05 2024-04-24T15:33:11.773 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:11.773 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:11.773 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:11.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.774 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43123 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 6717440~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:11.774 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094209+0000", 2024-04-24T15:33:11.774 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.5997688480000001, 2024-04-24T15:33:11.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033144583999999998, 2024-04-24T15:33:11.774 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:11.774 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:11.774 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:11.774 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:11.774 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:11.774 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43123 2024-04-24T15:33:11.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.775 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:11.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:11.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094209+0000", 2024-04-24T15:33:11.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:11.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:11.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094209+0000", 2024-04-24T15:33:11.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:11.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:11.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094209+0000", 2024-04-24T15:33:11.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:11.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:11.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094211+0000", 2024-04-24T15:33:11.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2089999999999999e-06 2024-04-24T15:33:11.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:11.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094217+0000", 2024-04-24T15:33:11.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6799999999999998e-06 2024-04-24T15:33:11.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:11.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094220+0000", 2024-04-24T15:33:11.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.444e-06 2024-04-24T15:33:11.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:11.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127190+0000", 2024-04-24T15:33:11.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032969471 2024-04-24T15:33:11.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:11.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127269+0000", 2024-04-24T15:33:11.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9246999999999995e-05 2024-04-24T15:33:11.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:11.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127354+0000", 2024-04-24T15:33:11.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4853999999999998e-05 2024-04-24T15:33:11.778 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:11.778 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:11.778 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:11.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.778 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43124 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 7143424~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:11.778 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094224+0000", 2024-04-24T15:33:11.778 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.5997538750000002, 2024-04-24T15:33:11.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033271955999999998, 2024-04-24T15:33:11.779 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:11.779 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:11.779 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:11.779 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:11.779 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:11.779 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43124 2024-04-24T15:33:11.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.779 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:11.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:11.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094224+0000", 2024-04-24T15:33:11.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:11.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:11.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094224+0000", 2024-04-24T15:33:11.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:11.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:11.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094224+0000", 2024-04-24T15:33:11.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:11.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:11.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094226+0000", 2024-04-24T15:33:11.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9039999999999999e-06 2024-04-24T15:33:11.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:11.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094233+0000", 2024-04-24T15:33:11.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7970000000000003e-06 2024-04-24T15:33:11.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:11.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094236+0000", 2024-04-24T15:33:11.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6339999999999999e-06 2024-04-24T15:33:11.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:11.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127390+0000", 2024-04-24T15:33:11.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.03315353 2024-04-24T15:33:11.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:11.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127439+0000", 2024-04-24T15:33:11.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8890000000000001e-05 2024-04-24T15:33:11.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:11.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127496+0000", 2024-04-24T15:33:11.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7476999999999999e-05 2024-04-24T15:33:11.782 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:11.782 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:11.782 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:11.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.782 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43125 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 7569408~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:11.782 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094240+0000", 2024-04-24T15:33:11.782 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.5997377879999997, 2024-04-24T15:33:11.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033446278000000003, 2024-04-24T15:33:11.783 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:11.783 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:11.783 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:11.783 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:11.783 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:11.783 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43125 2024-04-24T15:33:11.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.783 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:11.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:11.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094240+0000", 2024-04-24T15:33:11.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:11.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:11.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094240+0000", 2024-04-24T15:33:11.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:11.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:11.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094240+0000", 2024-04-24T15:33:11.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:11.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:11.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094247+0000", 2024-04-24T15:33:11.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6209999999999998e-06 2024-04-24T15:33:11.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:11.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094253+0000", 2024-04-24T15:33:11.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0739999999999998e-06 2024-04-24T15:33:11.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:11.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094256+0000", 2024-04-24T15:33:11.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7699999999999999e-06 2024-04-24T15:33:11.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:11.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127526+0000", 2024-04-24T15:33:11.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033269703999999997 2024-04-24T15:33:11.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:11.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127603+0000", 2024-04-24T15:33:11.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6964000000000004e-05 2024-04-24T15:33:11.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:11.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127687+0000", 2024-04-24T15:33:11.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3431999999999997e-05 2024-04-24T15:33:11.786 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:11.786 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:11.786 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:11.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.786 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43126 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 7995392~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:11.786 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094261+0000", 2024-04-24T15:33:11.786 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.5997172849999997, 2024-04-24T15:33:11.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033555201999999999, 2024-04-24T15:33:11.787 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:11.787 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:11.787 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:11.787 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:11.787 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:11.787 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43126 2024-04-24T15:33:11.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.787 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:11.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:11.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094261+0000", 2024-04-24T15:33:11.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:11.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:11.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094261+0000", 2024-04-24T15:33:11.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:11.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:11.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094260+0000", 2024-04-24T15:33:11.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:11.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:11.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094263+0000", 2024-04-24T15:33:11.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2579999999999998e-06 2024-04-24T15:33:11.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:11.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094269+0000", 2024-04-24T15:33:11.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.823e-06 2024-04-24T15:33:11.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:11.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094272+0000", 2024-04-24T15:33:11.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8999999999999999e-06 2024-04-24T15:33:11.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:11.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127720+0000", 2024-04-24T15:33:11.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033447521000000001 2024-04-24T15:33:11.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:11.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127765+0000", 2024-04-24T15:33:11.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5352000000000002e-05 2024-04-24T15:33:11.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:11.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:11.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:11.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127816+0000", 2024-04-24T15:33:11.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0689999999999997e-05 2024-04-24T15:33:11.790 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:11.790 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:11.790 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:11.790 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:11.790 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:11.791 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:11.792 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-24T15:33:11.792 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:33:11.793+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:33:11.815 INFO:teuthology.orchestra.run.smithi096.stderr:noscrub is unset 2024-04-24T15:33:11.869 INFO:teuthology.orchestra.run.smithi096.stdout:leaving image image_client.0-clone21 intact 2024-04-24T15:33:11.893 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:11.894+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:11.950 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:11.951 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:33:11.951 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-04-24T15:33:11.951 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:11.962 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-24T15:33:11.994 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:11.994+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:12.006 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #20, image image_client.0-clone20 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent21 2024-04-24T15:33:12.091 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-6 --journal-path /var/lib/ceph/osd/ceph-6/journal --op export --pgid 4.12 --file /var/log/ceph/exp.4.12.6 2024-04-24T15:33:12.133 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:12.133+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:12.144 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:12.144 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:33:12.144 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-04-24T15:33:12.144 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-04-24T15:33:12.144 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:12.154 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-24T15:33:12.196 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:33:12.197+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:33:12.294 INFO:teuthology.orchestra.run.smithi096.stdout:4029 punch from 0xa2c2017 to 0xa2ca0c4, (0x80ad bytes) 2024-04-24T15:33:12.295 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:12.296+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:12.298 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:12.306 INFO:teuthology.orchestra.run.smithi096.stdout:4032 trunc from 0xbfc6659 to 0x6164aee 2024-04-24T15:33:12.323 INFO:teuthology.orchestra.run.smithi096.stdout:4033 read 0x1a34f7c thru 0x1a395c5 (0x464a bytes) 2024-04-24T15:33:12.325 INFO:teuthology.orchestra.run.smithi096.stdout:4035 trunc from 0x6164aee to 0x487e4c6 2024-04-24T15:33:12.330 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:12.330 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-04-24T15:33:12.330 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-04-24T15:33:12.330 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-04-24T15:33:12.330 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.330 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14043 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2383872~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:12.330 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892347+0000", 2024-04-24T15:33:12.330 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 38.437516764999998, 2024-04-24T15:33:12.330 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038537279000000001, 2024-04-24T15:33:12.330 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:12.331 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:12.331 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:12.331 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:12.331 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:12.331 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14043 2024-04-24T15:33:12.331 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.331 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:12.331 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.331 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:12.331 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892347+0000", 2024-04-24T15:33:12.331 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:12.331 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.331 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.332 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:12.332 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892347+0000", 2024-04-24T15:33:12.332 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:12.332 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.332 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.332 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:12.332 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892347+0000", 2024-04-24T15:33:12.332 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:12.332 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.332 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.332 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:12.332 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892349+0000", 2024-04-24T15:33:12.332 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3400000000000001e-06 2024-04-24T15:33:12.332 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.332 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.333 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:12.333 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892357+0000", 2024-04-24T15:33:12.333 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1540000000000002e-06 2024-04-24T15:33:12.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.333 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.333 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:12.333 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892360+0000", 2024-04-24T15:33:12.333 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5829999999999998e-06 2024-04-24T15:33:12.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.333 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.333 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:12.333 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930795+0000", 2024-04-24T15:33:12.333 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038434590999999997 2024-04-24T15:33:12.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.334 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:12.334 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930836+0000", 2024-04-24T15:33:12.334 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0827000000000001e-05 2024-04-24T15:33:12.334 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.334 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:12.334 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930885+0000", 2024-04-24T15:33:12.334 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9029000000000001e-05 2024-04-24T15:33:12.334 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:12.334 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:12.334 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:12.334 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.335 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.335 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14044 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2424832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:12.335 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892372+0000", 2024-04-24T15:33:12.335 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 38.437492120999998, 2024-04-24T15:33:12.335 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038624890000000002, 2024-04-24T15:33:12.335 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:12.335 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:12.335 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:12.335 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:12.335 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:12.335 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14044 2024-04-24T15:33:12.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.335 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:12.335 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.335 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:12.335 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892372+0000", 2024-04-24T15:33:12.336 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:12.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.336 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.336 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:12.336 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892372+0000", 2024-04-24T15:33:12.336 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:12.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.336 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.336 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:12.336 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892372+0000", 2024-04-24T15:33:12.336 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:12.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.336 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.336 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:12.336 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892374+0000", 2024-04-24T15:33:12.336 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4899999999999999e-06 2024-04-24T15:33:12.337 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.337 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.337 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:12.337 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892380+0000", 2024-04-24T15:33:12.337 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.631e-06 2024-04-24T15:33:12.337 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.337 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.337 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:12.337 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892383+0000", 2024-04-24T15:33:12.337 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7400000000000002e-06 2024-04-24T15:33:12.337 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.337 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.337 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:12.337 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930905+0000", 2024-04-24T15:33:12.338 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038521396999999999 2024-04-24T15:33:12.338 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.338 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.338 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:12.338 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930949+0000", 2024-04-24T15:33:12.338 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4153999999999998e-05 2024-04-24T15:33:12.338 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.338 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.338 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:12.338 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930997+0000", 2024-04-24T15:33:12.338 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7998999999999999e-05 2024-04-24T15:33:12.338 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:12.338 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:12.339 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:12.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.339 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.339 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14045 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2465792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:12.339 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892387+0000", 2024-04-24T15:33:12.339 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 38.437476854000003, 2024-04-24T15:33:12.339 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038720933999999999, 2024-04-24T15:33:12.339 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:12.339 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:12.339 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:12.339 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:12.339 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:12.339 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14045 2024-04-24T15:33:12.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.339 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:12.340 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.340 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:12.340 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892387+0000", 2024-04-24T15:33:12.340 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:12.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.340 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.340 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:12.340 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892387+0000", 2024-04-24T15:33:12.340 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:12.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.340 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.340 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:12.340 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892387+0000", 2024-04-24T15:33:12.340 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:12.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.341 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.341 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:12.341 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892392+0000", 2024-04-24T15:33:12.341 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0039999999999999e-06 2024-04-24T15:33:12.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.341 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.341 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:12.341 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892398+0000", 2024-04-24T15:33:12.341 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0900000000000001e-06 2024-04-24T15:33:12.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.341 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.341 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:12.341 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892402+0000", 2024-04-24T15:33:12.341 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.399e-06 2024-04-24T15:33:12.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.342 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.342 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:12.342 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931018+0000", 2024-04-24T15:33:12.342 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038616551999999998 2024-04-24T15:33:12.342 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.342 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.342 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:12.342 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931059+0000", 2024-04-24T15:33:12.342 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1205000000000003e-05 2024-04-24T15:33:12.342 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.342 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.343 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:12.343 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931108+0000", 2024-04-24T15:33:12.343 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8996000000000003e-05 2024-04-24T15:33:12.343 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:12.343 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:12.343 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:12.343 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.343 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.343 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14046 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2506752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:12.343 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892406+0000", 2024-04-24T15:33:12.343 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 38.437458264, 2024-04-24T15:33:12.343 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038814507999999998, 2024-04-24T15:33:12.343 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:12.343 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:12.344 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:12.344 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:12.344 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:12.344 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14046 2024-04-24T15:33:12.344 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.344 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:12.344 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.344 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:12.344 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892406+0000", 2024-04-24T15:33:12.344 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:12.344 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.345 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.345 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:12.345 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892406+0000", 2024-04-24T15:33:12.345 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:12.345 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.345 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.345 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:12.345 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892406+0000", 2024-04-24T15:33:12.345 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:12.345 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.345 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.345 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:12.345 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892413+0000", 2024-04-24T15:33:12.346 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7449999999999995e-06 2024-04-24T15:33:12.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.346 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.346 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:12.346 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892419+0000", 2024-04-24T15:33:12.346 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5790000000000003e-06 2024-04-24T15:33:12.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.346 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.346 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:12.346 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892423+0000", 2024-04-24T15:33:12.346 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8199999999999998e-06 2024-04-24T15:33:12.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.346 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.347 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:12.347 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931128+0000", 2024-04-24T15:33:12.347 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038705502000000003 2024-04-24T15:33:12.347 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.347 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.347 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:12.347 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931172+0000", 2024-04-24T15:33:12.347 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4066999999999997e-05 2024-04-24T15:33:12.347 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.347 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.347 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:12.348 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931220+0000", 2024-04-24T15:33:12.348 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8167999999999997e-05 2024-04-24T15:33:12.348 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:12.348 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:12.348 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:12.348 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.348 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.348 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14047 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2547712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:12.348 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892427+0000", 2024-04-24T15:33:12.348 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 38.437437035000002, 2024-04-24T15:33:12.348 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038904506999999998, 2024-04-24T15:33:12.348 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:12.348 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:12.348 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:12.349 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:12.349 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:12.349 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14047 2024-04-24T15:33:12.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.349 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:12.349 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.349 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:12.349 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892427+0000", 2024-04-24T15:33:12.349 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:12.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.349 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.349 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:12.349 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892427+0000", 2024-04-24T15:33:12.349 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:12.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.350 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.350 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:12.350 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892427+0000", 2024-04-24T15:33:12.350 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:12.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.350 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.350 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:12.350 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892429+0000", 2024-04-24T15:33:12.350 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3099999999999999e-06 2024-04-24T15:33:12.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.350 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.350 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:12.350 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892435+0000", 2024-04-24T15:33:12.351 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5520000000000004e-06 2024-04-24T15:33:12.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.351 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.351 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:12.351 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892438+0000", 2024-04-24T15:33:12.351 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4240000000000002e-06 2024-04-24T15:33:12.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.351 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.351 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:12.351 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931242+0000", 2024-04-24T15:33:12.351 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038804162000000003 2024-04-24T15:33:12.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.351 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.351 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:12.351 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931284+0000", 2024-04-24T15:33:12.352 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1414e-05 2024-04-24T15:33:12.352 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.352 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.352 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:12.352 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931332+0000", 2024-04-24T15:33:12.352 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8024e-05 2024-04-24T15:33:12.352 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:12.352 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:12.352 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:12.352 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.352 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.352 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14048 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2588672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:12.353 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892442+0000", 2024-04-24T15:33:12.353 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 38.437422071999997, 2024-04-24T15:33:12.353 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039003132000000003, 2024-04-24T15:33:12.353 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:12.353 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:12.353 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:12.353 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:12.353 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:12.353 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14048 2024-04-24T15:33:12.353 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.353 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:12.353 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.353 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:12.354 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892442+0000", 2024-04-24T15:33:12.354 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:12.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.354 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.354 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:12.354 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892442+0000", 2024-04-24T15:33:12.354 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:12.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.354 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.354 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:12.354 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892442+0000", 2024-04-24T15:33:12.354 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:12.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.355 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.355 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:12.355 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892444+0000", 2024-04-24T15:33:12.355 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1169999999999998e-06 2024-04-24T15:33:12.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.355 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.355 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:12.355 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892449+0000", 2024-04-24T15:33:12.355 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1599999999999997e-06 2024-04-24T15:33:12.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.355 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.355 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:12.355 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892453+0000", 2024-04-24T15:33:12.355 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5930000000000002e-06 2024-04-24T15:33:12.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.356 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.356 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:12.356 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931352+0000", 2024-04-24T15:33:12.356 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038899361 2024-04-24T15:33:12.356 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.356 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.356 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:12.356 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931398+0000", 2024-04-24T15:33:12.356 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6315e-05 2024-04-24T15:33:12.356 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.356 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.356 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:12.356 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931445+0000", 2024-04-24T15:33:12.356 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6950000000000003e-05 2024-04-24T15:33:12.357 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:12.357 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:12.357 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:12.357 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.357 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.357 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14049 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2629632~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:12.357 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892458+0000", 2024-04-24T15:33:12.357 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 38.437406652, 2024-04-24T15:33:12.357 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039103322000000003, 2024-04-24T15:33:12.357 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:12.357 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:12.357 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:12.357 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:12.357 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:12.358 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14049 2024-04-24T15:33:12.358 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.358 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:12.358 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.358 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:12.358 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892458+0000", 2024-04-24T15:33:12.358 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:12.358 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.358 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.358 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:12.358 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892458+0000", 2024-04-24T15:33:12.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:12.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:12.359 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892457+0000", 2024-04-24T15:33:12.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:12.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:12.360 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892459+0000", 2024-04-24T15:33:12.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.305e-06 2024-04-24T15:33:12.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:12.360 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892465+0000", 2024-04-24T15:33:12.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4360000000000001e-06 2024-04-24T15:33:12.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:12.360 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892474+0000", 2024-04-24T15:33:12.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7639999999999994e-06 2024-04-24T15:33:12.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.361 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.361 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:12.361 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931466+0000", 2024-04-24T15:33:12.361 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038992644 2024-04-24T15:33:12.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.361 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.361 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:12.361 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931512+0000", 2024-04-24T15:33:12.361 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5200000000000001e-05 2024-04-24T15:33:12.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.361 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.361 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:12.361 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931561+0000", 2024-04-24T15:33:12.361 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9357999999999997e-05 2024-04-24T15:33:12.362 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:12.362 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:12.362 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:12.362 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.362 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.362 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14050 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2670592~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:12.362 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892478+0000", 2024-04-24T15:33:12.362 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 38.437386064999998, 2024-04-24T15:33:12.362 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039195102000000002, 2024-04-24T15:33:12.362 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:12.362 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:12.362 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:12.362 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:12.362 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:12.363 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14050 2024-04-24T15:33:12.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.363 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:12.363 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.363 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:12.363 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892478+0000", 2024-04-24T15:33:12.363 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:12.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.363 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.363 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:12.363 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892478+0000", 2024-04-24T15:33:12.363 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:12.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.363 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.363 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:12.364 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892478+0000", 2024-04-24T15:33:12.364 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:12.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.364 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:12.364 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892480+0000", 2024-04-24T15:33:12.364 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3300000000000001e-06 2024-04-24T15:33:12.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.364 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:12.364 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892486+0000", 2024-04-24T15:33:12.364 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9200000000000001e-06 2024-04-24T15:33:12.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.364 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:12.365 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892494+0000", 2024-04-24T15:33:12.365 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5159999999999997e-06 2024-04-24T15:33:12.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.365 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.365 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:12.365 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931581+0000", 2024-04-24T15:33:12.365 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039087894999999998 2024-04-24T15:33:12.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.365 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.365 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:12.365 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931626+0000", 2024-04-24T15:33:12.365 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4943e-05 2024-04-24T15:33:12.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.365 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.365 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:12.366 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931673+0000", 2024-04-24T15:33:12.366 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6906000000000001e-05 2024-04-24T15:33:12.366 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:12.366 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:12.366 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:12.366 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.366 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.366 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14051 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2711552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:12.366 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892507+0000", 2024-04-24T15:33:12.366 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 38.437357366000001, 2024-04-24T15:33:12.366 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039274393999999997, 2024-04-24T15:33:12.366 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:12.367 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:12.367 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:12.367 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:12.367 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:12.367 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14051 2024-04-24T15:33:12.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.367 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:12.367 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.367 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:12.367 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892507+0000", 2024-04-24T15:33:12.367 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:12.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.367 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.367 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:12.368 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892507+0000", 2024-04-24T15:33:12.368 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:12.368 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.368 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.368 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:12.368 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892506+0000", 2024-04-24T15:33:12.368 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:12.368 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.368 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.368 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:12.368 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892509+0000", 2024-04-24T15:33:12.368 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8760000000000001e-06 2024-04-24T15:33:12.368 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.368 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.368 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:12.369 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892515+0000", 2024-04-24T15:33:12.369 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1840000000000004e-06 2024-04-24T15:33:12.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.369 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.369 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:12.369 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892519+0000", 2024-04-24T15:33:12.369 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7570000000000002e-06 2024-04-24T15:33:12.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.369 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.369 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:12.369 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931694+0000", 2024-04-24T15:33:12.369 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039174960000000002 2024-04-24T15:33:12.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.369 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.370 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:12.370 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931734+0000", 2024-04-24T15:33:12.370 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0284000000000003e-05 2024-04-24T15:33:12.370 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.370 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.370 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:12.370 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931781+0000", 2024-04-24T15:33:12.370 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6813000000000001e-05 2024-04-24T15:33:12.370 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:12.370 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:12.370 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:12.370 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.370 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.370 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14052 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2752512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:12.371 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892524+0000", 2024-04-24T15:33:12.371 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 38.437340218999999, 2024-04-24T15:33:12.371 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039371122000000001, 2024-04-24T15:33:12.371 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:12.371 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:12.371 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:12.371 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:12.371 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:12.371 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14052 2024-04-24T15:33:12.371 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.371 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:12.372 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.372 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:12.372 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892524+0000", 2024-04-24T15:33:12.372 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:12.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.372 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.372 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:12.372 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892524+0000", 2024-04-24T15:33:12.372 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:12.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.372 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.372 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:12.372 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892524+0000", 2024-04-24T15:33:12.372 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:12.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.373 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.373 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:12.373 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892526+0000", 2024-04-24T15:33:12.373 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3630000000000001e-06 2024-04-24T15:33:12.373 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.373 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.373 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:12.373 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892538+0000", 2024-04-24T15:33:12.373 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1677e-05 2024-04-24T15:33:12.373 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.373 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.373 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:12.373 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892541+0000", 2024-04-24T15:33:12.373 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.822e-06 2024-04-24T15:33:12.374 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.374 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.374 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:12.374 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931802+0000", 2024-04-24T15:33:12.374 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039260911000000003 2024-04-24T15:33:12.374 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.374 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.374 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:12.374 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931847+0000", 2024-04-24T15:33:12.374 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4450999999999998e-05 2024-04-24T15:33:12.374 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.374 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.374 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:12.375 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931895+0000", 2024-04-24T15:33:12.375 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8257999999999998e-05 2024-04-24T15:33:12.375 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:12.375 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:12.375 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:12.375 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.375 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.375 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14053 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2793472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:12.375 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892546+0000", 2024-04-24T15:33:12.375 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 38.437318241, 2024-04-24T15:33:12.375 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039457712999999998, 2024-04-24T15:33:12.375 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:12.375 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:12.375 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:12.376 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:12.376 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:12.376 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14053 2024-04-24T15:33:12.376 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.376 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:12.376 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.376 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:12.376 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892546+0000", 2024-04-24T15:33:12.376 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:12.376 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.376 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.376 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:12.376 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892546+0000", 2024-04-24T15:33:12.377 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:12.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.377 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.377 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:12.377 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892546+0000", 2024-04-24T15:33:12.377 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:12.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.377 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.377 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:12.377 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892548+0000", 2024-04-24T15:33:12.377 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3120000000000001e-06 2024-04-24T15:33:12.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.377 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.377 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:12.377 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892553+0000", 2024-04-24T15:33:12.378 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2320000000000001e-06 2024-04-24T15:33:12.378 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.378 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.378 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:12.378 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892559+0000", 2024-04-24T15:33:12.378 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5480000000000001e-06 2024-04-24T15:33:12.378 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.378 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.378 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:12.378 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931916+0000", 2024-04-24T15:33:12.378 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039357180999999998 2024-04-24T15:33:12.378 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.378 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.379 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:12.379 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931956+0000", 2024-04-24T15:33:12.379 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0565000000000002e-05 2024-04-24T15:33:12.379 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.379 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.379 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:12.379 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932004+0000", 2024-04-24T15:33:12.379 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7224000000000001e-05 2024-04-24T15:33:12.379 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:12.379 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:12.379 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:12.379 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.379 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.379 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14054 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2834432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:12.379 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892563+0000", 2024-04-24T15:33:12.380 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 38.437301589, 2024-04-24T15:33:12.380 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039552406999999998, 2024-04-24T15:33:12.380 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:12.380 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:12.380 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:12.380 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:12.380 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:12.380 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14054 2024-04-24T15:33:12.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.380 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:12.380 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.380 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:12.380 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892563+0000", 2024-04-24T15:33:12.380 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:12.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.381 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:12.381 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892563+0000", 2024-04-24T15:33:12.381 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:12.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.381 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:12.381 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892562+0000", 2024-04-24T15:33:12.381 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:12.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.381 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:12.381 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892564+0000", 2024-04-24T15:33:12.382 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2029999999999999e-06 2024-04-24T15:33:12.382 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.382 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.382 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:12.382 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892570+0000", 2024-04-24T15:33:12.382 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1460000000000004e-06 2024-04-24T15:33:12.382 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.382 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.382 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:12.382 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892573+0000", 2024-04-24T15:33:12.382 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2030000000000001e-06 2024-04-24T15:33:12.382 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.382 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.382 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:12.382 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932024+0000", 2024-04-24T15:33:12.383 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039451696000000001 2024-04-24T15:33:12.383 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.383 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.383 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:12.383 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932068+0000", 2024-04-24T15:33:12.383 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3970000000000001e-05 2024-04-24T15:33:12.383 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.383 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.383 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:12.383 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932115+0000", 2024-04-24T15:33:12.383 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6569000000000001e-05 2024-04-24T15:33:12.383 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:12.383 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:12.383 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:12.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.384 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.384 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14055 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2875392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:12.384 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892591+0000", 2024-04-24T15:33:12.384 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 38.437273615999999, 2024-04-24T15:33:12.384 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039634309, 2024-04-24T15:33:12.384 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:12.384 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:12.384 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:12.384 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:12.384 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:12.384 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14055 2024-04-24T15:33:12.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.384 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:12.385 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.385 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:12.385 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892591+0000", 2024-04-24T15:33:12.385 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:12.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.385 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.385 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:12.385 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892591+0000", 2024-04-24T15:33:12.385 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:12.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.385 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.385 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:12.385 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892590+0000", 2024-04-24T15:33:12.385 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:12.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.386 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.386 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:12.386 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892592+0000", 2024-04-24T15:33:12.386 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7519999999999999e-06 2024-04-24T15:33:12.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.386 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.386 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:12.386 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892596+0000", 2024-04-24T15:33:12.386 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.608e-06 2024-04-24T15:33:12.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.386 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.386 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:12.386 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892598+0000", 2024-04-24T15:33:12.387 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.531e-06 2024-04-24T15:33:12.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.387 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.387 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:12.387 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932136+0000", 2024-04-24T15:33:12.387 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039537668999999998 2024-04-24T15:33:12.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.387 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.387 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:12.387 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932177+0000", 2024-04-24T15:33:12.387 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0880000000000002e-05 2024-04-24T15:33:12.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.387 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.387 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:12.388 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932225+0000", 2024-04-24T15:33:12.388 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8223999999999998e-05 2024-04-24T15:33:12.388 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:12.388 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:12.388 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:12.388 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.388 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.388 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14056 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2916352~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:12.388 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892600+0000", 2024-04-24T15:33:12.388 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 38.437263782000002, 2024-04-24T15:33:12.388 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039736447000000001, 2024-04-24T15:33:12.388 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:12.388 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:12.388 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:12.389 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:12.389 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:12.389 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14056 2024-04-24T15:33:12.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.389 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:12.389 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.389 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:12.389 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892600+0000", 2024-04-24T15:33:12.389 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:12.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.389 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.389 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:12.389 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892600+0000", 2024-04-24T15:33:12.389 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:12.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.390 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.390 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:12.390 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892600+0000", 2024-04-24T15:33:12.390 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967296 2024-04-24T15:33:12.390 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.390 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.390 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:12.390 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892602+0000", 2024-04-24T15:33:12.390 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.576e-06 2024-04-24T15:33:12.390 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.390 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.390 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:12.390 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892605+0000", 2024-04-24T15:33:12.390 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.574e-06 2024-04-24T15:33:12.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.391 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.391 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:12.391 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892608+0000", 2024-04-24T15:33:12.391 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3800000000000001e-06 2024-04-24T15:33:12.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.391 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.391 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:12.391 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932245+0000", 2024-04-24T15:33:12.391 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039637378000000001 2024-04-24T15:33:12.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.391 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.391 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:12.391 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932290+0000", 2024-04-24T15:33:12.392 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4734999999999998e-05 2024-04-24T15:33:12.392 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.392 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.392 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:12.392 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932337+0000", 2024-04-24T15:33:12.392 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7033000000000002e-05 2024-04-24T15:33:12.392 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:12.392 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:12.392 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:12.392 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.392 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.392 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14063 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 3203072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:12.392 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.894564+0000", 2024-04-24T15:33:12.392 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 38.435300191000003, 2024-04-24T15:33:12.392 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038554995000000002, 2024-04-24T15:33:12.393 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:12.393 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:12.393 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:12.393 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:12.393 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:12.393 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14063 2024-04-24T15:33:12.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.393 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:12.393 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.393 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:12.393 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894564+0000", 2024-04-24T15:33:12.393 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:12.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.393 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.394 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:12.394 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894564+0000", 2024-04-24T15:33:12.394 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:12.394 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.394 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.394 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:12.394 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894564+0000", 2024-04-24T15:33:12.394 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967296 2024-04-24T15:33:12.394 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.394 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.394 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:12.394 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894565+0000", 2024-04-24T15:33:12.394 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2640000000000001e-06 2024-04-24T15:33:12.394 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.395 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.395 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:12.395 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894569+0000", 2024-04-24T15:33:12.395 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1710000000000004e-06 2024-04-24T15:33:12.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.395 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.395 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:12.395 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894573+0000", 2024-04-24T15:33:12.395 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4410000000000002e-06 2024-04-24T15:33:12.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.395 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.395 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:12.395 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933030+0000", 2024-04-24T15:33:12.395 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038457709999999999 2024-04-24T15:33:12.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.396 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.396 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:12.396 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933072+0000", 2024-04-24T15:33:12.396 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.193e-05 2024-04-24T15:33:12.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.396 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.396 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:12.396 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933119+0000", 2024-04-24T15:33:12.396 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6715000000000003e-05 2024-04-24T15:33:12.396 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:12.396 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:12.396 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:12.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.396 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.396 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14064 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 3244032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:12.396 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.894591+0000", 2024-04-24T15:33:12.397 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 38.435273549000001, 2024-04-24T15:33:12.397 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038639806999999998, 2024-04-24T15:33:12.397 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:12.397 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:12.397 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:12.397 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:12.397 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:12.397 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14064 2024-04-24T15:33:12.397 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.397 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:12.397 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.397 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:12.397 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894591+0000", 2024-04-24T15:33:12.397 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:12.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.398 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.398 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:12.398 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894591+0000", 2024-04-24T15:33:12.398 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:12.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.398 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.398 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:12.398 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894590+0000", 2024-04-24T15:33:12.398 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:12.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.398 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.398 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:12.398 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894593+0000", 2024-04-24T15:33:12.398 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4719999999999998e-06 2024-04-24T15:33:12.399 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.399 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.399 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:12.399 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894598+0000", 2024-04-24T15:33:12.399 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.57e-06 2024-04-24T15:33:12.399 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.399 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.399 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:12.399 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894602+0000", 2024-04-24T15:33:12.399 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.614e-06 2024-04-24T15:33:12.399 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.399 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.399 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:12.399 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933139+0000", 2024-04-24T15:33:12.399 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038536929999999997 2024-04-24T15:33:12.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.400 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.400 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:12.400 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933183+0000", 2024-04-24T15:33:12.400 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4217e-05 2024-04-24T15:33:12.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.400 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.400 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:12.400 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933230+0000", 2024-04-24T15:33:12.400 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7434e-05 2024-04-24T15:33:12.400 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:12.400 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:12.400 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:12.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.401 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.401 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14065 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 3284992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:12.401 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.894606+0000", 2024-04-24T15:33:12.401 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 38.435258167000001, 2024-04-24T15:33:12.401 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038734126000000001, 2024-04-24T15:33:12.401 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:12.401 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:12.401 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:12.401 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:12.401 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:12.401 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14065 2024-04-24T15:33:12.401 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.401 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:12.401 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.402 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:12.402 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894606+0000", 2024-04-24T15:33:12.402 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:12.402 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.402 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.402 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:12.402 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894606+0000", 2024-04-24T15:33:12.402 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:12.402 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.402 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.402 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:12.402 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894606+0000", 2024-04-24T15:33:12.402 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:12.402 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.403 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.403 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:12.403 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894608+0000", 2024-04-24T15:33:12.403 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0880000000000002e-06 2024-04-24T15:33:12.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.403 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.403 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:12.403 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894614+0000", 2024-04-24T15:33:12.403 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7830000000000004e-06 2024-04-24T15:33:12.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.403 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.403 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:12.403 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894621+0000", 2024-04-24T15:33:12.403 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5809999999999999e-06 2024-04-24T15:33:12.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.403 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.404 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:12.404 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933251+0000", 2024-04-24T15:33:12.404 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038629995 2024-04-24T15:33:12.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.404 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.404 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:12.404 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933292+0000", 2024-04-24T15:33:12.404 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0380000000000003e-05 2024-04-24T15:33:12.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.404 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.404 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:12.404 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933340+0000", 2024-04-24T15:33:12.404 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8662999999999998e-05 2024-04-24T15:33:12.404 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:12.405 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:12.405 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:12.405 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.405 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.405 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14066 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 3325952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:12.405 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.894625+0000", 2024-04-24T15:33:12.405 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 38.435238728000002, 2024-04-24T15:33:12.405 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038825539999999999, 2024-04-24T15:33:12.405 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:12.405 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:12.405 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:12.405 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:12.405 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:12.405 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14066 2024-04-24T15:33:12.405 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.405 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:12.406 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.406 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:12.406 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894625+0000", 2024-04-24T15:33:12.406 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:12.406 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.406 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.406 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:12.406 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894625+0000", 2024-04-24T15:33:12.406 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:12.406 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.406 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.406 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:12.406 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894625+0000", 2024-04-24T15:33:12.406 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:12.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.407 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.407 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:12.407 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894627+0000", 2024-04-24T15:33:12.407 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.092e-06 2024-04-24T15:33:12.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.407 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.407 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:12.407 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894635+0000", 2024-04-24T15:33:12.407 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6660000000000008e-06 2024-04-24T15:33:12.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.407 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.407 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:12.407 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894639+0000", 2024-04-24T15:33:12.407 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.799e-06 2024-04-24T15:33:12.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.408 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.408 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:12.408 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933360+0000", 2024-04-24T15:33:12.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038721569999999997 2024-04-24T15:33:12.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.408 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.408 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:12.408 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933403+0000", 2024-04-24T15:33:12.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2605999999999998e-05 2024-04-24T15:33:12.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.408 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.409 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:12.409 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933451+0000", 2024-04-24T15:33:12.409 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8149999999999998e-05 2024-04-24T15:33:12.409 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:12.409 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:12.409 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:12.409 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.409 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.409 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14067 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 3366912~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:12.409 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.894644+0000", 2024-04-24T15:33:12.409 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 38.43522067, 2024-04-24T15:33:12.409 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038926068000000001, 2024-04-24T15:33:12.409 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:12.409 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:12.409 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:12.409 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:12.410 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:12.410 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14067 2024-04-24T15:33:12.410 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.410 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:12.410 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.410 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:12.410 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894644+0000", 2024-04-24T15:33:12.410 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:12.410 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.410 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.410 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:12.410 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894644+0000", 2024-04-24T15:33:12.410 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:12.410 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.410 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:12.411 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894643+0000", 2024-04-24T15:33:12.411 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:12.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:12.411 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894646+0000", 2024-04-24T15:33:12.411 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3379999999999999e-06 2024-04-24T15:33:12.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:12.411 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894651+0000", 2024-04-24T15:33:12.411 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.485e-06 2024-04-24T15:33:12.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:12.412 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894655+0000", 2024-04-24T15:33:12.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.709e-06 2024-04-24T15:33:12.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:12.412 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933472+0000", 2024-04-24T15:33:12.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038816992000000002 2024-04-24T15:33:12.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:12.412 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933521+0000", 2024-04-24T15:33:12.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8841999999999998e-05 2024-04-24T15:33:12.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:12.412 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933570+0000", 2024-04-24T15:33:12.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9063000000000001e-05 2024-04-24T15:33:12.413 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:12.413 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:12.413 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:12.413 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.413 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.413 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:30240 4.0 4:059b194c:::rbd_object_map.114a2e5eebc:head [read 0~43] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-04-24T15:33:12.413 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:47.244473+0000", 2024-04-24T15:33:12.413 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 25.085391483999999, 2024-04-24T15:33:12.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4146627299999999, 2024-04-24T15:33:12.413 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:12.413 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:12.413 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:12.413 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:12.413 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:12.414 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30240 2024-04-24T15:33:12.414 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.414 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:12.414 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.414 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:12.414 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.244473+0000", 2024-04-24T15:33:12.414 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:12.414 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.414 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.414 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:12.414 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.244473+0000", 2024-04-24T15:33:12.414 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:12.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.415 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.415 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:12.415 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.244471+0000", 2024-04-24T15:33:12.415 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999986 2024-04-24T15:33:12.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.415 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.415 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:12.415 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.244477+0000", 2024-04-24T15:33:12.415 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7819999999999999e-06 2024-04-24T15:33:12.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.415 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.415 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:12.415 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.244487+0000", 2024-04-24T15:33:12.415 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0247e-05 2024-04-24T15:33:12.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.416 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.416 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:12.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.244499+0000", 2024-04-24T15:33:12.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1324e-05 2024-04-24T15:33:12.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.416 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.416 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:12.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.244545+0000", 2024-04-24T15:33:12.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6186000000000001e-05 2024-04-24T15:33:12.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.416 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.416 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-04-24T15:33:12.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.244553+0000", 2024-04-24T15:33:12.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5380000000000002e-06 2024-04-24T15:33:12.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.417 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.417 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:12.417 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.631596+0000", 2024-04-24T15:33:12.417 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.38704289600000003 2024-04-24T15:33:12.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.417 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.417 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-04-24T15:33:12.417 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.631606+0000", 2024-04-24T15:33:12.417 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6390000000000004e-06 2024-04-24T15:33:12.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.417 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.417 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:12.417 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:48.658831+0000", 2024-04-24T15:33:12.417 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0272250540000001 2024-04-24T15:33:12.418 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.418 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.418 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:12.418 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:48.658991+0000", 2024-04-24T15:33:12.418 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015994800000000001 2024-04-24T15:33:12.418 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:12.418 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:12.418 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:12.418 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:48.659135+0000", 2024-04-24T15:33:12.418 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000144456 2024-04-24T15:33:12.418 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:12.418 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:12.418 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:12.418 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:12.419 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:12.419 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:12.420 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-24T15:33:12.423 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:12.423 INFO:teuthology.orchestra.run.smithi096.stdout:4037 read 0x76af79 thru 0x771e56 (0x6ede bytes) 2024-04-24T15:33:12.423 INFO:teuthology.orchestra.run.smithi096.stdout:4039 write 0x32e6081 thru 0x32ea1d0 (0x4150 bytes) 2024-04-24T15:33:12.423 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:12.423 INFO:teuthology.orchestra.run.smithi096.stdout:4042 write 0x2437ea2 thru 0x2442fa7 (0xb106 bytes) 2024-04-24T15:33:12.423 INFO:teuthology.orchestra.run.smithi096.stdout:4043 read 0xff8c14 thru 0xff9d35 (0x1122 bytes) 2024-04-24T15:33:12.423 INFO:teuthology.orchestra.run.smithi096.stdout:4044 write 0x6abc3c4 thru 0x6ac0a9b (0x46d8 bytes) 2024-04-24T15:33:12.423 INFO:teuthology.orchestra.run.smithi096.stdout:4045 write 0xeb52e65 thru 0xeb56172 (0x330e bytes) 2024-04-24T15:33:12.423 INFO:teuthology.orchestra.run.smithi096.stdout:4046 read 0x5caae5b thru 0x5cb6a6e (0xbc14 bytes) 2024-04-24T15:33:12.423 INFO:teuthology.orchestra.run.smithi096.stdout:4049 write 0x4c84ed6 thru 0x4c89304 (0x442f bytes) 2024-04-24T15:33:12.423 INFO:teuthology.orchestra.run.smithi096.stdout:4051 punch from 0xe97d9a0 to 0xe988c80, (0xb2e0 bytes) 2024-04-24T15:33:12.423 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:12.423 INFO:teuthology.orchestra.run.smithi096.stdout:4053 writesame 0x809c92d thru 0x80a189a data_size 0x3(0x4f6e bytes) 2024-04-24T15:33:12.424 INFO:teuthology.orchestra.run.smithi096.stdout:4054 read 0x8621ded thru 0x86260ab (0x42bf bytes) 2024-04-24T15:33:12.424 INFO:teuthology.orchestra.run.smithi096.stdout:4055 writesame 0x450b60e thru 0x4510a9c data_size 0x1(0x548f bytes) 2024-04-24T15:33:12.424 INFO:teuthology.orchestra.run.smithi096.stdout:4058 read 0xa71c07a thru 0xa71c5da (0x561 bytes) 2024-04-24T15:33:12.424 INFO:teuthology.orchestra.run.smithi096.stdout:4059 writesame 0x903f0be thru 0x903f7e8 data_size 0x1(0x72b bytes) 2024-04-24T15:33:12.424 INFO:teuthology.orchestra.run.smithi096.stdout:4060 trunc from 0xeb56173 to 0xde5e1f0 2024-04-24T15:33:12.424 INFO:teuthology.orchestra.run.smithi096.stdout:4061 write 0x467ac46 thru 0x468a8e1 (0xfc9c bytes) 2024-04-24T15:33:12.424 INFO:teuthology.orchestra.run.smithi096.stdout:4062 punch from 0x559d7e0 to 0x55a8d25, (0xb545 bytes) 2024-04-24T15:33:12.424 INFO:teuthology.orchestra.run.smithi096.stdout:4064 read 0x29f4dce thru 0x29f9fa6 (0x51d9 bytes) 2024-04-24T15:33:12.424 INFO:teuthology.orchestra.run.smithi096.stdout:4065 writesame 0x3b8a704 thru 0x3b998ea data_size 0x1(0xf1e7 bytes) 2024-04-24T15:33:12.424 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:33:12.421+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:33:12.436 INFO:teuthology.orchestra.run.smithi096.stdout:4066 read 0x2288d71 thru 0x22915a1 (0x8831 bytes) 2024-04-24T15:33:12.436 INFO:teuthology.orchestra.run.smithi096.stdout:4067 read 0x8b2ea81 thru 0x8b36eaf (0x842f bytes) 2024-04-24T15:33:12.436 INFO:teuthology.orchestra.run.smithi096.stdout:4068 punch from 0xbc401c6 to 0xbc4e6c7, (0xe501 bytes) 2024-04-24T15:33:12.436 INFO:teuthology.orchestra.run.smithi096.stdout:4070 punch from 0xbd200c5 to 0xbd22b4b, (0x2a86 bytes) 2024-04-24T15:33:12.436 INFO:teuthology.orchestra.run.smithi096.stdout:4071 punch from 0xcda8cf2 to 0xcdb1fec, (0x92fa bytes) 2024-04-24T15:33:12.439 INFO:teuthology.orchestra.run.smithi096.stdout:4072 trunc from 0xde5e1f0 to 0xb082f92 2024-04-24T15:33:12.443 INFO:teuthology.orchestra.run.smithi096.stdout:4073 write 0x981933c thru 0x9826568 (0xd22d bytes) 2024-04-24T15:33:12.453 INFO:teuthology.orchestra.run.smithi096.stdout:4075 writesame 0xc7a9bd4 thru 0xc7b98c9 data_size 0x56(0xfcf6 bytes) 2024-04-24T15:33:12.453 INFO:teuthology.orchestra.run.smithi096.stdout:4077 read 0x46898ff thru 0x4696505 (0xcc07 bytes) 2024-04-24T15:33:12.457 INFO:teuthology.orchestra.run.smithi096.stdout:4079 trunc from 0xc7b98ca to 0xa436185 2024-04-24T15:33:12.464 INFO:teuthology.orchestra.run.smithi096.stdout:4080 writesame 0x494415b thru 0x494cae2 data_size 0x2(0x8988 bytes) 2024-04-24T15:33:12.481 INFO:teuthology.orchestra.run.smithi096.stdout:4081 write 0xb923308 thru 0xb931225 (0xdf1e bytes) 2024-04-24T15:33:12.483 INFO:teuthology.orchestra.run.smithi096.stdout:4082 read 0x3eafbb1 thru 0x3ebf5de (0xfa2e bytes) 2024-04-24T15:33:12.484 INFO:teuthology.orchestra.run.smithi096.stdout:4083 write 0x64e5a thru 0x6a999 (0x5b40 bytes) 2024-04-24T15:33:12.493 INFO:teuthology.orchestra.run.smithi096.stdout:4084 write 0x8afbc6e thru 0x8b0a1f4 (0xe587 bytes) 2024-04-24T15:33:12.496 INFO:teuthology.orchestra.run.smithi096.stdout:4085 punch from 0xb8b230a to 0xb8bafc1, (0x8cb7 bytes) 2024-04-24T15:33:12.496 INFO:teuthology.orchestra.run.smithi096.stdout:4086 punch from 0x445754a to 0x4463af2, (0xc5a8 bytes) 2024-04-24T15:33:12.496 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:12.496 INFO:teuthology.orchestra.run.smithi096.stdout:4090 read 0x1b37391 thru 0x1b3b5be (0x422e bytes) 2024-04-24T15:33:12.496 INFO:teuthology.orchestra.run.smithi096.stdout:4091 punch from 0x530b6fc to 0x531456c, (0x8e70 bytes) 2024-04-24T15:33:12.499 INFO:teuthology.orchestra.run.smithi096.stdout:4093 trunc from 0xb931226 to 0xe35cd77 2024-04-24T15:33:12.504 INFO:teuthology.orchestra.run.smithi096.stdout:4095 write 0x229855d thru 0x22a4ea6 (0xc94a bytes) 2024-04-24T15:33:12.520 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:12.521+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:12.528 INFO:teuthology.orchestra.run.smithi096.stdout:4096 punch from 0x5d97fb7 to 0x5da33c3, (0xb40c bytes) 2024-04-24T15:33:12.528 INFO:teuthology.orchestra.run.smithi096.stdout:4097 write 0xa5c81d4 thru 0xa5d28a9 (0xa6d6 bytes) 2024-04-24T15:33:12.531 INFO:teuthology.orchestra.run.smithi096.stdout:4098 writesame 0x9c9b00f thru 0x9c9f540 data_size 0x11(0x4532 bytes) 2024-04-24T15:33:12.534 INFO:teuthology.orchestra.run.smithi096.stdout:4099 read 0x21acb2b thru 0x21ad567 (0xa3d bytes) 2024-04-24T15:33:12.535 INFO:teuthology.orchestra.run.smithi096.stdout:4100 read 0x9e3371f thru 0x9e3a361 (0x6c43 bytes) 2024-04-24T15:33:12.535 INFO:teuthology.orchestra.run.smithi096.stdout:4101 write 0x260aa52 thru 0x261953a (0xeae9 bytes) 2024-04-24T15:33:12.540 INFO:teuthology.orchestra.run.smithi096.stdout:4102 read 0x649aafb thru 0x64a49a2 (0x9ea8 bytes) 2024-04-24T15:33:12.540 INFO:teuthology.orchestra.run.smithi096.stdout:4103 write 0x1cdbe76 thru 0x1ce71b5 (0xb340 bytes) 2024-04-24T15:33:12.553 INFO:teuthology.orchestra.run.smithi096.stdout:4104 read 0x13a8b1e thru 0x13b3ba9 (0xb08c bytes) 2024-04-24T15:33:12.553 INFO:teuthology.orchestra.run.smithi096.stdout:4106 read 0x5219971 thru 0x521cb69 (0x31f9 bytes) 2024-04-24T15:33:12.562 INFO:teuthology.orchestra.run.smithi096.stdout:4107 trunc from 0xe35cd77 to 0x757a260 2024-04-24T15:33:12.581 INFO:teuthology.orchestra.run.smithi096.stdout:4108 write 0xab9e3ba thru 0xaba4a06 (0x664d bytes) 2024-04-24T15:33:12.581 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:12.582 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:33:12.582 INFO:teuthology.orchestra.run.smithi151.stdout: "num_ops": 0 2024-04-24T15:33:12.582 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:12.584 INFO:teuthology.orchestra.run.smithi096.stdout:4109 write 0x71682b3 thru 0x716bc9f (0x39ed bytes) 2024-04-24T15:33:12.587 INFO:teuthology.orchestra.run.smithi096.stdout:4111 write 0xa7d56d thru 0xa7ffae (0x2a42 bytes) 2024-04-24T15:33:12.594 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-24T15:33:12.598 INFO:teuthology.orchestra.run.smithi096.stdout:4112 trunc from 0xaba4a07 to 0x546f401 2024-04-24T15:33:12.613 INFO:teuthology.orchestra.run.smithi096.stdout:4114 trunc from 0x546f401 to 0x4e91427 2024-04-24T15:33:12.621 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:12.622+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:12.623 INFO:teuthology.orchestra.run.smithi096.stdout:4115 trunc from 0x4e91427 to 0x482e664 2024-04-24T15:33:12.630 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:12.631 INFO:teuthology.orchestra.run.smithi096.stdout:4117 write 0x3b2d842 thru 0x3b3426d (0x6a2c bytes) 2024-04-24T15:33:12.632 INFO:teuthology.orchestra.run.smithi096.stdout:4118 read 0xf91b56 thru 0xf9c563 (0xaa0e bytes) 2024-04-24T15:33:12.632 INFO:teuthology.orchestra.run.smithi096.stdout:4120 read 0x38a9229 thru 0x38b2002 (0x8dda bytes) 2024-04-24T15:33:12.632 INFO:teuthology.orchestra.run.smithi096.stdout:4121 read 0x4336bf5 thru 0x433fce5 (0x90f1 bytes) 2024-04-24T15:33:12.632 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:12.637 INFO:teuthology.orchestra.run.smithi096.stdout:4124 trunc from 0x482e664 to 0x876308c 2024-04-24T15:33:12.642 INFO:teuthology.orchestra.run.smithi096.stdout:4125 read 0x5b38754 thru 0x5b3b70a (0x2fb7 bytes) 2024-04-24T15:33:12.642 INFO:teuthology.orchestra.run.smithi096.stdout:4126 write 0x12bc327 thru 0x12bd52b (0x1205 bytes) 2024-04-24T15:33:12.652 INFO:teuthology.orchestra.run.smithi096.stdout:4127 punch from 0x8149327 to 0x8153c39, (0xa912 bytes) 2024-04-24T15:33:12.663 INFO:teuthology.orchestra.run.smithi096.stdout:4128 write 0xe004af4 thru 0xe008ae5 (0x3ff2 bytes) 2024-04-24T15:33:12.666 INFO:teuthology.orchestra.run.smithi096.stdout:4130 write 0xbb1b981 thru 0xbb1d8a3 (0x1f23 bytes) 2024-04-24T15:33:12.668 INFO:teuthology.orchestra.run.smithi096.stdout:4133 read 0xc5d509b thru 0xc5dc06a (0x6fd0 bytes) 2024-04-24T15:33:12.669 INFO:teuthology.orchestra.run.smithi096.stdout:4135 writesame 0xaf07971 thru 0xaf0b955 data_size 0x1(0x3fe5 bytes) 2024-04-24T15:33:12.675 INFO:teuthology.orchestra.run.smithi096.stdout:4137 writesame 0x561f661 thru 0x562313d data_size 0x1(0x3add bytes) 2024-04-24T15:33:12.683 INFO:teuthology.orchestra.run.smithi096.stdout:4139 punch from 0xbb8d154 to 0xbb8ed76, (0x1c22 bytes) 2024-04-24T15:33:12.683 INFO:teuthology.orchestra.run.smithi096.stdout:4140 punch from 0xb3441ce to 0xb345bba, (0x19ec bytes) 2024-04-24T15:33:12.683 INFO:teuthology.orchestra.run.smithi096.stdout:4141 write 0x1353d71 thru 0x1360677 (0xc907 bytes) 2024-04-24T15:33:12.702 INFO:teuthology.orchestra.run.smithi096.stdout:4142 trunc from 0xe008ae6 to 0x16ae625 2024-04-24T15:33:12.722 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:33:12.724+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:33:12.732 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:12.732 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:12.732 INFO:teuthology.orchestra.run.smithi096.stdout:4147 punch from 0x82a3c5 to 0x8340a1, (0x9cdc bytes) 2024-04-24T15:33:12.743 INFO:teuthology.orchestra.run.smithi096.stdout:4151 trunc from 0x16ae625 to 0xb63e38c 2024-04-24T15:33:12.749 INFO:teuthology.orchestra.run.smithi096.stdout:4152 read 0x58e531 thru 0x59533a (0x6e0a bytes) 2024-04-24T15:33:12.750 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:12.755 INFO:teuthology.orchestra.run.smithi096.stdout:4155 trunc from 0xb63e38c to 0x7335199 2024-04-24T15:33:12.760 INFO:teuthology.orchestra.run.smithi096.stdout:4156 write 0x226eb2a thru 0x227539f (0x6876 bytes) 2024-04-24T15:33:12.765 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:12.765 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:33:12.765 INFO:teuthology.orchestra.run.smithi151.stdout: "complaint_time": 30, 2024-04-24T15:33:12.765 INFO:teuthology.orchestra.run.smithi151.stdout: "num_blocked_ops": 0 2024-04-24T15:33:12.765 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:12.773 INFO:teuthology.orchestra.run.smithi096.stdout:4157 writesame 0xc003c4a thru 0xc012a33 data_size 0x3(0xedea bytes) 2024-04-24T15:33:12.773 INFO:teuthology.orchestra.run.smithi096.stdout:4158 write 0x91087fc thru 0x910c137 (0x393c bytes) 2024-04-24T15:33:12.776 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-24T15:33:12.776 INFO:teuthology.orchestra.run.smithi096.stdout:4159 writesame 0x2acf119 thru 0x2ada88b data_size 0x1(0xb773 bytes) 2024-04-24T15:33:12.799 INFO:teuthology.orchestra.run.smithi096.stdout:4160 punch from 0xac99fcb to 0xaca1fa5, (0x7fda bytes) 2024-04-24T15:33:12.809 INFO:teuthology.orchestra.run.smithi096.stdout:4163 trunc from 0xc012a34 to 0x39f9d7e 2024-04-24T15:33:12.819 INFO:teuthology.orchestra.run.smithi096.stdout:4164 read 0x6b386a thru 0x6b7450 (0x3be7 bytes) 2024-04-24T15:33:12.820 INFO:teuthology.orchestra.run.smithi096.stdout:4167 write 0xb7bb9c thru 0xb88f59 (0xd3be bytes) 2024-04-24T15:33:12.823 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:33:12.824+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:33:12.824 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:12.830 INFO:teuthology.orchestra.run.smithi096.stdout:4169 trunc from 0x39f9d7e to 0x7fd53ee 2024-04-24T15:33:12.834 INFO:teuthology.orchestra.run.smithi096.stdout:4171 read 0x60fda6d thru 0x60ff18c (0x1720 bytes) 2024-04-24T15:33:12.834 INFO:teuthology.orchestra.run.smithi096.stdout:4172 read 0x732f4d4 thru 0x7333e95 (0x49c2 bytes) 2024-04-24T15:33:12.835 INFO:teuthology.orchestra.run.smithi096.stdout:4173 read 0x3109173 thru 0x310aea2 (0x1d30 bytes) 2024-04-24T15:33:12.835 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:12.835 INFO:teuthology.orchestra.run.smithi096.stdout:4175 read 0x65debaa thru 0x65e178e (0x2be5 bytes) 2024-04-24T15:33:12.835 INFO:teuthology.orchestra.run.smithi096.stdout:4176 punch from 0x5babffe to 0x5bb8f83, (0xcf85 bytes) 2024-04-24T15:33:12.835 INFO:teuthology.orchestra.run.smithi096.stdout:4177 punch from 0xad8280 to 0xae3ef8, (0xbc78 bytes) 2024-04-24T15:33:12.835 INFO:teuthology.orchestra.run.smithi096.stdout:4184 writesame 0x4ec15bf thru 0x4ecf8dd data_size 0x1(0xe31f bytes) 2024-04-24T15:33:12.864 INFO:teuthology.orchestra.run.smithi096.stdout:4185 read 0x647ff11 thru 0x648a4c2 (0xa5b2 bytes) 2024-04-24T15:33:12.864 INFO:teuthology.orchestra.run.smithi096.stdout:4186 punch from 0x5bc156e to 0x5bc9684, (0x8116 bytes) 2024-04-24T15:33:12.864 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:12.865 INFO:teuthology.orchestra.run.smithi096.stdout:4188 write 0x314714 thru 0x317c36 (0x3523 bytes) 2024-04-24T15:33:12.875 INFO:teuthology.orchestra.run.smithi096.stdout:4189 write 0xb684757 thru 0xb68de91 (0x973b bytes) 2024-04-24T15:33:12.878 INFO:teuthology.orchestra.run.smithi096.stdout:4190 punch from 0xb242d28 to 0xb24f119, (0xc3f1 bytes) 2024-04-24T15:33:12.878 INFO:teuthology.orchestra.run.smithi096.stdout:4191 write 0x57361b3 thru 0x5737e0a (0x1c58 bytes) 2024-04-24T15:33:12.880 INFO:teuthology.orchestra.run.smithi096.stdout:4193 write 0xb64a464 thru 0xb64c56b (0x2108 bytes) 2024-04-24T15:33:12.882 INFO:teuthology.orchestra.run.smithi096.stdout:4194 writesame 0x5fdf616 thru 0x5fe3628 data_size 0x1(0x4013 bytes) 2024-04-24T15:33:12.889 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:12.889 INFO:teuthology.orchestra.run.smithi096.stdout:4197 writesame 0x9c9c3f9 thru 0x9c9cffd data_size 0x1(0xc05 bytes) 2024-04-24T15:33:12.892 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:12.892 INFO:teuthology.orchestra.run.smithi096.stdout:4199 writesame 0xb28bfda thru 0xb292e04 data_size 0x15(0x6e2b bytes) 2024-04-24T15:33:12.895 INFO:teuthology.orchestra.run.smithi096.stdout:4203 read 0xb05e284 thru 0xb06d4b7 (0xf234 bytes) 2024-04-24T15:33:12.902 INFO:teuthology.orchestra.run.smithi096.stdout:4204 write 0xc3e0a47 thru 0xc3eb0e6 (0xa6a0 bytes) 2024-04-24T15:33:12.905 INFO:teuthology.orchestra.run.smithi096.stdout:4205 writesame 0x4a12edb thru 0x4a1dceb data_size 0x1(0xae11 bytes) 2024-04-24T15:33:12.910 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:12.910 INFO:teuthology.orchestra.run.smithi151.stdout: "size": 20, 2024-04-24T15:33:12.910 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 600, 2024-04-24T15:33:12.911 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [ 2024-04-24T15:33:12.911 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.911 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:39041 4.1 4:82c1a902:::rbd_data.114abe589119.000000000000032c:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e68)", 2024-04-24T15:33:12.911 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:01.760552+0000", 2024-04-24T15:33:12.911 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 11.149648538999999, 2024-04-24T15:33:12.911 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.87570748300000001, 2024-04-24T15:33:12.911 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:12.911 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "reached pg", 2024-04-24T15:33:12.911 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:12.911 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:12.911 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:12.911 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 39041 2024-04-24T15:33:12.911 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.911 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:12.912 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.912 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:12.912 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760552+0000", 2024-04-24T15:33:12.912 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:12.912 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.912 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.912 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:12.912 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760552+0000", 2024-04-24T15:33:12.912 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:12.912 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.912 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.912 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:12.912 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760550+0000", 2024-04-24T15:33:12.912 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999986 2024-04-24T15:33:12.913 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.913 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.913 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:12.913 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760557+0000", 2024-04-24T15:33:12.913 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1420000000000004e-06 2024-04-24T15:33:12.913 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.913 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.913 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:12.913 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760572+0000", 2024-04-24T15:33:12.913 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.483e-05 2024-04-24T15:33:12.913 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.913 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.913 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:12.913 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760578+0000", 2024-04-24T15:33:12.914 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.2820000000000002e-06 2024-04-24T15:33:12.914 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.914 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.914 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:12.914 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760628+0000", 2024-04-24T15:33:12.914 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.0040000000000002e-05 2024-04-24T15:33:12.914 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.914 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.914 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op must wait for map", 2024-04-24T15:33:12.914 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760637+0000", 2024-04-24T15:33:12.914 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.6899999999999998e-06 2024-04-24T15:33:12.914 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.914 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.915 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:12.915 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.770562+0000", 2024-04-24T15:33:12.915 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0099252739999999996 2024-04-24T15:33:12.915 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.915 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.915 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for peered", 2024-04-24T15:33:12.915 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.770572+0000", 2024-04-24T15:33:12.915 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0105999999999999e-05 2024-04-24T15:33:12.915 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.915 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.915 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:12.915 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:02.636170+0000", 2024-04-24T15:33:12.915 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.86559722299999997 2024-04-24T15:33:12.915 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.916 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.916 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:12.916 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:02.636259+0000", 2024-04-24T15:33:12.916 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.9446999999999999e-05 2024-04-24T15:33:12.916 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:12.916 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:12.916 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:12.916 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.916 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.916 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48675 4.d 4:b465d49b:::rbd_data.114a858e6641.0000000000000002:17 [read 2768896~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:12.916 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.926403+0000", 2024-04-24T15:33:12.916 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 1.9837969690000001, 2024-04-24T15:33:12.916 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035742807000000001, 2024-04-24T15:33:12.917 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:12.917 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:12.917 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:12.917 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:12.917 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:12.917 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48675 2024-04-24T15:33:12.917 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.917 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:12.917 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.917 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:12.917 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926403+0000", 2024-04-24T15:33:12.917 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:12.917 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.917 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.918 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:12.918 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926403+0000", 2024-04-24T15:33:12.918 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:12.918 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.918 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.918 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:12.918 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926403+0000", 2024-04-24T15:33:12.918 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:12.918 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.918 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.918 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:12.918 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926404+0000", 2024-04-24T15:33:12.918 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.424e-06 2024-04-24T15:33:12.919 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.919 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.919 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:12.921 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926413+0000", 2024-04-24T15:33:12.921 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.6850000000000007e-06 2024-04-24T15:33:12.921 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.921 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.921 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:12.921 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926417+0000", 2024-04-24T15:33:12.921 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.7859999999999998e-06 2024-04-24T15:33:12.921 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.922 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.922 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:12.922 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.961986+0000", 2024-04-24T15:33:12.922 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035568862 2024-04-24T15:33:12.922 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.922 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.922 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:12.922 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.962063+0000", 2024-04-24T15:33:12.922 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7257999999999998e-05 2024-04-24T15:33:12.922 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.922 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.922 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:12.922 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.962146+0000", 2024-04-24T15:33:12.923 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3094000000000001e-05 2024-04-24T15:33:12.923 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:12.923 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:12.923 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:12.923 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.923 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.923 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48676 4.d 4:b465d49b:::rbd_data.114a858e6641.0000000000000002:17 [read 2834432~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:12.923 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.926421+0000", 2024-04-24T15:33:12.923 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 1.9837794259999999, 2024-04-24T15:33:12.923 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035829002999999998, 2024-04-24T15:33:12.923 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:12.923 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:12.923 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:12.924 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:12.924 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:12.924 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48676 2024-04-24T15:33:12.924 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.924 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:12.924 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.924 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:12.924 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926421+0000", 2024-04-24T15:33:12.924 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:12.924 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.924 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.924 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:12.924 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926421+0000", 2024-04-24T15:33:12.924 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:12.924 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.925 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.925 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:12.925 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926420+0000", 2024-04-24T15:33:12.925 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:12.925 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.925 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.925 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:12.925 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926423+0000", 2024-04-24T15:33:12.925 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.356e-06 2024-04-24T15:33:12.925 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.925 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.925 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:12.925 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926428+0000", 2024-04-24T15:33:12.925 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.3340000000000001e-06 2024-04-24T15:33:12.925 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.925 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.926 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:12.926 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926431+0000", 2024-04-24T15:33:12.926 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.2899999999999998e-06 2024-04-24T15:33:12.926 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.926 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.926 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:12.926 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.962156+0000", 2024-04-24T15:33:12.926 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035725159999999999 2024-04-24T15:33:12.926 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.926 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.926 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:12.926 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.962197+0000", 2024-04-24T15:33:12.926 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0457999999999998e-05 2024-04-24T15:33:12.927 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.927 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.927 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:12.927 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.962250+0000", 2024-04-24T15:33:12.927 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.2812000000000002e-05 2024-04-24T15:33:12.927 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:12.927 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:12.927 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:12.927 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.927 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.927 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48703 4.d 4:b59d5735:::rbd_data.114a858e6641.0000000000000004:17 [read 2637824~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:12.927 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928531+0000", 2024-04-24T15:33:12.927 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 1.9816688, 2024-04-24T15:33:12.927 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035675198999999998, 2024-04-24T15:33:12.927 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:12.928 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:12.928 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:12.928 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:12.928 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:12.928 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48703 2024-04-24T15:33:12.928 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.928 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:12.928 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.928 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:12.928 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928531+0000", 2024-04-24T15:33:12.928 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:12.928 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.928 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.928 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:12.928 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928531+0000", 2024-04-24T15:33:12.928 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:12.929 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.929 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.929 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:12.929 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928531+0000", 2024-04-24T15:33:12.929 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:12.929 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.929 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.929 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:12.929 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928532+0000", 2024-04-24T15:33:12.929 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.33e-06 2024-04-24T15:33:12.929 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.929 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.929 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:12.929 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928536+0000", 2024-04-24T15:33:12.930 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.4419999999999998e-06 2024-04-24T15:33:12.930 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.930 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.930 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:12.930 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928538+0000", 2024-04-24T15:33:12.930 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.0669999999999999e-06 2024-04-24T15:33:12.930 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.930 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.930 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:12.930 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964040+0000", 2024-04-24T15:33:12.930 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035501900000000003 2024-04-24T15:33:12.930 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.930 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.931 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:12.931 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964118+0000", 2024-04-24T15:33:12.931 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.8490000000000002e-05 2024-04-24T15:33:12.931 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.931 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.931 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:12.931 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964207+0000", 2024-04-24T15:33:12.931 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.8202999999999996e-05 2024-04-24T15:33:12.931 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:12.931 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:12.931 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:12.931 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.931 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.931 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48704 4.d 4:b59d5735:::rbd_data.114a858e6641.0000000000000004:17 [read 2703360~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:12.931 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928541+0000", 2024-04-24T15:33:12.932 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 1.981659456, 2024-04-24T15:33:12.932 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035763016000000002, 2024-04-24T15:33:12.932 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:12.932 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:12.932 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:12.932 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:12.932 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:12.932 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48704 2024-04-24T15:33:12.932 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.932 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:12.932 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.932 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:12.932 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928541+0000", 2024-04-24T15:33:12.932 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:12.932 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.932 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.933 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:12.933 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928541+0000", 2024-04-24T15:33:12.933 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:12.933 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.933 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.933 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:12.933 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928540+0000", 2024-04-24T15:33:12.933 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:12.933 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.933 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.933 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:12.933 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928542+0000", 2024-04-24T15:33:12.933 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.37e-06 2024-04-24T15:33:12.934 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.934 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.934 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:12.934 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928545+0000", 2024-04-24T15:33:12.934 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8229999999999999e-06 2024-04-24T15:33:12.934 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.934 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.934 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:12.934 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928547+0000", 2024-04-24T15:33:12.934 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.9360000000000002e-06 2024-04-24T15:33:12.934 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.934 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.934 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:12.934 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964219+0000", 2024-04-24T15:33:12.935 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035672813999999997 2024-04-24T15:33:12.935 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.935 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.935 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:12.935 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964259+0000", 2024-04-24T15:33:12.935 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0070000000000001e-05 2024-04-24T15:33:12.935 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.935 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.935 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:12.935 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964304+0000", 2024-04-24T15:33:12.935 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4229e-05 2024-04-24T15:33:12.935 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:12.935 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:12.935 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:12.935 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.935 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.936 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48719 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 1851392~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:12.936 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928549+0000", 2024-04-24T15:33:12.936 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 1.9816513090000001, 2024-04-24T15:33:12.936 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035890645999999998, 2024-04-24T15:33:12.936 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:12.936 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:12.936 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:12.936 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:12.936 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:12.936 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48719 2024-04-24T15:33:12.936 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.936 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:12.936 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.936 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:12.936 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928549+0000", 2024-04-24T15:33:12.936 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:12.937 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.937 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.937 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:12.937 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928549+0000", 2024-04-24T15:33:12.937 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:12.937 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.937 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.937 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:12.937 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928549+0000", 2024-04-24T15:33:12.937 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:12.937 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.937 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.937 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:12.937 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928550+0000", 2024-04-24T15:33:12.937 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3829999999999999e-06 2024-04-24T15:33:12.938 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.938 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.938 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:12.938 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928553+0000", 2024-04-24T15:33:12.938 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7369999999999998e-06 2024-04-24T15:33:12.938 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.938 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.938 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:12.938 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928555+0000", 2024-04-24T15:33:12.938 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.8920000000000001e-06 2024-04-24T15:33:12.938 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.938 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.938 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:12.938 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964240+0000", 2024-04-24T15:33:12.939 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035685537000000003 2024-04-24T15:33:12.939 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.939 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.939 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:12.939 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964318+0000", 2024-04-24T15:33:12.939 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7838999999999998e-05 2024-04-24T15:33:12.939 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.939 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.939 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:12.939 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964439+0000", 2024-04-24T15:33:12.939 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000121564 2024-04-24T15:33:12.939 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:12.939 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:12.939 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:12.939 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.939 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.940 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48720 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 1916928~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:12.940 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928557+0000", 2024-04-24T15:33:12.940 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 1.981643464, 2024-04-24T15:33:12.940 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036005219999999998, 2024-04-24T15:33:12.940 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:12.940 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:12.940 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:12.940 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:12.940 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:12.940 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48720 2024-04-24T15:33:12.940 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.940 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:12.940 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.940 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:12.940 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928557+0000", 2024-04-24T15:33:12.941 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:12.941 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.941 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.941 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:12.941 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928557+0000", 2024-04-24T15:33:12.941 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:12.941 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.941 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.941 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:12.941 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928556+0000", 2024-04-24T15:33:12.941 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:12.941 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.941 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.941 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:12.941 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928558+0000", 2024-04-24T15:33:12.942 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.13e-06 2024-04-24T15:33:12.942 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.942 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.942 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:12.942 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928561+0000", 2024-04-24T15:33:12.942 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5530000000000001e-06 2024-04-24T15:33:12.942 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.942 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.942 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:12.942 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928563+0000", 2024-04-24T15:33:12.942 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.8810000000000001e-06 2024-04-24T15:33:12.942 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.942 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.942 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:12.942 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964450+0000", 2024-04-24T15:33:12.943 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035887040000000002 2024-04-24T15:33:12.943 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.943 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.943 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:12.943 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964494+0000", 2024-04-24T15:33:12.943 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3699000000000003e-05 2024-04-24T15:33:12.943 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.943 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.943 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:12.943 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964562+0000", 2024-04-24T15:33:12.943 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.8149000000000001e-05 2024-04-24T15:33:12.943 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:12.943 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:12.943 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:12.944 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.944 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.944 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48721 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 1982464~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:12.944 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928565+0000", 2024-04-24T15:33:12.944 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 1.98163494, 2024-04-24T15:33:12.944 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036233466999999998, 2024-04-24T15:33:12.944 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:12.944 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:12.944 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:12.944 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:12.944 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:12.944 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48721 2024-04-24T15:33:12.944 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.944 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:12.944 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.944 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:12.945 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928565+0000", 2024-04-24T15:33:12.945 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:12.945 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.945 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.945 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:12.945 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928565+0000", 2024-04-24T15:33:12.945 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:12.945 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.945 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.945 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:12.945 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928565+0000", 2024-04-24T15:33:12.945 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:12.945 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.945 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.945 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:12.946 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928566+0000", 2024-04-24T15:33:12.946 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.336e-06 2024-04-24T15:33:12.946 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.946 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.946 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:12.946 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928569+0000", 2024-04-24T15:33:12.946 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.9189999999999999e-06 2024-04-24T15:33:12.946 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.946 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.946 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:12.946 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928571+0000", 2024-04-24T15:33:12.946 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.1909999999999999e-06 2024-04-24T15:33:12.946 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.946 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.946 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:12.947 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964592+0000", 2024-04-24T15:33:12.947 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036020120000000003 2024-04-24T15:33:12.947 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.947 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.947 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:12.947 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964673+0000", 2024-04-24T15:33:12.947 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.1254000000000005e-05 2024-04-24T15:33:12.947 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.947 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.947 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:12.947 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964799+0000", 2024-04-24T15:33:12.947 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00012588700000000001 2024-04-24T15:33:12.947 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:12.947 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:12.947 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:12.948 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.948 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.948 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48722 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2048000~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:12.948 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928586+0000", 2024-04-24T15:33:12.948 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 1.981613815, 2024-04-24T15:33:12.948 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036331347999999999, 2024-04-24T15:33:12.948 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:12.948 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:12.948 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:12.948 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:12.948 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:12.948 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48722 2024-04-24T15:33:12.948 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.948 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:12.948 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.948 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:12.949 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928586+0000", 2024-04-24T15:33:12.949 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:12.949 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.949 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.949 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:12.949 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928586+0000", 2024-04-24T15:33:12.949 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:12.949 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.949 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.949 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:12.949 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928586+0000", 2024-04-24T15:33:12.949 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:12.949 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.949 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.949 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:12.949 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928587+0000", 2024-04-24T15:33:12.950 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3519999999999999e-06 2024-04-24T15:33:12.950 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.950 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.950 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:12.950 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928590+0000", 2024-04-24T15:33:12.950 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.9699999999999999e-06 2024-04-24T15:33:12.950 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.950 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.950 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:12.950 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928592+0000", 2024-04-24T15:33:12.950 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.097e-06 2024-04-24T15:33:12.950 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.950 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.950 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:12.951 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964810+0000", 2024-04-24T15:33:12.951 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036217249 2024-04-24T15:33:12.951 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.951 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.951 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:12.951 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964851+0000", 2024-04-24T15:33:12.951 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.1151e-05 2024-04-24T15:33:12.951 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.951 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.951 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:12.951 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964918+0000", 2024-04-24T15:33:12.951 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.6898999999999998e-05 2024-04-24T15:33:12.951 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:12.951 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:12.951 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:12.952 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.952 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.952 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48723 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2113536~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:12.952 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928598+0000", 2024-04-24T15:33:12.952 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 1.981602426, 2024-04-24T15:33:12.952 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036552125999999997, 2024-04-24T15:33:12.952 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:12.952 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:12.952 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:12.952 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:12.952 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:12.952 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48723 2024-04-24T15:33:12.952 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.952 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:12.952 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.953 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:12.953 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928598+0000", 2024-04-24T15:33:12.953 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:12.953 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.953 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.953 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:12.953 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928598+0000", 2024-04-24T15:33:12.953 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:12.953 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.953 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.953 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:12.953 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928597+0000", 2024-04-24T15:33:12.953 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:12.953 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.953 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.953 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:12.954 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928599+0000", 2024-04-24T15:33:12.954 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.621e-06 2024-04-24T15:33:12.954 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.954 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.954 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:12.954 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928602+0000", 2024-04-24T15:33:12.954 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.9629999999999998e-06 2024-04-24T15:33:12.954 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.954 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.954 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:12.954 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928607+0000", 2024-04-24T15:33:12.954 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8940000000000001e-06 2024-04-24T15:33:12.954 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.954 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.954 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:12.955 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964947+0000", 2024-04-24T15:33:12.955 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036340439000000002 2024-04-24T15:33:12.955 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.955 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.955 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:12.955 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965028+0000", 2024-04-24T15:33:12.955 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.1054e-05 2024-04-24T15:33:12.955 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.955 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.955 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:12.955 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965150+0000", 2024-04-24T15:33:12.955 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00012154000000000001 2024-04-24T15:33:12.955 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:12.955 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:12.955 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:12.956 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.956 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.956 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48724 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2179072~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:12.956 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928609+0000", 2024-04-24T15:33:12.956 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 1.9815910510000001, 2024-04-24T15:33:12.956 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036679614999999999, 2024-04-24T15:33:12.956 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:12.956 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:12.956 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:12.956 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:12.956 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:12.956 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48724 2024-04-24T15:33:12.956 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.956 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:12.956 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.957 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:12.957 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928609+0000", 2024-04-24T15:33:12.957 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:12.957 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.957 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.957 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:12.957 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928609+0000", 2024-04-24T15:33:12.957 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:12.957 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.957 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.957 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:12.957 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928609+0000", 2024-04-24T15:33:12.957 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:12.957 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.957 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.958 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:12.958 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928610+0000", 2024-04-24T15:33:12.958 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3030000000000001e-06 2024-04-24T15:33:12.958 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.958 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.958 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:12.958 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928613+0000", 2024-04-24T15:33:12.958 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.762e-06 2024-04-24T15:33:12.958 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.958 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.958 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:12.958 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928615+0000", 2024-04-24T15:33:12.958 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.832e-06 2024-04-24T15:33:12.958 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.958 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.959 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:12.959 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965174+0000", 2024-04-24T15:33:12.959 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036559633000000001 2024-04-24T15:33:12.959 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.959 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.959 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:12.959 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965221+0000", 2024-04-24T15:33:12.959 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6239999999999998e-05 2024-04-24T15:33:12.959 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.959 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.959 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:12.959 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965289+0000", 2024-04-24T15:33:12.959 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.8053999999999996e-05 2024-04-24T15:33:12.959 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:12.959 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:12.960 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:12.960 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.960 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.960 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48725 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2244608~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:12.960 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928617+0000", 2024-04-24T15:33:12.960 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 1.981583281, 2024-04-24T15:33:12.960 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036902659999999997, 2024-04-24T15:33:12.960 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:12.960 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:12.960 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:12.960 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:12.960 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:12.960 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48725 2024-04-24T15:33:12.960 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.960 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:12.961 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.961 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:12.961 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928617+0000", 2024-04-24T15:33:12.961 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:12.961 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.961 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.961 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:12.961 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928617+0000", 2024-04-24T15:33:12.961 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:12.961 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.961 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.961 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:12.961 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928617+0000", 2024-04-24T15:33:12.961 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:12.962 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.962 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.962 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:12.962 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928618+0000", 2024-04-24T15:33:12.962 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.206e-06 2024-04-24T15:33:12.962 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.962 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.962 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:12.962 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928621+0000", 2024-04-24T15:33:12.962 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5130000000000001e-06 2024-04-24T15:33:12.962 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.962 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.963 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:12.963 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928623+0000", 2024-04-24T15:33:12.963 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.9640000000000002e-06 2024-04-24T15:33:12.963 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.963 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.963 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:12.963 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965318+0000", 2024-04-24T15:33:12.963 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036694953000000002 2024-04-24T15:33:12.963 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.963 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.963 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:12.963 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965398+0000", 2024-04-24T15:33:12.963 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.9986000000000003e-05 2024-04-24T15:33:12.963 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.963 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.963 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:12.964 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965520+0000", 2024-04-24T15:33:12.964 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000121247 2024-04-24T15:33:12.964 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:12.964 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:12.964 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:12.964 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.964 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.964 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48726 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2310144~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:12.964 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928625+0000", 2024-04-24T15:33:12.964 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 1.9815747340000001, 2024-04-24T15:33:12.964 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037015521000000003, 2024-04-24T15:33:12.964 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:12.964 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:12.964 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:12.964 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:12.964 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:12.965 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48726 2024-04-24T15:33:12.965 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.965 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:12.965 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.965 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:12.965 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928625+0000", 2024-04-24T15:33:12.965 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:12.965 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.965 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.965 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:12.965 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928625+0000", 2024-04-24T15:33:12.965 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:12.965 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.965 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.966 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:12.966 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928625+0000", 2024-04-24T15:33:12.966 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:12.966 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.966 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.966 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:12.966 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928627+0000", 2024-04-24T15:33:12.966 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3179999999999999e-06 2024-04-24T15:33:12.966 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.966 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.966 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:12.966 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928629+0000", 2024-04-24T15:33:12.966 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.819e-06 2024-04-24T15:33:12.966 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.966 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.967 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:12.967 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928634+0000", 2024-04-24T15:33:12.967 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6999999999999999e-06 2024-04-24T15:33:12.967 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.967 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.967 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:12.967 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965531+0000", 2024-04-24T15:33:12.967 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036897040999999998 2024-04-24T15:33:12.967 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.967 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.967 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:12.967 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965573+0000", 2024-04-24T15:33:12.967 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.189e-05 2024-04-24T15:33:12.967 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.967 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.968 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:12.968 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965641+0000", 2024-04-24T15:33:12.968 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.7959000000000004e-05 2024-04-24T15:33:12.968 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:12.968 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:12.968 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:12.968 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.968 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.968 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48727 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2375680~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:12.968 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928637+0000", 2024-04-24T15:33:12.968 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 1.981563373, 2024-04-24T15:33:12.968 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037235129999999998, 2024-04-24T15:33:12.968 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:12.968 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:12.968 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:12.969 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:12.969 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:12.969 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48727 2024-04-24T15:33:12.969 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.969 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:12.969 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.969 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:12.969 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928637+0000", 2024-04-24T15:33:12.969 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:12.969 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.969 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.969 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:12.969 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928637+0000", 2024-04-24T15:33:12.969 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:12.969 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.969 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.970 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:12.970 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928637+0000", 2024-04-24T15:33:12.970 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:12.970 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.970 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.970 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:12.970 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928638+0000", 2024-04-24T15:33:12.970 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.4279999999999999e-06 2024-04-24T15:33:12.970 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.970 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.971 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:12.971 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928641+0000", 2024-04-24T15:33:12.971 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.909e-06 2024-04-24T15:33:12.971 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.971 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.971 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:12.971 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928643+0000", 2024-04-24T15:33:12.971 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.9269999999999999e-06 2024-04-24T15:33:12.971 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.971 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.971 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:12.971 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965671+0000", 2024-04-24T15:33:12.971 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037028393 2024-04-24T15:33:12.971 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.972 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.972 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:12.972 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965752+0000", 2024-04-24T15:33:12.972 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0921999999999995e-05 2024-04-24T15:33:12.972 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.972 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.972 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:12.972 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965872+0000", 2024-04-24T15:33:12.972 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000119759 2024-04-24T15:33:12.972 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:12.972 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:12.972 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:12.972 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.972 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.972 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48728 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2441216~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:12.972 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928645+0000", 2024-04-24T15:33:12.973 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 1.9815551899999999, 2024-04-24T15:33:12.973 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037346852999999999, 2024-04-24T15:33:12.973 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:12.973 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:12.973 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:12.973 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:12.973 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:12.973 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48728 2024-04-24T15:33:12.973 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.973 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:12.973 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.973 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:12.973 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928645+0000", 2024-04-24T15:33:12.973 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:12.973 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.974 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.974 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:12.974 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928645+0000", 2024-04-24T15:33:12.974 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:12.974 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.974 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.974 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:12.974 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928645+0000", 2024-04-24T15:33:12.974 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:12.974 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.974 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.974 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:12.974 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928646+0000", 2024-04-24T15:33:12.974 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.291e-06 2024-04-24T15:33:12.975 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.975 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.975 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:12.975 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928649+0000", 2024-04-24T15:33:12.975 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8509999999999999e-06 2024-04-24T15:33:12.975 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.975 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.975 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:12.975 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928651+0000", 2024-04-24T15:33:12.975 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.809e-06 2024-04-24T15:33:12.975 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.975 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.975 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:12.975 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965882+0000", 2024-04-24T15:33:12.975 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037231715999999998 2024-04-24T15:33:12.976 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.976 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.976 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:12.976 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965925+0000", 2024-04-24T15:33:12.976 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2493999999999997e-05 2024-04-24T15:33:12.976 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.976 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.976 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:12.976 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965992+0000", 2024-04-24T15:33:12.976 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.6907000000000002e-05 2024-04-24T15:33:12.976 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:12.976 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:12.976 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:12.976 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.976 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.977 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48729 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2506752~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:12.977 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928653+0000", 2024-04-24T15:33:12.977 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 1.981547315, 2024-04-24T15:33:12.977 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037576904000000001, 2024-04-24T15:33:12.977 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:12.977 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:12.977 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:12.977 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:12.977 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:12.977 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48729 2024-04-24T15:33:12.977 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.977 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:12.977 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.977 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:12.977 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928653+0000", 2024-04-24T15:33:12.977 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:12.978 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.978 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.978 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:12.978 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928653+0000", 2024-04-24T15:33:12.978 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:12.978 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.978 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.978 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:12.978 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928653+0000", 2024-04-24T15:33:12.978 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:12.978 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.978 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.978 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:12.978 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928654+0000", 2024-04-24T15:33:12.978 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3209999999999999e-06 2024-04-24T15:33:12.979 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.979 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.979 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:12.979 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928657+0000", 2024-04-24T15:33:12.979 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8559999999999998e-06 2024-04-24T15:33:12.979 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.979 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.979 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:12.979 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928659+0000", 2024-04-24T15:33:12.979 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7080000000000002e-06 2024-04-24T15:33:12.979 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.979 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.979 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:12.979 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966021+0000", 2024-04-24T15:33:12.979 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037361296000000002 2024-04-24T15:33:12.980 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.980 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.980 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:12.980 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966102+0000", 2024-04-24T15:33:12.980 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0939999999999994e-05 2024-04-24T15:33:12.980 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.980 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.980 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:12.980 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966230+0000", 2024-04-24T15:33:12.980 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00012799999999999999 2024-04-24T15:33:12.980 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:12.980 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:12.980 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:12.980 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.981 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.981 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48730 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2572288~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:12.981 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928662+0000", 2024-04-24T15:33:12.981 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 1.9815382270000002, 2024-04-24T15:33:12.981 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037687246000000001, 2024-04-24T15:33:12.981 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:12.981 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:12.981 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:12.981 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:12.981 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:12.981 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48730 2024-04-24T15:33:12.981 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.981 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:12.981 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.981 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:12.982 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928662+0000", 2024-04-24T15:33:12.982 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:12.982 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.982 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.982 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:12.982 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928662+0000", 2024-04-24T15:33:12.982 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:12.982 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.982 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.982 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:12.982 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928662+0000", 2024-04-24T15:33:12.982 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:12.982 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.982 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.982 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:12.983 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928663+0000", 2024-04-24T15:33:12.983 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.381e-06 2024-04-24T15:33:12.983 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.983 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.983 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:12.983 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928666+0000", 2024-04-24T15:33:12.983 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.796e-06 2024-04-24T15:33:12.983 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.983 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.983 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:12.983 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928668+0000", 2024-04-24T15:33:12.983 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.8920000000000001e-06 2024-04-24T15:33:12.983 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.983 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.983 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:12.984 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966240+0000", 2024-04-24T15:33:12.984 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037572886999999999 2024-04-24T15:33:12.984 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.984 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.984 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:12.984 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966281+0000", 2024-04-24T15:33:12.984 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0450000000000001e-05 2024-04-24T15:33:12.984 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.984 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.984 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:12.984 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966349+0000", 2024-04-24T15:33:12.984 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.8215999999999999e-05 2024-04-24T15:33:12.984 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:12.984 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:12.984 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:12.984 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.985 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.985 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48731 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2637824~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:12.985 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928747+0000", 2024-04-24T15:33:12.985 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 1.9814532219999998, 2024-04-24T15:33:12.985 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037833602000000001, 2024-04-24T15:33:12.985 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:12.985 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:12.985 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:12.985 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:12.985 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:12.985 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48731 2024-04-24T15:33:12.985 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.985 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:12.985 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.985 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:12.986 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928747+0000", 2024-04-24T15:33:12.986 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:12.986 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.986 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.986 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:12.986 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928747+0000", 2024-04-24T15:33:12.986 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:12.986 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.986 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.986 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:12.986 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928670+0000", 2024-04-24T15:33:12.986 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999228 2024-04-24T15:33:12.986 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.986 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.986 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:12.987 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928750+0000", 2024-04-24T15:33:12.987 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0527999999999999e-05 2024-04-24T15:33:12.987 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.987 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.987 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:12.987 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928757+0000", 2024-04-24T15:33:12.987 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.9419999999999998e-06 2024-04-24T15:33:12.987 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.987 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.987 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:12.987 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928762+0000", 2024-04-24T15:33:12.987 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4240000000000003e-06 2024-04-24T15:33:12.987 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.987 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.987 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:12.988 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966379+0000", 2024-04-24T15:33:12.988 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037617267000000003 2024-04-24T15:33:12.988 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.988 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.988 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:12.988 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966461+0000", 2024-04-24T15:33:12.988 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.2720999999999996e-05 2024-04-24T15:33:12.988 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.988 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.988 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:12.988 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966581+0000", 2024-04-24T15:33:12.988 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000119014 2024-04-24T15:33:12.988 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:12.988 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:12.988 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:12.989 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.989 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.989 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48732 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2703360~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:12.989 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928766+0000", 2024-04-24T15:33:12.989 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 1.981433738, 2024-04-24T15:33:12.989 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037930924999999997, 2024-04-24T15:33:12.989 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:12.989 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:12.989 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:12.989 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:12.989 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:12.989 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48732 2024-04-24T15:33:12.989 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.989 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:12.989 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.989 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:12.990 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928766+0000", 2024-04-24T15:33:12.990 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:12.990 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.990 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.990 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:12.990 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928766+0000", 2024-04-24T15:33:12.990 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:12.990 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.990 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.990 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:12.990 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928766+0000", 2024-04-24T15:33:12.990 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:12.990 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.990 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.991 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:12.991 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928768+0000", 2024-04-24T15:33:12.991 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.3420000000000002e-06 2024-04-24T15:33:12.991 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.991 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.991 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:12.991 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928783+0000", 2024-04-24T15:33:12.991 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.4813e-05 2024-04-24T15:33:12.991 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.991 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.991 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:12.991 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928787+0000", 2024-04-24T15:33:12.991 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.1999999999999996e-06 2024-04-24T15:33:12.991 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.991 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.992 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:12.992 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966592+0000", 2024-04-24T15:33:12.992 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037804484999999999 2024-04-24T15:33:12.992 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.992 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.992 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:12.992 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966632+0000", 2024-04-24T15:33:12.992 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0014e-05 2024-04-24T15:33:12.992 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.992 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.992 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:12.992 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966697+0000", 2024-04-24T15:33:12.992 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.5489999999999998e-05 2024-04-24T15:33:12.992 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:12.992 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:12.993 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:12.993 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.993 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.993 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48813 4.d 4:b39400aa:::rbd_data.114a8b543e91.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:12.993 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.936015+0000", 2024-04-24T15:33:12.993 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 1.974185404, 2024-04-24T15:33:12.993 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.039312294999999997, 2024-04-24T15:33:12.993 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:12.993 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:12.993 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:12.993 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:12.993 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:12.993 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48813 2024-04-24T15:33:12.993 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.993 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:12.993 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.994 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:12.994 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.936015+0000", 2024-04-24T15:33:12.994 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:12.994 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.994 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.994 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:12.994 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.936015+0000", 2024-04-24T15:33:12.994 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:12.994 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.994 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.994 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:12.994 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.936014+0000", 2024-04-24T15:33:12.994 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:12.994 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.994 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.995 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:12.995 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.942618+0000", 2024-04-24T15:33:12.995 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0066038499999999997 2024-04-24T15:33:12.995 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.995 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.995 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:12.995 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.942629+0000", 2024-04-24T15:33:12.995 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0606e-05 2024-04-24T15:33:12.995 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.995 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.995 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:12.995 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.942634+0000", 2024-04-24T15:33:12.995 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.2000000000000002e-06 2024-04-24T15:33:12.995 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.995 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.995 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:12.996 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.969006+0000", 2024-04-24T15:33:12.996 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.026371564 2024-04-24T15:33:12.996 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.996 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.996 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:12.996 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.969126+0000", 2024-04-24T15:33:12.996 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000120085 2024-04-24T15:33:12.996 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.996 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.996 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 2", 2024-04-24T15:33:12.996 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.969469+0000", 2024-04-24T15:33:12.996 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00034295400000000002 2024-04-24T15:33:12.996 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.996 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.996 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:33:12.997 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.973118+0000", 2024-04-24T15:33:12.997 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0036489700000000001 2024-04-24T15:33:12.997 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.997 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.997 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:33:12.997 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.975250+0000", 2024-04-24T15:33:12.997 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0021327030000000001 2024-04-24T15:33:12.997 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.997 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.997 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:33:12.997 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.975293+0000", 2024-04-24T15:33:12.997 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2951999999999998e-05 2024-04-24T15:33:12.997 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:12.997 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:12.997 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:12.997 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.975327+0000", 2024-04-24T15:33:12.998 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.3840000000000001e-05 2024-04-24T15:33:12.998 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:12.998 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:12.998 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:12.998 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:12.998 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:12.998 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:13.000 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-24T15:33:13.000 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:12.999+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:13.002 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:13.002 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:13.002 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:13.002 INFO:teuthology.orchestra.run.smithi096.stdout:4209 read 0x8c401f3 thru 0x8c42622 (0x2430 bytes) 2024-04-24T15:33:13.002 INFO:teuthology.orchestra.run.smithi096.stdout:4210 write 0x9cb642b thru 0x9cbc977 (0x654d bytes) 2024-04-24T15:33:13.002 INFO:teuthology.orchestra.run.smithi096.stdout:4211 write 0x84adde9 thru 0x84b2d66 (0x4f7e bytes) 2024-04-24T15:33:13.002 INFO:teuthology.orchestra.run.smithi096.stdout:4213 writesame 0xe8ed594 thru 0xe8f8586 data_size 0x1(0xaff3 bytes) 2024-04-24T15:33:13.003 INFO:teuthology.orchestra.run.smithi096.stdout:4215 read 0xcff87d4 thru 0xcffa0ce (0x18fb bytes) 2024-04-24T15:33:13.003 INFO:teuthology.orchestra.run.smithi096.stdout:4218 read 0x630f428 thru 0x631ca34 (0xd60d bytes) 2024-04-24T15:33:13.003 INFO:teuthology.orchestra.run.smithi096.stdout:4220 writesame 0xa2ff373 thru 0xa30d204 data_size 0x1f(0xde92 bytes) 2024-04-24T15:33:13.003 INFO:teuthology.orchestra.run.smithi096.stdout:4221 writesame 0xbf08705 thru 0xbf0885f data_size 0x1(0x15b bytes) 2024-04-24T15:33:13.003 INFO:teuthology.orchestra.run.smithi096.stdout:4222 write 0xb87ca03 thru 0xb889aa8 (0xd0a6 bytes) 2024-04-24T15:33:13.003 INFO:teuthology.orchestra.run.smithi096.stdout:4224 writesame 0xd6a237 thru 0xd6f897 data_size 0x5b(0x5661 bytes) 2024-04-24T15:33:13.003 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:13.003 INFO:teuthology.orchestra.run.smithi096.stdout:4229 punch from 0xa7dee73 to 0xa7e21c5, (0x3352 bytes) 2024-04-24T15:33:13.003 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:13.003 INFO:teuthology.orchestra.run.smithi096.stdout:4234 write 0xbe59e86 thru 0xbe606d8 (0x6853 bytes) 2024-04-24T15:33:13.003 INFO:teuthology.orchestra.run.smithi096.stdout:4235 writesame 0x43e6875 thru 0x43ee7d4 data_size 0x10(0x7f60 bytes) 2024-04-24T15:33:13.003 INFO:teuthology.orchestra.run.smithi096.stdout:4237 trunc from 0xe8f8587 to 0x8ec1a95 2024-04-24T15:33:13.003 INFO:teuthology.orchestra.run.smithi096.stdout:4238 punch from 0x8ca9cc7 to 0x8cb0a61, (0x6d9a bytes) 2024-04-24T15:33:13.003 INFO:teuthology.orchestra.run.smithi096.stdout:4240 read 0x3370af4 thru 0x3375a03 (0x4f10 bytes) 2024-04-24T15:33:13.003 INFO:teuthology.orchestra.run.smithi096.stdout:4241 trunc from 0x8ec1a95 to 0xbe8bab0 2024-04-24T15:33:13.004 INFO:teuthology.orchestra.run.smithi096.stdout:4243 read 0x20802bb thru 0x208303c (0x2d82 bytes) 2024-04-24T15:33:13.004 INFO:teuthology.orchestra.run.smithi096.stdout:4244 punch from 0x407c227 to 0x40863c3, (0xa19c bytes) 2024-04-24T15:33:13.004 INFO:teuthology.orchestra.run.smithi096.stdout:4245 write 0x59b93da thru 0x59b94c5 (0xec bytes) 2024-04-24T15:33:13.004 INFO:teuthology.orchestra.run.smithi096.stdout:4246 writesame 0xa01a9d5 thru 0xa01b2e2 data_size 0x13(0x90e bytes) 2024-04-24T15:33:13.004 INFO:teuthology.orchestra.run.smithi096.stdout:4247 read 0x60b27db thru 0x60bc225 (0x9a4b bytes) 2024-04-24T15:33:13.004 INFO:teuthology.orchestra.run.smithi096.stdout:4248 read 0xf13fb3 thru 0xf1d2bc (0x930a bytes) 2024-04-24T15:33:13.004 INFO:teuthology.orchestra.run.smithi096.stdout:4249 trunc from 0xbe8bab0 to 0x82c7c9b 2024-04-24T15:33:13.019 INFO:teuthology.orchestra.run.smithi096.stdout:4250 write 0x98d6e44 thru 0x98e4379 (0xd536 bytes) 2024-04-24T15:33:13.029 INFO:teuthology.orchestra.run.smithi096.stdout:4251 writesame 0xa8fb687 thru 0xa90829f data_size 0x1(0xcc19 bytes) 2024-04-24T15:33:13.031 INFO:teuthology.orchestra.run.smithi096.stdout:4254 read 0x7c9327f thru 0x7c986c7 (0x5449 bytes) 2024-04-24T15:33:13.031 INFO:teuthology.orchestra.run.smithi096.stdout:4255 read 0x5989a83 thru 0x598ddd1 (0x434f bytes) 2024-04-24T15:33:13.037 INFO:teuthology.orchestra.run.smithi096.stdout:4256 writesame 0xb052cd7 thru 0xb052fb3 data_size 0x1(0x2dd bytes) 2024-04-24T15:33:13.037 INFO:teuthology.orchestra.run.smithi096.stdout:4257 punch from 0x7463d41 to 0x7467250, (0x350f bytes) 2024-04-24T15:33:13.037 INFO:teuthology.orchestra.run.smithi096.stdout:4259 writesame 0x4611ee1 thru 0x461431f data_size 0x3(0x243f bytes) 2024-04-24T15:33:13.041 INFO:teuthology.orchestra.run.smithi096.stdout:4261 write 0x9f91d5c thru 0x9fa1377 (0xf61c bytes) 2024-04-24T15:33:13.050 INFO:teuthology.orchestra.run.smithi096.stdout:4262 writesame 0xc1c29b5 thru 0xc1cd998 data_size 0x2(0xafe4 bytes) 2024-04-24T15:33:13.050 INFO:teuthology.orchestra.run.smithi096.stdout:4263 punch from 0x69c1628 to 0x69cd09f, (0xba77 bytes) 2024-04-24T15:33:13.050 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:13.051 INFO:teuthology.orchestra.run.smithi096.stdout:4266 read 0x83866a2 thru 0x8388b66 (0x24c5 bytes) 2024-04-24T15:33:13.051 INFO:teuthology.orchestra.run.smithi096.stdout:4268 clone 23 order 25 su 524288 sc 13 2024-04-24T15:33:13.100 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:13.101+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:13.162 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:13.162 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:33:13.162 INFO:teuthology.orchestra.run.smithi151.stdout: "num_ops": 0 2024-04-24T15:33:13.163 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:13.172 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-24T15:33:13.201 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:13.202+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:13.230 INFO:teuthology.orchestra.run.smithi151.stderr:Exporting 4.12 info 4.12( v 75'275 (0'0,75'275] local-lis/les=27/28 n=23 ec=26/26 lis/c=27/27 les/c/f=28/28/0 sis=27) 2024-04-24T15:33:13.238 INFO:teuthology.orchestra.run.smithi151.stderr:Read #4:48599db1:::rbd_data.114ab4f6fc67.0000000000000004:12# 2024-04-24T15:33:13.240 INFO:teuthology.orchestra.run.smithi151.stderr:Read #4:48599db1:::rbd_data.114ab4f6fc67.0000000000000004:head# 2024-04-24T15:33:13.240 INFO:teuthology.orchestra.run.smithi151.stderr:Read #4:48a8a567:::rbd_data.114a8b543e91.0000000000000014:head# 2024-04-24T15:33:13.241 INFO:teuthology.orchestra.run.smithi151.stderr:Read #4:49009fc0:::rbd_data.114ad3a18a61.000000000000000b:head# 2024-04-24T15:33:13.243 INFO:teuthology.orchestra.run.smithi151.stderr:Read #4:4a25d4cb:::rbd_data.114a858e6641.0000000000000026:17# 2024-04-24T15:33:13.245 INFO:teuthology.orchestra.run.smithi151.stderr:Read #4:4a25d4cb:::rbd_data.114a858e6641.0000000000000026:head# 2024-04-24T15:33:13.245 INFO:teuthology.orchestra.run.smithi151.stderr:Read #4:4ab5cdb0:::rbd_data.114aaa0086f7.0000000000000002:head# 2024-04-24T15:33:13.253 INFO:teuthology.orchestra.run.smithi151.stderr:Read #4:4b4f242e:::rbd_data.114a1f60cb66.00000000000002b4:head# 2024-04-24T15:33:13.254 INFO:teuthology.orchestra.run.smithi151.stderr:Read #4:4b7ae65e:::rbd_data.114ab4f6fc67.0000000000000056:12# 2024-04-24T15:33:13.254 INFO:teuthology.orchestra.run.smithi151.stderr:Read #4:4b7ae65e:::rbd_data.114ab4f6fc67.0000000000000056:head# 2024-04-24T15:33:13.254 INFO:teuthology.orchestra.run.smithi151.stderr:Read #4:4c04ad06:::rbd_data.114a60efb398.0000000000000031:c# 2024-04-24T15:33:13.254 INFO:teuthology.orchestra.run.smithi151.stderr:Read #4:4c04ad06:::rbd_data.114a60efb398.0000000000000031:head# 2024-04-24T15:33:13.254 INFO:teuthology.orchestra.run.smithi151.stderr:Read #4:4c93b7f9:::rbd_data.114a1f60cb66.0000000000000037:head# 2024-04-24T15:33:13.255 INFO:teuthology.orchestra.run.smithi151.stderr:Read #4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7# 2024-04-24T15:33:13.269 INFO:teuthology.orchestra.run.smithi151.stderr:Read #4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:head# 2024-04-24T15:33:13.269 INFO:teuthology.orchestra.run.smithi151.stderr:Read #4:4ccc81f4:::rbd_data.114a46f1ce44.0000000000000009:14# 2024-04-24T15:33:13.278 INFO:teuthology.orchestra.run.smithi151.stderr:Read #4:4ccc81f4:::rbd_data.114a46f1ce44.0000000000000009:head# 2024-04-24T15:33:13.279 INFO:teuthology.orchestra.run.smithi151.stderr:Read #4:4dddff5c:::rbd_data.114a60efb398.0000000000000032:c# 2024-04-24T15:33:13.279 INFO:teuthology.orchestra.run.smithi151.stderr:Read #4:4dddff5c:::rbd_data.114a60efb398.0000000000000032:head# 2024-04-24T15:33:13.279 INFO:teuthology.orchestra.run.smithi151.stderr:Read #4:4e5034f1:::rbd_data.114a8e7208a5.0000000000000000:b# 2024-04-24T15:33:13.282 INFO:teuthology.orchestra.run.smithi151.stderr:Read #4:4e5034f1:::rbd_data.114a8e7208a5.0000000000000000:head# 2024-04-24T15:33:13.282 INFO:teuthology.orchestra.run.smithi151.stderr:Read #4:4e515771:::rbd_data.114ae15b9a20.0000000000000024:head# 2024-04-24T15:33:13.282 INFO:teuthology.orchestra.run.smithi151.stderr:Read #4:4ee02263:::rbd_data.114a1f60cb66.0000000000000055:head# 2024-04-24T15:33:13.283 INFO:teuthology.orchestra.run.smithi151.stderr:Export successful 2024-04-24T15:33:13.301 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:13.302 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:33:13.302 INFO:teuthology.orchestra.run.smithi151.stdout: "complaint_time": 30, 2024-04-24T15:33:13.302 INFO:teuthology.orchestra.run.smithi151.stdout: "num_blocked_ops": 0 2024-04-24T15:33:13.302 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:13.302 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:33:13.302+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:33:13.312 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-24T15:33:13.402 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:13.403+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:13.478 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:13.478 INFO:teuthology.orchestra.run.smithi151.stdout: "size": 20, 2024-04-24T15:33:13.478 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 600, 2024-04-24T15:33:13.478 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [ 2024-04-24T15:33:13.478 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.478 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23613 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 14688256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:13.479 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.792522+0000", 2024-04-24T15:33:13.479 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 38.685561487000001, 2024-04-24T15:33:13.479 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040507356000000001, 2024-04-24T15:33:13.479 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:13.479 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:13.479 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:13.479 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:13.479 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:13.479 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23613 2024-04-24T15:33:13.479 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.479 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:13.479 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.479 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:13.479 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792522+0000", 2024-04-24T15:33:13.480 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:13.480 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.480 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.480 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:13.480 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792522+0000", 2024-04-24T15:33:13.480 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:13.480 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.480 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.480 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:13.480 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792522+0000", 2024-04-24T15:33:13.480 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:13.480 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.480 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.480 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:13.480 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792523+0000", 2024-04-24T15:33:13.481 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.576e-06 2024-04-24T15:33:13.481 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.481 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.481 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:13.481 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792526+0000", 2024-04-24T15:33:13.481 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7439999999999999e-06 2024-04-24T15:33:13.481 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.481 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.481 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:13.481 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792528+0000", 2024-04-24T15:33:13.481 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.0600000000000002e-06 2024-04-24T15:33:13.481 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.481 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.481 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:13.481 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.832864+0000", 2024-04-24T15:33:13.481 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040335742000000001 2024-04-24T15:33:13.482 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.482 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.482 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:13.482 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.832953+0000", 2024-04-24T15:33:13.482 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.9281000000000001e-05 2024-04-24T15:33:13.482 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.482 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.482 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:13.482 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.833029+0000", 2024-04-24T15:33:13.482 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.6259000000000003e-05 2024-04-24T15:33:13.482 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:13.482 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:13.482 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:13.483 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.483 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.483 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23614 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 14729216~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:13.483 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.792530+0000", 2024-04-24T15:33:13.483 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 38.685553222999999, 2024-04-24T15:33:13.483 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040633957999999998, 2024-04-24T15:33:13.483 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:13.483 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:13.483 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:13.483 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:13.483 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:13.483 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23614 2024-04-24T15:33:13.483 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.483 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:13.483 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.484 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:13.484 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792530+0000", 2024-04-24T15:33:13.484 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:13.484 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.484 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.484 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:13.484 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792530+0000", 2024-04-24T15:33:13.484 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:13.484 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.484 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.484 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:13.484 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792530+0000", 2024-04-24T15:33:13.484 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:13.484 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.484 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.484 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:13.485 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792531+0000", 2024-04-24T15:33:13.485 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.46e-06 2024-04-24T15:33:13.485 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.485 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.485 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:13.485 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792534+0000", 2024-04-24T15:33:13.485 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8959999999999999e-06 2024-04-24T15:33:13.485 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.485 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.485 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:13.485 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792536+0000", 2024-04-24T15:33:13.485 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.841e-06 2024-04-24T15:33:13.485 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.485 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.485 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:13.486 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.833060+0000", 2024-04-24T15:33:13.486 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040524125000000001 2024-04-24T15:33:13.486 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.486 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.486 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:13.486 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.833108+0000", 2024-04-24T15:33:13.486 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.7547999999999999e-05 2024-04-24T15:33:13.486 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.486 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.486 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:13.486 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.833164+0000", 2024-04-24T15:33:13.486 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6297e-05 2024-04-24T15:33:13.486 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:13.486 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:13.487 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:13.487 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.487 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.487 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23623 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15097856~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:13.487 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.793887+0000", 2024-04-24T15:33:13.487 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 38.684196262, 2024-04-24T15:33:13.487 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040432784999999999, 2024-04-24T15:33:13.487 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:13.487 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:13.487 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:13.487 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:13.487 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:13.487 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23623 2024-04-24T15:33:13.487 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.487 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:13.488 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.488 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:13.488 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793887+0000", 2024-04-24T15:33:13.488 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:13.488 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.488 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.488 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:13.488 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793887+0000", 2024-04-24T15:33:13.488 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:13.488 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.488 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.488 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:13.488 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793887+0000", 2024-04-24T15:33:13.488 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:13.488 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.488 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.489 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:13.489 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793888+0000", 2024-04-24T15:33:13.489 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3659999999999999e-06 2024-04-24T15:33:13.489 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.489 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.489 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:13.489 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793891+0000", 2024-04-24T15:33:13.489 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8389999999999998e-06 2024-04-24T15:33:13.489 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.489 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.489 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:13.489 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793893+0000", 2024-04-24T15:33:13.489 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.9549999999999999e-06 2024-04-24T15:33:13.489 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.490 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.490 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:13.490 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834239+0000", 2024-04-24T15:33:13.490 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040345487999999999 2024-04-24T15:33:13.490 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.490 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.490 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:13.490 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834280+0000", 2024-04-24T15:33:13.490 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.1878000000000001e-05 2024-04-24T15:33:13.490 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.490 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.490 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:13.490 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834320+0000", 2024-04-24T15:33:13.491 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9480000000000001e-05 2024-04-24T15:33:13.491 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:13.491 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:13.491 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:13.491 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.491 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.491 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23624 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15138816~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:13.491 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.794053+0000", 2024-04-24T15:33:13.491 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 38.684030313999997, 2024-04-24T15:33:13.491 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040370801999999997, 2024-04-24T15:33:13.491 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:13.491 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:13.491 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:13.491 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:13.491 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:13.492 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23624 2024-04-24T15:33:13.492 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.492 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:13.492 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.492 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:13.492 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794053+0000", 2024-04-24T15:33:13.492 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:13.492 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.492 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.492 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:13.492 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794053+0000", 2024-04-24T15:33:13.492 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:13.492 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.492 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.492 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:13.492 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793895+0000", 2024-04-24T15:33:13.493 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998422 2024-04-24T15:33:13.493 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.493 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.493 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:13.493 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794057+0000", 2024-04-24T15:33:13.493 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00016149400000000001 2024-04-24T15:33:13.493 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.493 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.493 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:13.493 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794065+0000", 2024-04-24T15:33:13.493 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.5259999999999993e-06 2024-04-24T15:33:13.493 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.493 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.493 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:13.493 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794071+0000", 2024-04-24T15:33:13.493 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6149999999999996e-06 2024-04-24T15:33:13.494 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.494 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.494 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:13.494 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834341+0000", 2024-04-24T15:33:13.494 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040270366000000002 2024-04-24T15:33:13.494 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.494 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.494 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:13.494 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834381+0000", 2024-04-24T15:33:13.494 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0102999999999999e-05 2024-04-24T15:33:13.494 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.494 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.495 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:13.495 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834424+0000", 2024-04-24T15:33:13.495 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2713000000000003e-05 2024-04-24T15:33:13.495 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:13.495 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:13.495 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:13.495 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.495 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.495 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23625 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15179776~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:13.495 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.794160+0000", 2024-04-24T15:33:13.495 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 38.683923432, 2024-04-24T15:33:13.495 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040413004000000002, 2024-04-24T15:33:13.495 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:13.495 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:13.495 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:13.496 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:13.496 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:13.496 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23625 2024-04-24T15:33:13.496 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.496 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:13.496 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.496 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:13.496 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794160+0000", 2024-04-24T15:33:13.496 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:13.496 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.496 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.496 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:13.496 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794160+0000", 2024-04-24T15:33:13.496 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:13.497 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.497 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.497 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:13.497 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794086+0000", 2024-04-24T15:33:13.497 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999256 2024-04-24T15:33:13.497 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.497 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.497 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:13.497 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794163+0000", 2024-04-24T15:33:13.497 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7896000000000001e-05 2024-04-24T15:33:13.497 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.497 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.497 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:13.497 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794171+0000", 2024-04-24T15:33:13.498 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.9500000000000001e-06 2024-04-24T15:33:13.498 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.498 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.498 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:13.498 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794179+0000", 2024-04-24T15:33:13.498 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7970000000000001e-06 2024-04-24T15:33:13.498 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.498 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.498 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:13.498 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834451+0000", 2024-04-24T15:33:13.498 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040271492999999998 2024-04-24T15:33:13.498 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.498 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.499 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:13.499 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834519+0000", 2024-04-24T15:33:13.499 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.7984000000000005e-05 2024-04-24T15:33:13.499 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.499 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.499 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:13.499 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834573+0000", 2024-04-24T15:33:13.499 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.4339999999999998e-05 2024-04-24T15:33:13.499 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:13.499 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:13.499 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:13.499 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.499 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.499 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23629 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15343616~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:13.500 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.794748+0000", 2024-04-24T15:33:13.500 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 38.683335630000002, 2024-04-24T15:33:13.500 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040419718, 2024-04-24T15:33:13.500 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:13.500 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:13.500 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:13.500 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:13.500 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:13.500 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23629 2024-04-24T15:33:13.500 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.500 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:13.500 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.500 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:13.500 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794748+0000", 2024-04-24T15:33:13.500 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:13.501 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.501 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.501 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:13.501 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794748+0000", 2024-04-24T15:33:13.501 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:13.501 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.501 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.501 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:13.501 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794673+0000", 2024-04-24T15:33:13.502 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999251 2024-04-24T15:33:13.502 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.502 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.502 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:13.502 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794751+0000", 2024-04-24T15:33:13.502 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.8480999999999996e-05 2024-04-24T15:33:13.502 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.502 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.502 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:13.502 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794760+0000", 2024-04-24T15:33:13.502 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3429999999999996e-06 2024-04-24T15:33:13.502 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.502 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.503 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:13.503 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794764+0000", 2024-04-24T15:33:13.503 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5600000000000004e-06 2024-04-24T15:33:13.503 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.503 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.503 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:13.503 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835040+0000", 2024-04-24T15:33:13.503 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040275188000000003 2024-04-24T15:33:13.503 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.503 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.503 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:13.503 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835105+0000", 2024-04-24T15:33:13.503 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.5992000000000001e-05 2024-04-24T15:33:13.503 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.504 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.504 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:13.504 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835167+0000", 2024-04-24T15:33:13.504 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.1981000000000002e-05 2024-04-24T15:33:13.504 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:13.504 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:13.504 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:13.504 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.504 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.505 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23630 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15384576~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:13.505 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.794839+0000", 2024-04-24T15:33:13.505 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 38.683244561000002, 2024-04-24T15:33:13.505 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040523894999999997, 2024-04-24T15:33:13.505 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:13.505 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:13.505 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:13.505 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:13.505 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:13.505 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23630 2024-04-24T15:33:13.506 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.506 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:13.506 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.506 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:13.506 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794839+0000", 2024-04-24T15:33:13.506 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:13.506 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.506 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.506 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:13.506 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794839+0000", 2024-04-24T15:33:13.506 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:13.506 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.507 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.507 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:13.507 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794768+0000", 2024-04-24T15:33:13.507 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999294 2024-04-24T15:33:13.507 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.507 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.507 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:13.507 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794841+0000", 2024-04-24T15:33:13.507 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2793000000000006e-05 2024-04-24T15:33:13.507 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.507 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.507 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:13.507 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794846+0000", 2024-04-24T15:33:13.507 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.7230000000000004e-06 2024-04-24T15:33:13.508 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.508 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.508 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:13.508 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794848+0000", 2024-04-24T15:33:13.508 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7659999999999999e-06 2024-04-24T15:33:13.508 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.508 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.508 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:13.508 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835223+0000", 2024-04-24T15:33:13.508 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040374275000000001 2024-04-24T15:33:13.508 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.508 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.508 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:13.508 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835295+0000", 2024-04-24T15:33:13.509 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2633999999999996e-05 2024-04-24T15:33:13.509 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.509 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.509 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:13.509 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835363+0000", 2024-04-24T15:33:13.509 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.7355999999999997e-05 2024-04-24T15:33:13.509 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:13.509 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:13.509 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:13.509 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.509 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.509 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23631 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15425536~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:13.509 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.794956+0000", 2024-04-24T15:33:13.509 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 38.683127720000002, 2024-04-24T15:33:13.509 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040562383, 2024-04-24T15:33:13.510 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:13.510 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:13.510 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:13.510 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:13.510 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:13.510 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23631 2024-04-24T15:33:13.510 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.510 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:13.510 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.510 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:13.510 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794956+0000", 2024-04-24T15:33:13.510 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:13.510 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.510 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.510 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:13.511 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794956+0000", 2024-04-24T15:33:13.511 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:13.511 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.511 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.511 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:13.511 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794859+0000", 2024-04-24T15:33:13.511 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999032 2024-04-24T15:33:13.511 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.511 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.511 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:13.511 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794959+0000", 2024-04-24T15:33:13.511 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000100207 2024-04-24T15:33:13.511 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.511 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.511 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:13.512 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794967+0000", 2024-04-24T15:33:13.512 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7640000000000005e-06 2024-04-24T15:33:13.512 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.512 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.512 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:13.512 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794971+0000", 2024-04-24T15:33:13.512 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.335e-06 2024-04-24T15:33:13.512 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.512 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.512 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:13.512 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835388+0000", 2024-04-24T15:33:13.512 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040417340000000003 2024-04-24T15:33:13.512 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.512 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.512 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:13.512 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835459+0000", 2024-04-24T15:33:13.512 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.0118999999999997e-05 2024-04-24T15:33:13.513 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.513 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.513 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:13.513 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835518+0000", 2024-04-24T15:33:13.513 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.9438999999999998e-05 2024-04-24T15:33:13.513 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:13.513 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:13.513 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:13.513 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.513 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.513 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23632 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15466496~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:13.513 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795060+0000", 2024-04-24T15:33:13.513 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 38.683023009999999, 2024-04-24T15:33:13.513 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040667993, 2024-04-24T15:33:13.513 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:13.513 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:13.514 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:13.514 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:13.514 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:13.514 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23632 2024-04-24T15:33:13.514 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.514 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:13.514 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.514 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:13.514 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795060+0000", 2024-04-24T15:33:13.514 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:13.514 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.514 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.514 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:13.515 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795060+0000", 2024-04-24T15:33:13.515 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:13.515 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.515 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.515 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:13.515 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794976+0000", 2024-04-24T15:33:13.515 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999156 2024-04-24T15:33:13.515 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.515 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.515 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:13.515 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795064+0000", 2024-04-24T15:33:13.515 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.7709999999999999e-05 2024-04-24T15:33:13.515 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.515 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.515 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:13.515 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795072+0000", 2024-04-24T15:33:13.516 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3769999999999996e-06 2024-04-24T15:33:13.516 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.516 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.516 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:13.516 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795078+0000", 2024-04-24T15:33:13.516 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.5079999999999996e-06 2024-04-24T15:33:13.516 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.517 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.517 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:13.517 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835541+0000", 2024-04-24T15:33:13.517 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040463440000000003 2024-04-24T15:33:13.517 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.517 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.517 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:13.517 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835657+0000", 2024-04-24T15:33:13.517 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00011572499999999999 2024-04-24T15:33:13.518 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.518 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.518 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:13.518 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835728+0000", 2024-04-24T15:33:13.518 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1402000000000005e-05 2024-04-24T15:33:13.518 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:13.518 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:13.518 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:13.518 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.518 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.518 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23633 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15507456~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:13.519 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795134+0000", 2024-04-24T15:33:13.519 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 38.682949354999998, 2024-04-24T15:33:13.519 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040748591000000001, 2024-04-24T15:33:13.519 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:13.519 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:13.519 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:13.519 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:13.519 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:13.519 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23633 2024-04-24T15:33:13.519 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.519 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:13.519 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.519 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:13.519 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795134+0000", 2024-04-24T15:33:13.519 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:13.519 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.520 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.520 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:13.520 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795134+0000", 2024-04-24T15:33:13.520 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:13.520 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.520 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.520 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:13.520 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795083+0000", 2024-04-24T15:33:13.520 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999949 2024-04-24T15:33:13.520 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.520 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.520 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:13.520 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795138+0000", 2024-04-24T15:33:13.520 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.4472000000000003e-05 2024-04-24T15:33:13.521 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.521 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.521 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:13.521 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795145+0000", 2024-04-24T15:33:13.521 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2779999999999996e-06 2024-04-24T15:33:13.521 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.521 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.521 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:13.521 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795149+0000", 2024-04-24T15:33:13.521 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.568e-06 2024-04-24T15:33:13.521 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.521 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.521 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:13.521 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835757+0000", 2024-04-24T15:33:13.522 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040608894999999999 2024-04-24T15:33:13.522 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.522 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.522 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:13.522 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835829+0000", 2024-04-24T15:33:13.522 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1879999999999996e-05 2024-04-24T15:33:13.522 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.522 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.522 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:13.522 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835883+0000", 2024-04-24T15:33:13.522 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.3346000000000001e-05 2024-04-24T15:33:13.522 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:13.522 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:13.523 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:13.523 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.523 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.523 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23634 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15548416~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:13.523 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795271+0000", 2024-04-24T15:33:13.523 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 38.682812321, 2024-04-24T15:33:13.523 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040755845999999998, 2024-04-24T15:33:13.523 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:13.523 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:13.523 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:13.523 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:13.523 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:13.523 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23634 2024-04-24T15:33:13.523 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.524 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:13.524 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.524 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:13.524 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795271+0000", 2024-04-24T15:33:13.524 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:13.524 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.524 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.524 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:13.524 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795271+0000", 2024-04-24T15:33:13.524 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:13.524 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.524 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.524 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:13.524 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795152+0000", 2024-04-24T15:33:13.525 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998813 2024-04-24T15:33:13.525 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.525 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.525 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:13.525 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795276+0000", 2024-04-24T15:33:13.525 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00012401799999999999 2024-04-24T15:33:13.525 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.525 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.525 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:13.525 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795281+0000", 2024-04-24T15:33:13.525 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8860000000000003e-06 2024-04-24T15:33:13.525 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.525 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.526 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:13.526 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795284+0000", 2024-04-24T15:33:13.526 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.0450000000000001e-06 2024-04-24T15:33:13.526 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.526 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.526 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:13.526 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835905+0000", 2024-04-24T15:33:13.526 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040621035999999999 2024-04-24T15:33:13.526 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.526 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.527 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:13.527 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835971+0000", 2024-04-24T15:33:13.527 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.5314000000000005e-05 2024-04-24T15:33:13.527 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.527 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.527 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:13.527 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836027+0000", 2024-04-24T15:33:13.527 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6305999999999999e-05 2024-04-24T15:33:13.527 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:13.527 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:13.527 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:13.528 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.528 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.528 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23635 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15589376~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:13.528 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795382+0000", 2024-04-24T15:33:13.528 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 38.682701547999997, 2024-04-24T15:33:13.528 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040805363999999997, 2024-04-24T15:33:13.528 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:13.528 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:13.528 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:13.528 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:13.528 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:13.528 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23635 2024-04-24T15:33:13.528 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.528 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:13.529 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.529 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:13.529 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795382+0000", 2024-04-24T15:33:13.529 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:13.529 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.529 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.529 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:13.529 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795382+0000", 2024-04-24T15:33:13.529 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:13.529 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.529 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.529 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:13.529 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795288+0000", 2024-04-24T15:33:13.529 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999056 2024-04-24T15:33:13.530 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.530 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.530 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:13.530 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795385+0000", 2024-04-24T15:33:13.530 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.7288999999999996e-05 2024-04-24T15:33:13.530 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.530 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.530 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:13.530 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795392+0000", 2024-04-24T15:33:13.530 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.452e-06 2024-04-24T15:33:13.530 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.530 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.530 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:13.530 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795397+0000", 2024-04-24T15:33:13.530 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2860000000000001e-06 2024-04-24T15:33:13.531 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.531 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.531 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:13.531 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836054+0000", 2024-04-24T15:33:13.531 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040657581999999998 2024-04-24T15:33:13.531 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.531 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.531 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:13.531 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836124+0000", 2024-04-24T15:33:13.531 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.0020999999999999e-05 2024-04-24T15:33:13.531 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.531 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.531 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:13.531 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836187+0000", 2024-04-24T15:33:13.531 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.3009999999999995e-05 2024-04-24T15:33:13.531 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:13.532 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:13.532 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:13.532 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.532 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.532 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23636 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15630336~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:13.532 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795610+0000", 2024-04-24T15:33:13.532 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 38.682473190000003, 2024-04-24T15:33:13.532 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040730942999999999, 2024-04-24T15:33:13.532 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:13.532 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:13.532 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:13.532 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:13.532 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:13.532 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23636 2024-04-24T15:33:13.533 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.533 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:13.533 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.533 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:13.533 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795610+0000", 2024-04-24T15:33:13.533 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:13.533 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.533 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.533 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:13.533 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795610+0000", 2024-04-24T15:33:13.533 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:13.533 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.533 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.533 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:13.533 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795401+0000", 2024-04-24T15:33:13.533 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9997911 2024-04-24T15:33:13.534 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.534 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.534 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:13.534 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795614+0000", 2024-04-24T15:33:13.534 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00021228600000000001 2024-04-24T15:33:13.534 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.534 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.534 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:13.534 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795622+0000", 2024-04-24T15:33:13.534 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3920000000000003e-06 2024-04-24T15:33:13.534 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.534 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.534 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:13.535 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795627+0000", 2024-04-24T15:33:13.535 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8199999999999996e-06 2024-04-24T15:33:13.535 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.535 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.535 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:13.535 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836211+0000", 2024-04-24T15:33:13.535 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040583803000000002 2024-04-24T15:33:13.535 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.535 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.535 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:13.535 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836284+0000", 2024-04-24T15:33:13.535 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.3617999999999999e-05 2024-04-24T15:33:13.535 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.536 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.536 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:13.536 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836341+0000", 2024-04-24T15:33:13.536 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6959999999999997e-05 2024-04-24T15:33:13.536 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:13.536 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:13.536 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:13.536 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.536 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.536 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23637 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15671296~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:13.536 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795699+0000", 2024-04-24T15:33:13.536 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 38.682384022999997, 2024-04-24T15:33:13.536 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040775556999999997, 2024-04-24T15:33:13.536 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:13.536 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:13.536 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:13.537 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:13.537 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:13.537 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23637 2024-04-24T15:33:13.537 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.537 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:13.537 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.537 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:13.537 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795699+0000", 2024-04-24T15:33:13.537 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:13.537 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.537 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.537 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:13.537 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795699+0000", 2024-04-24T15:33:13.538 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:13.538 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.538 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.538 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:13.538 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795632+0000", 2024-04-24T15:33:13.538 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999323 2024-04-24T15:33:13.538 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.538 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.538 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:13.538 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795704+0000", 2024-04-24T15:33:13.538 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2428000000000005e-05 2024-04-24T15:33:13.538 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.538 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.539 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:13.539 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795715+0000", 2024-04-24T15:33:13.539 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.1532000000000001e-05 2024-04-24T15:33:13.539 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.539 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.539 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:13.539 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795721+0000", 2024-04-24T15:33:13.539 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.8200000000000002e-06 2024-04-24T15:33:13.539 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.539 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.539 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:13.539 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836364+0000", 2024-04-24T15:33:13.539 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040642997 2024-04-24T15:33:13.539 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.539 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.540 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:13.540 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836435+0000", 2024-04-24T15:33:13.540 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.0821000000000005e-05 2024-04-24T15:33:13.540 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.540 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.540 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:13.540 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836475+0000", 2024-04-24T15:33:13.540 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9796999999999999e-05 2024-04-24T15:33:13.540 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:13.540 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:13.540 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:13.540 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.540 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.540 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23638 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15712256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:13.540 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795932+0000", 2024-04-24T15:33:13.540 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 38.682150993, 2024-04-24T15:33:13.541 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040668682999999997, 2024-04-24T15:33:13.541 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:13.541 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:13.541 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:13.541 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:13.541 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:13.541 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23638 2024-04-24T15:33:13.541 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.541 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:13.541 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.541 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:13.541 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795932+0000", 2024-04-24T15:33:13.541 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:13.541 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.541 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.541 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:13.542 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795932+0000", 2024-04-24T15:33:13.542 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:13.542 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.542 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.542 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:13.542 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795728+0000", 2024-04-24T15:33:13.542 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9997959 2024-04-24T15:33:13.542 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.542 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.542 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:13.542 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795936+0000", 2024-04-24T15:33:13.542 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000208128 2024-04-24T15:33:13.542 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.542 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.542 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:13.543 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795945+0000", 2024-04-24T15:33:13.543 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.7110000000000001e-06 2024-04-24T15:33:13.543 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.543 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.543 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:13.543 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795957+0000", 2024-04-24T15:33:13.543 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.1800999999999999e-05 2024-04-24T15:33:13.543 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.543 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.543 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:13.543 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836505+0000", 2024-04-24T15:33:13.543 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040547622999999998 2024-04-24T15:33:13.543 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.543 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.543 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:13.544 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836561+0000", 2024-04-24T15:33:13.544 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6598999999999998e-05 2024-04-24T15:33:13.544 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.544 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.544 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:13.544 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836601+0000", 2024-04-24T15:33:13.544 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9907000000000003e-05 2024-04-24T15:33:13.544 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:13.544 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:13.544 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:13.544 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.544 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.544 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23639 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15753216~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:13.544 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.796071+0000", 2024-04-24T15:33:13.544 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 38.682012731999997, 2024-04-24T15:33:13.544 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040648199000000003, 2024-04-24T15:33:13.545 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:13.545 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:13.545 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:13.545 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:13.545 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:13.545 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23639 2024-04-24T15:33:13.545 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.545 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:13.545 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.545 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:13.545 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796071+0000", 2024-04-24T15:33:13.545 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:13.545 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.545 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.545 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:13.545 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796071+0000", 2024-04-24T15:33:13.546 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:13.546 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.546 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.546 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:13.546 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795967+0000", 2024-04-24T15:33:13.546 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998965 2024-04-24T15:33:13.546 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.546 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.546 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:13.546 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796074+0000", 2024-04-24T15:33:13.546 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000107013 2024-04-24T15:33:13.546 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.546 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.547 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:13.547 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796081+0000", 2024-04-24T15:33:13.547 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2590000000000003e-06 2024-04-24T15:33:13.547 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.547 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.547 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:13.547 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796085+0000", 2024-04-24T15:33:13.547 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5269999999999999e-06 2024-04-24T15:33:13.547 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.547 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.547 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:13.547 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836624+0000", 2024-04-24T15:33:13.547 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040539324000000002 2024-04-24T15:33:13.547 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.548 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.548 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:13.548 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836667+0000", 2024-04-24T15:33:13.548 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2562999999999999e-05 2024-04-24T15:33:13.548 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.548 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.548 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:13.548 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836719+0000", 2024-04-24T15:33:13.548 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.2015999999999998e-05 2024-04-24T15:33:13.548 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:13.548 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:13.548 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:13.548 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.548 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.549 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23640 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15794176~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:13.549 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.796492+0000", 2024-04-24T15:33:13.549 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 38.681590974000002, 2024-04-24T15:33:13.549 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040420000999999997, 2024-04-24T15:33:13.549 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:13.549 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:13.549 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:13.549 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:13.549 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:13.549 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23640 2024-04-24T15:33:13.549 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.549 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:13.549 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.549 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:13.550 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796492+0000", 2024-04-24T15:33:13.550 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:13.550 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.550 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.550 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:13.550 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796492+0000", 2024-04-24T15:33:13.550 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:13.550 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.550 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.550 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:13.550 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796088+0000", 2024-04-24T15:33:13.550 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9995961 2024-04-24T15:33:13.550 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.550 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.551 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:13.551 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796496+0000", 2024-04-24T15:33:13.551 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00040785700000000002 2024-04-24T15:33:13.551 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.551 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.551 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:13.551 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796503+0000", 2024-04-24T15:33:13.551 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1049999999999997e-06 2024-04-24T15:33:13.551 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.551 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.551 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:13.551 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796508+0000", 2024-04-24T15:33:13.551 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.155e-06 2024-04-24T15:33:13.551 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.551 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.551 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:13.552 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836756+0000", 2024-04-24T15:33:13.552 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040248850000000003 2024-04-24T15:33:13.552 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.552 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.552 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:13.552 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836841+0000", 2024-04-24T15:33:13.552 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.4735999999999996e-05 2024-04-24T15:33:13.552 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.552 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.552 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:13.552 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836912+0000", 2024-04-24T15:33:13.552 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1229999999999994e-05 2024-04-24T15:33:13.552 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:13.552 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:13.553 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:13.553 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.553 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.553 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23641 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15835136~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:13.553 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.796658+0000", 2024-04-24T15:33:13.553 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 38.681424898000003, 2024-04-24T15:33:13.553 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040420706000000001, 2024-04-24T15:33:13.553 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:13.553 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:13.553 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:13.553 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:13.553 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:13.553 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23641 2024-04-24T15:33:13.553 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.554 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:13.554 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.554 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:13.554 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796658+0000", 2024-04-24T15:33:13.554 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:13.554 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.554 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.554 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:13.554 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796658+0000", 2024-04-24T15:33:13.554 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:13.554 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.554 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.554 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:13.554 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796512+0000", 2024-04-24T15:33:13.554 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998531 2024-04-24T15:33:13.555 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.555 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.555 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:13.555 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796669+0000", 2024-04-24T15:33:13.555 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000157577 2024-04-24T15:33:13.555 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.555 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.555 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:13.555 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796677+0000", 2024-04-24T15:33:13.555 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7800000000000001e-06 2024-04-24T15:33:13.555 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.555 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.555 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:13.555 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796681+0000", 2024-04-24T15:33:13.555 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.721e-06 2024-04-24T15:33:13.556 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.556 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.556 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:13.556 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836941+0000", 2024-04-24T15:33:13.556 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040259907999999997 2024-04-24T15:33:13.556 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.556 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.556 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:13.556 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837017+0000", 2024-04-24T15:33:13.556 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.6071999999999999e-05 2024-04-24T15:33:13.556 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.556 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.556 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:13.556 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837079+0000", 2024-04-24T15:33:13.556 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.2358000000000002e-05 2024-04-24T15:33:13.557 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:13.557 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:13.557 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:13.557 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.557 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.557 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23645 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15998976~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:13.557 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.797360+0000", 2024-04-24T15:33:13.557 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 38.680723399999998, 2024-04-24T15:33:13.557 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040387645999999999, 2024-04-24T15:33:13.557 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:13.557 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:13.557 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:13.557 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:13.557 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:13.557 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23645 2024-04-24T15:33:13.558 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.558 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:13.558 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.558 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:13.558 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797360+0000", 2024-04-24T15:33:13.558 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:13.558 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.558 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.558 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:13.558 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797360+0000", 2024-04-24T15:33:13.558 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:13.558 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.558 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.558 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:13.559 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797288+0000", 2024-04-24T15:33:13.559 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999928 2024-04-24T15:33:13.559 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.559 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.559 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:13.559 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797364+0000", 2024-04-24T15:33:13.559 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.5347999999999998e-05 2024-04-24T15:33:13.559 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.559 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.559 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:13.559 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797372+0000", 2024-04-24T15:33:13.559 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.2409999999999995e-06 2024-04-24T15:33:13.559 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.559 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.560 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:13.560 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797376+0000", 2024-04-24T15:33:13.560 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.515e-06 2024-04-24T15:33:13.560 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.560 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.560 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:13.560 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837586+0000", 2024-04-24T15:33:13.560 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040210047999999998 2024-04-24T15:33:13.560 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.560 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.560 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:13.560 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837672+0000", 2024-04-24T15:33:13.560 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.5997999999999999e-05 2024-04-24T15:33:13.561 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.561 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.561 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:13.561 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837748+0000", 2024-04-24T15:33:13.561 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.5328999999999997e-05 2024-04-24T15:33:13.561 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:13.561 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:13.561 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:13.561 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.561 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.561 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23646 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 16039936~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:13.561 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.797498+0000", 2024-04-24T15:33:13.561 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 38.680585299000001, 2024-04-24T15:33:13.561 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040433139, 2024-04-24T15:33:13.561 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:13.562 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:13.562 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:13.562 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:13.562 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:13.562 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23646 2024-04-24T15:33:13.562 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.562 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:13.562 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.562 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:13.562 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797498+0000", 2024-04-24T15:33:13.562 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:13.562 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.563 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.563 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:13.563 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797498+0000", 2024-04-24T15:33:13.563 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:13.563 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.563 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.563 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:13.563 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797381+0000", 2024-04-24T15:33:13.563 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998832 2024-04-24T15:33:13.563 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.563 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.563 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:13.563 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797501+0000", 2024-04-24T15:33:13.563 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000119759 2024-04-24T15:33:13.564 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.564 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.564 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:13.564 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797510+0000", 2024-04-24T15:33:13.564 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.4770000000000003e-06 2024-04-24T15:33:13.564 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.564 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.564 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:13.564 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797515+0000", 2024-04-24T15:33:13.564 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8330000000000001e-06 2024-04-24T15:33:13.564 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.564 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.564 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:13.564 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837777+0000", 2024-04-24T15:33:13.564 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040262502999999998 2024-04-24T15:33:13.564 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.565 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.565 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:13.565 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837860+0000", 2024-04-24T15:33:13.565 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3002000000000003e-05 2024-04-24T15:33:13.565 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:13.565 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:13.565 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:13.565 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837931+0000", 2024-04-24T15:33:13.565 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1222000000000004e-05 2024-04-24T15:33:13.565 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:13.565 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:13.565 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:13.565 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:13.565 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:13.565 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:13.567 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-24T15:33:13.567 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:13.568+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:13.668 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:33:13.668+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:33:13.690 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:13.690 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:13.700 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-24T15:33:13.784 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:13.785+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:13.813 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-6 --journal-path /var/lib/ceph/osd/ceph-6/journal --force --op remove --pgid 4.12 2024-04-24T15:33:13.831 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-24T15:33:13.871 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:13.872 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:13.875 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:33:13.876+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:33:13.883 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-24T15:33:13.885 INFO:teuthology.orchestra.run.smithi096.stdout:truncating image image_client.0-clone22 from 0xc1cd999 (overlap 0x16ae625) to 0x12251b7 2024-04-24T15:33:13.967 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #21, image image_client.0-clone21 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent22 2024-04-24T15:33:13.970 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:13.971+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:14.065 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:14.065 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:14.071 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:33:14.073+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:33:14.076 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-24T15:33:14.173 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:14.173+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:14.210 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:14.210 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:33:14.210 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-04-24T15:33:14.210 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:14.220 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-24T15:33:14.237 INFO:teuthology.orchestra.run.smithi096.stdout:4270 read 0x6916cf1 thru 0x6921105 (0xa415 bytes) 2024-04-24T15:33:14.238 INFO:teuthology.orchestra.run.smithi096.stdout:4272 read 0xc92c8 thru 0xc9469 (0x1a2 bytes) 2024-04-24T15:33:14.239 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:14.239 INFO:teuthology.orchestra.run.smithi096.stdout:4274 read 0x5164cb7 thru 0x516a177 (0x54c1 bytes) 2024-04-24T15:33:14.240 INFO:teuthology.orchestra.run.smithi096.stdout:4276 write 0x3001d97 thru 0x300d955 (0xbbbf bytes) 2024-04-24T15:33:14.249 INFO:teuthology.orchestra.run.smithi096.stdout:4280 write 0x84b5075 thru 0x84bbe79 (0x6e05 bytes) 2024-04-24T15:33:14.250 INFO:teuthology.orchestra.run.smithi096.stdout:4281 writesame 0xa5cf6b thru 0xa635a9 data_size 0x19(0x663f bytes) 2024-04-24T15:33:14.259 INFO:teuthology.orchestra.run.smithi096.stdout:4282 read 0x2a53fa thru 0x2ad9d2 (0x85d9 bytes) 2024-04-24T15:33:14.259 INFO:teuthology.orchestra.run.smithi096.stdout:4284 punch from 0x20412ed to 0x204aabb, (0x97ce bytes) 2024-04-24T15:33:14.259 INFO:teuthology.orchestra.run.smithi096.stdout:4285 writesame 0x62160aa thru 0x6223a63 data_size 0x3a(0xd9ba bytes) 2024-04-24T15:33:14.269 INFO:teuthology.orchestra.run.smithi096.stdout:4286 write 0x9ce47de thru 0x9cf1582 (0xcda5 bytes) 2024-04-24T15:33:14.270 INFO:teuthology.orchestra.run.smithi096.stdout:4287 write 0x440656c thru 0x440a1b3 (0x3c48 bytes) 2024-04-24T15:33:14.273 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:14.275+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:14.278 INFO:teuthology.orchestra.run.smithi096.stdout:4289 writesame 0x4a1bec3 thru 0x4a26def data_size 0x1(0xaf2d bytes) 2024-04-24T15:33:14.292 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:14.292 INFO:teuthology.orchestra.run.smithi096.stdout:4291 write 0x6cd6efe thru 0x6ce0193 (0x9296 bytes) 2024-04-24T15:33:14.297 INFO:teuthology.orchestra.run.smithi096.stdout:4292 punch from 0x1ce02a0 to 0x1ceb3e3, (0xb143 bytes) 2024-04-24T15:33:14.298 INFO:teuthology.orchestra.run.smithi096.stdout:4293 read 0xa5213b3 thru 0xa530b5f (0xf7ad bytes) 2024-04-24T15:33:14.298 INFO:teuthology.orchestra.run.smithi096.stdout:4294 writesame 0x25f081 thru 0x26749e data_size 0x9(0x841e bytes) 2024-04-24T15:33:14.304 INFO:teuthology.orchestra.run.smithi096.stdout:4295 writesame 0x221d00f thru 0x2220ce5 data_size 0x23(0x3cd7 bytes) 2024-04-24T15:33:14.323 INFO:teuthology.orchestra.run.smithi096.stdout:4296 trunc from 0xc1cd999 to 0x18ece40 2024-04-24T15:33:14.365 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:14.365 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:33:14.365 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-04-24T15:33:14.365 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-04-24T15:33:14.365 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:14.373 INFO:teuthology.orchestra.run.smithi096.stdout:4298 write 0xaf0b07e thru 0xaf0d78f (0x2712 bytes) 2024-04-24T15:33:14.375 INFO:teuthology.orchestra.run.smithi096.stdout:4299 punch from 0x8bee0aa to 0x8bf96a0, (0xb5f6 bytes) 2024-04-24T15:33:14.375 INFO:teuthology.orchestra.run.smithi096.stdout:4302 read 0x3405fe9 thru 0x340718f (0x11a7 bytes) 2024-04-24T15:33:14.375 INFO:teuthology.orchestra.run.smithi096.stdout:4303 read 0x42497c8 thru 0x424ff84 (0x67bd bytes) 2024-04-24T15:33:14.375 INFO:teuthology.orchestra.run.smithi096.stdout:4304 write 0x648dd41 thru 0x6494354 (0x6614 bytes) 2024-04-24T15:33:14.375 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:33:14.376+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:33:14.377 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-24T15:33:14.378 INFO:teuthology.orchestra.run.smithi096.stdout:4305 punch from 0x93fdf4c to 0x940ccb0, (0xed64 bytes) 2024-04-24T15:33:14.378 INFO:teuthology.orchestra.run.smithi096.stdout:4306 punch from 0x6092fd1 to 0x609e676, (0xb6a5 bytes) 2024-04-24T15:33:14.378 INFO:teuthology.orchestra.run.smithi096.stdout:4308 write 0x7919d21 thru 0x791c561 (0x2841 bytes) 2024-04-24T15:33:14.378 INFO:teuthology.orchestra.run.smithi096.stdout:4309 write 0xae2c7ca thru 0xae35170 (0x89a7 bytes) 2024-04-24T15:33:14.380 INFO:teuthology.orchestra.run.smithi096.stdout:4310 read 0x8383ed0 thru 0x838f5ed (0xb71e bytes) 2024-04-24T15:33:14.380 INFO:teuthology.orchestra.run.smithi096.stdout:4313 read 0x1bcebba thru 0x1bcf46d (0x8b4 bytes) 2024-04-24T15:33:14.380 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:14.380 INFO:teuthology.orchestra.run.smithi096.stdout:4315 writesame 0x85a8334 thru 0x85afc5c data_size 0x31(0x7929 bytes) 2024-04-24T15:33:14.382 INFO:teuthology.orchestra.run.smithi096.stdout:4316 read 0xae0df21 thru 0xae13ded (0x5ecd bytes) 2024-04-24T15:33:14.382 INFO:teuthology.orchestra.run.smithi096.stdout:4317 punch from 0xabbc69c to 0xabc0980, (0x42e4 bytes) 2024-04-24T15:33:14.383 INFO:teuthology.orchestra.run.smithi096.stdout:4320 punch from 0x4ab9ce2 to 0x4abf425, (0x5743 bytes) 2024-04-24T15:33:14.383 INFO:teuthology.orchestra.run.smithi096.stdout:4321 read 0xa5eeec9 thru 0xa5f36f5 (0x482d bytes) 2024-04-24T15:33:14.390 INFO:teuthology.orchestra.run.smithi096.stdout:4322 write 0xc7dae9a thru 0xc7e9c62 (0xedc9 bytes) 2024-04-24T15:33:14.392 INFO:teuthology.orchestra.run.smithi096.stdout:4323 writesame 0x5815058 thru 0x581d2c7 data_size 0x8(0x8270 bytes) 2024-04-24T15:33:14.397 INFO:teuthology.orchestra.run.smithi096.stdout:4324 writesame 0x236de41 thru 0x237bea6 data_size 0x1(0xe066 bytes) 2024-04-24T15:33:14.415 INFO:teuthology.orchestra.run.smithi096.stdout:4325 read 0x5f24023 thru 0x5f32184 (0xe162 bytes) 2024-04-24T15:33:14.415 INFO:teuthology.orchestra.run.smithi096.stdout:4327 punch from 0xa2b9636 to 0xa2c7509, (0xded3 bytes) 2024-04-24T15:33:14.429 INFO:teuthology.orchestra.run.smithi096.stdout:4328 trunc from 0xc7e9c63 to 0xfd4dc7 2024-04-24T15:33:14.461 INFO:teuthology.orchestra.run.smithi096.stdout:4329 write 0xb1322a5 thru 0xb138137 (0x5e93 bytes) 2024-04-24T15:33:14.462 INFO:teuthology.orchestra.run.smithi096.stdout:4330 punch from 0x6c2421b to 0x6c2a45f, (0x6244 bytes) 2024-04-24T15:33:14.463 INFO:teuthology.orchestra.run.smithi096.stdout:4331 read 0x43bc2f9 thru 0x43cbff9 (0xfd01 bytes) 2024-04-24T15:33:14.463 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:14.463 INFO:teuthology.orchestra.run.smithi096.stdout:4334 punch from 0x87fae7 to 0x882f0f, (0x3428 bytes) 2024-04-24T15:33:14.468 INFO:teuthology.orchestra.run.smithi096.stdout:4335 trunc from 0xb138138 to 0x61da30b 2024-04-24T15:33:14.476 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:14.476+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:14.479 INFO:teuthology.orchestra.run.smithi096.stdout:4337 write 0x9b48ba thru 0x9b9783 (0x4eca bytes) 2024-04-24T15:33:14.491 INFO:teuthology.orchestra.run.smithi096.stdout:4338 writesame 0x98505d5 thru 0x98527a3 data_size 0x5(0x21cf bytes) 2024-04-24T15:33:14.492 INFO:teuthology.orchestra.run.smithi096.stdout:4339 flatten 2024-04-24T15:33:14.496 INFO:teuthology.orchestra.run.smithi096.stdout:4341 write 0x6805298 thru 0x68146bb (0xf424 bytes) 2024-04-24T15:33:14.509 INFO:teuthology.orchestra.run.smithi096.stdout:4342 trunc from 0x98527a4 to 0x11f9088 2024-04-24T15:33:14.519 INFO:teuthology.orchestra.run.smithi096.stdout:4343 read 0xe486c9 thru 0xe50343 (0x7c7b bytes) 2024-04-24T15:33:14.523 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:14.523 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-04-24T15:33:14.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-04-24T15:33:14.523 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-04-24T15:33:14.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.523 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:47384 4.1f 4:fdac2fcc:::rbd_header.114a8b543e91:head [watch reconnect cookie 140672527126976 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-04-24T15:33:14.523 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:09.802333+0000", 2024-04-24T15:33:14.523 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.7207036789999997, 2024-04-24T15:33:14.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0093213050000001, 2024-04-24T15:33:14.524 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:14.524 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:14.524 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:14.524 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:14.524 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:14.524 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 47384 2024-04-24T15:33:14.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.524 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:14.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.524 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:14.524 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.802333+0000", 2024-04-24T15:33:14.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:14.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.525 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:14.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.802333+0000", 2024-04-24T15:33:14.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:14.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.525 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:14.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.802332+0000", 2024-04-24T15:33:14.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999981 2024-04-24T15:33:14.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.525 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:14.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.802340+0000", 2024-04-24T15:33:14.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7700000000000007e-06 2024-04-24T15:33:14.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:14.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.802360+0000", 2024-04-24T15:33:14.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9959999999999999e-05 2024-04-24T15:33:14.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:14.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.802371+0000", 2024-04-24T15:33:14.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0473e-05 2024-04-24T15:33:14.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:14.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.802447+0000", 2024-04-24T15:33:14.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6089999999999998e-05 2024-04-24T15:33:14.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.527 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op must wait for map", 2024-04-24T15:33:14.527 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.802456+0000", 2024-04-24T15:33:14.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0599999999999997e-06 2024-04-24T15:33:14.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.527 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:14.527 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.808378+0000", 2024-04-24T15:33:14.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0059224910000000002 2024-04-24T15:33:14.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.527 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-04-24T15:33:14.527 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.808384+0000", 2024-04-24T15:33:14.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7740000000000001e-06 2024-04-24T15:33:14.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:14.528 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.809812+0000", 2024-04-24T15:33:14.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.001428043 2024-04-24T15:33:14.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:14.528 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.810082+0000", 2024-04-24T15:33:14.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00026971800000000001 2024-04-24T15:33:14.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:14.528 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.811655+0000", 2024-04-24T15:33:14.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015726869999999999 2024-04-24T15:33:14.528 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:14.528 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:14.529 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:14.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.529 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.529 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:47385 4.1f 4:fdac2fcc:::rbd_header.114a8b543e91:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-04-24T15:33:14.529 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:09.803953+0000", 2024-04-24T15:33:14.529 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.7190837649999997, 2024-04-24T15:33:14.529 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0080910729999999, 2024-04-24T15:33:14.529 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:14.529 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:14.529 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:14.529 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:14.529 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:14.529 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 47385 2024-04-24T15:33:14.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.529 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:14.530 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.530 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:14.530 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.803953+0000", 2024-04-24T15:33:14.530 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:14.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.530 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.530 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:14.530 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.803953+0000", 2024-04-24T15:33:14.530 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:14.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.530 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.530 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:14.530 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.803952+0000", 2024-04-24T15:33:14.530 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999981 2024-04-24T15:33:14.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:14.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.803961+0000", 2024-04-24T15:33:14.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9549999999999998e-06 2024-04-24T15:33:14.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:14.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.803977+0000", 2024-04-24T15:33:14.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6498999999999999e-05 2024-04-24T15:33:14.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:14.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.803985+0000", 2024-04-24T15:33:14.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5000000000000002e-06 2024-04-24T15:33:14.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:14.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.804032+0000", 2024-04-24T15:33:14.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7657999999999997e-05 2024-04-24T15:33:14.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2024-04-24T15:33:14.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.804039+0000", 2024-04-24T15:33:14.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5719999999999999e-06 2024-04-24T15:33:14.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:14.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.808397+0000", 2024-04-24T15:33:14.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0043579860000000003 2024-04-24T15:33:14.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-04-24T15:33:14.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.808401+0000", 2024-04-24T15:33:14.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3089999999999997e-06 2024-04-24T15:33:14.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:14.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.810700+0000", 2024-04-24T15:33:14.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0022984699999999 2024-04-24T15:33:14.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:14.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.810758+0000", 2024-04-24T15:33:14.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8121000000000001e-05 2024-04-24T15:33:14.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:33:14.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.811932+0000", 2024-04-24T15:33:14.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001174542 2024-04-24T15:33:14.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:14.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.811978+0000", 2024-04-24T15:33:14.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5645000000000001e-05 2024-04-24T15:33:14.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:14.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.812044+0000", 2024-04-24T15:33:14.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6575999999999994e-05 2024-04-24T15:33:14.535 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:14.535 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:14.535 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:14.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.535 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48589 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2621440~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:14.535 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889196+0000", 2024-04-24T15:33:14.535 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.6338408719999999, 2024-04-24T15:33:14.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016538913999999998, 2024-04-24T15:33:14.535 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:14.535 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:14.535 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:14.535 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:14.536 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:14.536 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48589 2024-04-24T15:33:14.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.536 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:14.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:14.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889196+0000", 2024-04-24T15:33:14.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:14.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:14.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889196+0000", 2024-04-24T15:33:14.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:14.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:14.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889196+0000", 2024-04-24T15:33:14.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:14.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:14.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889203+0000", 2024-04-24T15:33:14.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8499999999999996e-06 2024-04-24T15:33:14.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:14.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889210+0000", 2024-04-24T15:33:14.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8450000000000003e-06 2024-04-24T15:33:14.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.538 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:14.538 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889215+0000", 2024-04-24T15:33:14.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1139999999999999e-06 2024-04-24T15:33:14.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.538 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:14.538 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.905610+0000", 2024-04-24T15:33:14.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016395404999999998 2024-04-24T15:33:14.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.538 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:14.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.905668+0000", 2024-04-24T15:33:14.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8269000000000001e-05 2024-04-24T15:33:14.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.539 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:14.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.905735+0000", 2024-04-24T15:33:14.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6841999999999995e-05 2024-04-24T15:33:14.539 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:14.539 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:14.539 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:14.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.539 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48590 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2686976~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:14.539 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889219+0000", 2024-04-24T15:33:14.539 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.6338178330000002, 2024-04-24T15:33:14.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016760856000000001, 2024-04-24T15:33:14.540 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:14.540 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:14.540 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:14.540 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:14.540 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:14.540 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48590 2024-04-24T15:33:14.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.540 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:14.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.540 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:14.540 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889219+0000", 2024-04-24T15:33:14.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:14.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.540 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:14.541 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889219+0000", 2024-04-24T15:33:14.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:14.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.541 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:14.541 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889219+0000", 2024-04-24T15:33:14.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:14.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.541 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:14.541 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889221+0000", 2024-04-24T15:33:14.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4499999999999998e-06 2024-04-24T15:33:14.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.542 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:14.542 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889227+0000", 2024-04-24T15:33:14.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6869999999999999e-06 2024-04-24T15:33:14.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.542 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.542 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:14.542 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889231+0000", 2024-04-24T15:33:14.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6320000000000001e-06 2024-04-24T15:33:14.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.542 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.542 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:14.542 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.905769+0000", 2024-04-24T15:33:14.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016538850000000001 2024-04-24T15:33:14.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.542 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.543 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:14.543 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.905855+0000", 2024-04-24T15:33:14.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5296000000000005e-05 2024-04-24T15:33:14.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.543 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:14.543 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.905980+0000", 2024-04-24T15:33:14.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000125406 2024-04-24T15:33:14.543 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:14.543 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:14.543 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:14.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.543 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48705 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 1851392~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:14.543 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889236+0000", 2024-04-24T15:33:14.543 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.633801225, 2024-04-24T15:33:14.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016984302, 2024-04-24T15:33:14.544 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:14.544 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:14.544 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:14.544 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:14.544 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:14.544 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48705 2024-04-24T15:33:14.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.544 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:14.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.544 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:14.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889236+0000", 2024-04-24T15:33:14.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:14.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.544 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:14.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889236+0000", 2024-04-24T15:33:14.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:14.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:14.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889235+0000", 2024-04-24T15:33:14.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:14.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:14.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889238+0000", 2024-04-24T15:33:14.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.249e-06 2024-04-24T15:33:14.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:14.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889243+0000", 2024-04-24T15:33:14.546 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6810000000000003e-06 2024-04-24T15:33:14.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.546 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:14.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889247+0000", 2024-04-24T15:33:14.546 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6780000000000002e-06 2024-04-24T15:33:14.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.546 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:14.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906008+0000", 2024-04-24T15:33:14.546 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01676099 2024-04-24T15:33:14.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.546 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:14.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906090+0000", 2024-04-24T15:33:14.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1508999999999995e-05 2024-04-24T15:33:14.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.547 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.547 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:14.547 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906220+0000", 2024-04-24T15:33:14.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013056999999999999 2024-04-24T15:33:14.547 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:14.547 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:14.547 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:14.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.547 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.547 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48706 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 1916928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:14.547 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889251+0000", 2024-04-24T15:33:14.547 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.6337857919999998, 2024-04-24T15:33:14.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01718693, 2024-04-24T15:33:14.547 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:14.548 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:14.548 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:14.548 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:14.548 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:14.548 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48706 2024-04-24T15:33:14.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.548 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:14.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.548 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:14.548 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889251+0000", 2024-04-24T15:33:14.548 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:14.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.548 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:14.548 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889251+0000", 2024-04-24T15:33:14.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:14.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.549 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:14.549 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889251+0000", 2024-04-24T15:33:14.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:14.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.549 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:14.549 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889253+0000", 2024-04-24T15:33:14.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.244e-06 2024-04-24T15:33:14.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.549 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:14.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889259+0000", 2024-04-24T15:33:14.550 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3640000000000004e-06 2024-04-24T15:33:14.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.550 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:14.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889263+0000", 2024-04-24T15:33:14.550 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3720000000000001e-06 2024-04-24T15:33:14.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.550 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:14.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906258+0000", 2024-04-24T15:33:14.550 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016995310999999999 2024-04-24T15:33:14.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.551 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:14.551 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906343+0000", 2024-04-24T15:33:14.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4536000000000005e-05 2024-04-24T15:33:14.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.551 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:14.551 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906438+0000", 2024-04-24T15:33:14.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5502999999999997e-05 2024-04-24T15:33:14.551 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:14.551 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:14.551 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:14.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.551 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48707 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 1982464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:14.551 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889267+0000", 2024-04-24T15:33:14.552 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.6337702319999998, 2024-04-24T15:33:14.552 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017425395, 2024-04-24T15:33:14.552 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:14.552 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:14.552 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:14.552 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:14.552 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:14.552 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48707 2024-04-24T15:33:14.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.552 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:14.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.552 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:14.552 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889267+0000", 2024-04-24T15:33:14.552 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:14.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:14.553 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889267+0000", 2024-04-24T15:33:14.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:14.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:14.553 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889266+0000", 2024-04-24T15:33:14.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:14.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:14.553 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889275+0000", 2024-04-24T15:33:14.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.568e-06 2024-04-24T15:33:14.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.554 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:14.554 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889281+0000", 2024-04-24T15:33:14.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3779999999999998e-06 2024-04-24T15:33:14.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.554 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:14.554 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889291+0000", 2024-04-24T15:33:14.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2769999999999994e-06 2024-04-24T15:33:14.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.554 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:14.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906476+0000", 2024-04-24T15:33:14.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017185277999999998 2024-04-24T15:33:14.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.555 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:14.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906567+0000", 2024-04-24T15:33:14.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0902999999999993e-05 2024-04-24T15:33:14.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.555 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:14.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906692+0000", 2024-04-24T15:33:14.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012541700000000001 2024-04-24T15:33:14.555 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:14.555 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:14.555 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:14.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.556 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48708 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2048000~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:14.556 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889295+0000", 2024-04-24T15:33:14.556 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.6337417410000001, 2024-04-24T15:33:14.556 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017534984999999999, 2024-04-24T15:33:14.556 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:14.556 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:14.556 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:14.556 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:14.556 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:14.556 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48708 2024-04-24T15:33:14.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.556 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:14.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:14.557 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889295+0000", 2024-04-24T15:33:14.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:14.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:14.557 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889295+0000", 2024-04-24T15:33:14.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:14.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:14.557 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889295+0000", 2024-04-24T15:33:14.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:14.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.558 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.558 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:14.558 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889297+0000", 2024-04-24T15:33:14.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3479999999999998e-06 2024-04-24T15:33:14.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.558 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.558 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:14.558 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889304+0000", 2024-04-24T15:33:14.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3720000000000001e-06 2024-04-24T15:33:14.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.558 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.559 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:14.559 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889307+0000", 2024-04-24T15:33:14.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7160000000000001e-06 2024-04-24T15:33:14.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.559 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:14.559 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906711+0000", 2024-04-24T15:33:14.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017403452999999999 2024-04-24T15:33:14.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.559 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:14.559 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906765+0000", 2024-04-24T15:33:14.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4219999999999999e-05 2024-04-24T15:33:14.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.559 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:14.560 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906830+0000", 2024-04-24T15:33:14.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5277000000000005e-05 2024-04-24T15:33:14.560 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:14.560 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:14.560 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:14.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.560 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.560 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48709 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2113536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:14.560 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889312+0000", 2024-04-24T15:33:14.560 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.633725434, 2024-04-24T15:33:14.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017772013, 2024-04-24T15:33:14.560 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:14.560 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:14.560 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:14.560 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:14.560 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:14.560 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48709 2024-04-24T15:33:14.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.561 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:14.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.561 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:14.561 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889312+0000", 2024-04-24T15:33:14.561 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:14.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.561 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:14.561 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889312+0000", 2024-04-24T15:33:14.561 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:14.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.561 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:14.561 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889311+0000", 2024-04-24T15:33:14.561 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:14.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.562 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.562 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:14.562 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889314+0000", 2024-04-24T15:33:14.562 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2759999999999999e-06 2024-04-24T15:33:14.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.562 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.562 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:14.562 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889322+0000", 2024-04-24T15:33:14.562 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8589999999999994e-06 2024-04-24T15:33:14.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.562 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.562 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:14.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889326+0000", 2024-04-24T15:33:14.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6380000000000002e-06 2024-04-24T15:33:14.563 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.563 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.563 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:14.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906862+0000", 2024-04-24T15:33:14.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017536403999999998 2024-04-24T15:33:14.563 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.563 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.563 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:14.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906949+0000", 2024-04-24T15:33:14.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6217000000000005e-05 2024-04-24T15:33:14.563 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.563 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.563 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:14.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907084+0000", 2024-04-24T15:33:14.564 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013497100000000001 2024-04-24T15:33:14.564 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:14.564 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:14.564 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:14.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.564 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.564 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48710 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2179072~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:14.564 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889332+0000", 2024-04-24T15:33:14.564 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.6337046200000001, 2024-04-24T15:33:14.564 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017944720000000001, 2024-04-24T15:33:14.564 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:14.564 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:14.564 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:14.564 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:14.564 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:14.565 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48710 2024-04-24T15:33:14.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.565 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:14.565 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.565 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:14.565 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889332+0000", 2024-04-24T15:33:14.565 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:14.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.565 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.565 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:14.565 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889332+0000", 2024-04-24T15:33:14.565 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:14.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.565 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.565 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:14.565 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889332+0000", 2024-04-24T15:33:14.566 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:14.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.566 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.566 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:14.566 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889334+0000", 2024-04-24T15:33:14.566 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4210000000000002e-06 2024-04-24T15:33:14.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.566 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.566 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:14.566 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889340+0000", 2024-04-24T15:33:14.566 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7209999999999999e-06 2024-04-24T15:33:14.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.566 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.566 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:14.566 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889344+0000", 2024-04-24T15:33:14.567 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.726e-06 2024-04-24T15:33:14.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.567 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.567 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:14.567 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907119+0000", 2024-04-24T15:33:14.567 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017775551000000001 2024-04-24T15:33:14.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.567 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.567 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:14.567 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907190+0000", 2024-04-24T15:33:14.567 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0634000000000002e-05 2024-04-24T15:33:14.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.567 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.567 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:14.567 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907277+0000", 2024-04-24T15:33:14.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7120999999999995e-05 2024-04-24T15:33:14.568 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:14.568 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:14.568 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:14.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.568 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48711 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2244608~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:14.568 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889348+0000", 2024-04-24T15:33:14.568 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.6336890029999998, 2024-04-24T15:33:14.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018188109000000001, 2024-04-24T15:33:14.568 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:14.568 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:14.568 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:14.568 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:14.568 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:14.569 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48711 2024-04-24T15:33:14.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.569 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:14.569 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.569 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:14.569 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889348+0000", 2024-04-24T15:33:14.569 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:14.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.569 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.569 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:14.569 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889348+0000", 2024-04-24T15:33:14.569 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:14.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.569 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.569 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:14.570 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889348+0000", 2024-04-24T15:33:14.570 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:14.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.570 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.570 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:14.570 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889350+0000", 2024-04-24T15:33:14.570 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2330000000000001e-06 2024-04-24T15:33:14.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.570 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.570 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:14.570 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889356+0000", 2024-04-24T15:33:14.570 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8769999999999998e-06 2024-04-24T15:33:14.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.571 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.571 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:14.571 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889359+0000", 2024-04-24T15:33:14.571 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3699999999999999e-06 2024-04-24T15:33:14.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.571 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.571 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:14.571 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907316+0000", 2024-04-24T15:33:14.571 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017956857999999999 2024-04-24T15:33:14.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.571 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.571 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:14.571 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907402+0000", 2024-04-24T15:33:14.571 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5927000000000006e-05 2024-04-24T15:33:14.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.572 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:14.572 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907536+0000", 2024-04-24T15:33:14.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000134258 2024-04-24T15:33:14.572 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:14.572 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:14.572 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:14.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.572 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48712 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2310144~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:14.572 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889363+0000", 2024-04-24T15:33:14.572 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.6336736840000001, 2024-04-24T15:33:14.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018314815000000002, 2024-04-24T15:33:14.573 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:14.573 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:14.573 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:14.573 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:14.573 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:14.573 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48712 2024-04-24T15:33:14.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.573 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:14.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.573 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:14.573 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889363+0000", 2024-04-24T15:33:14.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:14.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.573 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:14.574 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889363+0000", 2024-04-24T15:33:14.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:14.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.574 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:14.574 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889363+0000", 2024-04-24T15:33:14.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:14.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.574 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:14.574 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889366+0000", 2024-04-24T15:33:14.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6709999999999999e-06 2024-04-24T15:33:14.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:14.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889373+0000", 2024-04-24T15:33:14.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1269999999999997e-06 2024-04-24T15:33:14.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:14.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889377+0000", 2024-04-24T15:33:14.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1219999999999997e-06 2024-04-24T15:33:14.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:14.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907560+0000", 2024-04-24T15:33:14.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018183357000000001 2024-04-24T15:33:14.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:14.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907612+0000", 2024-04-24T15:33:14.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2172999999999997e-05 2024-04-24T15:33:14.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:14.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907678+0000", 2024-04-24T15:33:14.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5736999999999998e-05 2024-04-24T15:33:14.576 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:14.576 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:14.576 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:14.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.577 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48713 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2375680~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:14.577 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889381+0000", 2024-04-24T15:33:14.577 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.6336557909999998, 2024-04-24T15:33:14.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018556019, 2024-04-24T15:33:14.577 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:14.577 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:14.577 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:14.577 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:14.577 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:14.577 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48713 2024-04-24T15:33:14.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.577 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:14.578 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.578 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:14.578 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889381+0000", 2024-04-24T15:33:14.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:14.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.578 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.578 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:14.578 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889381+0000", 2024-04-24T15:33:14.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:14.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.578 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.578 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:14.578 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889381+0000", 2024-04-24T15:33:14.579 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:14.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.579 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:14.579 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889388+0000", 2024-04-24T15:33:14.579 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8809999999999998e-06 2024-04-24T15:33:14.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.579 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:14.579 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889394+0000", 2024-04-24T15:33:14.579 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0410000000000003e-06 2024-04-24T15:33:14.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.579 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:14.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889397+0000", 2024-04-24T15:33:14.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5939999999999998e-06 2024-04-24T15:33:14.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:14.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907711+0000", 2024-04-24T15:33:14.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018313949999999999 2024-04-24T15:33:14.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:14.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907802+0000", 2024-04-24T15:33:14.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0307000000000001e-05 2024-04-24T15:33:14.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:14.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907937+0000", 2024-04-24T15:33:14.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013561899999999999 2024-04-24T15:33:14.581 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:14.581 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:14.581 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:14.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.581 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48714 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2441216~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:14.581 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889435+0000", 2024-04-24T15:33:14.581 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.633602443, 2024-04-24T15:33:14.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018636040999999999, 2024-04-24T15:33:14.581 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:14.581 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:14.582 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:14.582 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:14.582 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:14.582 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48714 2024-04-24T15:33:14.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.582 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:14.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.582 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:14.582 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889435+0000", 2024-04-24T15:33:14.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:14.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.582 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:14.582 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889435+0000", 2024-04-24T15:33:14.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:14.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:14.583 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889434+0000", 2024-04-24T15:33:14.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:14.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:14.583 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889437+0000", 2024-04-24T15:33:14.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.875e-06 2024-04-24T15:33:14.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:14.584 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889444+0000", 2024-04-24T15:33:14.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9870000000000002e-06 2024-04-24T15:33:14.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:14.584 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889448+0000", 2024-04-24T15:33:14.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3030000000000001e-06 2024-04-24T15:33:14.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:14.584 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907952+0000", 2024-04-24T15:33:14.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018504245999999999 2024-04-24T15:33:14.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:14.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908007+0000", 2024-04-24T15:33:14.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4579e-05 2024-04-24T15:33:14.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:14.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908071+0000", 2024-04-24T15:33:14.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3615999999999995e-05 2024-04-24T15:33:14.585 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:14.585 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:14.585 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:14.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.585 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48715 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2506752~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:14.585 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889453+0000", 2024-04-24T15:33:14.586 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.6335840790000002, 2024-04-24T15:33:14.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018873935000000001, 2024-04-24T15:33:14.586 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:14.586 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:14.586 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:14.586 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:14.586 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:14.586 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48715 2024-04-24T15:33:14.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.586 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:14.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:14.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889453+0000", 2024-04-24T15:33:14.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:14.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.587 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:14.587 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889453+0000", 2024-04-24T15:33:14.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:14.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.587 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:14.587 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889452+0000", 2024-04-24T15:33:14.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:14.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.587 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:14.588 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889459+0000", 2024-04-24T15:33:14.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9269999999999999e-06 2024-04-24T15:33:14.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:14.588 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889468+0000", 2024-04-24T15:33:14.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2390000000000002e-06 2024-04-24T15:33:14.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:14.588 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889475+0000", 2024-04-24T15:33:14.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3939999999999998e-06 2024-04-24T15:33:14.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:14.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908103+0000", 2024-04-24T15:33:14.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018628388999999999 2024-04-24T15:33:14.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.589 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:14.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908193+0000", 2024-04-24T15:33:14.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9343000000000002e-05 2024-04-24T15:33:14.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.589 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:14.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908327+0000", 2024-04-24T15:33:14.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013415899999999999 2024-04-24T15:33:14.589 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:14.589 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:14.589 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:14.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.590 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48716 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2572288~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:14.590 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889480+0000", 2024-04-24T15:33:14.590 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.633557229, 2024-04-24T15:33:14.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018992862999999999, 2024-04-24T15:33:14.590 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:14.590 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:14.590 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:14.590 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:14.590 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:14.590 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48716 2024-04-24T15:33:14.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.591 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:14.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.591 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:14.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889480+0000", 2024-04-24T15:33:14.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:14.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.591 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:14.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889480+0000", 2024-04-24T15:33:14.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:14.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.591 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:14.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889479+0000", 2024-04-24T15:33:14.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:14.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:14.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889482+0000", 2024-04-24T15:33:14.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4729999999999999e-06 2024-04-24T15:33:14.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:14.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889488+0000", 2024-04-24T15:33:14.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1750000000000002e-06 2024-04-24T15:33:14.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:14.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889501+0000", 2024-04-24T15:33:14.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.288e-05 2024-04-24T15:33:14.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:14.593 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908345+0000", 2024-04-24T15:33:14.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018843835999999999 2024-04-24T15:33:14.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:14.593 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908395+0000", 2024-04-24T15:33:14.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9765999999999998e-05 2024-04-24T15:33:14.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:14.594 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908473+0000", 2024-04-24T15:33:14.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8159999999999997e-05 2024-04-24T15:33:14.594 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:14.594 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:14.594 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:14.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.594 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.594 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48717 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2637824~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:14.594 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889506+0000", 2024-04-24T15:33:14.594 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.633530978, 2024-04-24T15:33:14.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.019241864000000001, 2024-04-24T15:33:14.594 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:14.594 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:14.594 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:14.595 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:14.595 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:14.595 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48717 2024-04-24T15:33:14.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.595 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:14.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.595 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:14.595 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889506+0000", 2024-04-24T15:33:14.595 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:14.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.595 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:14.595 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889506+0000", 2024-04-24T15:33:14.595 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:14.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:14.596 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889506+0000", 2024-04-24T15:33:14.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:14.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:14.596 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889508+0000", 2024-04-24T15:33:14.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6900000000000001e-06 2024-04-24T15:33:14.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:14.596 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889515+0000", 2024-04-24T15:33:14.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5409999999999997e-06 2024-04-24T15:33:14.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:14.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889519+0000", 2024-04-24T15:33:14.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7129999999999999e-06 2024-04-24T15:33:14.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:14.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908520+0000", 2024-04-24T15:33:14.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.019001606000000001 2024-04-24T15:33:14.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:14.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908612+0000", 2024-04-24T15:33:14.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1909000000000004e-05 2024-04-24T15:33:14.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.598 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:14.598 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908748+0000", 2024-04-24T15:33:14.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000135847 2024-04-24T15:33:14.598 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:14.598 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:14.598 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:14.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.598 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48718 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2703360~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:14.598 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889523+0000", 2024-04-24T15:33:14.598 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.6335139860000001, 2024-04-24T15:33:14.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.019451725999999999, 2024-04-24T15:33:14.599 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:14.599 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:14.599 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:14.599 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:14.599 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:14.599 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48718 2024-04-24T15:33:14.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.599 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:14.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:14.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889523+0000", 2024-04-24T15:33:14.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:14.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:14.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889523+0000", 2024-04-24T15:33:14.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:14.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:14.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889523+0000", 2024-04-24T15:33:14.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:14.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:14.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889525+0000", 2024-04-24T15:33:14.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9970000000000001e-06 2024-04-24T15:33:14.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:14.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889530+0000", 2024-04-24T15:33:14.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7610000000000004e-06 2024-04-24T15:33:14.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:14.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889534+0000", 2024-04-24T15:33:14.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6119999999999999e-06 2024-04-24T15:33:14.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:14.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908775+0000", 2024-04-24T15:33:14.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.019240709000000002 2024-04-24T15:33:14.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:14.602 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908856+0000", 2024-04-24T15:33:14.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0788e-05 2024-04-24T15:33:14.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:14.602 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908975+0000", 2024-04-24T15:33:14.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011927800000000001 2024-04-24T15:33:14.602 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:14.602 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:14.602 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:14.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.602 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:49872 4.1f 4:fe48f4c1:::rbd_data.114ab402c616.0000000000000014:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-04-24T15:33:14.602 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:12.416730+0000", 2024-04-24T15:33:14.603 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 2.1063070079999999, 2024-04-24T15:33:14.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.020522327999999999, 2024-04-24T15:33:14.603 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:14.603 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:14.603 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:14.603 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:14.603 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:14.603 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 49872 2024-04-24T15:33:14.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.603 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:14.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:14.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.416730+0000", 2024-04-24T15:33:14.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:14.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:14.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.416730+0000", 2024-04-24T15:33:14.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:14.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:14.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.416729+0000", 2024-04-24T15:33:14.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:33:14.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:14.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.416734+0000", 2024-04-24T15:33:14.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0180000000000001e-06 2024-04-24T15:33:14.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:14.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.416744+0000", 2024-04-24T15:33:14.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0069e-05 2024-04-24T15:33:14.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:14.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.416749+0000", 2024-04-24T15:33:14.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7190000000000001e-06 2024-04-24T15:33:14.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:14.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.416777+0000", 2024-04-24T15:33:14.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7532999999999999e-05 2024-04-24T15:33:14.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:14.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.416824+0000", 2024-04-24T15:33:14.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7738000000000003e-05 2024-04-24T15:33:14.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:33:14.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.437145+0000", 2024-04-24T15:33:14.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.020320416000000001 2024-04-24T15:33:14.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:14.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.437189+0000", 2024-04-24T15:33:14.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4493000000000002e-05 2024-04-24T15:33:14.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:14.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.437252+0000", 2024-04-24T15:33:14.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3110999999999999e-05 2024-04-24T15:33:14.607 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:14.607 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:14.607 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:14.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.607 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:52545 4.a 4:5331820a:::rbd_data.114ab867d54d.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-04-24T15:33:14.607 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:14.399696+0000", 2024-04-24T15:33:14.607 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.12334060500000001, 2024-04-24T15:33:14.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016877893000000001, 2024-04-24T15:33:14.608 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:14.608 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:14.608 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:14.608 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:14.608 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:14.608 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 52545 2024-04-24T15:33:14.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.608 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:14.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:14.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.399696+0000", 2024-04-24T15:33:14.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:14.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:14.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.399696+0000", 2024-04-24T15:33:14.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:14.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:14.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.399695+0000", 2024-04-24T15:33:14.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999981 2024-04-24T15:33:14.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:14.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.399704+0000", 2024-04-24T15:33:14.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.098e-06 2024-04-24T15:33:14.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:14.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.399721+0000", 2024-04-24T15:33:14.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7088999999999999e-05 2024-04-24T15:33:14.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:14.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.399728+0000", 2024-04-24T15:33:14.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5699999999999998e-06 2024-04-24T15:33:14.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:14.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.399775+0000", 2024-04-24T15:33:14.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7445999999999999e-05 2024-04-24T15:33:14.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:14.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.399840+0000", 2024-04-24T15:33:14.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5492000000000002e-05 2024-04-24T15:33:14.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:33:14.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.416468+0000", 2024-04-24T15:33:14.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016627527 2024-04-24T15:33:14.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:14.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.416521+0000", 2024-04-24T15:33:14.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2988000000000002e-05 2024-04-24T15:33:14.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:14.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:14.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:14.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.416574+0000", 2024-04-24T15:33:14.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3363999999999999e-05 2024-04-24T15:33:14.612 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:14.612 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:14.612 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:14.612 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:14.612 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:14.612 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:14.613 INFO:teuthology.orchestra.run.smithi096.stdout:4344 write 0x47aa3ce thru 0x47b6b1e (0xc751 bytes) 2024-04-24T15:33:14.613 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:14.614 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-24T15:33:14.616 INFO:teuthology.orchestra.run.smithi096.stdout:4347 write 0xd65fcff thru 0xd6647b8 (0x4aba bytes) 2024-04-24T15:33:14.616 INFO:teuthology.orchestra.run.smithi096.stdout:4348 write 0x73894cb thru 0x7391e36 (0x896c bytes) 2024-04-24T15:33:14.616 INFO:teuthology.orchestra.run.smithi096.stdout:4349 read 0x351291 thru 0x356025 (0x4d95 bytes) 2024-04-24T15:33:14.616 INFO:teuthology.orchestra.run.smithi096.stdout:4350 read 0x557d6df thru 0x5588991 (0xb2b3 bytes) 2024-04-24T15:33:14.616 INFO:teuthology.orchestra.run.smithi096.stdout:4351 punch from 0x161dcc7 to 0x161ecb1, (0xfea bytes) 2024-04-24T15:33:14.616 INFO:teuthology.orchestra.run.smithi096.stdout:4352 read 0xc2fada6 thru 0xc308d30 (0xdf8b bytes) 2024-04-24T15:33:14.616 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:14.616 INFO:teuthology.orchestra.run.smithi096.stdout:4356 write 0xde78aaf thru 0xde7f1de (0x6730 bytes) 2024-04-24T15:33:14.616 INFO:teuthology.orchestra.run.smithi096.stdout:4357 write 0xbab302a thru 0xbabb776 (0x874d bytes) 2024-04-24T15:33:14.616 INFO:teuthology.orchestra.run.smithi096.stdout:4358 trunc from 0xde7f1df to 0xc540163 2024-04-24T15:33:14.616 INFO:teuthology.orchestra.run.smithi096.stdout:4360 trunc from 0xc540163 to 0xde72c93 2024-04-24T15:33:14.616 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:14.616 INFO:teuthology.orchestra.run.smithi096.stdout:4364 clone 24 order 23 su 131072 sc 15 2024-04-24T15:33:14.617 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:14.614+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:14.714 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-04-24T15:33:14.715 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T15:33:14.715 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T15:33:14.715 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T15:33:14.715 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T15:33:14.715 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T15:33:14.715 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T15:33:14.715 INFO:teuthology.orchestra.run.smithi096.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-04-24T15:33:14.715 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T15:33:14.716 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:14.716+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:14.717 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-24T15:33:14.718 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-24T15:33:14.781 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:14.781 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:33:14.781 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-04-24T15:33:14.781 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:14.791 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-24T15:33:14.816 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:14.816+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:14.825 INFO:teuthology.orchestra.run.smithi096.stderr:nodeep-scrub is unset 2024-04-24T15:33:14.916 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:14.916+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:14.930 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:14.931 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:33:14.931 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-04-24T15:33:14.931 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-04-24T15:33:14.931 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:14.943 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-24T15:33:14.952 INFO:teuthology.orchestra.run.smithi151.stdout: marking collection for removal 2024-04-24T15:33:14.952 INFO:teuthology.orchestra.run.smithi151.stdout:setting '_remove' omap key 2024-04-24T15:33:14.956 INFO:teuthology.orchestra.run.smithi151.stdout:finish_remove_pgs 4.12_head removing 4.12 2024-04-24T15:33:14.960 INFO:teuthology.orchestra.run.smithi151.stdout:Remove successful 2024-04-24T15:33:15.016 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:33:15.017+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:33:15.020 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #22, image image_client.0-clone22 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent23 2024-04-24T15:33:15.047 INFO:teuthology.orchestra.run.smithi096.stdout:4365 read 0x890b930 thru 0x890d741 (0x1e12 bytes) 2024-04-24T15:33:15.048 INFO:teuthology.orchestra.run.smithi096.stdout:4366 punch from 0x497079 to 0x49c4fb, (0x5482 bytes) 2024-04-24T15:33:15.057 INFO:teuthology.orchestra.run.smithi096.stdout:4367 trunc from 0xde72c93 to 0x8791c39 2024-04-24T15:33:15.093 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:15.094 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-04-24T15:33:15.094 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-04-24T15:33:15.094 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [] 2024-04-24T15:33:15.094 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:15.105 INFO:teuthology.orchestra.run.smithi096.stdout:4368 read 0x3f5d9f2 thru 0x3f6c2e7 (0xe8f6 bytes) 2024-04-24T15:33:15.106 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-24T15:33:15.113 INFO:teuthology.orchestra.run.smithi096.stdout:4369 trunc from 0x8791c39 to 0x1ad709d 2024-04-24T15:33:15.117 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:33:15.118+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:33:15.191 INFO:teuthology.orchestra.run.smithi096.stdout:4370 punch from 0x1953b74 to 0x1961602, (0xda8e bytes) 2024-04-24T15:33:15.205 INFO:teuthology.orchestra.run.smithi096.stdout:4371 trunc from 0x1ad709d to 0xd1e5cc7 2024-04-24T15:33:15.210 INFO:teuthology.orchestra.run.smithi096.stdout:4372 write 0x71ac9a thru 0x726be5 (0xbf4c bytes) 2024-04-24T15:33:15.211 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:15.212 INFO:teuthology.orchestra.run.smithi096.stdout:4374 read 0xc5a6885 thru 0xc5b0c13 (0xa38f bytes) 2024-04-24T15:33:15.212 INFO:teuthology.orchestra.run.smithi096.stdout:4376 read 0x7ef3477 thru 0x7efa61a (0x71a4 bytes) 2024-04-24T15:33:15.212 INFO:teuthology.orchestra.run.smithi096.stdout:4377 write 0xa8bcbb7 thru 0xa8c603b (0x9485 bytes) 2024-04-24T15:33:15.218 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:15.219+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:15.220 INFO:teuthology.orchestra.run.smithi096.stdout:4378 trunc from 0xd1e5cc7 to 0x8a0b164 2024-04-24T15:33:15.226 INFO:teuthology.orchestra.run.smithi096.stdout:4380 read 0xbde79e thru 0xbe53af (0x6c12 bytes) 2024-04-24T15:33:15.232 INFO:teuthology.orchestra.run.smithi096.stdout:4381 trunc from 0x8a0b164 to 0x4ad7115 2024-04-24T15:33:15.252 INFO:teuthology.orchestra.run.smithi096.stdout:4387 write 0x5a340e7 thru 0x5a36046 (0x1f60 bytes) 2024-04-24T15:33:15.269 INFO:teuthology.orchestra.run.smithi096.stdout:4388 writesame 0xeb9589f thru 0xeb9e670 data_size 0x9(0x8dd2 bytes) 2024-04-24T15:33:15.269 INFO:teuthology.orchestra.run.smithi096.stdout:4389 punch from 0x4efd1ff to 0x4f03104, (0x5f05 bytes) 2024-04-24T15:33:15.271 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:15.272 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:33:15.272 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-04-24T15:33:15.272 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:15.286 INFO:teuthology.orchestra.run.smithi096.stdout:4391 trunc from 0xeb9e671 to 0xc5d30d 2024-04-24T15:33:15.290 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-24T15:33:15.307 INFO:teuthology.orchestra.run.smithi096.stdout:4393 write 0x684f242 thru 0x685a7ff (0xb5be bytes) 2024-04-24T15:33:15.309 INFO:teuthology.orchestra.run.smithi096.stdout:4394 write 0x637967 thru 0x63c80b (0x4ea5 bytes) 2024-04-24T15:33:15.310 INFO:teuthology.orchestra.run.smithi096.stdout:4395 write 0x5369436 thru 0x5379350 (0xff1b bytes) 2024-04-24T15:33:15.314 INFO:teuthology.orchestra.run.smithi096.stdout:4397 trunc from 0x685a800 to 0x53a7b4b 2024-04-24T15:33:15.319 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:33:15.320+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:33:15.323 INFO:teuthology.orchestra.run.smithi096.stdout:4398 punch from 0x45085cd to 0x450d359, (0x4d8c bytes) 2024-04-24T15:33:15.323 INFO:teuthology.orchestra.run.smithi096.stdout:4400 punch from 0x41e2ec4 to 0x41f162b, (0xe767 bytes) 2024-04-24T15:33:15.323 INFO:teuthology.orchestra.run.smithi096.stdout:4401 read 0x97aab2 thru 0x988c1d (0xe16c bytes) 2024-04-24T15:33:15.332 INFO:teuthology.orchestra.run.smithi096.stdout:4402 trunc from 0x53a7b4b to 0xca2ba97 2024-04-24T15:33:15.337 INFO:teuthology.orchestra.run.smithi096.stdout:4403 write 0x442ee4a thru 0x4432feb (0x41a2 bytes) 2024-04-24T15:33:15.339 INFO:teuthology.orchestra.run.smithi096.stdout:4404 punch from 0x15cbd2c to 0x15cd1f5, (0x14c9 bytes) 2024-04-24T15:33:15.339 INFO:teuthology.orchestra.run.smithi096.stdout:4405 write 0x3153c66 thru 0x3160163 (0xc4fe bytes) 2024-04-24T15:33:15.341 INFO:teuthology.orchestra.run.smithi096.stdout:4406 writesame 0x53a9b8a thru 0x53ad1e0 data_size 0x1(0x3657 bytes) 2024-04-24T15:33:15.348 INFO:teuthology.orchestra.run.smithi096.stdout:4407 write 0x61cc88d thru 0x61d404e (0x77c2 bytes) 2024-04-24T15:33:15.350 INFO:teuthology.orchestra.run.smithi096.stdout:4408 write 0x5a73e39 thru 0x5a8262e (0xe7f6 bytes) 2024-04-24T15:33:15.351 INFO:teuthology.orchestra.run.smithi096.stdout:4410 write 0x40f97a9 thru 0x40fb40c (0x1c64 bytes) 2024-04-24T15:33:15.353 INFO:teuthology.orchestra.run.smithi096.stdout:4411 write 0x72cb97 thru 0x72f73b (0x2ba5 bytes) 2024-04-24T15:33:15.354 INFO:teuthology.orchestra.run.smithi096.stdout:4412 write 0x5299f01 thru 0x52a3c43 (0x9d43 bytes) 2024-04-24T15:33:15.356 INFO:teuthology.orchestra.run.smithi096.stdout:4413 read 0x2c37adb thru 0x2c41dbb (0xa2e1 bytes) 2024-04-24T15:33:15.356 INFO:teuthology.orchestra.run.smithi096.stdout:4414 read 0x9d1afec thru 0x9d2044a (0x545f bytes) 2024-04-24T15:33:15.356 INFO:teuthology.orchestra.run.smithi096.stdout:4415 punch from 0x8562de4 to 0x8564e05, (0x2021 bytes) 2024-04-24T15:33:15.357 INFO:teuthology.orchestra.run.smithi096.stdout:4416 write 0xc997cc6 thru 0xc99f5a1 (0x78dc bytes) 2024-04-24T15:33:15.359 INFO:teuthology.orchestra.run.smithi096.stdout:4417 punch from 0xa49b8c4 to 0xa4a1ece, (0x660a bytes) 2024-04-24T15:33:15.359 INFO:teuthology.orchestra.run.smithi096.stdout:4418 write 0xf70c49 thru 0xf7a91d (0x9cd5 bytes) 2024-04-24T15:33:15.361 INFO:teuthology.orchestra.run.smithi096.stdout:4419 writesame 0x776ecda thru 0x77749d1 data_size 0x8(0x5cf8 bytes) 2024-04-24T15:33:15.362 INFO:teuthology.orchestra.run.smithi096.stdout:4420 punch from 0x541fb4b to 0x542c9b5, (0xce6a bytes) 2024-04-24T15:33:15.362 INFO:teuthology.orchestra.run.smithi096.stdout:4421 write 0xa268bd0 thru 0xa270246 (0x7677 bytes) 2024-04-24T15:33:15.370 INFO:teuthology.orchestra.run.smithi096.stdout:4424 write 0xca91aa4 thru 0xca93ae5 (0x2042 bytes) 2024-04-24T15:33:15.372 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:15.372 INFO:teuthology.orchestra.run.smithi096.stdout:4426 trunc from 0xca93ae6 to 0xc8e3cc6 2024-04-24T15:33:15.380 INFO:teuthology.orchestra.run.smithi096.stdout:4427 write 0x287f934 thru 0x287ffd1 (0x69e bytes) 2024-04-24T15:33:15.382 INFO:teuthology.orchestra.run.smithi096.stdout:4428 writesame 0x8203dfd thru 0x8213725 data_size 0x1(0xf929 bytes) 2024-04-24T15:33:15.416 INFO:teuthology.orchestra.run.smithi096.stdout:4429 punch from 0x1c79a69 to 0x1c838d8, (0x9e6f bytes) 2024-04-24T15:33:15.416 INFO:teuthology.orchestra.run.smithi096.stdout:4430 write 0x34f7087 thru 0x34f983f (0x27b9 bytes) 2024-04-24T15:33:15.418 INFO:teuthology.orchestra.run.smithi096.stdout:4431 write 0x7b55e35 thru 0x7b6471e (0xe8ea bytes) 2024-04-24T15:33:15.420 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:15.420+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:15.421 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:15.421 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:15.428 INFO:teuthology.orchestra.run.smithi096.stdout:4435 write 0xe091bad thru 0xe092543 (0x997 bytes) 2024-04-24T15:33:15.429 INFO:teuthology.orchestra.run.smithi096.stdout:4436 write 0xc484742 thru 0xc48ae1d (0x66dc bytes) 2024-04-24T15:33:15.430 INFO:teuthology.orchestra.run.smithi096.stdout:4438 read 0x3cae924 thru 0x3cb10eb (0x27c8 bytes) 2024-04-24T15:33:15.431 INFO:teuthology.orchestra.run.smithi096.stdout:4439 write 0xdcc6e47 thru 0xdccb8f4 (0x4aae bytes) 2024-04-24T15:33:15.432 INFO:teuthology.orchestra.run.smithi096.stdout:4440 write 0x8d357b2 thru 0x8d3bfde (0x682d bytes) 2024-04-24T15:33:15.435 INFO:teuthology.orchestra.run.smithi096.stdout:4441 punch from 0xc569bbd to 0xc573b94, (0x9fd7 bytes) 2024-04-24T15:33:15.435 INFO:teuthology.orchestra.run.smithi096.stdout:4442 write 0x4466ce4 thru 0x446b936 (0x4c53 bytes) 2024-04-24T15:33:15.436 INFO:teuthology.orchestra.run.smithi096.stdout:4443 write 0x2d56e2a thru 0x2d65259 (0xe430 bytes) 2024-04-24T15:33:15.439 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:15.439 INFO:teuthology.orchestra.run.smithi096.stdout:4448 read 0x9414a0a thru 0x941ed2f (0xa326 bytes) 2024-04-24T15:33:15.439 INFO:teuthology.orchestra.run.smithi096.stdout:4449 read 0x6457db3 thru 0x646572a (0xd978 bytes) 2024-04-24T15:33:15.439 INFO:teuthology.orchestra.run.smithi096.stdout:4450 write 0x4c1b546 thru 0x4c255bf (0xa07a bytes) 2024-04-24T15:33:15.441 INFO:teuthology.orchestra.run.smithi096.stdout:4451 writesame 0xab453f7 thru 0xab4741f data_size 0x1(0x2029 bytes) 2024-04-24T15:33:15.445 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:15.445 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:33:15.445 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-04-24T15:33:15.445 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-04-24T15:33:15.445 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:15.446 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:15.448 INFO:teuthology.orchestra.run.smithi096.stdout:4455 trunc from 0xe092544 to 0xd075a18 2024-04-24T15:33:15.457 INFO:teuthology.orchestra.run.smithi096.stdout:4456 read 0x508f25e thru 0x509dc03 (0xe9a6 bytes) 2024-04-24T15:33:15.457 INFO:teuthology.orchestra.run.smithi096.stdout:4459 punch from 0xa76226 to 0xa7f0ce, (0x8ea8 bytes) 2024-04-24T15:33:15.457 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:15.457 INFO:teuthology.orchestra.run.smithi096.stdout:4464 read 0x8347a24 thru 0x83561f6 (0xe7d3 bytes) 2024-04-24T15:33:15.457 INFO:teuthology.orchestra.run.smithi096.stdout:4467 write 0xcc35a8 thru 0xcc9820 (0x6279 bytes) 2024-04-24T15:33:15.458 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-24T15:33:15.459 INFO:teuthology.orchestra.run.smithi096.stdout:4468 write 0x7cda92d thru 0x7cded66 (0x443a bytes) 2024-04-24T15:33:15.462 INFO:teuthology.orchestra.run.smithi096.stdout:4469 punch from 0x8c60598 to 0x8c6846d, (0x7ed5 bytes) 2024-04-24T15:33:15.462 INFO:teuthology.orchestra.run.smithi096.stdout:4470 writesame 0x92ff723 thru 0x930682e data_size 0xa(0x710c bytes) 2024-04-24T15:33:15.466 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-6 --journal-path /var/lib/ceph/osd/ceph-6/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.4.12.6 2024-04-24T15:33:15.468 INFO:teuthology.orchestra.run.smithi096.stdout:4471 write 0x66672bb thru 0x6675a6b (0xe7b1 bytes) 2024-04-24T15:33:15.470 INFO:teuthology.orchestra.run.smithi096.stdout:4474 write 0x2c42443 thru 0x2c445e9 (0x21a7 bytes) 2024-04-24T15:33:15.471 INFO:teuthology.orchestra.run.smithi096.stdout:4477 write 0x9c01cd2 thru 0x9c03b87 (0x1eb6 bytes) 2024-04-24T15:33:15.475 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:15.475 INFO:teuthology.orchestra.run.smithi096.stdout:4481 read 0x3b5b935 thru 0x3b5d81b (0x1ee7 bytes) 2024-04-24T15:33:15.475 INFO:teuthology.orchestra.run.smithi096.stdout:4484 read 0x1348305 thru 0x1353b5c (0xb858 bytes) 2024-04-24T15:33:15.475 INFO:teuthology.orchestra.run.smithi096.stdout:4485 read 0x28e219c thru 0x28e9b99 (0x79fe bytes) 2024-04-24T15:33:15.476 INFO:teuthology.orchestra.run.smithi096.stdout:4487 punch from 0xfa03cd to 0xfae151, (0xdd84 bytes) 2024-04-24T15:33:15.476 INFO:teuthology.orchestra.run.smithi096.stdout:4488 write 0xcfac5f5 thru 0xcfb1a5c (0x5468 bytes) 2024-04-24T15:33:15.477 INFO:teuthology.orchestra.run.smithi096.stdout:4490 read 0xc7aa165 thru 0xc7b67fc (0xc698 bytes) 2024-04-24T15:33:15.478 INFO:teuthology.orchestra.run.smithi096.stdout:4491 read 0xc7451b6 thru 0xc752733 (0xd57e bytes) 2024-04-24T15:33:15.478 INFO:teuthology.orchestra.run.smithi096.stdout:4492 read 0x15e1ed3 thru 0x15efec0 (0xdfee bytes) 2024-04-24T15:33:15.478 INFO:teuthology.orchestra.run.smithi096.stdout:4493 read 0xccc1867 thru 0xccc8168 (0x6902 bytes) 2024-04-24T15:33:15.479 INFO:teuthology.orchestra.run.smithi096.stdout:4495 punch from 0x260e960 to 0x2610ac4, (0x2164 bytes) 2024-04-24T15:33:15.479 INFO:teuthology.orchestra.run.smithi096.stdout:4496 punch from 0x82bd09f to 0x82cce21, (0xfd82 bytes) 2024-04-24T15:33:15.479 INFO:teuthology.orchestra.run.smithi096.stdout:4497 write 0xae44be3 thru 0xae4c2c3 (0x76e1 bytes) 2024-04-24T15:33:15.482 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:15.482 INFO:teuthology.orchestra.run.smithi096.stdout:4499 write 0x8d74f0f thru 0x8d810b6 (0xc1a8 bytes) 2024-04-24T15:33:15.484 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:15.501 INFO:teuthology.orchestra.run.smithi096.stdout:4502 write 0xe454064 thru 0xe45e50a (0xa4a7 bytes) 2024-04-24T15:33:15.502 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:15.502 INFO:teuthology.orchestra.run.smithi096.stdout:4504 write 0x69c136c thru 0x69c1b35 (0x7ca bytes) 2024-04-24T15:33:15.504 INFO:teuthology.orchestra.run.smithi096.stdout:4505 read 0x608c7b7 thru 0x609c1a3 (0xf9ed bytes) 2024-04-24T15:33:15.504 INFO:teuthology.orchestra.run.smithi096.stdout:4506 punch from 0x975f7f3 to 0x976b201, (0xba0e bytes) 2024-04-24T15:33:15.505 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:15.505 INFO:teuthology.orchestra.run.smithi096.stdout:4510 write 0x8c0345e thru 0x8c0dd15 (0xa8b8 bytes) 2024-04-24T15:33:15.508 INFO:teuthology.orchestra.run.smithi096.stdout:4511 punch from 0xdc0cd46 to 0xdc11e8f, (0x5149 bytes) 2024-04-24T15:33:15.508 INFO:teuthology.orchestra.run.smithi096.stdout:4512 writesame 0x4d0f2b4 thru 0x4d18a99 data_size 0x3(0x97e6 bytes) 2024-04-24T15:33:15.516 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:15.516 INFO:teuthology.orchestra.run.smithi096.stdout:4514 writesame 0xe364b1 thru 0xe4259b data_size 0x1d(0xc0eb bytes) 2024-04-24T15:33:15.518 INFO:teuthology.orchestra.run.smithi096.stdout:4515 write 0x454f26e thru 0x45539d0 (0x4763 bytes) 2024-04-24T15:33:15.519 INFO:teuthology.orchestra.run.smithi096.stdout:4516 punch from 0x8437e6c to 0x844301b, (0xb1af bytes) 2024-04-24T15:33:15.519 INFO:teuthology.orchestra.run.smithi096.stdout:4517 read 0x3d8c315 thru 0x3d8da85 (0x1771 bytes) 2024-04-24T15:33:15.529 INFO:teuthology.orchestra.run.smithi096.stdout:4518 trunc from 0xe45e50b to 0x61e1b0c 2024-04-24T15:33:15.548 INFO:teuthology.orchestra.run.smithi096.stdout:4519 write 0x2158594 thru 0x215a63d (0x20aa bytes) 2024-04-24T15:33:15.550 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:15.551+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:15.555 INFO:teuthology.orchestra.run.smithi096.stdout:4520 trunc from 0x61e1b0c to 0xad592e7 2024-04-24T15:33:15.562 INFO:teuthology.orchestra.run.smithi096.stdout:4523 write 0x48d969c thru 0x48e672e (0xd093 bytes) 2024-04-24T15:33:15.564 INFO:teuthology.orchestra.run.smithi096.stdout:4524 read 0x1548cc4 thru 0x1551825 (0x8b62 bytes) 2024-04-24T15:33:15.564 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:15.569 INFO:teuthology.orchestra.run.smithi096.stdout:4526 trunc from 0xad592e7 to 0x6365811 2024-04-24T15:33:15.598 INFO:teuthology.orchestra.run.smithi096.stdout:4527 write 0xd0afd0b thru 0xd0bda1f (0xdd15 bytes) 2024-04-24T15:33:15.600 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:15.600 INFO:teuthology.orchestra.run.smithi096.stdout:4529 read 0x31ac918 thru 0x31b8ade (0xc1c7 bytes) 2024-04-24T15:33:15.601 INFO:teuthology.orchestra.run.smithi096.stdout:4530 read 0x457495c thru 0x45847e9 (0xfe8e bytes) 2024-04-24T15:33:15.601 INFO:teuthology.orchestra.run.smithi096.stdout:4531 read 0x64e0323 thru 0x64e29f1 (0x26cf bytes) 2024-04-24T15:33:15.602 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:15.605 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:15.605 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-04-24T15:33:15.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-04-24T15:33:15.605 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-04-24T15:33:15.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.605 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:2522 4.6 4:60ceedf0:::rbd_data.114a1f60cb66.000000000000011d:head [set-alloc-hint object_size 262144 write_size 262144,write 110592~2048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-24T15:33:15.606 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:23.088240+0000", 2024-04-24T15:33:15.606 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 52.516775596000002, 2024-04-24T15:33:15.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.988232797, 2024-04-24T15:33:15.606 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:15.606 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:15.606 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:15.606 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:15.606 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:15.606 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2522 2024-04-24T15:33:15.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.606 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:15.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:15.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088240+0000", 2024-04-24T15:33:15.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:15.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:15.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088240+0000", 2024-04-24T15:33:15.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:15.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:15.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088238+0000", 2024-04-24T15:33:15.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999981 2024-04-24T15:33:15.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:15.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088254+0000", 2024-04-24T15:33:15.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5562000000000001e-05 2024-04-24T15:33:15.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:15.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088275+0000", 2024-04-24T15:33:15.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1256000000000001e-05 2024-04-24T15:33:15.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:15.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088283+0000", 2024-04-24T15:33:15.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3219999999999993e-06 2024-04-24T15:33:15.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:15.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088351+0000", 2024-04-24T15:33:15.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7595000000000006e-05 2024-04-24T15:33:15.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-04-24T15:33:15.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088362+0000", 2024-04-24T15:33:15.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1764e-05 2024-04-24T15:33:15.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:15.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.073748+0000", 2024-04-24T15:33:15.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98538541899999998 2024-04-24T15:33:15.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:15.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.073925+0000", 2024-04-24T15:33:15.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017747600000000001 2024-04-24T15:33:15.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-24T15:33:15.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.074092+0000", 2024-04-24T15:33:15.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000166622 2024-04-24T15:33:15.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:33:15.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.074774+0000", 2024-04-24T15:33:15.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00068225000000000002 2024-04-24T15:33:15.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:33:15.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.076412+0000", 2024-04-24T15:33:15.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001638189 2024-04-24T15:33:15.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:15.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.076446+0000", 2024-04-24T15:33:15.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3587000000000002e-05 2024-04-24T15:33:15.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:15.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.076473+0000", 2024-04-24T15:33:15.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6886999999999999e-05 2024-04-24T15:33:15.611 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:15.611 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:15.611 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:15.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.612 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43108 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 327680~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:15.612 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.093966+0000", 2024-04-24T15:33:15.612 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 8.5110493280000004, 2024-04-24T15:33:15.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.030814699000000001, 2024-04-24T15:33:15.612 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:15.612 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:15.612 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:15.612 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:15.612 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:15.612 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43108 2024-04-24T15:33:15.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.612 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:15.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:15.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.093966+0000", 2024-04-24T15:33:15.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:15.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:15.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.093966+0000", 2024-04-24T15:33:15.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:15.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:15.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.093966+0000", 2024-04-24T15:33:15.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:15.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:15.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.093968+0000", 2024-04-24T15:33:15.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2120000000000002e-06 2024-04-24T15:33:15.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.614 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:15.614 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.093981+0000", 2024-04-24T15:33:15.614 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2797e-05 2024-04-24T15:33:15.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.614 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:15.614 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.093985+0000", 2024-04-24T15:33:15.614 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0539999999999996e-06 2024-04-24T15:33:15.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.614 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:15.614 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.124662+0000", 2024-04-24T15:33:15.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.030676717999999999 2024-04-24T15:33:15.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:15.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.124712+0000", 2024-04-24T15:33:15.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0523999999999999e-05 2024-04-24T15:33:15.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:15.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.124781+0000", 2024-04-24T15:33:15.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8753999999999999e-05 2024-04-24T15:33:15.615 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:15.615 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:15.615 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:15.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.616 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43109 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 753664~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:15.616 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.093990+0000", 2024-04-24T15:33:15.616 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 8.5110258309999995, 2024-04-24T15:33:15.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.031011216000000001, 2024-04-24T15:33:15.616 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:15.616 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:15.616 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:15.616 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:15.616 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:15.616 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43109 2024-04-24T15:33:15.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.616 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:15.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:15.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.093990+0000", 2024-04-24T15:33:15.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:15.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:15.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.093990+0000", 2024-04-24T15:33:15.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:15.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:15.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.093990+0000", 2024-04-24T15:33:15.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:15.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.618 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:15.618 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.093992+0000", 2024-04-24T15:33:15.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8990000000000001e-06 2024-04-24T15:33:15.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.618 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:15.618 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.093998+0000", 2024-04-24T15:33:15.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7019999999999998e-06 2024-04-24T15:33:15.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.618 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:15.618 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094002+0000", 2024-04-24T15:33:15.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5829999999999998e-06 2024-04-24T15:33:15.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:15.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.124822+0000", 2024-04-24T15:33:15.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.030820276000000001 2024-04-24T15:33:15.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:15.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.124909+0000", 2024-04-24T15:33:15.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6762999999999996e-05 2024-04-24T15:33:15.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:15.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.125001+0000", 2024-04-24T15:33:15.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2356999999999997e-05 2024-04-24T15:33:15.619 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:15.620 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:15.620 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:15.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.620 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43110 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 1179648~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:15.620 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094006+0000", 2024-04-24T15:33:15.620 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 8.5110096090000003, 2024-04-24T15:33:15.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.031113297000000002, 2024-04-24T15:33:15.620 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:15.620 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:15.620 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:15.620 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:15.620 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:15.620 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43110 2024-04-24T15:33:15.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.621 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:15.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.621 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:15.621 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094006+0000", 2024-04-24T15:33:15.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:15.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.621 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:15.621 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094006+0000", 2024-04-24T15:33:15.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:15.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.622 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:15.622 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094006+0000", 2024-04-24T15:33:15.622 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:15.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.622 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.622 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:15.622 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094008+0000", 2024-04-24T15:33:15.622 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6089999999999999e-06 2024-04-24T15:33:15.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.622 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.622 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:15.622 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094014+0000", 2024-04-24T15:33:15.622 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3940000000000004e-06 2024-04-24T15:33:15.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.623 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:15.623 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094017+0000", 2024-04-24T15:33:15.623 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4709999999999999e-06 2024-04-24T15:33:15.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.623 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:15.623 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.125013+0000", 2024-04-24T15:33:15.623 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.030995742 2024-04-24T15:33:15.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.623 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:15.623 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.125060+0000", 2024-04-24T15:33:15.623 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6894999999999997e-05 2024-04-24T15:33:15.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.624 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:15.624 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.125119+0000", 2024-04-24T15:33:15.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9521000000000002e-05 2024-04-24T15:33:15.624 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:15.624 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:15.624 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:15.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.624 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43111 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 1605632~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:15.624 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094022+0000", 2024-04-24T15:33:15.624 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 8.5109939079999997, 2024-04-24T15:33:15.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.031292357999999999, 2024-04-24T15:33:15.624 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:15.625 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:15.625 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:15.625 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:15.625 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:15.625 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43111 2024-04-24T15:33:15.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.625 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:15.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.625 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:15.625 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094022+0000", 2024-04-24T15:33:15.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:15.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.625 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:15.625 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094022+0000", 2024-04-24T15:33:15.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:15.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.626 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:15.626 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094021+0000", 2024-04-24T15:33:15.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:15.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.626 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:15.626 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094024+0000", 2024-04-24T15:33:15.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2749999999999998e-06 2024-04-24T15:33:15.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.626 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:15.626 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094029+0000", 2024-04-24T15:33:15.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7270000000000004e-06 2024-04-24T15:33:15.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:15.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094033+0000", 2024-04-24T15:33:15.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3239999999999999e-06 2024-04-24T15:33:15.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:15.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.125138+0000", 2024-04-24T15:33:15.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.031105738000000001 2024-04-24T15:33:15.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:15.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.125216+0000", 2024-04-24T15:33:15.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7824000000000006e-05 2024-04-24T15:33:15.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:15.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.125314+0000", 2024-04-24T15:33:15.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7853e-05 2024-04-24T15:33:15.628 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:15.628 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:15.628 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:15.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.628 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43112 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 2031616~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:15.628 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094037+0000", 2024-04-24T15:33:15.628 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 8.510978927, 2024-04-24T15:33:15.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.031503482999999999, 2024-04-24T15:33:15.628 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:15.629 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:15.629 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:15.629 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:15.629 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:15.629 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43112 2024-04-24T15:33:15.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.629 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:15.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:15.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094037+0000", 2024-04-24T15:33:15.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:15.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:15.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094037+0000", 2024-04-24T15:33:15.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:15.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:15.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094036+0000", 2024-04-24T15:33:15.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:15.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:15.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094039+0000", 2024-04-24T15:33:15.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0590000000000001e-06 2024-04-24T15:33:15.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:15.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094044+0000", 2024-04-24T15:33:15.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7710000000000003e-06 2024-04-24T15:33:15.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:15.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094048+0000", 2024-04-24T15:33:15.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9419999999999997e-06 2024-04-24T15:33:15.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:15.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.125352+0000", 2024-04-24T15:33:15.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.031304037999999999 2024-04-24T15:33:15.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:15.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.125438+0000", 2024-04-24T15:33:15.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5681999999999997e-05 2024-04-24T15:33:15.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:15.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.125540+0000", 2024-04-24T15:33:15.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010229399999999999 2024-04-24T15:33:15.632 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:15.632 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:15.632 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:15.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.632 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43113 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 2457600~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:15.632 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094056+0000", 2024-04-24T15:33:15.633 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 8.5109593189999995, 2024-04-24T15:33:15.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.031639255999999998, 2024-04-24T15:33:15.633 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:15.633 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:15.633 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:15.633 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:15.633 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:15.633 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43113 2024-04-24T15:33:15.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.633 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:15.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.633 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:15.633 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094056+0000", 2024-04-24T15:33:15.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:15.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:15.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094056+0000", 2024-04-24T15:33:15.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:15.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:15.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094056+0000", 2024-04-24T15:33:15.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:15.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:15.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094058+0000", 2024-04-24T15:33:15.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2510000000000001e-06 2024-04-24T15:33:15.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:15.635 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094064+0000", 2024-04-24T15:33:15.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0070000000000003e-06 2024-04-24T15:33:15.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:15.635 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094068+0000", 2024-04-24T15:33:15.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.912e-06 2024-04-24T15:33:15.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:15.635 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.125568+0000", 2024-04-24T15:33:15.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.031499857999999999 2024-04-24T15:33:15.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:15.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.125629+0000", 2024-04-24T15:33:15.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0877e-05 2024-04-24T15:33:15.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:15.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.125696+0000", 2024-04-24T15:33:15.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6680999999999994e-05 2024-04-24T15:33:15.636 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:15.636 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:15.636 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:15.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.637 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43114 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 2883584~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:15.637 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094073+0000", 2024-04-24T15:33:15.637 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 8.5109430390000007, 2024-04-24T15:33:15.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.031834029999999999, 2024-04-24T15:33:15.637 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:15.637 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:15.637 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:15.637 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:15.637 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:15.637 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43114 2024-04-24T15:33:15.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.637 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:15.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.637 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:15.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094073+0000", 2024-04-24T15:33:15.638 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:15.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.638 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:15.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094073+0000", 2024-04-24T15:33:15.638 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:15.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.638 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:15.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094072+0000", 2024-04-24T15:33:15.638 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:15.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.639 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:15.639 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094074+0000", 2024-04-24T15:33:15.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0899999999999999e-06 2024-04-24T15:33:15.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.639 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:15.639 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094080+0000", 2024-04-24T15:33:15.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4949999999999999e-06 2024-04-24T15:33:15.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.639 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:15.639 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094084+0000", 2024-04-24T15:33:15.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2570000000000001e-06 2024-04-24T15:33:15.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:15.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.125730+0000", 2024-04-24T15:33:15.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.031646035000000003 2024-04-24T15:33:15.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:15.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.125815+0000", 2024-04-24T15:33:15.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4628000000000004e-05 2024-04-24T15:33:15.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:15.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.125907+0000", 2024-04-24T15:33:15.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1848000000000006e-05 2024-04-24T15:33:15.641 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:15.641 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:15.641 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:15.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.641 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43115 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 3309568~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:15.641 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094088+0000", 2024-04-24T15:33:15.641 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 8.510927379, 2024-04-24T15:33:15.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.031951305999999999, 2024-04-24T15:33:15.641 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:15.641 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:15.641 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:15.641 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:15.642 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:15.642 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43115 2024-04-24T15:33:15.642 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.642 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:15.642 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.642 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:15.642 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094088+0000", 2024-04-24T15:33:15.642 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:15.642 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.642 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.642 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:15.642 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094088+0000", 2024-04-24T15:33:15.643 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:15.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.643 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:15.643 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094088+0000", 2024-04-24T15:33:15.643 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:15.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.643 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:15.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094090+0000", 2024-04-24T15:33:15.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2189999999999998e-06 2024-04-24T15:33:15.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:15.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094096+0000", 2024-04-24T15:33:15.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9560000000000002e-06 2024-04-24T15:33:15.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:15.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094100+0000", 2024-04-24T15:33:15.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5710000000000002e-06 2024-04-24T15:33:15.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:15.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.125924+0000", 2024-04-24T15:33:15.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.031823885000000003 2024-04-24T15:33:15.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:15.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.125986+0000", 2024-04-24T15:33:15.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2558999999999995e-05 2024-04-24T15:33:15.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.646 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:15.646 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.126040+0000", 2024-04-24T15:33:15.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3439000000000001e-05 2024-04-24T15:33:15.646 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:15.646 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:15.646 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:15.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.646 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43116 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 3735552~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:15.647 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094104+0000", 2024-04-24T15:33:15.647 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 8.5109117790000006, 2024-04-24T15:33:15.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032126963000000001, 2024-04-24T15:33:15.647 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:15.647 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:15.647 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:15.647 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:15.647 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:15.647 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43116 2024-04-24T15:33:15.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.647 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:15.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.648 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:15.648 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094104+0000", 2024-04-24T15:33:15.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:15.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.648 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:15.648 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094104+0000", 2024-04-24T15:33:15.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:15.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.648 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:15.648 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094104+0000", 2024-04-24T15:33:15.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:15.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:15.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094106+0000", 2024-04-24T15:33:15.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.99e-06 2024-04-24T15:33:15.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:15.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094111+0000", 2024-04-24T15:33:15.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2739999999999998e-06 2024-04-24T15:33:15.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:15.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094114+0000", 2024-04-24T15:33:15.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2729999999999998e-06 2024-04-24T15:33:15.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.650 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:15.650 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.126069+0000", 2024-04-24T15:33:15.650 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.031954713000000003 2024-04-24T15:33:15.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.650 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:15.650 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.126147+0000", 2024-04-24T15:33:15.650 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7706000000000004e-05 2024-04-24T15:33:15.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.650 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:15.651 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.126231+0000", 2024-04-24T15:33:15.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4331000000000003e-05 2024-04-24T15:33:15.651 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:15.651 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:15.651 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:15.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.651 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43117 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 4161536~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:15.651 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094118+0000", 2024-04-24T15:33:15.651 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 8.5108976070000004, 2024-04-24T15:33:15.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032239986999999998, 2024-04-24T15:33:15.651 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:15.651 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:15.651 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:15.652 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:15.652 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:15.652 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43117 2024-04-24T15:33:15.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.652 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:15.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:15.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094118+0000", 2024-04-24T15:33:15.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:15.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:15.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094118+0000", 2024-04-24T15:33:15.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:15.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:15.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094118+0000", 2024-04-24T15:33:15.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:15.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:15.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094120+0000", 2024-04-24T15:33:15.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1500000000000002e-06 2024-04-24T15:33:15.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:15.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094126+0000", 2024-04-24T15:33:15.654 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9490000000000001e-06 2024-04-24T15:33:15.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.654 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:15.654 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094129+0000", 2024-04-24T15:33:15.654 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6009999999999999e-06 2024-04-24T15:33:15.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.654 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:15.654 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.126248+0000", 2024-04-24T15:33:15.654 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032118531999999998 2024-04-24T15:33:15.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.654 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:15.655 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.126306+0000", 2024-04-24T15:33:15.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7843999999999997e-05 2024-04-24T15:33:15.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.655 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:15.655 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.126358+0000", 2024-04-24T15:33:15.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2240999999999997e-05 2024-04-24T15:33:15.655 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:15.655 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:15.655 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:15.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.655 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43118 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 4587520~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:15.656 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094134+0000", 2024-04-24T15:33:15.656 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 8.5108821940000006, 2024-04-24T15:33:15.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032417003, 2024-04-24T15:33:15.656 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:15.656 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:15.656 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:15.656 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:15.656 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:15.656 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43118 2024-04-24T15:33:15.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.656 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:15.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.656 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:15.656 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094134+0000", 2024-04-24T15:33:15.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:15.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:15.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094134+0000", 2024-04-24T15:33:15.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:15.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:15.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094133+0000", 2024-04-24T15:33:15.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:15.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:15.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094135+0000", 2024-04-24T15:33:15.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0590000000000001e-06 2024-04-24T15:33:15.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:15.658 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094141+0000", 2024-04-24T15:33:15.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3190000000000002e-06 2024-04-24T15:33:15.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:15.658 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094144+0000", 2024-04-24T15:33:15.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2430000000000001e-06 2024-04-24T15:33:15.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.659 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:15.659 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.126387+0000", 2024-04-24T15:33:15.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032243124999999997 2024-04-24T15:33:15.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.659 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:15.659 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.126464+0000", 2024-04-24T15:33:15.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6607999999999996e-05 2024-04-24T15:33:15.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.659 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:15.659 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.126551+0000", 2024-04-24T15:33:15.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6953000000000006e-05 2024-04-24T15:33:15.659 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:15.659 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:15.660 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:15.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.660 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43119 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 5013504~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:15.660 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094148+0000", 2024-04-24T15:33:15.660 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 8.510867953, 2024-04-24T15:33:15.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032531142999999998, 2024-04-24T15:33:15.660 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:15.660 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:15.660 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:15.660 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:15.660 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:15.660 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43119 2024-04-24T15:33:15.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.661 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:15.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:15.661 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094148+0000", 2024-04-24T15:33:15.661 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:15.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:15.661 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094148+0000", 2024-04-24T15:33:15.661 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:15.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:15.661 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094147+0000", 2024-04-24T15:33:15.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:15.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.662 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:15.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094150+0000", 2024-04-24T15:33:15.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1100000000000001e-06 2024-04-24T15:33:15.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.662 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:15.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094155+0000", 2024-04-24T15:33:15.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4480000000000002e-06 2024-04-24T15:33:15.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.663 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:15.663 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094159+0000", 2024-04-24T15:33:15.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5499999999999999e-06 2024-04-24T15:33:15.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.663 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:15.663 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.126567+0000", 2024-04-24T15:33:15.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032407997000000001 2024-04-24T15:33:15.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.663 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:15.663 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.126622+0000", 2024-04-24T15:33:15.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5847000000000002e-05 2024-04-24T15:33:15.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.664 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:15.664 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.126679+0000", 2024-04-24T15:33:15.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6508999999999997e-05 2024-04-24T15:33:15.664 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:15.664 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:15.664 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:15.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.664 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43120 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 5439488~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:15.664 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094163+0000", 2024-04-24T15:33:15.664 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 8.5108528440000004, 2024-04-24T15:33:15.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032680305999999999, 2024-04-24T15:33:15.664 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:15.664 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:15.665 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:15.665 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:15.665 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:15.665 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43120 2024-04-24T15:33:15.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.665 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:15.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:15.665 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094163+0000", 2024-04-24T15:33:15.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:15.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:15.666 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094163+0000", 2024-04-24T15:33:15.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:15.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.666 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:15.666 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094163+0000", 2024-04-24T15:33:15.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:15.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.666 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:15.666 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094165+0000", 2024-04-24T15:33:15.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.181e-06 2024-04-24T15:33:15.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.667 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:15.667 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094170+0000", 2024-04-24T15:33:15.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6019999999999999e-06 2024-04-24T15:33:15.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.667 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:15.667 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094174+0000", 2024-04-24T15:33:15.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4410000000000002e-06 2024-04-24T15:33:15.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.667 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:15.667 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.126731+0000", 2024-04-24T15:33:15.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032557599999999999 2024-04-24T15:33:15.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.668 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:15.668 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.126783+0000", 2024-04-24T15:33:15.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1378000000000002e-05 2024-04-24T15:33:15.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.668 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:15.668 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.126843+0000", 2024-04-24T15:33:15.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0433000000000002e-05 2024-04-24T15:33:15.668 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:15.668 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:15.668 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:15.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.669 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43121 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 5865472~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:15.669 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094178+0000", 2024-04-24T15:33:15.669 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 8.5108379129999996, 2024-04-24T15:33:15.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032860598999999997, 2024-04-24T15:33:15.669 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:15.669 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:15.669 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:15.669 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:15.669 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:15.669 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43121 2024-04-24T15:33:15.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.669 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:15.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:15.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094178+0000", 2024-04-24T15:33:15.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:15.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:15.670 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094178+0000", 2024-04-24T15:33:15.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:15.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:15.670 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094178+0000", 2024-04-24T15:33:15.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:15.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:15.671 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094180+0000", 2024-04-24T15:33:15.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9989999999999998e-06 2024-04-24T15:33:15.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.671 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:15.671 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094185+0000", 2024-04-24T15:33:15.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.541e-06 2024-04-24T15:33:15.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.671 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:15.671 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094189+0000", 2024-04-24T15:33:15.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0509999999999998e-06 2024-04-24T15:33:15.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.672 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:15.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.126877+0000", 2024-04-24T15:33:15.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032687473000000002 2024-04-24T15:33:15.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.672 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:15.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.126952+0000", 2024-04-24T15:33:15.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5924e-05 2024-04-24T15:33:15.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.672 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:15.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127038+0000", 2024-04-24T15:33:15.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5908999999999994e-05 2024-04-24T15:33:15.672 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:15.673 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:15.673 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:15.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.673 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43122 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 6291456~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:15.673 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094193+0000", 2024-04-24T15:33:15.673 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 8.5108226689999995, 2024-04-24T15:33:15.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032978383999999999, 2024-04-24T15:33:15.673 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:15.673 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:15.673 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:15.673 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:15.673 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:15.673 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43122 2024-04-24T15:33:15.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.674 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:15.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.674 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:15.674 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094193+0000", 2024-04-24T15:33:15.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:15.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.674 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:15.674 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094193+0000", 2024-04-24T15:33:15.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:15.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:15.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094193+0000", 2024-04-24T15:33:15.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:15.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:15.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094195+0000", 2024-04-24T15:33:15.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3269999999999999e-06 2024-04-24T15:33:15.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:15.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094201+0000", 2024-04-24T15:33:15.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9850000000000003e-06 2024-04-24T15:33:15.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:15.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094205+0000", 2024-04-24T15:33:15.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8430000000000003e-06 2024-04-24T15:33:15.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:15.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127068+0000", 2024-04-24T15:33:15.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032863305000000002 2024-04-24T15:33:15.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:15.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127113+0000", 2024-04-24T15:33:15.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4412000000000001e-05 2024-04-24T15:33:15.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.677 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:15.677 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127171+0000", 2024-04-24T15:33:15.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8807999999999997e-05 2024-04-24T15:33:15.677 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:15.677 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:15.677 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:15.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.677 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43123 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 6717440~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:15.677 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094209+0000", 2024-04-24T15:33:15.677 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 8.5108065029999995, 2024-04-24T15:33:15.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033144583999999998, 2024-04-24T15:33:15.678 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:15.678 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:15.678 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:15.678 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:15.678 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:15.678 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43123 2024-04-24T15:33:15.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.678 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:15.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:15.678 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094209+0000", 2024-04-24T15:33:15.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:15.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.679 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:15.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094209+0000", 2024-04-24T15:33:15.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:15.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.679 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:15.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094209+0000", 2024-04-24T15:33:15.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:15.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.679 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:15.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094211+0000", 2024-04-24T15:33:15.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2089999999999999e-06 2024-04-24T15:33:15.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.680 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:15.680 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094217+0000", 2024-04-24T15:33:15.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6799999999999998e-06 2024-04-24T15:33:15.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.680 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:15.680 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094220+0000", 2024-04-24T15:33:15.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.444e-06 2024-04-24T15:33:15.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.680 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:15.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127190+0000", 2024-04-24T15:33:15.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032969471 2024-04-24T15:33:15.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:15.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127269+0000", 2024-04-24T15:33:15.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9246999999999995e-05 2024-04-24T15:33:15.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:15.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127354+0000", 2024-04-24T15:33:15.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4853999999999998e-05 2024-04-24T15:33:15.681 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:15.682 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:15.682 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:15.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.682 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.682 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43124 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 7143424~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:15.682 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094224+0000", 2024-04-24T15:33:15.682 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 8.5107915300000005, 2024-04-24T15:33:15.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033271955999999998, 2024-04-24T15:33:15.682 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:15.682 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:15.682 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:15.682 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:15.682 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:15.683 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43124 2024-04-24T15:33:15.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.683 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:15.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.683 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:15.683 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094224+0000", 2024-04-24T15:33:15.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:15.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.683 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:15.683 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094224+0000", 2024-04-24T15:33:15.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:15.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:15.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094224+0000", 2024-04-24T15:33:15.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:15.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:15.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094226+0000", 2024-04-24T15:33:15.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9039999999999999e-06 2024-04-24T15:33:15.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:15.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094233+0000", 2024-04-24T15:33:15.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7970000000000003e-06 2024-04-24T15:33:15.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:15.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094236+0000", 2024-04-24T15:33:15.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6339999999999999e-06 2024-04-24T15:33:15.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:15.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127390+0000", 2024-04-24T15:33:15.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.03315353 2024-04-24T15:33:15.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:15.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127439+0000", 2024-04-24T15:33:15.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8890000000000001e-05 2024-04-24T15:33:15.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:15.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127496+0000", 2024-04-24T15:33:15.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7476999999999999e-05 2024-04-24T15:33:15.686 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:15.686 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:15.686 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:15.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.686 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43125 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 7569408~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:15.686 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094240+0000", 2024-04-24T15:33:15.686 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 8.510775443, 2024-04-24T15:33:15.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033446278000000003, 2024-04-24T15:33:15.686 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:15.686 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:15.687 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:15.687 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:15.687 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:15.687 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43125 2024-04-24T15:33:15.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.687 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:15.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.687 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:15.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094240+0000", 2024-04-24T15:33:15.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:15.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.687 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:15.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094240+0000", 2024-04-24T15:33:15.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:15.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:15.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094240+0000", 2024-04-24T15:33:15.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:15.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:15.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094247+0000", 2024-04-24T15:33:15.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6209999999999998e-06 2024-04-24T15:33:15.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:15.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094253+0000", 2024-04-24T15:33:15.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0739999999999998e-06 2024-04-24T15:33:15.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:15.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094256+0000", 2024-04-24T15:33:15.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7699999999999999e-06 2024-04-24T15:33:15.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:15.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127526+0000", 2024-04-24T15:33:15.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033269703999999997 2024-04-24T15:33:15.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:15.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127603+0000", 2024-04-24T15:33:15.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6964000000000004e-05 2024-04-24T15:33:15.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:15.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127687+0000", 2024-04-24T15:33:15.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3431999999999997e-05 2024-04-24T15:33:15.690 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:15.690 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:15.690 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:15.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.690 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43126 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 7995392~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:15.690 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094261+0000", 2024-04-24T15:33:15.690 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 8.51075494, 2024-04-24T15:33:15.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033555201999999999, 2024-04-24T15:33:15.691 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:15.691 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:15.691 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:15.691 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:15.691 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:15.691 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43126 2024-04-24T15:33:15.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.691 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:15.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:15.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094261+0000", 2024-04-24T15:33:15.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:15.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.692 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:15.692 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094261+0000", 2024-04-24T15:33:15.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:15.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.692 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:15.692 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094260+0000", 2024-04-24T15:33:15.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:15.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.692 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:15.692 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094263+0000", 2024-04-24T15:33:15.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2579999999999998e-06 2024-04-24T15:33:15.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:15.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094269+0000", 2024-04-24T15:33:15.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.823e-06 2024-04-24T15:33:15.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:15.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094272+0000", 2024-04-24T15:33:15.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8999999999999999e-06 2024-04-24T15:33:15.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:15.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127720+0000", 2024-04-24T15:33:15.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033447521000000001 2024-04-24T15:33:15.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.694 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:15.694 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127765+0000", 2024-04-24T15:33:15.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5352000000000002e-05 2024-04-24T15:33:15.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.694 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:15.694 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127816+0000", 2024-04-24T15:33:15.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0689999999999997e-05 2024-04-24T15:33:15.694 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:15.694 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:15.694 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:15.694 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:15.695 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:15.695 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:15.695 INFO:teuthology.orchestra.run.smithi096.stdout:4534 trunc from 0xd0bda20 to 0x26e25c0 2024-04-24T15:33:15.697 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-24T15:33:15.698 INFO:teuthology.orchestra.run.smithi096.stdout:4535 trunc from 0x26e25c0 to 0xb061799 2024-04-24T15:33:15.698 INFO:teuthology.orchestra.run.smithi096.stdout:4536 writesame 0xe1d35e0 thru 0xe1e094e data_size 0x1(0xd36f bytes) 2024-04-24T15:33:15.698 INFO:teuthology.orchestra.run.smithi096.stdout:4537 trunc from 0xe1e094f to 0x8fa2a05 2024-04-24T15:33:15.698 INFO:teuthology.orchestra.run.smithi096.stdout:4539 read 0x380831a thru 0x3809b84 (0x186b bytes) 2024-04-24T15:33:15.698 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:15.698 INFO:teuthology.orchestra.run.smithi096.stdout:4542 write 0x7d74be2 thru 0x7d82854 (0xdc73 bytes) 2024-04-24T15:33:15.698 INFO:teuthology.orchestra.run.smithi096.stdout:4543 write 0xbed4b29 thru 0xbedd708 (0x8be0 bytes) 2024-04-24T15:33:15.698 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:15.698 INFO:teuthology.orchestra.run.smithi096.stdout:4545 write 0xaa46f80 thru 0xaa51232 (0xa2b3 bytes) 2024-04-24T15:33:15.699 INFO:teuthology.orchestra.run.smithi096.stdout:4546 write 0xf46336 thru 0xf4e387 (0x8052 bytes) 2024-04-24T15:33:15.699 INFO:teuthology.orchestra.run.smithi096.stdout:4547 write 0xa938b97 thru 0xa945be7 (0xd051 bytes) 2024-04-24T15:33:15.699 INFO:teuthology.orchestra.run.smithi096.stdout:4548 trunc from 0xbedd709 to 0x4b07f45 2024-04-24T15:33:15.699 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:15.697+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:15.715 INFO:teuthology.orchestra.run.smithi096.stdout:4549 trunc from 0x4b07f45 to 0xbc6ac42 2024-04-24T15:33:15.721 INFO:teuthology.orchestra.run.smithi096.stdout:4550 writesame 0x9aa77ac thru 0x9ab2a3c data_size 0x1(0xb291 bytes) 2024-04-24T15:33:15.746 INFO:teuthology.orchestra.run.smithi096.stdout:4551 read 0x120f1b9 thru 0x121e4f6 (0xf33e bytes) 2024-04-24T15:33:15.747 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:15.747 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:15.747 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:15.747 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:15.747 INFO:teuthology.orchestra.run.smithi096.stdout:4557 read 0x835f8b6 thru 0x836ca91 (0xd1dc bytes) 2024-04-24T15:33:15.747 INFO:teuthology.orchestra.run.smithi096.stdout:4559 write 0x4564f97 thru 0x45725b8 (0xd622 bytes) 2024-04-24T15:33:15.749 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:15.749 INFO:teuthology.orchestra.run.smithi096.stdout:4562 writesame 0x8409cc4 thru 0x840cfc5 data_size 0x1(0x3302 bytes) 2024-04-24T15:33:15.755 INFO:teuthology.orchestra.run.smithi096.stdout:4563 write 0x3d765af thru 0x3d84cc9 (0xe71b bytes) 2024-04-24T15:33:15.756 INFO:teuthology.orchestra.run.smithi096.stdout:4564 write 0x408c7eb thru 0x408f827 (0x303d bytes) 2024-04-24T15:33:15.757 INFO:teuthology.orchestra.run.smithi096.stdout:4565 read 0x51b590a thru 0x51b87e4 (0x2edb bytes) 2024-04-24T15:33:15.768 INFO:teuthology.orchestra.run.smithi096.stdout:4566 trunc from 0xbc6ac42 to 0x2b455d6 2024-04-24T15:33:15.792 INFO:teuthology.orchestra.run.smithi096.stdout:4567 write 0x7dd3fb3 thru 0x7de2e83 (0xeed1 bytes) 2024-04-24T15:33:15.797 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:15.797+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:15.801 INFO:teuthology.orchestra.run.smithi096.stdout:4568 write 0x86774b9 thru 0x8678e33 (0x197b bytes) 2024-04-24T15:33:15.804 INFO:teuthology.orchestra.run.smithi096.stdout:4569 read 0x69946c5 thru 0x699fd29 (0xb665 bytes) 2024-04-24T15:33:15.804 INFO:teuthology.orchestra.run.smithi096.stdout:4571 writesame 0x132c27e thru 0x132dbdb data_size 0x2(0x195e bytes) 2024-04-24T15:33:15.806 INFO:teuthology.orchestra.run.smithi096.stdout:4573 read 0x456372c thru 0x4566cba (0x358f bytes) 2024-04-24T15:33:15.806 INFO:teuthology.orchestra.run.smithi096.stdout:4576 write 0x147b366 thru 0x148256c (0x7207 bytes) 2024-04-24T15:33:15.809 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:15.820 INFO:teuthology.orchestra.run.smithi096.stdout:4579 writesame 0xdbea441 thru 0xdbf8377 data_size 0x1(0xdf37 bytes) 2024-04-24T15:33:15.820 INFO:teuthology.orchestra.run.smithi096.stdout:4581 punch from 0x4cf98f0 to 0x4cff128, (0x5838 bytes) 2024-04-24T15:33:15.829 INFO:teuthology.orchestra.run.smithi096.stdout:4582 trunc from 0xdbf8378 to 0x5ef36f3 2024-04-24T15:33:15.846 INFO:teuthology.orchestra.run.smithi096.stdout:4583 read 0x5c1ec30 thru 0x5c256a3 (0x6a74 bytes) 2024-04-24T15:33:15.847 INFO:teuthology.orchestra.run.smithi096.stdout:4586 read 0x22aac0 thru 0x22fb68 (0x50a9 bytes) 2024-04-24T15:33:15.861 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:15.861 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-04-24T15:33:15.861 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.861 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:55355 4.1e 4.7f1751be (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-04-24T15:33:15.861 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:15.862915+0000", 2024-04-24T15:33:15.861 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.1989000000000002e-05, 2024-04-24T15:33:15.862 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.431e-05, 2024-04-24T15:33:15.862 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:15.862 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2024-04-24T15:33:15.862 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:15.862 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:15.862 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:15.862 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 55355 2024-04-24T15:33:15.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.862 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:15.862 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.862 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:15.862 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:15.862915+0000", 2024-04-24T15:33:15.862 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:15.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.862 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.863 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:15.863 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:15.862915+0000", 2024-04-24T15:33:15.863 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:15.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.863 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.863 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:15.863 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:15.862914+0000", 2024-04-24T15:33:15.863 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:15.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.863 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.863 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:15.863 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:15.862918+0000", 2024-04-24T15:33:15.863 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5870000000000001e-06 2024-04-24T15:33:15.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.863 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.863 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:15.864 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:15.862925+0000", 2024-04-24T15:33:15.864 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9580000000000001e-06 2024-04-24T15:33:15.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.864 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.864 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:15.864 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:15.862928+0000", 2024-04-24T15:33:15.864 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3129999999999999e-06 2024-04-24T15:33:15.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:15.864 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:15.864 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:15.864 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:15.862958+0000", 2024-04-24T15:33:15.864 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9326999999999999e-05 2024-04-24T15:33:15.864 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:15.864 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:15.864 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:15.864 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:15.865 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-24T15:33:15.865 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 1 2024-04-24T15:33:15.865 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:15.865 INFO:teuthology.orchestra.run.smithi096.stdout:4587 write 0xe724d7f thru 0xe733ffd (0xf27f bytes) 2024-04-24T15:33:15.866 INFO:teuthology.orchestra.run.smithi096.stdout:4588 write 0x749e4ef thru 0x74a529a (0x6dac bytes) 2024-04-24T15:33:15.868 INFO:teuthology.orchestra.run.smithi096.stdout:4589 writesame 0x728a5ff thru 0x7298375 data_size 0x11(0xdd77 bytes) 2024-04-24T15:33:15.870 INFO:teuthology.orchestra.run.smithi096.stdout:4591 clone 25 order 24 su 4194304 sc 7 2024-04-24T15:33:15.873 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-24T15:33:15.898 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:15.898+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:15.999 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:15.999+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:16.036 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:16.036 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:33:16.036 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-04-24T15:33:16.036 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-04-24T15:33:16.036 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:16.046 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-24T15:33:16.100 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:33:16.101+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:33:16.171 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:16.172 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-04-24T15:33:16.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-04-24T15:33:16.172 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-04-24T15:33:16.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.172 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14043 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2383872~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:16.172 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892347+0000", 2024-04-24T15:33:16.172 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 42.279237670000001, 2024-04-24T15:33:16.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038537279000000001, 2024-04-24T15:33:16.172 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:16.172 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:16.172 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:16.172 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:16.172 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:16.172 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14043 2024-04-24T15:33:16.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.173 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:16.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.173 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:16.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892347+0000", 2024-04-24T15:33:16.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:16.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.173 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:16.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892347+0000", 2024-04-24T15:33:16.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:16.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.173 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:16.174 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892347+0000", 2024-04-24T15:33:16.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:16.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.174 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:16.174 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892349+0000", 2024-04-24T15:33:16.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3400000000000001e-06 2024-04-24T15:33:16.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.174 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:16.174 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892357+0000", 2024-04-24T15:33:16.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1540000000000002e-06 2024-04-24T15:33:16.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:16.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892360+0000", 2024-04-24T15:33:16.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5829999999999998e-06 2024-04-24T15:33:16.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:16.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930795+0000", 2024-04-24T15:33:16.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038434590999999997 2024-04-24T15:33:16.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:16.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930836+0000", 2024-04-24T15:33:16.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0827000000000001e-05 2024-04-24T15:33:16.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:16.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930885+0000", 2024-04-24T15:33:16.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9029000000000001e-05 2024-04-24T15:33:16.176 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:16.176 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:16.176 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:16.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.176 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14044 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2424832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:16.176 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892372+0000", 2024-04-24T15:33:16.176 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 42.279213026000001, 2024-04-24T15:33:16.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038624890000000002, 2024-04-24T15:33:16.176 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:16.176 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:16.177 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:16.177 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:16.177 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:16.177 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14044 2024-04-24T15:33:16.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.177 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:16.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:16.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892372+0000", 2024-04-24T15:33:16.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:16.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:16.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892372+0000", 2024-04-24T15:33:16.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:16.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:16.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892372+0000", 2024-04-24T15:33:16.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:16.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:16.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892374+0000", 2024-04-24T15:33:16.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4899999999999999e-06 2024-04-24T15:33:16.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:16.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892380+0000", 2024-04-24T15:33:16.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.631e-06 2024-04-24T15:33:16.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.179 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:16.179 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892383+0000", 2024-04-24T15:33:16.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7400000000000002e-06 2024-04-24T15:33:16.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.179 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:16.179 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930905+0000", 2024-04-24T15:33:16.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038521396999999999 2024-04-24T15:33:16.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.179 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:16.179 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930949+0000", 2024-04-24T15:33:16.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4153999999999998e-05 2024-04-24T15:33:16.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:16.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930997+0000", 2024-04-24T15:33:16.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7998999999999999e-05 2024-04-24T15:33:16.180 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:16.180 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:16.180 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:16.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.180 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14045 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2465792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:16.180 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892387+0000", 2024-04-24T15:33:16.180 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 42.279197758999999, 2024-04-24T15:33:16.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038720933999999999, 2024-04-24T15:33:16.181 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:16.181 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:16.181 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:16.181 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:16.181 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:16.181 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14045 2024-04-24T15:33:16.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.181 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:16.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:16.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892387+0000", 2024-04-24T15:33:16.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:16.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:16.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892387+0000", 2024-04-24T15:33:16.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:16.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:16.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892387+0000", 2024-04-24T15:33:16.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:16.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:16.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892392+0000", 2024-04-24T15:33:16.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0039999999999999e-06 2024-04-24T15:33:16.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:16.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892398+0000", 2024-04-24T15:33:16.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0900000000000001e-06 2024-04-24T15:33:16.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:16.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892402+0000", 2024-04-24T15:33:16.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.399e-06 2024-04-24T15:33:16.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:16.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931018+0000", 2024-04-24T15:33:16.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038616551999999998 2024-04-24T15:33:16.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.184 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:16.184 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931059+0000", 2024-04-24T15:33:16.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1205000000000003e-05 2024-04-24T15:33:16.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.184 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:16.184 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931108+0000", 2024-04-24T15:33:16.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8996000000000003e-05 2024-04-24T15:33:16.184 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:16.184 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:16.184 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:16.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.185 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14046 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2506752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:16.185 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892406+0000", 2024-04-24T15:33:16.185 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 42.279179169000003, 2024-04-24T15:33:16.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038814507999999998, 2024-04-24T15:33:16.185 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:16.185 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:16.185 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:16.185 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:16.185 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:16.185 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14046 2024-04-24T15:33:16.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.185 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:16.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:16.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892406+0000", 2024-04-24T15:33:16.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:16.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:16.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892406+0000", 2024-04-24T15:33:16.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:16.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:16.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892406+0000", 2024-04-24T15:33:16.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:16.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:16.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892413+0000", 2024-04-24T15:33:16.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7449999999999995e-06 2024-04-24T15:33:16.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:16.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892419+0000", 2024-04-24T15:33:16.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5790000000000003e-06 2024-04-24T15:33:16.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:16.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892423+0000", 2024-04-24T15:33:16.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8199999999999998e-06 2024-04-24T15:33:16.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:16.188 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931128+0000", 2024-04-24T15:33:16.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038705502000000003 2024-04-24T15:33:16.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:16.188 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931172+0000", 2024-04-24T15:33:16.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4066999999999997e-05 2024-04-24T15:33:16.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:16.188 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931220+0000", 2024-04-24T15:33:16.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8167999999999997e-05 2024-04-24T15:33:16.189 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:16.189 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:16.189 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:16.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.189 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14047 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2547712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:16.189 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892427+0000", 2024-04-24T15:33:16.189 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 42.279157939999997, 2024-04-24T15:33:16.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038904506999999998, 2024-04-24T15:33:16.189 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:16.189 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:16.189 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:16.189 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:16.190 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:16.190 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14047 2024-04-24T15:33:16.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.190 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:16.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:16.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892427+0000", 2024-04-24T15:33:16.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:16.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:16.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892427+0000", 2024-04-24T15:33:16.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:16.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:16.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892427+0000", 2024-04-24T15:33:16.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:16.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:16.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892429+0000", 2024-04-24T15:33:16.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3099999999999999e-06 2024-04-24T15:33:16.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:16.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892435+0000", 2024-04-24T15:33:16.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5520000000000004e-06 2024-04-24T15:33:16.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:16.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892438+0000", 2024-04-24T15:33:16.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4240000000000002e-06 2024-04-24T15:33:16.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:16.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931242+0000", 2024-04-24T15:33:16.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038804162000000003 2024-04-24T15:33:16.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:16.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931284+0000", 2024-04-24T15:33:16.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1414e-05 2024-04-24T15:33:16.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:16.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931332+0000", 2024-04-24T15:33:16.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8024e-05 2024-04-24T15:33:16.193 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:16.193 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:16.193 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:16.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.193 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14048 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2588672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:16.193 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892442+0000", 2024-04-24T15:33:16.193 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 42.279142976999999, 2024-04-24T15:33:16.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039003132000000003, 2024-04-24T15:33:16.194 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:16.194 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:16.194 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:16.194 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:16.194 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:16.194 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14048 2024-04-24T15:33:16.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.194 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:16.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:16.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892442+0000", 2024-04-24T15:33:16.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:16.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:16.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892442+0000", 2024-04-24T15:33:16.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:16.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:16.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892442+0000", 2024-04-24T15:33:16.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:16.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:16.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892444+0000", 2024-04-24T15:33:16.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1169999999999998e-06 2024-04-24T15:33:16.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:16.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892449+0000", 2024-04-24T15:33:16.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1599999999999997e-06 2024-04-24T15:33:16.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:16.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892453+0000", 2024-04-24T15:33:16.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5930000000000002e-06 2024-04-24T15:33:16.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:16.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931352+0000", 2024-04-24T15:33:16.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038899361 2024-04-24T15:33:16.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:16.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931398+0000", 2024-04-24T15:33:16.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6315e-05 2024-04-24T15:33:16.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:16.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931445+0000", 2024-04-24T15:33:16.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6950000000000003e-05 2024-04-24T15:33:16.197 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:16.197 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:16.197 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:16.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.197 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14049 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2629632~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:16.198 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892458+0000", 2024-04-24T15:33:16.198 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 42.279127557000002, 2024-04-24T15:33:16.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039103322000000003, 2024-04-24T15:33:16.198 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:16.198 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:16.198 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:16.198 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:16.198 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:16.198 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14049 2024-04-24T15:33:16.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.198 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:16.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.198 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:16.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892458+0000", 2024-04-24T15:33:16.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:16.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:16.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892458+0000", 2024-04-24T15:33:16.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:16.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:16.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892457+0000", 2024-04-24T15:33:16.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:16.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:16.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892459+0000", 2024-04-24T15:33:16.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.305e-06 2024-04-24T15:33:16.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:16.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892465+0000", 2024-04-24T15:33:16.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4360000000000001e-06 2024-04-24T15:33:16.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:16.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892474+0000", 2024-04-24T15:33:16.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7639999999999994e-06 2024-04-24T15:33:16.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:16.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931466+0000", 2024-04-24T15:33:16.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038992644 2024-04-24T15:33:16.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:16.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931512+0000", 2024-04-24T15:33:16.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5200000000000001e-05 2024-04-24T15:33:16.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:16.202 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931561+0000", 2024-04-24T15:33:16.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9357999999999997e-05 2024-04-24T15:33:16.202 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:16.202 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:16.202 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:16.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.202 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14050 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2670592~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:16.202 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892478+0000", 2024-04-24T15:33:16.202 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 42.279106970000001, 2024-04-24T15:33:16.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039195102000000002, 2024-04-24T15:33:16.202 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:16.202 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:16.203 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:16.203 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:16.203 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:16.203 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14050 2024-04-24T15:33:16.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.203 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:16.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:16.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892478+0000", 2024-04-24T15:33:16.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:16.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:16.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892478+0000", 2024-04-24T15:33:16.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:16.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:16.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892478+0000", 2024-04-24T15:33:16.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:16.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:16.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892480+0000", 2024-04-24T15:33:16.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3300000000000001e-06 2024-04-24T15:33:16.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:16.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892486+0000", 2024-04-24T15:33:16.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9200000000000001e-06 2024-04-24T15:33:16.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.205 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:16.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892494+0000", 2024-04-24T15:33:16.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5159999999999997e-06 2024-04-24T15:33:16.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.205 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:16.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931581+0000", 2024-04-24T15:33:16.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039087894999999998 2024-04-24T15:33:16.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.205 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:16.206 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931626+0000", 2024-04-24T15:33:16.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4943e-05 2024-04-24T15:33:16.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.206 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.206 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:16.206 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931673+0000", 2024-04-24T15:33:16.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6906000000000001e-05 2024-04-24T15:33:16.206 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:16.206 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:16.206 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:16.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.206 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.206 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14051 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2711552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:16.206 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892507+0000", 2024-04-24T15:33:16.207 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 42.279078271000003, 2024-04-24T15:33:16.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039274393999999997, 2024-04-24T15:33:16.207 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:16.207 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:16.207 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:16.207 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:16.207 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:16.207 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14051 2024-04-24T15:33:16.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.207 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:16.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.207 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:16.207 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892507+0000", 2024-04-24T15:33:16.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:16.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:16.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892507+0000", 2024-04-24T15:33:16.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:16.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:16.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892506+0000", 2024-04-24T15:33:16.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:16.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:16.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892509+0000", 2024-04-24T15:33:16.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8760000000000001e-06 2024-04-24T15:33:16.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:16.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892515+0000", 2024-04-24T15:33:16.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1840000000000004e-06 2024-04-24T15:33:16.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:16.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892519+0000", 2024-04-24T15:33:16.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7570000000000002e-06 2024-04-24T15:33:16.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.210 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:16.210 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931694+0000", 2024-04-24T15:33:16.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039174960000000002 2024-04-24T15:33:16.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.210 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:16.210 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931734+0000", 2024-04-24T15:33:16.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0284000000000003e-05 2024-04-24T15:33:16.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.210 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:16.210 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931781+0000", 2024-04-24T15:33:16.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6813000000000001e-05 2024-04-24T15:33:16.211 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:16.211 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:16.211 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:16.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.211 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14052 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2752512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:16.211 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892524+0000", 2024-04-24T15:33:16.211 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 42.279061124000002, 2024-04-24T15:33:16.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039371122000000001, 2024-04-24T15:33:16.211 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:16.211 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:16.211 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:16.211 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:16.211 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:16.212 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14052 2024-04-24T15:33:16.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.212 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:16.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:16.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892524+0000", 2024-04-24T15:33:16.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:16.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:16.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892524+0000", 2024-04-24T15:33:16.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:16.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:16.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892524+0000", 2024-04-24T15:33:16.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:16.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:16.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892526+0000", 2024-04-24T15:33:16.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3630000000000001e-06 2024-04-24T15:33:16.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:16.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892538+0000", 2024-04-24T15:33:16.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1677e-05 2024-04-24T15:33:16.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:16.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892541+0000", 2024-04-24T15:33:16.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.822e-06 2024-04-24T15:33:16.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:16.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931802+0000", 2024-04-24T15:33:16.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039260911000000003 2024-04-24T15:33:16.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:16.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931847+0000", 2024-04-24T15:33:16.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4450999999999998e-05 2024-04-24T15:33:16.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:16.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931895+0000", 2024-04-24T15:33:16.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8257999999999998e-05 2024-04-24T15:33:16.215 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:16.215 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:16.215 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:16.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.215 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14053 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2793472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:16.215 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892546+0000", 2024-04-24T15:33:16.216 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 42.279039146000002, 2024-04-24T15:33:16.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039457712999999998, 2024-04-24T15:33:16.216 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:16.216 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:16.216 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:16.216 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:16.216 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:16.216 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14053 2024-04-24T15:33:16.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.216 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:16.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:16.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892546+0000", 2024-04-24T15:33:16.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:16.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:16.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892546+0000", 2024-04-24T15:33:16.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:16.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:16.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892546+0000", 2024-04-24T15:33:16.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:16.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:16.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892548+0000", 2024-04-24T15:33:16.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3120000000000001e-06 2024-04-24T15:33:16.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:16.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892553+0000", 2024-04-24T15:33:16.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2320000000000001e-06 2024-04-24T15:33:16.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:16.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892559+0000", 2024-04-24T15:33:16.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5480000000000001e-06 2024-04-24T15:33:16.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:16.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931916+0000", 2024-04-24T15:33:16.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039357180999999998 2024-04-24T15:33:16.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:16.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931956+0000", 2024-04-24T15:33:16.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0565000000000002e-05 2024-04-24T15:33:16.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:16.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932004+0000", 2024-04-24T15:33:16.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7224000000000001e-05 2024-04-24T15:33:16.220 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:16.220 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:16.220 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:16.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.220 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14054 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2834432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:16.220 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892563+0000", 2024-04-24T15:33:16.220 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 42.279022494000003, 2024-04-24T15:33:16.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039552406999999998, 2024-04-24T15:33:16.220 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:16.220 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:16.220 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:16.220 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:16.220 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:16.220 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14054 2024-04-24T15:33:16.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.221 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:16.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:16.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892563+0000", 2024-04-24T15:33:16.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:16.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:16.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892563+0000", 2024-04-24T15:33:16.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:16.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:16.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892562+0000", 2024-04-24T15:33:16.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:16.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:16.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892564+0000", 2024-04-24T15:33:16.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2029999999999999e-06 2024-04-24T15:33:16.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:16.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892570+0000", 2024-04-24T15:33:16.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1460000000000004e-06 2024-04-24T15:33:16.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:16.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892573+0000", 2024-04-24T15:33:16.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2030000000000001e-06 2024-04-24T15:33:16.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:16.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932024+0000", 2024-04-24T15:33:16.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039451696000000001 2024-04-24T15:33:16.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:16.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932068+0000", 2024-04-24T15:33:16.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3970000000000001e-05 2024-04-24T15:33:16.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:16.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932115+0000", 2024-04-24T15:33:16.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6569000000000001e-05 2024-04-24T15:33:16.223 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:16.223 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:16.223 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:16.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.224 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14055 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2875392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:16.224 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892591+0000", 2024-04-24T15:33:16.224 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 42.278994521000001, 2024-04-24T15:33:16.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039634309, 2024-04-24T15:33:16.224 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:16.224 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:16.224 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:16.224 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:16.224 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:16.224 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14055 2024-04-24T15:33:16.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.224 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:16.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:16.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892591+0000", 2024-04-24T15:33:16.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:16.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:16.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892591+0000", 2024-04-24T15:33:16.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:16.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:16.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892590+0000", 2024-04-24T15:33:16.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:16.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:16.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892592+0000", 2024-04-24T15:33:16.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7519999999999999e-06 2024-04-24T15:33:16.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:16.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892596+0000", 2024-04-24T15:33:16.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.608e-06 2024-04-24T15:33:16.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:16.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892598+0000", 2024-04-24T15:33:16.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.531e-06 2024-04-24T15:33:16.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:16.227 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932136+0000", 2024-04-24T15:33:16.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039537668999999998 2024-04-24T15:33:16.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:16.227 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932177+0000", 2024-04-24T15:33:16.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0880000000000002e-05 2024-04-24T15:33:16.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:16.227 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932225+0000", 2024-04-24T15:33:16.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8223999999999998e-05 2024-04-24T15:33:16.227 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:16.227 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:16.228 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:16.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.228 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14056 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2916352~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:16.228 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892600+0000", 2024-04-24T15:33:16.228 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 42.278984686999998, 2024-04-24T15:33:16.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039736447000000001, 2024-04-24T15:33:16.228 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:16.228 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:16.228 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:16.228 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:16.228 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:16.228 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14056 2024-04-24T15:33:16.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.228 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:16.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:16.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892600+0000", 2024-04-24T15:33:16.229 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:16.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:16.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892600+0000", 2024-04-24T15:33:16.229 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:16.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:16.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892600+0000", 2024-04-24T15:33:16.229 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967296 2024-04-24T15:33:16.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.230 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:16.230 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892602+0000", 2024-04-24T15:33:16.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.576e-06 2024-04-24T15:33:16.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.230 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:16.230 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892605+0000", 2024-04-24T15:33:16.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.574e-06 2024-04-24T15:33:16.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.230 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:16.230 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892608+0000", 2024-04-24T15:33:16.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3800000000000001e-06 2024-04-24T15:33:16.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.231 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:16.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932245+0000", 2024-04-24T15:33:16.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039637378000000001 2024-04-24T15:33:16.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.231 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:16.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932290+0000", 2024-04-24T15:33:16.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4734999999999998e-05 2024-04-24T15:33:16.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.231 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:16.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932337+0000", 2024-04-24T15:33:16.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7033000000000002e-05 2024-04-24T15:33:16.231 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:16.231 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:16.232 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:16.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.232 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14063 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 3203072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:16.232 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.894564+0000", 2024-04-24T15:33:16.232 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 42.277021095999999, 2024-04-24T15:33:16.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038554995000000002, 2024-04-24T15:33:16.232 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:16.232 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:16.232 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:16.232 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:16.232 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:16.232 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14063 2024-04-24T15:33:16.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.232 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:16.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:16.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894564+0000", 2024-04-24T15:33:16.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:16.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:16.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894564+0000", 2024-04-24T15:33:16.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:16.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:16.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894564+0000", 2024-04-24T15:33:16.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967296 2024-04-24T15:33:16.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:16.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894565+0000", 2024-04-24T15:33:16.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2640000000000001e-06 2024-04-24T15:33:16.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:16.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894569+0000", 2024-04-24T15:33:16.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1710000000000004e-06 2024-04-24T15:33:16.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:16.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894573+0000", 2024-04-24T15:33:16.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4410000000000002e-06 2024-04-24T15:33:16.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:16.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933030+0000", 2024-04-24T15:33:16.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038457709999999999 2024-04-24T15:33:16.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:16.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933072+0000", 2024-04-24T15:33:16.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.193e-05 2024-04-24T15:33:16.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:16.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933119+0000", 2024-04-24T15:33:16.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6715000000000003e-05 2024-04-24T15:33:16.235 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:16.236 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:16.236 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:16.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.236 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14064 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 3244032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:16.236 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.894591+0000", 2024-04-24T15:33:16.236 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 42.276994453999997, 2024-04-24T15:33:16.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038639806999999998, 2024-04-24T15:33:16.236 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:16.236 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:16.236 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:16.236 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:16.236 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:16.236 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14064 2024-04-24T15:33:16.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.236 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:16.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:16.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894591+0000", 2024-04-24T15:33:16.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:16.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:16.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894591+0000", 2024-04-24T15:33:16.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:16.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:16.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894590+0000", 2024-04-24T15:33:16.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:16.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.238 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:16.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894593+0000", 2024-04-24T15:33:16.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4719999999999998e-06 2024-04-24T15:33:16.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.238 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:16.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894598+0000", 2024-04-24T15:33:16.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.57e-06 2024-04-24T15:33:16.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.238 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:16.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894602+0000", 2024-04-24T15:33:16.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.614e-06 2024-04-24T15:33:16.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:16.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933139+0000", 2024-04-24T15:33:16.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038536929999999997 2024-04-24T15:33:16.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:16.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933183+0000", 2024-04-24T15:33:16.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4217e-05 2024-04-24T15:33:16.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:16.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933230+0000", 2024-04-24T15:33:16.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7434e-05 2024-04-24T15:33:16.240 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:16.240 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:16.240 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:16.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.240 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14065 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 3284992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:16.240 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.894606+0000", 2024-04-24T15:33:16.240 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 42.276979072000003, 2024-04-24T15:33:16.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038734126000000001, 2024-04-24T15:33:16.240 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:16.240 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:16.241 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:16.241 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:16.241 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:16.241 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14065 2024-04-24T15:33:16.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.241 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:16.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:16.241 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894606+0000", 2024-04-24T15:33:16.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:16.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:16.241 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894606+0000", 2024-04-24T15:33:16.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:16.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:16.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894606+0000", 2024-04-24T15:33:16.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:16.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:16.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894608+0000", 2024-04-24T15:33:16.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0880000000000002e-06 2024-04-24T15:33:16.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:16.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894614+0000", 2024-04-24T15:33:16.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7830000000000004e-06 2024-04-24T15:33:16.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:16.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894621+0000", 2024-04-24T15:33:16.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5809999999999999e-06 2024-04-24T15:33:16.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:16.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933251+0000", 2024-04-24T15:33:16.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038629995 2024-04-24T15:33:16.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:16.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933292+0000", 2024-04-24T15:33:16.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0380000000000003e-05 2024-04-24T15:33:16.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:16.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933340+0000", 2024-04-24T15:33:16.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8662999999999998e-05 2024-04-24T15:33:16.244 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:16.244 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:16.244 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:16.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.245 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14066 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 3325952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:16.245 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.894625+0000", 2024-04-24T15:33:16.245 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 42.276959632999997, 2024-04-24T15:33:16.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038825539999999999, 2024-04-24T15:33:16.245 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:16.245 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:16.245 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:16.245 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:16.245 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:16.245 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14066 2024-04-24T15:33:16.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.245 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:16.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.246 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:16.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894625+0000", 2024-04-24T15:33:16.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:16.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.246 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:16.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894625+0000", 2024-04-24T15:33:16.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:16.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.246 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:16.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894625+0000", 2024-04-24T15:33:16.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:16.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:16.247 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894627+0000", 2024-04-24T15:33:16.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.092e-06 2024-04-24T15:33:16.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:16.247 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894635+0000", 2024-04-24T15:33:16.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6660000000000008e-06 2024-04-24T15:33:16.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:16.247 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894639+0000", 2024-04-24T15:33:16.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.799e-06 2024-04-24T15:33:16.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:16.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933360+0000", 2024-04-24T15:33:16.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038721569999999997 2024-04-24T15:33:16.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:16.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933403+0000", 2024-04-24T15:33:16.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2605999999999998e-05 2024-04-24T15:33:16.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:16.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933451+0000", 2024-04-24T15:33:16.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8149999999999998e-05 2024-04-24T15:33:16.248 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:16.249 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:16.249 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:16.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.249 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14067 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 3366912~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:16.249 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.894644+0000", 2024-04-24T15:33:16.249 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 42.276941575000002, 2024-04-24T15:33:16.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038926068000000001, 2024-04-24T15:33:16.249 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:16.249 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:16.249 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:16.249 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:16.249 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:16.249 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14067 2024-04-24T15:33:16.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.250 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:16.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:16.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894644+0000", 2024-04-24T15:33:16.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:16.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:16.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894644+0000", 2024-04-24T15:33:16.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:16.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:16.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894643+0000", 2024-04-24T15:33:16.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:16.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:16.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894646+0000", 2024-04-24T15:33:16.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3379999999999999e-06 2024-04-24T15:33:16.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:16.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894651+0000", 2024-04-24T15:33:16.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.485e-06 2024-04-24T15:33:16.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:16.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894655+0000", 2024-04-24T15:33:16.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.709e-06 2024-04-24T15:33:16.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:16.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933472+0000", 2024-04-24T15:33:16.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038816992000000002 2024-04-24T15:33:16.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:16.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933521+0000", 2024-04-24T15:33:16.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8841999999999998e-05 2024-04-24T15:33:16.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:16.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933570+0000", 2024-04-24T15:33:16.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9063000000000001e-05 2024-04-24T15:33:16.252 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:16.253 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:16.253 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:16.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.253 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:30240 4.0 4:059b194c:::rbd_object_map.114a2e5eebc:head [read 0~43] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-04-24T15:33:16.253 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:47.244473+0000", 2024-04-24T15:33:16.253 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.927112389000001, 2024-04-24T15:33:16.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4146627299999999, 2024-04-24T15:33:16.253 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:16.253 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:16.253 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:16.253 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:16.253 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:16.253 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30240 2024-04-24T15:33:16.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.253 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:16.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:16.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.244473+0000", 2024-04-24T15:33:16.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:16.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:16.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.244473+0000", 2024-04-24T15:33:16.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:16.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:16.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.244471+0000", 2024-04-24T15:33:16.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999986 2024-04-24T15:33:16.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:16.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.244477+0000", 2024-04-24T15:33:16.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7819999999999999e-06 2024-04-24T15:33:16.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:16.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.244487+0000", 2024-04-24T15:33:16.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0247e-05 2024-04-24T15:33:16.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:16.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.244499+0000", 2024-04-24T15:33:16.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1324e-05 2024-04-24T15:33:16.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:16.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.244545+0000", 2024-04-24T15:33:16.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6186000000000001e-05 2024-04-24T15:33:16.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-04-24T15:33:16.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.244553+0000", 2024-04-24T15:33:16.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5380000000000002e-06 2024-04-24T15:33:16.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:16.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.631596+0000", 2024-04-24T15:33:16.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.38704289600000003 2024-04-24T15:33:16.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-04-24T15:33:16.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.631606+0000", 2024-04-24T15:33:16.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6390000000000004e-06 2024-04-24T15:33:16.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:16.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:48.658831+0000", 2024-04-24T15:33:16.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0272250540000001 2024-04-24T15:33:16.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:16.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:48.658991+0000", 2024-04-24T15:33:16.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015994800000000001 2024-04-24T15:33:16.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:16.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:16.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:16.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:48.659135+0000", 2024-04-24T15:33:16.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000144456 2024-04-24T15:33:16.258 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:16.258 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:16.258 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:16.258 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:16.258 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:16.258 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:16.260 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-24T15:33:16.260 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:33:16.261+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:33:16.361 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:16.362+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:16.393 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:16.393 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:33:16.393 INFO:teuthology.orchestra.run.smithi151.stdout: "num_ops": 0 2024-04-24T15:33:16.393 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:16.402 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-24T15:33:16.462 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:33:16.463+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:33:16.524 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:16.524 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:33:16.524 INFO:teuthology.orchestra.run.smithi151.stdout: "complaint_time": 30, 2024-04-24T15:33:16.524 INFO:teuthology.orchestra.run.smithi151.stdout: "num_blocked_ops": 0 2024-04-24T15:33:16.524 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:16.534 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-24T15:33:16.578 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:16.579+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:16.621 INFO:teuthology.orchestra.run.smithi151.stdout:get_pg_num_history pg_num_history pg_num_history(e75 pg_nums {1={14=1},2={15=8},3={18=4},4={26=32}} deleted_pools ) 2024-04-24T15:33:16.621 INFO:teuthology.orchestra.run.smithi151.stdout:Importing pgid 4.12 2024-04-24T15:33:16.621 INFO:teuthology.orchestra.run.smithi151.stdout:Write #4:48599db1:::rbd_data.114ab4f6fc67.0000000000000004:12# 2024-04-24T15:33:16.624 INFO:teuthology.orchestra.run.smithi151.stdout:Write #4:48599db1:::rbd_data.114ab4f6fc67.0000000000000004:head# 2024-04-24T15:33:16.624 INFO:teuthology.orchestra.run.smithi151.stdout:snapset 12=[]:{12=[12]} 2024-04-24T15:33:16.624 INFO:teuthology.orchestra.run.smithi151.stdout:Write #4:48a8a567:::rbd_data.114a8b543e91.0000000000000014:head# 2024-04-24T15:33:16.625 INFO:teuthology.orchestra.run.smithi151.stdout:snapset 0=[]:{} 2024-04-24T15:33:16.627 INFO:teuthology.orchestra.run.smithi151.stdout:Write #4:49009fc0:::rbd_data.114ad3a18a61.000000000000000b:head# 2024-04-24T15:33:16.627 INFO:teuthology.orchestra.run.smithi151.stdout:snapset 0=[]:{} 2024-04-24T15:33:16.628 INFO:teuthology.orchestra.run.smithi151.stdout:Write #4:4a25d4cb:::rbd_data.114a858e6641.0000000000000026:17# 2024-04-24T15:33:16.632 INFO:teuthology.orchestra.run.smithi151.stdout:Write #4:4a25d4cb:::rbd_data.114a858e6641.0000000000000026:head# 2024-04-24T15:33:16.632 INFO:teuthology.orchestra.run.smithi151.stdout:snapset 17=[]:{17=[17]} 2024-04-24T15:33:16.632 INFO:teuthology.orchestra.run.smithi151.stdout:Write #4:4ab5cdb0:::rbd_data.114aaa0086f7.0000000000000002:head# 2024-04-24T15:33:16.635 INFO:teuthology.orchestra.run.smithi151.stdout:snapset 0=[]:{} 2024-04-24T15:33:16.642 INFO:teuthology.orchestra.run.smithi151.stdout:Write #4:4b4f242e:::rbd_data.114a1f60cb66.00000000000002b4:head# 2024-04-24T15:33:16.642 INFO:teuthology.orchestra.run.smithi151.stdout:snapset 0=[]:{} 2024-04-24T15:33:16.642 INFO:teuthology.orchestra.run.smithi151.stdout:Write #4:4b7ae65e:::rbd_data.114ab4f6fc67.0000000000000056:12# 2024-04-24T15:33:16.643 INFO:teuthology.orchestra.run.smithi151.stdout:Write #4:4b7ae65e:::rbd_data.114ab4f6fc67.0000000000000056:head# 2024-04-24T15:33:16.643 INFO:teuthology.orchestra.run.smithi151.stdout:snapset 12=[]:{12=[12]} 2024-04-24T15:33:16.644 INFO:teuthology.orchestra.run.smithi151.stdout:Write #4:4c04ad06:::rbd_data.114a60efb398.0000000000000031:c# 2024-04-24T15:33:16.644 INFO:teuthology.orchestra.run.smithi151.stdout:Write #4:4c04ad06:::rbd_data.114a60efb398.0000000000000031:head# 2024-04-24T15:33:16.644 INFO:teuthology.orchestra.run.smithi151.stdout:snapset c=[]:{c=[c]} 2024-04-24T15:33:16.644 INFO:teuthology.orchestra.run.smithi151.stdout:Write #4:4c93b7f9:::rbd_data.114a1f60cb66.0000000000000037:head# 2024-04-24T15:33:16.644 INFO:teuthology.orchestra.run.smithi151.stdout:snapset 0=[]:{} 2024-04-24T15:33:16.645 INFO:teuthology.orchestra.run.smithi151.stdout:Write #4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:7# 2024-04-24T15:33:16.664 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:16.665+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:16.671 INFO:teuthology.orchestra.run.smithi151.stdout:Write #4:4cb7f5ad:::rbd_data.114a38cedbad.0000000000000000:head# 2024-04-24T15:33:16.671 INFO:teuthology.orchestra.run.smithi151.stdout:snapset 7=[]:{7=[7]} 2024-04-24T15:33:16.672 INFO:teuthology.orchestra.run.smithi151.stdout:Write #4:4ccc81f4:::rbd_data.114a46f1ce44.0000000000000009:14# 2024-04-24T15:33:16.688 INFO:teuthology.orchestra.run.smithi151.stdout:Write #4:4ccc81f4:::rbd_data.114a46f1ce44.0000000000000009:head# 2024-04-24T15:33:16.688 INFO:teuthology.orchestra.run.smithi151.stdout:snapset 14=[]:{14=[14]} 2024-04-24T15:33:16.688 INFO:teuthology.orchestra.run.smithi151.stdout:Write #4:4dddff5c:::rbd_data.114a60efb398.0000000000000032:c# 2024-04-24T15:33:16.689 INFO:teuthology.orchestra.run.smithi151.stdout:Write #4:4dddff5c:::rbd_data.114a60efb398.0000000000000032:head# 2024-04-24T15:33:16.689 INFO:teuthology.orchestra.run.smithi151.stdout:snapset c=[]:{c=[c]} 2024-04-24T15:33:16.689 INFO:teuthology.orchestra.run.smithi151.stdout:Write #4:4e5034f1:::rbd_data.114a8e7208a5.0000000000000000:b# 2024-04-24T15:33:16.692 INFO:teuthology.orchestra.run.smithi151.stdout:Write #4:4e5034f1:::rbd_data.114a8e7208a5.0000000000000000:head# 2024-04-24T15:33:16.692 INFO:teuthology.orchestra.run.smithi151.stdout:snapset b=[]:{b=[b]} 2024-04-24T15:33:16.693 INFO:teuthology.orchestra.run.smithi151.stdout:Write #4:4e515771:::rbd_data.114ae15b9a20.0000000000000024:head# 2024-04-24T15:33:16.693 INFO:teuthology.orchestra.run.smithi151.stdout:snapset 0=[]:{} 2024-04-24T15:33:16.693 INFO:teuthology.orchestra.run.smithi151.stdout:Write #4:4ee02263:::rbd_data.114a1f60cb66.0000000000000055:head# 2024-04-24T15:33:16.694 INFO:teuthology.orchestra.run.smithi151.stdout:snapset 0=[]:{} 2024-04-24T15:33:16.694 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:16.694 INFO:teuthology.orchestra.run.smithi151.stdout: "size": 20, 2024-04-24T15:33:16.694 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 600, 2024-04-24T15:33:16.695 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [ 2024-04-24T15:33:16.695 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.695 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:39041 4.1 4:82c1a902:::rbd_data.114abe589119.000000000000032c:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e68)", 2024-04-24T15:33:16.695 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:01.760552+0000", 2024-04-24T15:33:16.695 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 14.934300498000001, 2024-04-24T15:33:16.695 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.87570748300000001, 2024-04-24T15:33:16.695 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:16.695 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "reached pg", 2024-04-24T15:33:16.695 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:16.695 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:16.695 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:16.695 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 39041 2024-04-24T15:33:16.695 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.695 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:16.696 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.696 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:16.696 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760552+0000", 2024-04-24T15:33:16.696 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:16.696 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.696 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.696 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:16.696 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760552+0000", 2024-04-24T15:33:16.696 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:16.696 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.696 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.696 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:16.696 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760550+0000", 2024-04-24T15:33:16.697 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999986 2024-04-24T15:33:16.697 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.697 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.697 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:16.697 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760557+0000", 2024-04-24T15:33:16.697 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1420000000000004e-06 2024-04-24T15:33:16.697 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.697 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.697 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:16.697 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760572+0000", 2024-04-24T15:33:16.697 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.483e-05 2024-04-24T15:33:16.697 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.697 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.697 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:16.698 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760578+0000", 2024-04-24T15:33:16.698 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.2820000000000002e-06 2024-04-24T15:33:16.698 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.698 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.698 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:16.698 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760628+0000", 2024-04-24T15:33:16.698 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.0040000000000002e-05 2024-04-24T15:33:16.698 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.698 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.698 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op must wait for map", 2024-04-24T15:33:16.698 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760637+0000", 2024-04-24T15:33:16.698 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.6899999999999998e-06 2024-04-24T15:33:16.698 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.699 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.699 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:16.699 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.770562+0000", 2024-04-24T15:33:16.699 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0099252739999999996 2024-04-24T15:33:16.699 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.699 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.699 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for peered", 2024-04-24T15:33:16.699 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.770572+0000", 2024-04-24T15:33:16.699 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0105999999999999e-05 2024-04-24T15:33:16.699 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.699 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.699 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:16.699 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:02.636170+0000", 2024-04-24T15:33:16.700 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.86559722299999997 2024-04-24T15:33:16.700 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.700 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.700 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:16.700 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:02.636259+0000", 2024-04-24T15:33:16.700 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.9446999999999999e-05 2024-04-24T15:33:16.700 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:16.700 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:16.700 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:16.700 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.700 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.700 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48675 4.d 4:b465d49b:::rbd_data.114a858e6641.0000000000000002:17 [read 2768896~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:16.700 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.926403+0000", 2024-04-24T15:33:16.700 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 5.7684489279999998, 2024-04-24T15:33:16.700 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035742807000000001, 2024-04-24T15:33:16.701 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:16.702 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:16.702 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:16.702 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:16.702 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:16.702 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48675 2024-04-24T15:33:16.702 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.702 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:16.702 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.702 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:16.702 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926403+0000", 2024-04-24T15:33:16.702 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:16.702 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.703 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.703 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:16.703 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926403+0000", 2024-04-24T15:33:16.703 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:16.703 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.703 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.703 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:16.703 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926403+0000", 2024-04-24T15:33:16.703 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:16.703 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.703 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.703 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:16.703 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926404+0000", 2024-04-24T15:33:16.703 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.424e-06 2024-04-24T15:33:16.704 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.704 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.704 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:16.704 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926413+0000", 2024-04-24T15:33:16.704 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.6850000000000007e-06 2024-04-24T15:33:16.704 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.704 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.704 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:16.704 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926417+0000", 2024-04-24T15:33:16.704 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.7859999999999998e-06 2024-04-24T15:33:16.704 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.704 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.704 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:16.704 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.961986+0000", 2024-04-24T15:33:16.705 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035568862 2024-04-24T15:33:16.705 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.705 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.705 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:16.705 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.962063+0000", 2024-04-24T15:33:16.705 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7257999999999998e-05 2024-04-24T15:33:16.705 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.705 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.705 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:16.705 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.962146+0000", 2024-04-24T15:33:16.705 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3094000000000001e-05 2024-04-24T15:33:16.705 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:16.705 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:16.706 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:16.706 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.706 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.706 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48676 4.d 4:b465d49b:::rbd_data.114a858e6641.0000000000000002:17 [read 2834432~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:16.706 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.926421+0000", 2024-04-24T15:33:16.706 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 5.7684313849999995, 2024-04-24T15:33:16.706 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035829002999999998, 2024-04-24T15:33:16.706 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:16.706 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:16.706 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:16.706 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:16.706 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:16.706 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48676 2024-04-24T15:33:16.707 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.707 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:16.707 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.707 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:16.707 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926421+0000", 2024-04-24T15:33:16.707 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:16.707 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.707 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.707 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:16.707 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926421+0000", 2024-04-24T15:33:16.707 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:16.707 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.707 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.707 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:16.708 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926420+0000", 2024-04-24T15:33:16.708 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:16.708 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.708 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.708 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:16.708 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926423+0000", 2024-04-24T15:33:16.708 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.356e-06 2024-04-24T15:33:16.708 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.708 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.708 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:16.708 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926428+0000", 2024-04-24T15:33:16.708 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.3340000000000001e-06 2024-04-24T15:33:16.708 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.708 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.708 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:16.709 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926431+0000", 2024-04-24T15:33:16.709 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.2899999999999998e-06 2024-04-24T15:33:16.709 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.709 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.709 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:16.709 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.962156+0000", 2024-04-24T15:33:16.709 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035725159999999999 2024-04-24T15:33:16.709 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.709 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.709 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:16.709 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.962197+0000", 2024-04-24T15:33:16.709 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0457999999999998e-05 2024-04-24T15:33:16.709 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.710 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.710 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:16.710 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.962250+0000", 2024-04-24T15:33:16.710 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.2812000000000002e-05 2024-04-24T15:33:16.710 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:16.710 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:16.710 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:16.710 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.710 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.710 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48703 4.d 4:b59d5735:::rbd_data.114a858e6641.0000000000000004:17 [read 2637824~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:16.710 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928531+0000", 2024-04-24T15:33:16.710 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 5.7663207590000001, 2024-04-24T15:33:16.710 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035675198999999998, 2024-04-24T15:33:16.711 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:16.711 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:16.711 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:16.711 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:16.711 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:16.711 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48703 2024-04-24T15:33:16.711 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.711 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:16.711 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.711 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:16.711 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928531+0000", 2024-04-24T15:33:16.711 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:16.711 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.711 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.712 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:16.712 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928531+0000", 2024-04-24T15:33:16.712 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:16.712 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.712 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.712 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:16.712 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928531+0000", 2024-04-24T15:33:16.712 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:16.712 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.712 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.712 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:16.712 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928532+0000", 2024-04-24T15:33:16.712 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.33e-06 2024-04-24T15:33:16.712 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.712 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.712 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:16.713 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928536+0000", 2024-04-24T15:33:16.713 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.4419999999999998e-06 2024-04-24T15:33:16.713 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.713 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.713 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:16.713 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928538+0000", 2024-04-24T15:33:16.713 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.0669999999999999e-06 2024-04-24T15:33:16.713 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.713 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.713 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:16.713 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964040+0000", 2024-04-24T15:33:16.713 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035501900000000003 2024-04-24T15:33:16.713 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.713 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.714 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:16.714 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964118+0000", 2024-04-24T15:33:16.714 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.8490000000000002e-05 2024-04-24T15:33:16.714 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.714 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.714 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:16.714 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964207+0000", 2024-04-24T15:33:16.714 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.8202999999999996e-05 2024-04-24T15:33:16.714 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:16.714 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:16.714 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:16.714 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.714 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.715 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48704 4.d 4:b59d5735:::rbd_data.114a858e6641.0000000000000004:17 [read 2703360~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:16.715 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928541+0000", 2024-04-24T15:33:16.715 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 5.7663114149999997, 2024-04-24T15:33:16.715 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035763016000000002, 2024-04-24T15:33:16.715 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:16.715 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:16.715 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:16.715 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:16.715 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:16.715 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48704 2024-04-24T15:33:16.715 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.715 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:16.715 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.716 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:16.716 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928541+0000", 2024-04-24T15:33:16.716 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:16.716 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.716 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.716 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:16.716 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928541+0000", 2024-04-24T15:33:16.716 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:16.716 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.716 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.716 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:16.716 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928540+0000", 2024-04-24T15:33:16.716 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:16.716 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.717 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.717 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:16.717 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928542+0000", 2024-04-24T15:33:16.717 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.37e-06 2024-04-24T15:33:16.717 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.717 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.717 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:16.717 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928545+0000", 2024-04-24T15:33:16.717 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8229999999999999e-06 2024-04-24T15:33:16.717 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.717 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.717 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:16.717 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928547+0000", 2024-04-24T15:33:16.717 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.9360000000000002e-06 2024-04-24T15:33:16.717 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.718 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.718 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:16.718 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964219+0000", 2024-04-24T15:33:16.718 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035672813999999997 2024-04-24T15:33:16.718 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.718 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.718 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:16.718 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964259+0000", 2024-04-24T15:33:16.718 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0070000000000001e-05 2024-04-24T15:33:16.718 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.718 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.718 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:16.718 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964304+0000", 2024-04-24T15:33:16.719 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4229e-05 2024-04-24T15:33:16.719 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:16.719 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:16.719 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:16.719 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.719 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.719 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48719 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 1851392~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:16.719 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928549+0000", 2024-04-24T15:33:16.719 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 5.7663032679999997, 2024-04-24T15:33:16.719 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035890645999999998, 2024-04-24T15:33:16.719 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:16.719 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:16.719 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:16.719 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:16.720 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:16.720 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48719 2024-04-24T15:33:16.720 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.720 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:16.720 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.720 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:16.720 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928549+0000", 2024-04-24T15:33:16.720 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:16.720 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.720 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.720 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:16.720 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928549+0000", 2024-04-24T15:33:16.720 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:16.720 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.721 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.721 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:16.721 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928549+0000", 2024-04-24T15:33:16.721 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:16.721 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.721 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.721 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:16.721 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928550+0000", 2024-04-24T15:33:16.721 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3829999999999999e-06 2024-04-24T15:33:16.721 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.721 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.721 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:16.721 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928553+0000", 2024-04-24T15:33:16.721 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7369999999999998e-06 2024-04-24T15:33:16.721 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.721 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.722 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:16.722 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928555+0000", 2024-04-24T15:33:16.722 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.8920000000000001e-06 2024-04-24T15:33:16.722 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.722 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.722 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:16.722 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964240+0000", 2024-04-24T15:33:16.722 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035685537000000003 2024-04-24T15:33:16.722 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.722 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.722 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:16.722 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964318+0000", 2024-04-24T15:33:16.722 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7838999999999998e-05 2024-04-24T15:33:16.722 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.723 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.723 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:16.723 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964439+0000", 2024-04-24T15:33:16.723 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000121564 2024-04-24T15:33:16.723 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:16.723 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:16.723 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:16.723 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.723 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.723 INFO:teuthology.orchestra.run.smithi151.stdout:write_pg epoch 75 info 4.12( v 75'275 (0'0,75'275] local-lis/les=27/28 n=23 ec=26/26 lis/c=27/27 les/c/f=28/28/0 sis=27) 2024-04-24T15:33:16.724 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48720 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 1916928~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:16.724 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928557+0000", 2024-04-24T15:33:16.724 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 5.7662954229999999, 2024-04-24T15:33:16.724 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036005219999999998, 2024-04-24T15:33:16.724 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:16.725 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:16.725 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:16.725 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:16.725 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:16.725 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48720 2024-04-24T15:33:16.725 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.725 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:16.725 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.725 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:16.725 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928557+0000", 2024-04-24T15:33:16.725 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:16.725 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.725 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.726 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:16.726 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928557+0000", 2024-04-24T15:33:16.726 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:16.726 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.726 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.726 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:16.726 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928556+0000", 2024-04-24T15:33:16.726 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:16.726 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.726 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.726 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:16.726 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928558+0000", 2024-04-24T15:33:16.726 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.13e-06 2024-04-24T15:33:16.726 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.727 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.727 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:16.727 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928561+0000", 2024-04-24T15:33:16.727 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5530000000000001e-06 2024-04-24T15:33:16.727 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.727 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.727 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:16.727 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928563+0000", 2024-04-24T15:33:16.727 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.8810000000000001e-06 2024-04-24T15:33:16.727 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.727 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.727 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:16.727 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964450+0000", 2024-04-24T15:33:16.727 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035887040000000002 2024-04-24T15:33:16.727 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.728 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.728 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:16.728 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964494+0000", 2024-04-24T15:33:16.728 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3699000000000003e-05 2024-04-24T15:33:16.728 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.728 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.728 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:16.728 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964562+0000", 2024-04-24T15:33:16.728 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.8149000000000001e-05 2024-04-24T15:33:16.728 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:16.728 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:16.728 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:16.728 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.728 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.728 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48721 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 1982464~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:16.728 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928565+0000", 2024-04-24T15:33:16.729 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 5.7662868989999998, 2024-04-24T15:33:16.729 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036233466999999998, 2024-04-24T15:33:16.729 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:16.729 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:16.729 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:16.729 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:16.729 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:16.729 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48721 2024-04-24T15:33:16.729 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.729 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:16.729 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.729 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:16.729 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928565+0000", 2024-04-24T15:33:16.729 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:16.729 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.729 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.730 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:16.730 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928565+0000", 2024-04-24T15:33:16.730 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:16.730 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.730 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.730 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:16.730 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928565+0000", 2024-04-24T15:33:16.730 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:16.730 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.730 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.730 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:16.730 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928566+0000", 2024-04-24T15:33:16.730 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.336e-06 2024-04-24T15:33:16.730 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.730 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.731 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:16.731 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928569+0000", 2024-04-24T15:33:16.731 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.9189999999999999e-06 2024-04-24T15:33:16.731 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.731 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.731 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:16.731 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928571+0000", 2024-04-24T15:33:16.731 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.1909999999999999e-06 2024-04-24T15:33:16.731 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.731 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.731 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:16.731 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964592+0000", 2024-04-24T15:33:16.731 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036020120000000003 2024-04-24T15:33:16.731 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.731 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.731 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:16.732 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964673+0000", 2024-04-24T15:33:16.732 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.1254000000000005e-05 2024-04-24T15:33:16.732 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.732 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.732 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:16.732 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964799+0000", 2024-04-24T15:33:16.732 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00012588700000000001 2024-04-24T15:33:16.732 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:16.732 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:16.732 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:16.732 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.732 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.732 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48722 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2048000~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:16.732 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928586+0000", 2024-04-24T15:33:16.732 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 5.7662657739999998, 2024-04-24T15:33:16.732 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036331347999999999, 2024-04-24T15:33:16.733 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:16.733 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:16.733 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:16.733 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:16.733 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:16.733 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48722 2024-04-24T15:33:16.733 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.733 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:16.733 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.733 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:16.733 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928586+0000", 2024-04-24T15:33:16.733 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:16.733 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.733 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.733 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:16.734 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928586+0000", 2024-04-24T15:33:16.734 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:16.734 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.734 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.734 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:16.734 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928586+0000", 2024-04-24T15:33:16.734 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:16.734 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.734 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.734 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:16.734 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928587+0000", 2024-04-24T15:33:16.734 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3519999999999999e-06 2024-04-24T15:33:16.734 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.735 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.735 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:16.735 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928590+0000", 2024-04-24T15:33:16.735 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.9699999999999999e-06 2024-04-24T15:33:16.735 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.735 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.735 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:16.735 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928592+0000", 2024-04-24T15:33:16.735 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.097e-06 2024-04-24T15:33:16.735 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.735 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.735 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:16.735 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964810+0000", 2024-04-24T15:33:16.736 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036217249 2024-04-24T15:33:16.736 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.736 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.736 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:16.736 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964851+0000", 2024-04-24T15:33:16.736 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.1151e-05 2024-04-24T15:33:16.736 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.736 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.736 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:16.736 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964918+0000", 2024-04-24T15:33:16.736 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.6898999999999998e-05 2024-04-24T15:33:16.736 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:16.736 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:16.736 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:16.737 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.737 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.737 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48723 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2113536~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:16.737 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928598+0000", 2024-04-24T15:33:16.737 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 5.7662543849999999, 2024-04-24T15:33:16.737 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036552125999999997, 2024-04-24T15:33:16.737 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:16.737 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:16.737 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:16.737 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:16.737 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:16.737 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48723 2024-04-24T15:33:16.737 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.738 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:16.738 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.738 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:16.738 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928598+0000", 2024-04-24T15:33:16.738 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:16.738 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.738 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.738 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:16.738 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928598+0000", 2024-04-24T15:33:16.738 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:16.738 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.738 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.738 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:16.739 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928597+0000", 2024-04-24T15:33:16.739 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:16.739 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.739 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.739 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:16.739 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928599+0000", 2024-04-24T15:33:16.739 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.621e-06 2024-04-24T15:33:16.739 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.739 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.739 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:16.739 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928602+0000", 2024-04-24T15:33:16.739 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.9629999999999998e-06 2024-04-24T15:33:16.739 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.740 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.740 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:16.740 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928607+0000", 2024-04-24T15:33:16.740 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8940000000000001e-06 2024-04-24T15:33:16.740 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.740 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.740 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:16.740 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964947+0000", 2024-04-24T15:33:16.740 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036340439000000002 2024-04-24T15:33:16.740 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.740 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.740 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:16.740 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965028+0000", 2024-04-24T15:33:16.740 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.1054e-05 2024-04-24T15:33:16.741 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.741 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.741 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:16.741 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965150+0000", 2024-04-24T15:33:16.741 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00012154000000000001 2024-04-24T15:33:16.741 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:16.741 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:16.741 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:16.741 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.741 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.741 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48724 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2179072~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:16.741 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928609+0000", 2024-04-24T15:33:16.741 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 5.7662430100000002, 2024-04-24T15:33:16.741 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036679614999999999, 2024-04-24T15:33:16.742 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:16.742 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:16.742 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:16.742 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:16.742 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:16.742 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48724 2024-04-24T15:33:16.742 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.742 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:16.742 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.742 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:16.742 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928609+0000", 2024-04-24T15:33:16.742 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:16.742 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.743 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.743 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:16.743 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928609+0000", 2024-04-24T15:33:16.743 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:16.743 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.743 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.743 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:16.743 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928609+0000", 2024-04-24T15:33:16.743 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:16.743 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.743 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.743 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:16.743 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928610+0000", 2024-04-24T15:33:16.744 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3030000000000001e-06 2024-04-24T15:33:16.744 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.744 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.744 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:16.744 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928613+0000", 2024-04-24T15:33:16.744 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.762e-06 2024-04-24T15:33:16.744 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.744 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.744 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:16.744 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928615+0000", 2024-04-24T15:33:16.744 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.832e-06 2024-04-24T15:33:16.744 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.744 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.744 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:16.745 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965174+0000", 2024-04-24T15:33:16.745 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036559633000000001 2024-04-24T15:33:16.745 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.745 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.745 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:16.745 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965221+0000", 2024-04-24T15:33:16.745 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6239999999999998e-05 2024-04-24T15:33:16.745 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.745 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.745 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:16.745 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965289+0000", 2024-04-24T15:33:16.745 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.8053999999999996e-05 2024-04-24T15:33:16.745 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:16.745 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:16.746 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:16.746 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.746 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.746 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48725 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2244608~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:16.746 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928617+0000", 2024-04-24T15:33:16.746 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 5.7662352400000003, 2024-04-24T15:33:16.746 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036902659999999997, 2024-04-24T15:33:16.746 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:16.746 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:16.746 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:16.746 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:16.746 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:16.747 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48725 2024-04-24T15:33:16.747 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.747 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:16.747 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.747 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:16.747 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928617+0000", 2024-04-24T15:33:16.747 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:16.747 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.747 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.747 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:16.747 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928617+0000", 2024-04-24T15:33:16.747 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:16.747 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.747 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.748 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:16.748 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928617+0000", 2024-04-24T15:33:16.748 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:16.748 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.748 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.748 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:16.748 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928618+0000", 2024-04-24T15:33:16.748 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.206e-06 2024-04-24T15:33:16.748 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.748 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.748 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:16.748 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928621+0000", 2024-04-24T15:33:16.748 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5130000000000001e-06 2024-04-24T15:33:16.748 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.748 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.749 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:16.749 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928623+0000", 2024-04-24T15:33:16.749 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.9640000000000002e-06 2024-04-24T15:33:16.749 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.749 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.749 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:16.749 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965318+0000", 2024-04-24T15:33:16.749 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036694953000000002 2024-04-24T15:33:16.749 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.749 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.749 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:16.749 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965398+0000", 2024-04-24T15:33:16.749 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.9986000000000003e-05 2024-04-24T15:33:16.749 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.749 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.750 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:16.750 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965520+0000", 2024-04-24T15:33:16.750 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000121247 2024-04-24T15:33:16.750 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:16.750 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:16.750 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:16.750 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.750 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.750 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48726 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2310144~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:16.750 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928625+0000", 2024-04-24T15:33:16.750 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 5.7662266930000001, 2024-04-24T15:33:16.750 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037015521000000003, 2024-04-24T15:33:16.750 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:16.750 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:16.751 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:16.751 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:16.751 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:16.751 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48726 2024-04-24T15:33:16.751 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.751 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:16.751 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.751 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:16.751 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928625+0000", 2024-04-24T15:33:16.751 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:16.751 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.751 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.751 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:16.751 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928625+0000", 2024-04-24T15:33:16.751 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:16.751 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.752 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.752 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:16.752 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928625+0000", 2024-04-24T15:33:16.752 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:16.752 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.752 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.752 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:16.752 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928627+0000", 2024-04-24T15:33:16.752 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3179999999999999e-06 2024-04-24T15:33:16.752 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.752 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.752 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:16.752 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928629+0000", 2024-04-24T15:33:16.752 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.819e-06 2024-04-24T15:33:16.752 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.753 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.753 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:16.753 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928634+0000", 2024-04-24T15:33:16.753 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6999999999999999e-06 2024-04-24T15:33:16.753 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.753 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.753 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:16.753 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965531+0000", 2024-04-24T15:33:16.753 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036897040999999998 2024-04-24T15:33:16.753 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.753 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.753 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:16.753 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965573+0000", 2024-04-24T15:33:16.753 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.189e-05 2024-04-24T15:33:16.753 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.754 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.754 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:16.754 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965641+0000", 2024-04-24T15:33:16.754 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.7959000000000004e-05 2024-04-24T15:33:16.754 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:16.754 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:16.754 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:16.754 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.754 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.754 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48727 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2375680~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:16.754 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928637+0000", 2024-04-24T15:33:16.754 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 5.7662153319999998, 2024-04-24T15:33:16.754 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037235129999999998, 2024-04-24T15:33:16.755 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:16.755 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:16.755 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:16.755 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:16.755 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:16.755 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48727 2024-04-24T15:33:16.755 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.755 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:16.755 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.755 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:16.755 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928637+0000", 2024-04-24T15:33:16.755 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:16.755 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.755 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.755 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:16.756 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928637+0000", 2024-04-24T15:33:16.756 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:16.756 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.756 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.756 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:16.756 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928637+0000", 2024-04-24T15:33:16.756 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:16.756 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.756 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.756 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:16.756 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928638+0000", 2024-04-24T15:33:16.756 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.4279999999999999e-06 2024-04-24T15:33:16.756 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.756 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.756 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:16.756 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928641+0000", 2024-04-24T15:33:16.757 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.909e-06 2024-04-24T15:33:16.757 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.757 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.757 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:16.757 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928643+0000", 2024-04-24T15:33:16.757 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.9269999999999999e-06 2024-04-24T15:33:16.757 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.757 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.757 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:16.757 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965671+0000", 2024-04-24T15:33:16.757 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037028393 2024-04-24T15:33:16.757 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.757 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.757 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:16.758 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965752+0000", 2024-04-24T15:33:16.758 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0921999999999995e-05 2024-04-24T15:33:16.758 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.758 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.758 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:16.758 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965872+0000", 2024-04-24T15:33:16.758 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000119759 2024-04-24T15:33:16.758 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:16.758 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:16.758 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:16.758 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.758 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.758 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48728 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2441216~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:16.758 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928645+0000", 2024-04-24T15:33:16.759 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 5.7662071489999995, 2024-04-24T15:33:16.759 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037346852999999999, 2024-04-24T15:33:16.759 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:16.759 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:16.759 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:16.759 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:16.759 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:16.759 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48728 2024-04-24T15:33:16.759 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.759 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:16.759 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.759 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:16.759 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928645+0000", 2024-04-24T15:33:16.759 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:16.759 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.760 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.760 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:16.760 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928645+0000", 2024-04-24T15:33:16.760 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:16.760 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.760 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.760 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:16.760 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928645+0000", 2024-04-24T15:33:16.760 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:16.760 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.760 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.760 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:16.760 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928646+0000", 2024-04-24T15:33:16.760 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.291e-06 2024-04-24T15:33:16.760 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.760 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.761 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:16.761 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928649+0000", 2024-04-24T15:33:16.761 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8509999999999999e-06 2024-04-24T15:33:16.761 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.761 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.761 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:16.761 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928651+0000", 2024-04-24T15:33:16.761 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.809e-06 2024-04-24T15:33:16.761 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.761 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.761 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:16.761 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965882+0000", 2024-04-24T15:33:16.761 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037231715999999998 2024-04-24T15:33:16.761 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.761 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.761 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:16.762 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965925+0000", 2024-04-24T15:33:16.762 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2493999999999997e-05 2024-04-24T15:33:16.762 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.762 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.762 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:16.762 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965992+0000", 2024-04-24T15:33:16.762 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.6907000000000002e-05 2024-04-24T15:33:16.762 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:16.762 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:16.762 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:16.762 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.762 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.762 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48729 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2506752~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:16.762 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928653+0000", 2024-04-24T15:33:16.762 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 5.7661992739999999, 2024-04-24T15:33:16.763 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037576904000000001, 2024-04-24T15:33:16.763 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:16.763 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:16.763 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:16.763 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:16.763 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:16.763 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48729 2024-04-24T15:33:16.763 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.763 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:16.763 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.763 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:16.763 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928653+0000", 2024-04-24T15:33:16.763 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:16.763 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.763 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.763 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:16.764 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928653+0000", 2024-04-24T15:33:16.764 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:16.764 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.764 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.764 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:16.764 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928653+0000", 2024-04-24T15:33:16.764 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:16.764 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.764 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.764 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:16.764 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928654+0000", 2024-04-24T15:33:16.764 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3209999999999999e-06 2024-04-24T15:33:16.764 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.764 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.764 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:16.765 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928657+0000", 2024-04-24T15:33:16.765 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8559999999999998e-06 2024-04-24T15:33:16.765 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.765 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.765 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:16.765 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928659+0000", 2024-04-24T15:33:16.765 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7080000000000002e-06 2024-04-24T15:33:16.765 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.765 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.765 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:16.765 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966021+0000", 2024-04-24T15:33:16.765 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037361296000000002 2024-04-24T15:33:16.766 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.766 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.766 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:16.766 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966102+0000", 2024-04-24T15:33:16.766 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0939999999999994e-05 2024-04-24T15:33:16.766 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.766 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.766 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:16.766 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966230+0000", 2024-04-24T15:33:16.766 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00012799999999999999 2024-04-24T15:33:16.766 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:16.766 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:16.766 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:16.767 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.767 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.767 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48730 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2572288~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:16.767 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928662+0000", 2024-04-24T15:33:16.767 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 5.7661901860000002, 2024-04-24T15:33:16.767 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037687246000000001, 2024-04-24T15:33:16.767 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:16.767 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:16.767 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:16.767 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:16.767 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:16.767 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48730 2024-04-24T15:33:16.767 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.767 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:16.767 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.768 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:16.768 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928662+0000", 2024-04-24T15:33:16.768 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:16.768 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.768 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.768 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:16.768 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928662+0000", 2024-04-24T15:33:16.768 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:16.768 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.768 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.768 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:16.768 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928662+0000", 2024-04-24T15:33:16.768 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:16.768 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.768 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.769 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:16.769 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928663+0000", 2024-04-24T15:33:16.769 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.381e-06 2024-04-24T15:33:16.769 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.769 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.769 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:16.769 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928666+0000", 2024-04-24T15:33:16.769 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.796e-06 2024-04-24T15:33:16.769 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.769 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.769 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:16.769 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928668+0000", 2024-04-24T15:33:16.769 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.8920000000000001e-06 2024-04-24T15:33:16.769 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.769 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.770 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:16.770 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966240+0000", 2024-04-24T15:33:16.770 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037572886999999999 2024-04-24T15:33:16.770 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.770 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.770 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:16.770 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966281+0000", 2024-04-24T15:33:16.770 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0450000000000001e-05 2024-04-24T15:33:16.770 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.770 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.770 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:16.770 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966349+0000", 2024-04-24T15:33:16.770 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.8215999999999999e-05 2024-04-24T15:33:16.770 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:16.770 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:16.771 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:16.771 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.771 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.771 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48731 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2637824~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:16.771 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928747+0000", 2024-04-24T15:33:16.771 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 5.7661051810000004, 2024-04-24T15:33:16.771 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037833602000000001, 2024-04-24T15:33:16.771 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:16.771 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:16.771 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:16.771 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:16.771 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:16.771 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48731 2024-04-24T15:33:16.771 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.771 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:16.771 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.772 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:16.772 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928747+0000", 2024-04-24T15:33:16.772 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:16.772 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.772 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.772 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:16.772 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928747+0000", 2024-04-24T15:33:16.772 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:16.772 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.772 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.772 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:16.772 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928670+0000", 2024-04-24T15:33:16.772 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999228 2024-04-24T15:33:16.772 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.772 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.772 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:16.773 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928750+0000", 2024-04-24T15:33:16.773 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0527999999999999e-05 2024-04-24T15:33:16.773 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.773 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.773 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:16.773 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928757+0000", 2024-04-24T15:33:16.773 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.9419999999999998e-06 2024-04-24T15:33:16.773 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.773 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.773 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:16.773 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928762+0000", 2024-04-24T15:33:16.773 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4240000000000003e-06 2024-04-24T15:33:16.773 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.773 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.773 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:16.773 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966379+0000", 2024-04-24T15:33:16.774 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037617267000000003 2024-04-24T15:33:16.774 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.774 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.774 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:16.774 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966461+0000", 2024-04-24T15:33:16.774 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.2720999999999996e-05 2024-04-24T15:33:16.774 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.774 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.774 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:16.774 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966581+0000", 2024-04-24T15:33:16.774 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000119014 2024-04-24T15:33:16.774 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:16.774 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:16.774 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:16.775 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.775 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.775 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48732 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2703360~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:16.775 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928766+0000", 2024-04-24T15:33:16.775 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 5.7660856970000003, 2024-04-24T15:33:16.775 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037930924999999997, 2024-04-24T15:33:16.775 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:16.775 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:16.775 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:16.775 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:16.775 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:16.775 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48732 2024-04-24T15:33:16.775 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.775 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:16.776 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.776 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:16.776 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928766+0000", 2024-04-24T15:33:16.776 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:16.776 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.776 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.776 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:16.776 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928766+0000", 2024-04-24T15:33:16.776 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:16.776 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.776 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.776 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:16.776 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928766+0000", 2024-04-24T15:33:16.776 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:16.777 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.777 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.777 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:16.777 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928768+0000", 2024-04-24T15:33:16.777 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.3420000000000002e-06 2024-04-24T15:33:16.777 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.777 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.777 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:16.777 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928783+0000", 2024-04-24T15:33:16.777 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.4813e-05 2024-04-24T15:33:16.777 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.777 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.777 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:16.777 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928787+0000", 2024-04-24T15:33:16.777 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.1999999999999996e-06 2024-04-24T15:33:16.777 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.778 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.778 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:16.778 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966592+0000", 2024-04-24T15:33:16.778 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037804484999999999 2024-04-24T15:33:16.778 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.778 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.778 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:16.778 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966632+0000", 2024-04-24T15:33:16.778 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0014e-05 2024-04-24T15:33:16.778 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.778 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.778 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:16.778 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966697+0000", 2024-04-24T15:33:16.778 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.5489999999999998e-05 2024-04-24T15:33:16.779 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:16.779 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:16.779 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:16.779 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.779 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.779 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48813 4.d 4:b39400aa:::rbd_data.114a8b543e91.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:16.779 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.936015+0000", 2024-04-24T15:33:16.779 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 5.7588373629999996, 2024-04-24T15:33:16.779 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.039312294999999997, 2024-04-24T15:33:16.779 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:16.779 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:16.779 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:16.779 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:16.779 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:16.779 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48813 2024-04-24T15:33:16.779 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.780 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:16.780 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.780 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:16.780 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.936015+0000", 2024-04-24T15:33:16.780 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:16.780 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.780 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.780 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:16.780 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.936015+0000", 2024-04-24T15:33:16.780 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:16.780 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.780 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.780 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:16.780 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.936014+0000", 2024-04-24T15:33:16.780 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:16.780 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.781 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.781 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:16.781 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.942618+0000", 2024-04-24T15:33:16.781 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0066038499999999997 2024-04-24T15:33:16.781 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.781 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.781 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:16.781 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.942629+0000", 2024-04-24T15:33:16.781 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0606e-05 2024-04-24T15:33:16.781 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.781 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.781 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:16.781 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.942634+0000", 2024-04-24T15:33:16.781 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.2000000000000002e-06 2024-04-24T15:33:16.781 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.781 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.782 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:16.782 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.969006+0000", 2024-04-24T15:33:16.782 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.026371564 2024-04-24T15:33:16.782 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.782 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.782 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:16.782 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.969126+0000", 2024-04-24T15:33:16.782 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000120085 2024-04-24T15:33:16.782 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.782 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.782 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 2", 2024-04-24T15:33:16.782 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.969469+0000", 2024-04-24T15:33:16.782 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00034295400000000002 2024-04-24T15:33:16.782 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.782 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.783 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:33:16.783 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.973118+0000", 2024-04-24T15:33:16.783 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0036489700000000001 2024-04-24T15:33:16.783 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.783 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.783 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:33:16.783 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.975250+0000", 2024-04-24T15:33:16.783 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0021327030000000001 2024-04-24T15:33:16.783 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.783 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.783 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:33:16.783 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.975293+0000", 2024-04-24T15:33:16.783 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2951999999999998e-05 2024-04-24T15:33:16.783 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:16.783 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:16.783 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:16.784 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.975327+0000", 2024-04-24T15:33:16.784 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.3840000000000001e-05 2024-04-24T15:33:16.784 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:16.784 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:16.784 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:16.784 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:16.784 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:16.784 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:16.784 INFO:teuthology.orchestra.run.smithi151.stdout:Import successful 2024-04-24T15:33:16.785 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-24T15:33:16.786 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-24T15:33:16.787 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:16.788+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:16.838 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-24T15:33:16.888 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:33:16.889+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:33:16.900 INFO:teuthology.orchestra.run.smithi096.stdout:truncating image image_client.0-clone24 from 0xe733ffe (overlap 0xc5d30d) to 0x4f2138 2024-04-24T15:33:16.955 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:16.955 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:33:16.955 INFO:teuthology.orchestra.run.smithi151.stdout: "num_ops": 0 2024-04-24T15:33:16.955 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:16.958 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-04-24T15:33:16.965 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-24T15:33:16.966 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #23, image image_client.0-clone23 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent24 2024-04-24T15:33:16.979 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-04-24T15:33:16.989 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:16.990+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:17.004 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-04-24T15:33:17.024 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-04-24T15:33:17.045 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-04-24T15:33:17.065 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-04-24T15:33:17.084 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-04-24T15:33:17.084 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T15:33:17.084 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T15:33:17.084 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T15:33:17.085 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T15:33:17.085 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T15:33:17.085 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T15:33:17.085 INFO:teuthology.orchestra.run.smithi096.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-04-24T15:33:17.085 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T15:33:17.087 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-24T15:33:17.087 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-24T15:33:17.093 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:17.093 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:17.103 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-24T15:33:17.129 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:17.130+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:17.191 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:17.192+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:17.222 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm -f /var/log/ceph/exp.4.12.6 2024-04-24T15:33:17.225 INFO:teuthology.orchestra.run.smithi096.stdout:4592 read 0x1966627 thru 0x197517c (0xeb56 bytes) 2024-04-24T15:33:17.225 INFO:teuthology.orchestra.run.smithi096.stdout:4593 read 0x645933e thru 0x6466fe2 (0xdca5 bytes) 2024-04-24T15:33:17.225 INFO:teuthology.orchestra.run.smithi096.stdout:4594 read 0xc5230c5 thru 0xc5260bd (0x2ff9 bytes) 2024-04-24T15:33:17.228 INFO:teuthology.orchestra.run.smithi096.stdout:4595 trunc from 0xe733ffe to 0xc63318a 2024-04-24T15:33:17.229 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:17.229 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:17.241 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-24T15:33:17.246 INFO:teuthology.orchestra.run.smithi096.stdout:4596 writesame 0x5293304 thru 0x5296f0d data_size 0x6a(0x3c0a bytes) 2024-04-24T15:33:17.251 INFO:teuthology.orchestra.run.smithi096.stdout:4598 read 0x480ef65 thru 0x48182df (0x937b bytes) 2024-04-24T15:33:17.251 INFO:teuthology.orchestra.run.smithi096.stdout:4599 write 0x680921 thru 0x6872f7 (0x69d7 bytes) 2024-04-24T15:33:17.261 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T15:33:17.264 INFO:teuthology.orchestra.run.smithi096.stdout:4601 read 0x16eb870 thru 0x16f640a (0xab9b bytes) 2024-04-24T15:33:17.271 INFO:teuthology.orchestra.run.smithi096.stdout:4602 write 0xedc53b1 thru 0xedcaf47 (0x5b97 bytes) 2024-04-24T15:33:17.275 INFO:teuthology.orchestra.run.smithi096.stdout:4603 writesame 0x925a108 thru 0x9267334 data_size 0x11(0xd22d bytes) 2024-04-24T15:33:17.277 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:17.278 INFO:teuthology.orchestra.run.smithi096.stdout:4605 write 0x4abf787 thru 0x4ac0c81 (0x14fb bytes) 2024-04-24T15:33:17.283 INFO:teuthology.orchestra.run.smithi096.stdout:4608 write 0x83239eb thru 0x8328065 (0x467b bytes) 2024-04-24T15:33:17.285 INFO:teuthology.orchestra.run.smithi096.stdout:4609 write 0x2d2c645 thru 0x2d39a58 (0xd414 bytes) 2024-04-24T15:33:17.286 INFO:teuthology.orchestra.run.smithi096.stdout:4611 read 0x6b4cb89 thru 0x6b58324 (0xb79c bytes) 2024-04-24T15:33:17.287 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:17.287 INFO:teuthology.orchestra.run.smithi096.stdout:4614 write 0x47ffa8c thru 0x4804aeb (0x5060 bytes) 2024-04-24T15:33:17.292 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:17.293+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:17.295 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:17.295 INFO:teuthology.orchestra.run.smithi096.stdout:4616 read 0x1794105 thru 0x17a3f55 (0xfe51 bytes) 2024-04-24T15:33:17.295 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:17.295 INFO:teuthology.orchestra.run.smithi096.stdout:4618 writesame 0xa2ff1ea thru 0xa300be1 data_size 0xc(0x19f8 bytes) 2024-04-24T15:33:17.297 INFO:teuthology.orchestra.run.smithi096.stdout:4619 writesame 0x81bd6b0 thru 0x81c7d9b data_size 0x12(0xa6ec bytes) 2024-04-24T15:33:17.299 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:17.299 INFO:teuthology.orchestra.run.smithi096.stdout:4621 write 0x576ea8 thru 0x57da86 (0x6bdf bytes) 2024-04-24T15:33:17.300 INFO:teuthology.orchestra.run.smithi096.stdout:4623 read 0x7b2796b thru 0x7b29b9f (0x2235 bytes) 2024-04-24T15:33:17.301 INFO:teuthology.orchestra.run.smithi096.stdout:4625 read 0xc2cb16f thru 0xc2d1574 (0x6406 bytes) 2024-04-24T15:33:17.301 INFO:teuthology.orchestra.run.smithi096.stdout:4627 read 0xba11557 thru 0xba1c030 (0xaada bytes) 2024-04-24T15:33:17.301 INFO:teuthology.orchestra.run.smithi096.stdout:4628 read 0xa4ee392 thru 0xa4f34c1 (0x5130 bytes) 2024-04-24T15:33:17.302 INFO:teuthology.orchestra.run.smithi096.stdout:4629 read 0xb1cb902 thru 0xb1cc605 (0xd04 bytes) 2024-04-24T15:33:17.302 INFO:teuthology.orchestra.run.smithi096.stdout:4630 read 0xdd13e88 thru 0xdd188c0 (0x4a39 bytes) 2024-04-24T15:33:17.304 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:17.304 INFO:teuthology.orchestra.run.smithi096.stdout:4634 read 0x36c8aa4 thru 0x36d2cb7 (0xa214 bytes) 2024-04-24T15:33:17.304 INFO:teuthology.orchestra.run.smithi096.stdout:4635 read 0xe5b855a thru 0xe5c5bfb (0xd6a2 bytes) 2024-04-24T15:33:17.304 INFO:teuthology.orchestra.run.smithi096.stdout:4636 read 0xcf4b6b1 thru 0xcf4f1cd (0x3b1d bytes) 2024-04-24T15:33:17.304 INFO:teuthology.orchestra.run.smithi096.stdout:4637 read 0x929ffa5 thru 0x92a52e1 (0x533d bytes) 2024-04-24T15:33:17.307 INFO:teuthology.orchestra.run.smithi096.stdout:4638 trunc from 0xedcaf48 to 0xc1c7026 2024-04-24T15:33:17.323 INFO:teuthology.orchestra.run.smithi096.stdout:4640 punch from 0x2b7a038 to 0x2b81610, (0x75d8 bytes) 2024-04-24T15:33:17.323 INFO:teuthology.orchestra.run.smithi096.stdout:4642 write 0xa914779 thru 0xa915ce6 (0x156e bytes) 2024-04-24T15:33:17.325 INFO:teuthology.orchestra.run.smithi096.stdout:4645 read 0xada222d thru 0xada23de (0x1b2 bytes) 2024-04-24T15:33:17.325 INFO:teuthology.orchestra.run.smithi096.stdout:4646 write 0x954a892 thru 0x954adc1 (0x530 bytes) 2024-04-24T15:33:17.326 INFO:teuthology.orchestra.run.smithi096.stdout:4649 read 0x9a9fe3a thru 0x9aa90ed (0x92b4 bytes) 2024-04-24T15:33:17.331 INFO:teuthology.orchestra.run.smithi096.stdout:4650 trunc from 0xc1c7026 to 0x7a2b357 2024-04-24T15:33:17.352 INFO:teuthology.orchestra.run.smithi096.stdout:4651 punch from 0x39ff30b to 0x3a07d55, (0x8a4a bytes) 2024-04-24T15:33:17.362 INFO:teuthology.orchestra.run.smithi096.stdout:4652 writesame 0xb248a17 thru 0xb24e5d1 data_size 0x1(0x5bbb bytes) 2024-04-24T15:33:17.362 INFO:teuthology.orchestra.run.smithi096.stdout:4655 read 0x5351e1a thru 0x535fe0c (0xdff3 bytes) 2024-04-24T15:33:17.363 INFO:teuthology.orchestra.run.smithi096.stdout:4657 read 0x4026a3a thru 0x40349a0 (0xdf67 bytes) 2024-04-24T15:33:17.363 INFO:teuthology.orchestra.run.smithi096.stdout:4658 read 0x6e463d0 thru 0x6e47858 (0x1489 bytes) 2024-04-24T15:33:17.364 INFO:teuthology.orchestra.run.smithi096.stdout:4659 punch from 0xa9ca604 to 0xa9d82d4, (0xdcd0 bytes) 2024-04-24T15:33:17.366 INFO:teuthology.orchestra.run.smithi096.stdout:4660 trunc from 0xb24e5d2 to 0xcc2676d 2024-04-24T15:33:17.377 INFO:teuthology.orchestra.run.smithi096.stdout:4661 writesame 0xd2adcdc thru 0xd2af03a data_size 0x9(0x135f bytes) 2024-04-24T15:33:17.377 INFO:teuthology.orchestra.run.smithi096.stdout:4662 punch from 0x917efcf to 0x918bcd2, (0xcd03 bytes) 2024-04-24T15:33:17.377 INFO:teuthology.orchestra.run.smithi096.stdout:4664 write 0x8422fc9 thru 0x843214c (0xf184 bytes) 2024-04-24T15:33:17.381 INFO:teuthology.orchestra.run.smithi096.stdout:4667 punch from 0xba79431 to 0xba7cd4f, (0x391e bytes) 2024-04-24T15:33:17.381 INFO:teuthology.orchestra.run.smithi096.stdout:4668 read 0x969478b thru 0x969e06d (0x98e3 bytes) 2024-04-24T15:33:17.381 INFO:teuthology.orchestra.run.smithi096.stdout:4669 read 0x9c6d34b thru 0x9c77231 (0x9ee7 bytes) 2024-04-24T15:33:17.381 INFO:teuthology.orchestra.run.smithi096.stdout:4670 read 0x67cd07 thru 0x67f597 (0x2891 bytes) 2024-04-24T15:33:17.381 INFO:teuthology.orchestra.run.smithi096.stdout:4671 writesame 0xa67a657 thru 0xa686a62 data_size 0x2(0xc40c bytes) 2024-04-24T15:33:17.393 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:17.394+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:17.395 INFO:teuthology.orchestra.run.smithi096.stdout:4672 read 0x93530a5 thru 0x935ef9f (0xbefb bytes) 2024-04-24T15:33:17.395 INFO:teuthology.orchestra.run.smithi096.stdout:4673 write 0x8eba1a7 thru 0x8ec78e0 (0xd73a bytes) 2024-04-24T15:33:17.397 INFO:teuthology.orchestra.run.smithi096.stdout:4674 punch from 0x6ad0bde to 0x6add738, (0xcb5a bytes) 2024-04-24T15:33:17.397 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:17.397 INFO:teuthology.orchestra.run.smithi096.stdout:4676 read 0x479d3ea thru 0x47a1da5 (0x49bc bytes) 2024-04-24T15:33:17.397 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:17.397 INFO:teuthology.orchestra.run.smithi096.stdout:4679 writesame 0x576d6a6 thru 0x57741c1 data_size 0x1e(0x6b1c bytes) 2024-04-24T15:33:17.415 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:17.428 INFO:teuthology.orchestra.run.smithi096.stdout:4681 trunc from 0xd2af03b to 0x24f8734 2024-04-24T15:33:17.434 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:17.434 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:17.446 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-24T15:33:17.463 INFO:teuthology.orchestra.run.smithi096.stdout:4684 writesame 0x2d2a7f2 thru 0x2d384af data_size 0x5(0xdcbe bytes) 2024-04-24T15:33:17.464 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:17.471 INFO:teuthology.orchestra.run.smithi096.stdout:4688 write 0x45ec781 thru 0x45ed571 (0xdf1 bytes) 2024-04-24T15:33:17.472 INFO:teuthology.orchestra.run.smithi096.stdout:4689 punch from 0x115fc2f to 0x116bc6e, (0xc03f bytes) 2024-04-24T15:33:17.473 INFO:teuthology.orchestra.run.smithi096.stdout:4690 writesame 0x2b934d3 thru 0x2ba0317 data_size 0x3b(0xce45 bytes) 2024-04-24T15:33:17.474 INFO:teuthology.orchestra.run.smithi096.stdout:4691 read 0x54833c thru 0x54dcfa (0x59bf bytes) 2024-04-24T15:33:17.486 INFO:teuthology.orchestra.run.smithi096.stdout:4694 write 0xa042fbf thru 0xa049349 (0x638b bytes) 2024-04-24T15:33:17.489 INFO:teuthology.orchestra.run.smithi096.stdout:4696 read 0x9163d1b thru 0x916ad31 (0x7017 bytes) 2024-04-24T15:33:17.489 INFO:teuthology.orchestra.run.smithi096.stdout:4697 write 0x9abe716 thru 0x9ac35fd (0x4ee8 bytes) 2024-04-24T15:33:17.492 INFO:teuthology.orchestra.run.smithi096.stdout:4698 read 0x88e9da thru 0x8923de (0x3a05 bytes) 2024-04-24T15:33:17.492 INFO:teuthology.orchestra.run.smithi096.stdout:4699 write 0x3187c7c thru 0x319433b (0xc6c0 bytes) 2024-04-24T15:33:17.494 INFO:teuthology.orchestra.run.smithi096.stdout:4700 read 0x574acfe thru 0x5754e74 (0xa177 bytes) 2024-04-24T15:33:17.494 INFO:teuthology.orchestra.run.smithi096.stdout:4701 writesame 0x6b1e453 thru 0x6b2a4ba data_size 0x83(0xc068 bytes) 2024-04-24T15:33:17.494 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:33:17.495+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:33:17.495 INFO:teuthology.orchestra.run.smithi096.stdout:4703 write 0x9370fa0 thru 0x937632e (0x538f bytes) 2024-04-24T15:33:17.497 INFO:teuthology.orchestra.run.smithi096.stdout:4704 read 0x343cb6a thru 0x3445824 (0x8cbb bytes) 2024-04-24T15:33:17.498 INFO:teuthology.orchestra.run.smithi096.stdout:4705 write 0x22bcd2e thru 0x22bce9c (0x16f bytes) 2024-04-24T15:33:17.504 INFO:teuthology.orchestra.run.smithi096.stdout:4706 trunc from 0xa04934a to 0x5a56904 2024-04-24T15:33:17.517 INFO:teuthology.orchestra.run.smithi096.stdout:4707 punch from 0x43f35cd to 0x440343f, (0xfe72 bytes) 2024-04-24T15:33:17.532 INFO:teuthology.orchestra.run.smithi096.stdout:4708 writesame 0xcfddf2c thru 0xcfe7a60 data_size 0x1(0x9b35 bytes) 2024-04-24T15:33:17.532 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:17.532 INFO:teuthology.orchestra.run.smithi096.stdout:4711 read 0x9a66eaf thru 0x9a6c96a (0x5abc bytes) 2024-04-24T15:33:17.532 INFO:teuthology.orchestra.run.smithi096.stdout:4713 punch from 0x6de5bbe to 0x6df1e13, (0xc255 bytes) 2024-04-24T15:33:17.532 INFO:teuthology.orchestra.run.smithi096.stdout:4714 punch from 0x6eaa243 to 0x6eb041e, (0x61db bytes) 2024-04-24T15:33:17.532 INFO:teuthology.orchestra.run.smithi096.stdout:4715 punch from 0x262750c to 0x262fca2, (0x8796 bytes) 2024-04-24T15:33:17.541 INFO:teuthology.orchestra.run.smithi096.stdout:4716 trunc from 0xcfe7a61 to 0x590bda8 2024-04-24T15:33:17.564 INFO:teuthology.orchestra.run.smithi096.stdout:4717 write 0xe63e676 thru 0xe6454bf (0x6e4a bytes) 2024-04-24T15:33:17.567 INFO:teuthology.orchestra.run.smithi096.stdout:4718 punch from 0xd61be20 to 0xd62bc27, (0xfe07 bytes) 2024-04-24T15:33:17.567 INFO:teuthology.orchestra.run.smithi096.stdout:4719 read 0x62533f thru 0x631ce1 (0xc9a3 bytes) 2024-04-24T15:33:17.567 INFO:teuthology.orchestra.run.smithi096.stdout:4720 writesame 0xcf84a70 thru 0xcf8ca90 data_size 0x1(0x8021 bytes) 2024-04-24T15:33:17.573 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:17.573 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:17.580 INFO:teuthology.orchestra.run.smithi096.stdout:4721 read 0x3d8a7be thru 0x3d8d1dd (0x2a20 bytes) 2024-04-24T15:33:17.580 INFO:teuthology.orchestra.run.smithi096.stdout:4722 read 0xe4e0aaa thru 0xe4ecf1f (0xc476 bytes) 2024-04-24T15:33:17.581 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:17.581 INFO:teuthology.orchestra.run.smithi096.stdout:4727 write 0xba45cf6 thru 0xba4d57c (0x7887 bytes) 2024-04-24T15:33:17.583 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-24T15:33:17.584 INFO:teuthology.orchestra.run.smithi096.stdout:4729 writesame 0xbe28290 thru 0xbe28336 data_size 0x1(0xa7 bytes) 2024-04-24T15:33:17.586 INFO:teuthology.orchestra.run.smithi096.stdout:4731 write 0xc22928b thru 0xc23398e (0xa704 bytes) 2024-04-24T15:33:17.589 INFO:teuthology.orchestra.run.smithi096.stdout:4736 punch from 0xa2d2ff9 to 0xa2e1cb8, (0xecbf bytes) 2024-04-24T15:33:17.589 INFO:teuthology.orchestra.run.smithi096.stdout:4737 writesame 0x53dbae1 thru 0x53dfa7f data_size 0x3d(0x3f9f bytes) 2024-04-24T15:33:17.595 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:33:17.596+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:33:17.598 INFO:teuthology.orchestra.run.smithi096.stdout:4739 trunc from 0xe6454c0 to 0x8a4364a 2024-04-24T15:33:17.611 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:17.611 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:17.611 INFO:teuthology.orchestra.run.smithi096.stdout:4742 write 0x72c3baa thru 0x72c9cb3 (0x610a bytes) 2024-04-24T15:33:17.615 INFO:teuthology.orchestra.run.smithi096.stdout:4743 read 0x51ae0d3 thru 0x51bbf0f (0xde3d bytes) 2024-04-24T15:33:17.616 INFO:teuthology.orchestra.run.smithi096.stdout:4744 read 0x673180e thru 0x6738e01 (0x75f4 bytes) 2024-04-24T15:33:17.616 INFO:teuthology.orchestra.run.smithi096.stdout:4745 punch from 0x13f6e98 to 0x13fb8fb, (0x4a63 bytes) 2024-04-24T15:33:17.616 INFO:teuthology.orchestra.run.smithi096.stdout:4746 writesame 0x17a1094 thru 0x17a87ff data_size 0x2(0x776c bytes) 2024-04-24T15:33:17.621 INFO:teuthology.orchestra.run.smithi096.stdout:4747 read 0x8978cf2 thru 0x8985d26 (0xd035 bytes) 2024-04-24T15:33:17.622 INFO:teuthology.orchestra.run.smithi096.stdout:4749 read 0x5c987e5 thru 0x5c9eb42 (0x635e bytes) 2024-04-24T15:33:17.622 INFO:teuthology.orchestra.run.smithi096.stdout:4750 punch from 0x435d4f8 to 0x4367505, (0xa00d bytes) 2024-04-24T15:33:17.622 INFO:teuthology.orchestra.run.smithi096.stdout:4751 read 0x8d1d09 thru 0x8e0e13 (0xf10b bytes) 2024-04-24T15:33:17.623 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:17.623 INFO:teuthology.orchestra.run.smithi096.stdout:4753 write 0x55dc11d thru 0x55e78d7 (0xb7bb bytes) 2024-04-24T15:33:17.631 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:33:17.631 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":81,"fsid":"253d426c-62cb-4194-8514-6cd970fd85c7","created":"2024-04-24T15:31:47.870327+0000","modified":"2024-04-24T15:33:16.829964+0000","last_up_change":"2024-04-24T15:33:09.796895+0000","last_in_change":"2024-04-24T15:32:58.722778+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T15:31:59.589762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-24T15:31:59.971601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"21","last_force_op_resend_prenautilus":"21","last_force_op_resend_preluminous":"21","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[3],"tier_of":-1,"read_tier":3,"write_tier":3,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cache","create_time":"2024-04-24T15:32:06.914463+0000","flags":9,"flags_names":"hashpspool,incomplete_clones","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":4,"pg_placement_num":4,"pg_placement_num_target":4,"pg_num_target":4,"pg_num_pending":4,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"21","last_force_op_resend_prenautilus":"21","last_force_op_resend_preluminous":"21","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":2,"read_tier":-1,"write_tier":-1,"cache_mode":"writeback","target_max_bytes":0,"target_max_objects":250,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"bloom","false_positive_probability":0.050000000000000003,"target_size":0,"seed":0},"hit_set_period":60,"hit_set_count":8,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"pool_client.0","create_time":"2024-04-24T15:32:15.929114+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":28,"snap_epoch":81,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"92941fb9-96ee-4b2b-8147-3c1d61192f94","up":1,"in":1,"weight":1,"primary_affinity":0,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":44106},{"type":"v1","addr":"172.21.15.96:6817","nonce":44106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":44106},{"type":"v1","addr":"172.21.15.96:6819","nonce":44106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":44106},{"type":"v1","addr":"172.21.15.96:6823","nonce":44106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":44106},{"type":"v1","addr":"172.21.15.96:6821","nonce":44106}]},"public_addr":"172.21.15.96:6817/44106","cluster_addr":"172.21.15.96:6819/44106","heartbeat_back_addr":"172.21.15.96:6823/44106","heartbeat_front_addr":"172.21.15.96:6821/44106","state":["exists","up"]},{"osd":1,"uuid":"628e868a-9884-42e6-9331-caef97cb6967","up":1,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":44089},{"type":"v1","addr":"172.21.15.96:6801","nonce":44089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":44089},{"type":"v1","addr":"172.21.15.96:6803","nonce":44089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":44089},{"type":"v1","addr":"172.21.15.96:6807","nonce":44089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":44089},{"type":"v1","addr":"172.21.15.96:6805","nonce":44089}]},"public_addr":"172.21.15.96:6801/44089","cluster_addr":"172.21.15.96:6803/44089","heartbeat_back_addr":"172.21.15.96:6807/44089","heartbeat_front_addr":"172.21.15.96:6805/44089","state":["exists","up"]},{"osd":2,"uuid":"46de3f65-87a8-460c-b57a-4468d9c89a57","up":1,"in":1,"weight":1,"primary_affinity":0.3755035400390625,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":44090},{"type":"v1","addr":"172.21.15.96:6809","nonce":44090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":44090},{"type":"v1","addr":"172.21.15.96:6811","nonce":44090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":44090},{"type":"v1","addr":"172.21.15.96:6815","nonce":44090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":44090},{"type":"v1","addr":"172.21.15.96:6813","nonce":44090}]},"public_addr":"172.21.15.96:6809/44090","cluster_addr":"172.21.15.96:6811/44090","heartbeat_back_addr":"172.21.15.96:6815/44090","heartbeat_front_addr":"172.21.15.96:6813/44090","state":["exists","up"]},{"osd":3,"uuid":"5507e063-10bc-4e32-b1ec-00aa3e47d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":44121},{"type":"v1","addr":"172.21.15.96:6825","nonce":44121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":44121},{"type":"v1","addr":"172.21.15.96:6827","nonce":44121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":44121},{"type":"v1","addr":"172.21.15.96:6831","nonce":44121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":44121},{"type":"v1","addr":"172.21.15.96:6829","nonce":44121}]},"public_addr":"172.21.15.96:6825/44121","cluster_addr":"172.21.15.96:6827/44121","heartbeat_back_addr":"172.21.15.96:6831/44121","heartbeat_front_addr":"172.21.15.96:6829/44121","state":["exists","up"]},{"osd":4,"uuid":"6a74d453-1048-44ef-a5b0-4277583d76c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":44025},{"type":"v1","addr":"172.21.15.151:6817","nonce":44025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":44025},{"type":"v1","addr":"172.21.15.151:6819","nonce":44025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":44025},{"type":"v1","addr":"172.21.15.151:6823","nonce":44025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":44025},{"type":"v1","addr":"172.21.15.151:6821","nonce":44025}]},"public_addr":"172.21.15.151:6817/44025","cluster_addr":"172.21.15.151:6819/44025","heartbeat_back_addr":"172.21.15.151:6823/44025","heartbeat_front_addr":"172.21.15.151:6821/44025","state":["exists","up"]},{"osd":5,"uuid":"3645ef18-5574-49f8-ab81-b40651208309","up":1,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":44023},{"type":"v1","addr":"172.21.15.151:6801","nonce":44023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":44023},{"type":"v1","addr":"172.21.15.151:6803","nonce":44023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":44023},{"type":"v1","addr":"172.21.15.151:6807","nonce":44023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":44023},{"type":"v1","addr":"172.21.15.151:6805","nonce":44023}]},"public_addr":"172.21.15.151:6801/44023","cluster_addr":"172.21.15.151:6803/44023","heartbeat_back_addr":"172.21.15.151:6807/44023","heartbeat_front_addr":"172.21.15.151:6805/44023","state":["exists","up"]},{"osd":6,"uuid":"6b8b0b36-9246-4241-a197-c69fa48a6202","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":76,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":44057},{"type":"v1","addr":"172.21.15.151:6825","nonce":44057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":44057},{"type":"v1","addr":"172.21.15.151:6827","nonce":44057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":44057},{"type":"v1","addr":"172.21.15.151:6831","nonce":44057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":44057},{"type":"v1","addr":"172.21.15.151:6829","nonce":44057}]},"public_addr":"172.21.15.151:6825/44057","cluster_addr":"172.21.15.151:6827/44057","heartbeat_back_addr":"172.21.15.151:6831/44057","heartbeat_front_addr":"172.21.15.151:6829/44057","state":["exists"]},{"osd":7,"uuid":"4501cd9f-e437-4ddd-af3d-129fdf1ee61a","up":1,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":44024},{"type":"v1","addr":"172.21.15.151:6809","nonce":44024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":44024},{"type":"v1","addr":"172.21.15.151:6811","nonce":44024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":44024},{"type":"v1","addr":"172.21.15.151:6815","nonce":44024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":44024},{"type":"v1","addr":"172.21.15.151:6813","nonce":44024}]},"public_addr":"172.21.15.151:6809/44024","cluster_addr":"172.21.15.151:6811/44024","heartbeat_back_addr":"172.21.15.151:6815/44024","heartbeat_front_addr":"172.21.15.151:6813/44024","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T15:31:56.300910+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":65536,"last_purged_snaps_scrub":"2024-04-24T15:31:56.202064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T15:31:56.198961+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T15:31:56.346777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T15:31:56.282585+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":65536,"last_purged_snaps_scrub":"2024-04-24T15:31:56.257943+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T15:31:56.260479+0000","dead_epoch":76},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":65536,"last_purged_snaps_scrub":"2024-04-24T15:31:56.242093+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T15:33:17.632 INFO:teuthology.orchestra.run.smithi096.stdout:4757 writesame 0x9ad1adc thru 0x9ad642a data_size 0x7(0x494f bytes) 2024-04-24T15:33:17.632 INFO:teuthology.orchestra.run.smithi096.stdout:4758 read 0x4f86cfb thru 0x4f8f29b (0x85a1 bytes) 2024-04-24T15:33:17.632 INFO:teuthology.orchestra.run.smithi096.stdout:4759 read 0x1730587 thru 0x1735750 (0x51ca bytes) 2024-04-24T15:33:17.632 INFO:teuthology.orchestra.run.smithi096.stdout:4760 clone 26 order 21 su 16384 sc 7 2024-04-24T15:33:17.645 INFO:tasks.ceph.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T15:31:59.589762+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-04-24T15:31:59.971601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '21', 'last_force_op_resend_prenautilus': '21', 'last_force_op_resend_preluminous': '21', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 19, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [3], 'tier_of': -1, 'read_tier': 3, 'write_tier': 3, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cache', 'create_time': '2024-04-24T15:32:06.914463+0000', 'flags': 9, 'flags_names': 'hashpspool,incomplete_clones', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 4, 'pg_placement_num': 4, 'pg_placement_num_target': 4, 'pg_num_target': 4, 'pg_num_pending': 4, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '21', 'last_force_op_resend_prenautilus': '21', 'last_force_op_resend_preluminous': '21', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 19, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': 2, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'writeback', 'target_max_bytes': 0, 'target_max_objects': 250, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'bloom', 'false_positive_probability': 0.05, 'target_size': 0, 'seed': 0}, 'hit_set_period': 60, 'hit_set_count': 8, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [1000000, 1000000, 1000000, 1000000, 1000000, 1000000, 1000000, 1000000], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 4, 'pool_name': 'pool_client.0', 'create_time': '2024-04-24T15:32:15.929114+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 28, 'snap_epoch': 81, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2024-04-24T15:33:17.646 DEBUG:teuthology.orchestra.run.smithi151:> CEPH_ARGS='--filestore-merge-threshold 1 --filestore-split-multiple 1' sudo -E ceph-objectstore-tool --no-mon-config --log-file=/var/log/ceph/objectstore_tool.$pid.log --data-path /var/lib/ceph/osd/ceph-6 --journal-path /var/lib/ceph/osd/ceph-6/journal --op apply-layout-settings --pool .mgr 2024-04-24T15:33:17.695 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:33:17.697+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:33:17.713 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:17.713 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:17.724 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-24T15:33:17.796 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:33:17.797+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:33:17.850 INFO:teuthology.orchestra.run.smithi096.stderr:noscrub is set 2024-04-24T15:33:17.851 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:17.851 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:17.865 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-24T15:33:17.909 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:17.910+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:17.909 INFO:teuthology.orchestra.run.smithi096.stdout:truncating image image_client.0-clone25 from 0x9ad642b (overlap 0x24f8734) to 0x127c39a 2024-04-24T15:33:17.994 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #24, image image_client.0-clone24 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent25 2024-04-24T15:33:17.998 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:33:17.999+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:33:18.006 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:18.006 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:18.017 INFO:teuthology.orchestra.run.smithi096.stdout:4761 writesame 0xd491c10 thru 0xd492271 data_size 0x13(0x662 bytes) 2024-04-24T15:33:18.017 INFO:teuthology.orchestra.run.smithi096.stdout:4762 punch from 0x23c5323 to 0x23d2ff1, (0xdcce bytes) 2024-04-24T15:33:18.017 INFO:teuthology.orchestra.run.smithi096.stdout:4763 punch from 0x12887f3 to 0x129530b, (0xcb18 bytes) 2024-04-24T15:33:18.017 INFO:teuthology.orchestra.run.smithi096.stdout:4764 read 0xc901a44 thru 0xc907210 (0x57cd bytes) 2024-04-24T15:33:18.018 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-24T15:33:18.028 INFO:teuthology.orchestra.run.smithi096.stdout:4765 trunc from 0xd492272 to 0x337d7c0 2024-04-24T15:33:18.098 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:33:18.100+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:33:18.124 INFO:teuthology.orchestra.run.smithi096.stdout:4766 punch from 0x23cc961 to 0x23cd3c8, (0xa67 bytes) 2024-04-24T15:33:18.124 INFO:teuthology.orchestra.run.smithi096.stdout:4767 write 0x1bfc668 thru 0x1c0ab69 (0xe502 bytes) 2024-04-24T15:33:18.165 INFO:teuthology.orchestra.run.smithi096.stdout:4768 writesame 0x5dfc4f1 thru 0x5e0b46e data_size 0x5(0xef7e bytes) 2024-04-24T15:33:18.165 INFO:teuthology.orchestra.run.smithi096.stdout:4769 punch from 0x342ad6 to 0x34d067, (0xa591 bytes) 2024-04-24T15:33:18.165 INFO:teuthology.orchestra.run.smithi096.stdout:4770 punch from 0xe9a5c7 to 0xea9307, (0xed40 bytes) 2024-04-24T15:33:18.165 INFO:teuthology.orchestra.run.smithi096.stdout:4772 read 0x5bf68f4 thru 0x5bfd526 (0x6c33 bytes) 2024-04-24T15:33:18.166 INFO:teuthology.orchestra.run.smithi096.stdout:4773 trunc from 0x5e0b46f to 0x5d524fb 2024-04-24T15:33:18.179 INFO:teuthology.orchestra.run.smithi096.stdout:4774 trunc from 0x5d524fb to 0xdb01b36 2024-04-24T15:33:18.184 INFO:teuthology.orchestra.run.smithi096.stdout:4775 punch from 0x322d7fa to 0x322fad1, (0x22d7 bytes) 2024-04-24T15:33:18.184 INFO:teuthology.orchestra.run.smithi096.stdout:4776 writesame 0x58de3a2 thru 0x58e27d2 data_size 0x21(0x4431 bytes) 2024-04-24T15:33:18.188 INFO:teuthology.orchestra.run.smithi096.stdout:4778 read 0xb994b47 thru 0xb99fa27 (0xaee1 bytes) 2024-04-24T15:33:18.188 INFO:teuthology.orchestra.run.smithi096.stdout:4780 read 0x619a2c9 thru 0x61a2a30 (0x8768 bytes) 2024-04-24T15:33:18.188 INFO:teuthology.orchestra.run.smithi096.stdout:4782 read 0xb6ef7e3 thru 0xb6f4452 (0x4c70 bytes) 2024-04-24T15:33:18.188 INFO:teuthology.orchestra.run.smithi096.stdout:4784 read 0x2954769 thru 0x295f326 (0xabbe bytes) 2024-04-24T15:33:18.191 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:18.192 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:18.194 INFO:teuthology.orchestra.run.smithi096.stdout:4785 write 0xe46cee2 thru 0xe474b17 (0x7c36 bytes) 2024-04-24T15:33:18.199 INFO:teuthology.orchestra.run.smithi096.stdout:4788 punch from 0xc97b980 to 0xc983a74, (0x80f4 bytes) 2024-04-24T15:33:18.199 INFO:teuthology.orchestra.run.smithi096.stdout:4789 write 0xb3b0e78 thru 0xb3bc0a0 (0xb229 bytes) 2024-04-24T15:33:18.199 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:18.200+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:18.204 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-24T15:33:18.216 INFO:teuthology.orchestra.run.smithi096.stdout:4791 trunc from 0xe474b18 to 0x4a32ee0 2024-04-24T15:33:18.227 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:18.237 INFO:teuthology.orchestra.run.smithi096.stdout:4796 write 0x885da40 thru 0x8862a31 (0x4ff2 bytes) 2024-04-24T15:33:18.241 INFO:teuthology.orchestra.run.smithi096.stdout:4798 punch from 0x7d8f363 to 0x7d935e8, (0x4285 bytes) 2024-04-24T15:33:18.241 INFO:teuthology.orchestra.run.smithi096.stdout:4799 read 0xc9aab9 thru 0xca1334 (0x687c bytes) 2024-04-24T15:33:18.242 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:18.242 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:18.242 INFO:teuthology.orchestra.run.smithi096.stdout:4802 punch from 0x4fcbe78 to 0x4fcf0c4, (0x324c bytes) 2024-04-24T15:33:18.242 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:18.242 INFO:teuthology.orchestra.run.smithi096.stdout:4805 read 0x6946008 thru 0x6952af9 (0xcaf2 bytes) 2024-04-24T15:33:18.242 INFO:teuthology.orchestra.run.smithi096.stdout:4808 read 0x6b95525 thru 0x6b9b74f (0x622b bytes) 2024-04-24T15:33:18.242 INFO:teuthology.orchestra.run.smithi096.stdout:4809 read 0x37052a5 thru 0x37145f8 (0xf354 bytes) 2024-04-24T15:33:18.254 INFO:teuthology.orchestra.run.smithi096.stdout:4810 write 0xd8c1fc2 thru 0xd8c96c9 (0x7708 bytes) 2024-04-24T15:33:18.259 INFO:teuthology.orchestra.run.smithi096.stdout:4813 write 0x8e51b5c thru 0x8e60c93 (0xf138 bytes) 2024-04-24T15:33:18.271 INFO:teuthology.orchestra.run.smithi096.stdout:4814 punch from 0x6765ba3 to 0x677192a, (0xbd87 bytes) 2024-04-24T15:33:18.271 INFO:teuthology.orchestra.run.smithi096.stdout:4815 write 0x549c554 thru 0x549fe94 (0x3941 bytes) 2024-04-24T15:33:18.274 INFO:teuthology.orchestra.run.smithi096.stdout:4816 write 0x284f025 thru 0x2855c5d (0x6c39 bytes) 2024-04-24T15:33:18.300 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:18.300+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:18.301 INFO:teuthology.orchestra.run.smithi096.stdout:4817 write 0x2917ec8 thru 0x2920d28 (0x8e61 bytes) 2024-04-24T15:33:18.322 INFO:teuthology.orchestra.run.smithi096.stdout:4818 trunc from 0xd8c96ca to 0xa9d0eef 2024-04-24T15:33:18.337 INFO:teuthology.orchestra.run.smithi096.stdout:4821 trunc from 0xa9d0eef to 0x78ecb98 2024-04-24T15:33:18.350 INFO:teuthology.orchestra.run.smithi096.stdout:4823 trunc from 0x78ecb98 to 0x8e35869 2024-04-24T15:33:18.355 INFO:teuthology.orchestra.run.smithi096.stdout:4824 write 0x1a03d7b thru 0x1a0afc8 (0x724e bytes) 2024-04-24T15:33:18.373 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:18.373 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:18.385 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-24T15:33:18.387 INFO:teuthology.orchestra.run.smithi096.stdout:4825 read 0x355cc68 thru 0x35688d2 (0xbc6b bytes) 2024-04-24T15:33:18.387 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:18.387 INFO:teuthology.orchestra.run.smithi096.stdout:4828 read 0x780c4fe thru 0x781b7e3 (0xf2e6 bytes) 2024-04-24T15:33:18.387 INFO:teuthology.orchestra.run.smithi096.stdout:4830 read 0x721ab27 thru 0x7220190 (0x566a bytes) 2024-04-24T15:33:18.388 INFO:teuthology.orchestra.run.smithi096.stdout:4831 read 0x7e6be89 thru 0x7e6c2c1 (0x439 bytes) 2024-04-24T15:33:18.388 INFO:teuthology.orchestra.run.smithi096.stdout:4832 read 0x371ca70 thru 0x3722969 (0x5efa bytes) 2024-04-24T15:33:18.388 INFO:teuthology.orchestra.run.smithi096.stdout:4833 read 0x6c0f291 thru 0x6c0fe3c (0xbac bytes) 2024-04-24T15:33:18.389 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:18.389 INFO:teuthology.orchestra.run.smithi096.stdout:4837 punch from 0x2d9f2c7 to 0x2dab3a0, (0xc0d9 bytes) 2024-04-24T15:33:18.389 INFO:teuthology.orchestra.run.smithi096.stdout:4839 punch from 0x6f5888b to 0x6f5ef1c, (0x6691 bytes) 2024-04-24T15:33:18.399 INFO:teuthology.orchestra.run.smithi096.stdout:4841 write 0xdb2724b thru 0xdb2a9e6 (0x379c bytes) 2024-04-24T15:33:18.401 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:18.402+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:18.403 INFO:teuthology.orchestra.run.smithi096.stdout:4843 read 0x5f43892 thru 0x5f494aa (0x5c19 bytes) 2024-04-24T15:33:18.403 INFO:teuthology.orchestra.run.smithi096.stdout:4844 punch from 0x326e686 to 0x32713fa, (0x2d74 bytes) 2024-04-24T15:33:18.403 INFO:teuthology.orchestra.run.smithi096.stdout:4845 punch from 0x9c600ea to 0x9c679fb, (0x7911 bytes) 2024-04-24T15:33:18.404 INFO:teuthology.orchestra.run.smithi096.stdout:4849 write 0x42aab9b thru 0x42aeaef (0x3f55 bytes) 2024-04-24T15:33:18.408 INFO:teuthology.orchestra.run.smithi096.stdout:4850 punch from 0x92a271b to 0x92ad8c6, (0xb1ab bytes) 2024-04-24T15:33:18.408 INFO:teuthology.orchestra.run.smithi096.stdout:4851 writesame 0x1f965fa thru 0x1f97067 data_size 0xf(0xa6e bytes) 2024-04-24T15:33:18.409 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:18.409 INFO:teuthology.orchestra.run.smithi096.stdout:4854 clone 27 order 22 su 16384 sc 5 2024-04-24T15:33:18.502 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:18.504+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:18.511 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:18.512 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:18.522 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-24T15:33:18.605 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:33:18.607+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:33:18.684 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:18.685 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:18.694 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-24T15:33:18.705 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:18.705+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:18.738 INFO:teuthology.orchestra.run.smithi151.stderr:Nothing to do for non-filestore backend 2024-04-24T15:33:18.807 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:18.807+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:18.864 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:18.864 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:18.875 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-24T15:33:18.908 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:18.908+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:18.916 INFO:teuthology.orchestra.run.smithi096.stdout:truncating image image_client.0-clone26 from 0xdb2a9e7 (overlap 0x337d7c0) to 0x526260 2024-04-24T15:33:19.008 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:19.009+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:19.088 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-24T15:33:19.095 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:19.095 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:19.108 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-24T15:33:19.109 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:33:19.110+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:33:19.143 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #25, image image_client.0-clone25 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent26 2024-04-24T15:33:19.167 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:19.167 INFO:teuthology.orchestra.run.smithi096.stdout:4856 read 0x7ffb0d6 thru 0x7ffc06c (0xf97 bytes) 2024-04-24T15:33:19.168 INFO:teuthology.orchestra.run.smithi096.stdout:4857 read 0xad92e14 thru 0xad96f03 (0x40f0 bytes) 2024-04-24T15:33:19.169 INFO:teuthology.orchestra.run.smithi096.stdout:4858 read 0x7c3a108 thru 0x7c3ed9d (0x4c96 bytes) 2024-04-24T15:33:19.169 INFO:teuthology.orchestra.run.smithi096.stdout:4859 trunc from 0xdb2a9e7 to 0xd8b3baa 2024-04-24T15:33:19.200 INFO:teuthology.orchestra.run.smithi096.stdout:4860 write 0x88b1e9e thru 0x88bbe56 (0x9fb9 bytes) 2024-04-24T15:33:19.207 INFO:teuthology.orchestra.run.smithi096.stdout:4861 read 0x71a806c thru 0x71b4352 (0xc2e7 bytes) 2024-04-24T15:33:19.207 INFO:teuthology.orchestra.run.smithi096.stdout:4863 write 0x53a359a thru 0x53a9a17 (0x647e bytes) 2024-04-24T15:33:19.210 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:33:19.211+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:33:19.217 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:19.217 INFO:teuthology.orchestra.run.smithi096.stdout:4867 read 0xc705065 thru 0xc70c81e (0x77ba bytes) 2024-04-24T15:33:19.217 INFO:teuthology.orchestra.run.smithi096.stdout:4868 punch from 0x5006ea4 to 0x50128b7, (0xba13 bytes) 2024-04-24T15:33:19.218 INFO:teuthology.orchestra.run.smithi096.stdout:4869 write 0x10d743a thru 0x10e378a (0xc351 bytes) 2024-04-24T15:33:19.258 DEBUG:teuthology.orchestra.run.smithi151:> CEPH_ARGS='--filestore-merge-threshold 1 --filestore-split-multiple 1' sudo -E ceph-objectstore-tool --no-mon-config --log-file=/var/log/ceph/objectstore_tool.$pid.log --data-path /var/lib/ceph/osd/ceph-6 --journal-path /var/lib/ceph/osd/ceph-6/journal --op apply-layout-settings --pool rbd 2024-04-24T15:33:19.272 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-04-24T15:33:19.300 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-04-24T15:33:19.309 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:19.309 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:19.310 INFO:teuthology.orchestra.run.smithi096.stdout:4871 write 0x4c751c5 thru 0x4c77df0 (0x2c2c bytes) 2024-04-24T15:33:19.311 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:33:19.312+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:33:19.319 INFO:teuthology.orchestra.run.smithi096.stdout:4872 write 0xa7ae35c thru 0xa7bbbdb (0xd880 bytes) 2024-04-24T15:33:19.321 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-24T15:33:19.322 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-04-24T15:33:19.324 INFO:teuthology.orchestra.run.smithi096.stdout:4873 punch from 0x5df87bd to 0x5dfa6db, (0x1f1e bytes) 2024-04-24T15:33:19.325 INFO:teuthology.orchestra.run.smithi096.stdout:4874 writesame 0x97f15e7 thru 0x97f5786 data_size 0x46(0x41a0 bytes) 2024-04-24T15:33:19.334 INFO:teuthology.orchestra.run.smithi096.stdout:4875 trunc from 0xd8b3baa to 0x8c094d9 2024-04-24T15:33:19.342 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-04-24T15:33:19.360 INFO:teuthology.orchestra.run.smithi096.stdout:4876 writesame 0x3d3ded0 thru 0x3d4c3d2 data_size 0x1(0xe503 bytes) 2024-04-24T15:33:19.363 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-04-24T15:33:19.387 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-04-24T15:33:19.390 INFO:teuthology.orchestra.run.smithi096.stdout:4877 read 0x58dd08c thru 0x58e4039 (0x6fae bytes) 2024-04-24T15:33:19.390 INFO:teuthology.orchestra.run.smithi096.stdout:4878 write 0x3d3e6f7 thru 0x3d4050f (0x1e19 bytes) 2024-04-24T15:33:19.392 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:19.392 INFO:teuthology.orchestra.run.smithi096.stdout:4880 writesame 0x73d665c thru 0x73de084 data_size 0x1(0x7a29 bytes) 2024-04-24T15:33:19.405 INFO:teuthology.orchestra.run.smithi096.stdout:4881 write 0x6a51867 thru 0x6a5829b (0x6a35 bytes) 2024-04-24T15:33:19.407 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-04-24T15:33:19.407 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T15:33:19.407 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T15:33:19.407 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T15:33:19.407 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T15:33:19.408 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T15:33:19.408 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T15:33:19.408 INFO:teuthology.orchestra.run.smithi096.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-04-24T15:33:19.408 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T15:33:19.411 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-24T15:33:19.412 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-24T15:33:19.412 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:33:19.412+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:33:19.416 INFO:teuthology.orchestra.run.smithi096.stdout:4882 write 0xad29a4a thru 0xad32ce5 (0x929c bytes) 2024-04-24T15:33:19.419 INFO:teuthology.orchestra.run.smithi096.stdout:4883 read 0x866d060 thru 0x8677bf8 (0xab99 bytes) 2024-04-24T15:33:19.419 INFO:teuthology.orchestra.run.smithi096.stdout:4884 write 0x169f29d thru 0x16a15c8 (0x232c bytes) 2024-04-24T15:33:19.469 INFO:teuthology.orchestra.run.smithi096.stdout:4886 trunc from 0xad32ce6 to 0x4f09424 2024-04-24T15:33:19.478 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:19.479 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:33:19.479 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-04-24T15:33:19.479 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:19.492 INFO:teuthology.orchestra.run.smithi096.stdout:4887 write 0x9934fd3 thru 0x9938e0d (0x3e3b bytes) 2024-04-24T15:33:19.493 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-24T15:33:19.498 INFO:teuthology.orchestra.run.smithi096.stdout:4888 read 0x69113e3 thru 0x691dfe1 (0xcbff bytes) 2024-04-24T15:33:19.498 INFO:teuthology.orchestra.run.smithi096.stdout:4890 punch from 0x34599b5 to 0x3467af0, (0xe13b bytes) 2024-04-24T15:33:19.500 INFO:teuthology.orchestra.run.smithi096.stdout:4891 trunc from 0x9938e0e to 0x796fc8e 2024-04-24T15:33:19.508 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:19.509 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:19.509 INFO:teuthology.orchestra.run.smithi096.stdout:4894 read 0x4e2af1d thru 0x4e3a235 (0xf319 bytes) 2024-04-24T15:33:19.509 INFO:teuthology.orchestra.run.smithi096.stdout:4895 write 0x2140061 thru 0x2144174 (0x4114 bytes) 2024-04-24T15:33:19.512 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:19.513+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:19.540 INFO:teuthology.orchestra.run.smithi096.stdout:4896 punch from 0x4621b46 to 0x4623d95, (0x224f bytes) 2024-04-24T15:33:19.542 INFO:teuthology.orchestra.run.smithi096.stdout:4897 trunc from 0x796fc8e to 0x5a57e95 2024-04-24T15:33:19.547 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:19.547 INFO:teuthology.orchestra.run.smithi096.stdout:4900 write 0x29cb31e thru 0x29d1cb6 (0x6999 bytes) 2024-04-24T15:33:19.607 INFO:teuthology.orchestra.run.smithi096.stdout:4901 write 0xd5bf290 thru 0xd5cbb49 (0xc8ba bytes) 2024-04-24T15:33:19.613 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:33:19.615+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:33:19.614 INFO:teuthology.orchestra.run.smithi096.stdout:4904 write 0x537481f thru 0x537e87e (0xa060 bytes) 2024-04-24T15:33:19.620 INFO:teuthology.orchestra.run.smithi096.stdout:4907 write 0x99f6c1a thru 0x99fe054 (0x743b bytes) 2024-04-24T15:33:19.627 INFO:teuthology.orchestra.run.smithi096.stdout:4908 writesame 0x4ee6e31 thru 0x4ef6cef data_size 0x1(0xfebf bytes) 2024-04-24T15:33:19.640 INFO:teuthology.orchestra.run.smithi096.stdout:4909 read 0x68e741e thru 0x68f63fc (0xefdf bytes) 2024-04-24T15:33:19.641 INFO:teuthology.orchestra.run.smithi096.stdout:4910 read 0x1cc726b thru 0x1ccf8df (0x8675 bytes) 2024-04-24T15:33:19.642 INFO:teuthology.orchestra.run.smithi096.stdout:4911 trunc from 0xd5cbb4a to 0xe379097 2024-04-24T15:33:19.647 INFO:teuthology.orchestra.run.smithi096.stdout:4912 write 0xcf695d thru 0xd00c37 (0xa2db bytes) 2024-04-24T15:33:19.655 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:19.655 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:33:19.655 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-04-24T15:33:19.655 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-04-24T15:33:19.655 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:19.666 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-24T15:33:19.674 INFO:teuthology.orchestra.run.smithi096.stdout:4915 read 0x8b4b24d thru 0x8b4bdda (0xb8e bytes) 2024-04-24T15:33:19.674 INFO:teuthology.orchestra.run.smithi096.stdout:4917 read 0x58a919a thru 0x58aa329 (0x1190 bytes) 2024-04-24T15:33:19.675 INFO:teuthology.orchestra.run.smithi096.stdout:4918 write 0x53b29c2 thru 0x53b6467 (0x3aa6 bytes) 2024-04-24T15:33:19.676 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:19.677 INFO:teuthology.orchestra.run.smithi096.stdout:4922 punch from 0xb4031c9 to 0xb40fa82, (0xc8b9 bytes) 2024-04-24T15:33:19.677 INFO:teuthology.orchestra.run.smithi096.stdout:4923 write 0x3ab1111 thru 0x3ab8186 (0x7076 bytes) 2024-04-24T15:33:19.707 INFO:teuthology.orchestra.run.smithi096.stdout:4924 write 0x9a6fc23 thru 0x9a7d6b6 (0xda94 bytes) 2024-04-24T15:33:19.712 INFO:teuthology.orchestra.run.smithi096.stdout:4925 read 0x75cbd9e thru 0x75d4367 (0x85ca bytes) 2024-04-24T15:33:19.712 INFO:teuthology.orchestra.run.smithi096.stdout:4927 read 0x3a41563 thru 0x3a47cbe (0x675c bytes) 2024-04-24T15:33:19.712 INFO:teuthology.orchestra.run.smithi096.stdout:4929 read 0xdd0422 thru 0xdd094f (0x52e bytes) 2024-04-24T15:33:19.713 INFO:teuthology.orchestra.run.smithi096.stdout:4930 read 0xb252a4f thru 0xb25ff32 (0xd4e4 bytes) 2024-04-24T15:33:19.713 INFO:teuthology.orchestra.run.smithi096.stdout:4931 write 0xda759f thru 0xdb1d6e (0xa7d0 bytes) 2024-04-24T15:33:19.714 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:19.715+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:19.729 INFO:teuthology.orchestra.run.smithi096.stdout:4933 trunc from 0xe379097 to 0x3d3e357 2024-04-24T15:33:19.743 INFO:teuthology.orchestra.run.smithi096.stdout:4934 read 0x9fffe7 thru 0xa07c3a (0x7c54 bytes) 2024-04-24T15:33:19.746 INFO:teuthology.orchestra.run.smithi096.stdout:4935 trunc from 0x3d3e357 to 0x2205435 2024-04-24T15:33:19.787 INFO:teuthology.orchestra.run.smithi096.stdout:4937 punch from 0x12fdc11 to 0x1300f2a, (0x3319 bytes) 2024-04-24T15:33:19.788 INFO:teuthology.orchestra.run.smithi096.stdout:4938 punch from 0x16ea92b to 0x16f1af1, (0x71c6 bytes) 2024-04-24T15:33:19.788 INFO:teuthology.orchestra.run.smithi096.stdout:4939 clone 28 order 22 su 262144 sc 5 2024-04-24T15:33:19.815 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:33:19.816+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:33:19.826 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:19.826 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-04-24T15:33:19.826 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-04-24T15:33:19.826 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-04-24T15:33:19.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.826 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14043 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2383872~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:19.826 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892347+0000", 2024-04-24T15:33:19.826 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 45.933459921999997, 2024-04-24T15:33:19.826 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038537279000000001, 2024-04-24T15:33:19.827 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:19.827 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:19.827 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:19.827 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:19.827 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:19.827 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14043 2024-04-24T15:33:19.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.827 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:19.827 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.827 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:19.827 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892347+0000", 2024-04-24T15:33:19.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:19.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.827 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.828 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:19.828 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892347+0000", 2024-04-24T15:33:19.828 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:19.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.828 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.828 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:19.828 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892347+0000", 2024-04-24T15:33:19.828 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:19.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.828 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.828 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:19.828 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892349+0000", 2024-04-24T15:33:19.828 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3400000000000001e-06 2024-04-24T15:33:19.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.829 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:19.829 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892357+0000", 2024-04-24T15:33:19.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1540000000000002e-06 2024-04-24T15:33:19.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.829 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:19.829 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892360+0000", 2024-04-24T15:33:19.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5829999999999998e-06 2024-04-24T15:33:19.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.829 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:19.830 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930795+0000", 2024-04-24T15:33:19.830 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038434590999999997 2024-04-24T15:33:19.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.830 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:19.830 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930836+0000", 2024-04-24T15:33:19.830 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0827000000000001e-05 2024-04-24T15:33:19.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.830 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:19.830 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930885+0000", 2024-04-24T15:33:19.830 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9029000000000001e-05 2024-04-24T15:33:19.830 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:19.831 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:19.831 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:19.831 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.831 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.831 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14044 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2424832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:19.831 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892372+0000", 2024-04-24T15:33:19.831 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 45.933435277999997, 2024-04-24T15:33:19.831 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038624890000000002, 2024-04-24T15:33:19.831 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:19.831 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:19.831 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:19.831 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:19.831 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:19.831 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14044 2024-04-24T15:33:19.832 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.832 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:19.832 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.832 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:19.832 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892372+0000", 2024-04-24T15:33:19.832 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:19.832 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.832 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.832 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:19.832 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892372+0000", 2024-04-24T15:33:19.832 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:19.832 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.832 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.832 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:19.833 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892372+0000", 2024-04-24T15:33:19.833 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:19.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.833 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:19.833 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892374+0000", 2024-04-24T15:33:19.833 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4899999999999999e-06 2024-04-24T15:33:19.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.833 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:19.833 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892380+0000", 2024-04-24T15:33:19.833 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.631e-06 2024-04-24T15:33:19.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.833 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:19.833 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892383+0000", 2024-04-24T15:33:19.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7400000000000002e-06 2024-04-24T15:33:19.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.834 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:19.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930905+0000", 2024-04-24T15:33:19.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038521396999999999 2024-04-24T15:33:19.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.834 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:19.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930949+0000", 2024-04-24T15:33:19.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4153999999999998e-05 2024-04-24T15:33:19.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.834 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:19.835 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930997+0000", 2024-04-24T15:33:19.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7998999999999999e-05 2024-04-24T15:33:19.835 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:19.835 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:19.835 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:19.835 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.835 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.835 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14045 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2465792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:19.835 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892387+0000", 2024-04-24T15:33:19.835 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 45.933420011000003, 2024-04-24T15:33:19.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038720933999999999, 2024-04-24T15:33:19.835 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:19.835 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:19.835 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:19.835 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:19.836 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:19.836 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14045 2024-04-24T15:33:19.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.836 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:19.836 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.836 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:19.836 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892387+0000", 2024-04-24T15:33:19.836 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:19.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.836 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.836 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:19.836 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892387+0000", 2024-04-24T15:33:19.836 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:19.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.836 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.836 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:19.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892387+0000", 2024-04-24T15:33:19.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:19.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.837 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:19.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892392+0000", 2024-04-24T15:33:19.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0039999999999999e-06 2024-04-24T15:33:19.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.837 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:19.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892398+0000", 2024-04-24T15:33:19.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0900000000000001e-06 2024-04-24T15:33:19.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.837 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:19.838 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892402+0000", 2024-04-24T15:33:19.838 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.399e-06 2024-04-24T15:33:19.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.838 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:19.838 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931018+0000", 2024-04-24T15:33:19.838 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038616551999999998 2024-04-24T15:33:19.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.838 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:19.838 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931059+0000", 2024-04-24T15:33:19.838 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1205000000000003e-05 2024-04-24T15:33:19.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.839 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:19.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931108+0000", 2024-04-24T15:33:19.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8996000000000003e-05 2024-04-24T15:33:19.839 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:19.839 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:19.839 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:19.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.839 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14046 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2506752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:19.839 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892406+0000", 2024-04-24T15:33:19.839 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 45.933401420999999, 2024-04-24T15:33:19.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038814507999999998, 2024-04-24T15:33:19.839 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:19.840 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:19.840 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:19.840 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:19.840 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:19.840 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14046 2024-04-24T15:33:19.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.840 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:19.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.840 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:19.840 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892406+0000", 2024-04-24T15:33:19.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:19.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.840 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:19.841 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892406+0000", 2024-04-24T15:33:19.841 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:19.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.841 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:19.841 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892406+0000", 2024-04-24T15:33:19.841 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:19.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.841 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:19.841 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892413+0000", 2024-04-24T15:33:19.841 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7449999999999995e-06 2024-04-24T15:33:19.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.842 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:19.842 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892419+0000", 2024-04-24T15:33:19.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5790000000000003e-06 2024-04-24T15:33:19.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.842 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.842 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:19.842 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892423+0000", 2024-04-24T15:33:19.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8199999999999998e-06 2024-04-24T15:33:19.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.842 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.842 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:19.842 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931128+0000", 2024-04-24T15:33:19.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038705502000000003 2024-04-24T15:33:19.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.843 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:19.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931172+0000", 2024-04-24T15:33:19.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4066999999999997e-05 2024-04-24T15:33:19.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.843 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:19.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931220+0000", 2024-04-24T15:33:19.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8167999999999997e-05 2024-04-24T15:33:19.843 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:19.843 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:19.843 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:19.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.844 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14047 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2547712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:19.844 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892427+0000", 2024-04-24T15:33:19.844 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 45.933380192000001, 2024-04-24T15:33:19.844 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038904506999999998, 2024-04-24T15:33:19.844 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:19.844 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:19.844 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:19.844 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:19.844 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:19.844 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14047 2024-04-24T15:33:19.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.844 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:19.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.845 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:19.845 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892427+0000", 2024-04-24T15:33:19.845 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:19.845 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.845 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.845 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:19.845 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892427+0000", 2024-04-24T15:33:19.845 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:19.845 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.845 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.845 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:19.845 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892427+0000", 2024-04-24T15:33:19.845 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:19.846 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.846 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.846 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:19.846 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892429+0000", 2024-04-24T15:33:19.846 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3099999999999999e-06 2024-04-24T15:33:19.846 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.846 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.846 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:19.846 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892435+0000", 2024-04-24T15:33:19.846 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5520000000000004e-06 2024-04-24T15:33:19.846 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.846 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.846 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:19.846 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892438+0000", 2024-04-24T15:33:19.847 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4240000000000002e-06 2024-04-24T15:33:19.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.847 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.847 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:19.847 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931242+0000", 2024-04-24T15:33:19.847 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038804162000000003 2024-04-24T15:33:19.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.847 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.847 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:19.847 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931284+0000", 2024-04-24T15:33:19.847 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1414e-05 2024-04-24T15:33:19.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.847 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.847 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:19.848 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931332+0000", 2024-04-24T15:33:19.848 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8024e-05 2024-04-24T15:33:19.848 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:19.848 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:19.848 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:19.848 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.848 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.848 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14048 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2588672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:19.848 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892442+0000", 2024-04-24T15:33:19.848 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 45.933365229000003, 2024-04-24T15:33:19.848 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039003132000000003, 2024-04-24T15:33:19.848 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:19.848 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:19.849 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:19.849 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:19.849 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:19.849 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14048 2024-04-24T15:33:19.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.849 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:19.849 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.849 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:19.849 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892442+0000", 2024-04-24T15:33:19.849 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:19.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.849 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.849 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:19.849 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892442+0000", 2024-04-24T15:33:19.850 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:19.850 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.850 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.850 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:19.850 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892442+0000", 2024-04-24T15:33:19.850 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:19.850 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.850 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.850 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:19.850 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892444+0000", 2024-04-24T15:33:19.850 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1169999999999998e-06 2024-04-24T15:33:19.850 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.850 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.851 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:19.851 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892449+0000", 2024-04-24T15:33:19.851 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1599999999999997e-06 2024-04-24T15:33:19.851 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.851 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.851 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:19.851 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892453+0000", 2024-04-24T15:33:19.851 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5930000000000002e-06 2024-04-24T15:33:19.851 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.851 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.851 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:19.851 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931352+0000", 2024-04-24T15:33:19.851 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038899361 2024-04-24T15:33:19.851 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.852 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.852 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:19.852 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931398+0000", 2024-04-24T15:33:19.852 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6315e-05 2024-04-24T15:33:19.852 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.852 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.852 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:19.852 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931445+0000", 2024-04-24T15:33:19.852 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6950000000000003e-05 2024-04-24T15:33:19.852 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:19.852 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:19.852 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:19.852 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.852 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.852 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14049 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2629632~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:19.853 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892458+0000", 2024-04-24T15:33:19.853 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 45.933349808999999, 2024-04-24T15:33:19.853 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039103322000000003, 2024-04-24T15:33:19.853 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:19.853 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:19.853 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:19.853 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:19.853 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:19.853 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14049 2024-04-24T15:33:19.853 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.853 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:19.853 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.853 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:19.853 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892458+0000", 2024-04-24T15:33:19.853 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:19.854 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.854 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.854 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:19.854 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892458+0000", 2024-04-24T15:33:19.854 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:19.854 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.854 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.854 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:19.854 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892457+0000", 2024-04-24T15:33:19.854 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:19.854 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.854 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.854 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:19.855 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892459+0000", 2024-04-24T15:33:19.855 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.305e-06 2024-04-24T15:33:19.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.855 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.855 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:19.855 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892465+0000", 2024-04-24T15:33:19.855 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4360000000000001e-06 2024-04-24T15:33:19.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.855 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.855 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:19.855 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892474+0000", 2024-04-24T15:33:19.855 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7639999999999994e-06 2024-04-24T15:33:19.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.856 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.856 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:19.856 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931466+0000", 2024-04-24T15:33:19.856 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038992644 2024-04-24T15:33:19.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.856 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.856 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:19.856 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931512+0000", 2024-04-24T15:33:19.856 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5200000000000001e-05 2024-04-24T15:33:19.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.856 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.856 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:19.856 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931561+0000", 2024-04-24T15:33:19.857 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9357999999999997e-05 2024-04-24T15:33:19.857 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:19.857 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:19.857 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:19.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.857 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.857 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14050 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2670592~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:19.857 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892478+0000", 2024-04-24T15:33:19.857 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 45.933329221999998, 2024-04-24T15:33:19.857 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039195102000000002, 2024-04-24T15:33:19.857 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:19.857 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:19.857 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:19.857 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:19.857 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:19.858 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14050 2024-04-24T15:33:19.858 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.858 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:19.858 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.858 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:19.858 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892478+0000", 2024-04-24T15:33:19.858 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:19.858 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.858 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.858 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:19.858 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892478+0000", 2024-04-24T15:33:19.858 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:19.858 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.858 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.859 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:19.859 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892478+0000", 2024-04-24T15:33:19.859 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:19.859 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.859 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.859 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:19.859 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892480+0000", 2024-04-24T15:33:19.859 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3300000000000001e-06 2024-04-24T15:33:19.859 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.859 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.859 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:19.859 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892486+0000", 2024-04-24T15:33:19.859 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9200000000000001e-06 2024-04-24T15:33:19.859 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.860 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.860 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:19.860 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892494+0000", 2024-04-24T15:33:19.860 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5159999999999997e-06 2024-04-24T15:33:19.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.860 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.860 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:19.860 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931581+0000", 2024-04-24T15:33:19.860 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039087894999999998 2024-04-24T15:33:19.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.860 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.860 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:19.860 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931626+0000", 2024-04-24T15:33:19.860 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4943e-05 2024-04-24T15:33:19.861 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.861 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.861 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:19.861 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931673+0000", 2024-04-24T15:33:19.861 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6906000000000001e-05 2024-04-24T15:33:19.861 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:19.861 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:19.861 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:19.861 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.861 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.861 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14051 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2711552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:19.861 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892507+0000", 2024-04-24T15:33:19.861 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 45.933300523, 2024-04-24T15:33:19.861 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039274393999999997, 2024-04-24T15:33:19.861 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:19.862 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:19.862 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:19.862 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:19.862 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:19.862 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14051 2024-04-24T15:33:19.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.862 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:19.862 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.862 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:19.862 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892507+0000", 2024-04-24T15:33:19.862 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:19.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.862 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.862 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:19.863 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892507+0000", 2024-04-24T15:33:19.863 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:19.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.863 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.863 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:19.863 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892506+0000", 2024-04-24T15:33:19.863 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:19.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.863 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.863 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:19.863 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892509+0000", 2024-04-24T15:33:19.863 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8760000000000001e-06 2024-04-24T15:33:19.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.864 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.864 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:19.864 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892515+0000", 2024-04-24T15:33:19.864 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1840000000000004e-06 2024-04-24T15:33:19.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.864 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.864 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:19.864 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892519+0000", 2024-04-24T15:33:19.864 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7570000000000002e-06 2024-04-24T15:33:19.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.864 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.864 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:19.864 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931694+0000", 2024-04-24T15:33:19.864 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039174960000000002 2024-04-24T15:33:19.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.865 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.865 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:19.865 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931734+0000", 2024-04-24T15:33:19.865 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0284000000000003e-05 2024-04-24T15:33:19.865 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.865 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.865 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:19.865 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931781+0000", 2024-04-24T15:33:19.865 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6813000000000001e-05 2024-04-24T15:33:19.865 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:19.865 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:19.865 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:19.865 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.865 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.865 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14052 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2752512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:19.866 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892524+0000", 2024-04-24T15:33:19.866 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 45.933283375999999, 2024-04-24T15:33:19.866 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039371122000000001, 2024-04-24T15:33:19.866 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:19.866 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:19.866 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:19.866 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:19.866 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:19.866 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14052 2024-04-24T15:33:19.866 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.866 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:19.866 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.866 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:19.866 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892524+0000", 2024-04-24T15:33:19.867 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:19.867 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.867 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.867 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:19.867 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892524+0000", 2024-04-24T15:33:19.867 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:19.867 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.867 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.867 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:19.867 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892524+0000", 2024-04-24T15:33:19.867 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:19.867 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.867 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.867 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:19.867 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892526+0000", 2024-04-24T15:33:19.868 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3630000000000001e-06 2024-04-24T15:33:19.868 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.868 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.868 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:19.868 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892538+0000", 2024-04-24T15:33:19.868 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1677e-05 2024-04-24T15:33:19.868 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.868 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.868 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:19.868 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892541+0000", 2024-04-24T15:33:19.868 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.822e-06 2024-04-24T15:33:19.868 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.868 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.868 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:19.868 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931802+0000", 2024-04-24T15:33:19.869 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039260911000000003 2024-04-24T15:33:19.869 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.869 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.869 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:19.869 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931847+0000", 2024-04-24T15:33:19.869 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4450999999999998e-05 2024-04-24T15:33:19.869 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.869 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.869 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:19.869 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931895+0000", 2024-04-24T15:33:19.869 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8257999999999998e-05 2024-04-24T15:33:19.869 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:19.869 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:19.869 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:19.869 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.870 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.870 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14053 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2793472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:19.870 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892546+0000", 2024-04-24T15:33:19.870 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 45.933261397999999, 2024-04-24T15:33:19.870 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039457712999999998, 2024-04-24T15:33:19.870 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:19.870 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:19.870 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:19.870 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:19.870 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:19.870 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14053 2024-04-24T15:33:19.870 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.870 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:19.870 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.871 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:19.871 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892546+0000", 2024-04-24T15:33:19.871 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:19.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.871 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.871 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:19.871 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892546+0000", 2024-04-24T15:33:19.871 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:19.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.871 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.871 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:19.871 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892546+0000", 2024-04-24T15:33:19.871 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:19.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.871 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.871 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:19.872 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892548+0000", 2024-04-24T15:33:19.872 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3120000000000001e-06 2024-04-24T15:33:19.872 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.872 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.872 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:19.872 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892553+0000", 2024-04-24T15:33:19.872 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2320000000000001e-06 2024-04-24T15:33:19.872 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.872 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.872 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:19.872 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892559+0000", 2024-04-24T15:33:19.872 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5480000000000001e-06 2024-04-24T15:33:19.873 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.873 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.873 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:19.873 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931916+0000", 2024-04-24T15:33:19.873 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039357180999999998 2024-04-24T15:33:19.873 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.873 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.873 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:19.873 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931956+0000", 2024-04-24T15:33:19.873 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0565000000000002e-05 2024-04-24T15:33:19.873 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.873 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.873 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:19.874 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932004+0000", 2024-04-24T15:33:19.874 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7224000000000001e-05 2024-04-24T15:33:19.874 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:19.874 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:19.874 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:19.874 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.874 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.874 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14054 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2834432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:19.874 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892563+0000", 2024-04-24T15:33:19.874 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 45.933244746, 2024-04-24T15:33:19.874 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039552406999999998, 2024-04-24T15:33:19.874 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:19.874 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:19.874 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:19.875 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:19.875 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:19.875 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14054 2024-04-24T15:33:19.875 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.875 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:19.875 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.875 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:19.875 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892563+0000", 2024-04-24T15:33:19.875 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:19.875 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.875 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.875 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:19.875 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892563+0000", 2024-04-24T15:33:19.876 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:19.876 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.876 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.876 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:19.876 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892562+0000", 2024-04-24T15:33:19.876 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:19.876 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.876 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.876 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:19.876 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892564+0000", 2024-04-24T15:33:19.876 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2029999999999999e-06 2024-04-24T15:33:19.876 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.876 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.876 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:19.877 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892570+0000", 2024-04-24T15:33:19.877 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1460000000000004e-06 2024-04-24T15:33:19.877 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.877 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.877 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:19.877 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892573+0000", 2024-04-24T15:33:19.877 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2030000000000001e-06 2024-04-24T15:33:19.877 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.877 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.877 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:19.877 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932024+0000", 2024-04-24T15:33:19.877 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039451696000000001 2024-04-24T15:33:19.877 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.877 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.878 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:19.878 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932068+0000", 2024-04-24T15:33:19.878 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3970000000000001e-05 2024-04-24T15:33:19.878 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.878 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.878 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:19.878 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932115+0000", 2024-04-24T15:33:19.878 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6569000000000001e-05 2024-04-24T15:33:19.878 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:19.878 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:19.878 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:19.878 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.879 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.879 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14055 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2875392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:19.879 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892591+0000", 2024-04-24T15:33:19.879 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 45.933216772999998, 2024-04-24T15:33:19.879 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039634309, 2024-04-24T15:33:19.879 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:19.879 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:19.879 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:19.879 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:19.879 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:19.879 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14055 2024-04-24T15:33:19.879 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.879 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:19.879 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.880 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:19.880 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892591+0000", 2024-04-24T15:33:19.880 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:19.880 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.880 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.880 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:19.880 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892591+0000", 2024-04-24T15:33:19.880 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:19.880 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.880 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.880 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:19.880 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892590+0000", 2024-04-24T15:33:19.880 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:19.880 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.880 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.880 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:19.881 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892592+0000", 2024-04-24T15:33:19.881 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7519999999999999e-06 2024-04-24T15:33:19.881 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.881 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.881 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:19.881 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892596+0000", 2024-04-24T15:33:19.881 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.608e-06 2024-04-24T15:33:19.881 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.881 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.881 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:19.881 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892598+0000", 2024-04-24T15:33:19.881 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.531e-06 2024-04-24T15:33:19.881 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.882 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.882 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:19.882 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932136+0000", 2024-04-24T15:33:19.882 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039537668999999998 2024-04-24T15:33:19.882 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.882 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.882 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:19.882 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932177+0000", 2024-04-24T15:33:19.882 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0880000000000002e-05 2024-04-24T15:33:19.882 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.882 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.882 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:19.882 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932225+0000", 2024-04-24T15:33:19.882 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8223999999999998e-05 2024-04-24T15:33:19.883 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:19.883 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:19.883 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:19.883 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.883 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.883 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14056 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2916352~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:19.883 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892600+0000", 2024-04-24T15:33:19.883 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 45.933206939000002, 2024-04-24T15:33:19.883 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039736447000000001, 2024-04-24T15:33:19.883 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:19.883 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:19.883 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:19.883 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:19.883 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:19.883 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14056 2024-04-24T15:33:19.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.884 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:19.884 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.884 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:19.884 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892600+0000", 2024-04-24T15:33:19.884 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:19.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.884 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.884 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:19.884 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892600+0000", 2024-04-24T15:33:19.884 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:19.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.884 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.884 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:19.884 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892600+0000", 2024-04-24T15:33:19.885 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967296 2024-04-24T15:33:19.885 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.885 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.885 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:19.885 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892602+0000", 2024-04-24T15:33:19.885 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.576e-06 2024-04-24T15:33:19.885 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.885 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.885 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:19.885 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892605+0000", 2024-04-24T15:33:19.885 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.574e-06 2024-04-24T15:33:19.885 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.885 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.885 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:19.885 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892608+0000", 2024-04-24T15:33:19.885 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3800000000000001e-06 2024-04-24T15:33:19.886 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.886 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.886 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:19.886 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932245+0000", 2024-04-24T15:33:19.886 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039637378000000001 2024-04-24T15:33:19.886 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.886 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.886 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:19.886 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932290+0000", 2024-04-24T15:33:19.886 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4734999999999998e-05 2024-04-24T15:33:19.886 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.886 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.886 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:19.887 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932337+0000", 2024-04-24T15:33:19.887 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7033000000000002e-05 2024-04-24T15:33:19.887 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:19.887 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:19.887 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:19.887 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.887 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.887 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14063 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 3203072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:19.887 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.894564+0000", 2024-04-24T15:33:19.887 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 45.931243348000002, 2024-04-24T15:33:19.887 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038554995000000002, 2024-04-24T15:33:19.887 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:19.887 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:19.887 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:19.888 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:19.888 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:19.888 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14063 2024-04-24T15:33:19.888 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.888 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:19.888 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.888 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:19.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894564+0000", 2024-04-24T15:33:19.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:19.888 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.888 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.888 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:19.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894564+0000", 2024-04-24T15:33:19.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:19.888 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:19.889 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894564+0000", 2024-04-24T15:33:19.889 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967296 2024-04-24T15:33:19.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:19.889 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894565+0000", 2024-04-24T15:33:19.889 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2640000000000001e-06 2024-04-24T15:33:19.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:19.889 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894569+0000", 2024-04-24T15:33:19.889 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1710000000000004e-06 2024-04-24T15:33:19.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.890 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.890 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:19.890 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894573+0000", 2024-04-24T15:33:19.890 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4410000000000002e-06 2024-04-24T15:33:19.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.890 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.890 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:19.890 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933030+0000", 2024-04-24T15:33:19.890 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038457709999999999 2024-04-24T15:33:19.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.890 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.890 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:19.890 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933072+0000", 2024-04-24T15:33:19.891 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.193e-05 2024-04-24T15:33:19.891 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.891 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.891 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:19.891 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933119+0000", 2024-04-24T15:33:19.891 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6715000000000003e-05 2024-04-24T15:33:19.891 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:19.891 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:19.891 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:19.891 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.891 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.891 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14064 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 3244032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:19.891 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.894591+0000", 2024-04-24T15:33:19.891 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 45.931216706000001, 2024-04-24T15:33:19.892 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038639806999999998, 2024-04-24T15:33:19.892 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:19.892 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:19.892 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:19.892 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:19.892 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:19.892 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14064 2024-04-24T15:33:19.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.892 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:19.892 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.892 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:19.892 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894591+0000", 2024-04-24T15:33:19.892 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:19.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.892 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.893 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:19.893 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894591+0000", 2024-04-24T15:33:19.893 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:19.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.893 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.893 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:19.893 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894590+0000", 2024-04-24T15:33:19.893 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:19.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.893 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.893 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:19.893 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894593+0000", 2024-04-24T15:33:19.893 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4719999999999998e-06 2024-04-24T15:33:19.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.893 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.893 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:19.894 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894598+0000", 2024-04-24T15:33:19.894 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.57e-06 2024-04-24T15:33:19.894 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.894 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.894 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:19.894 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894602+0000", 2024-04-24T15:33:19.894 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.614e-06 2024-04-24T15:33:19.894 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.894 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.894 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:19.894 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933139+0000", 2024-04-24T15:33:19.894 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038536929999999997 2024-04-24T15:33:19.894 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.895 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.895 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:19.895 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933183+0000", 2024-04-24T15:33:19.895 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4217e-05 2024-04-24T15:33:19.895 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.895 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.895 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:19.895 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933230+0000", 2024-04-24T15:33:19.895 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7434e-05 2024-04-24T15:33:19.895 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:19.895 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:19.895 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:19.895 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.895 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.895 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14065 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 3284992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:19.896 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.894606+0000", 2024-04-24T15:33:19.896 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 45.931201324, 2024-04-24T15:33:19.896 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038734126000000001, 2024-04-24T15:33:19.896 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:19.896 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:19.896 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:19.896 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:19.896 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:19.896 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14065 2024-04-24T15:33:19.896 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.896 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:19.896 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.896 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:19.897 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894606+0000", 2024-04-24T15:33:19.897 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:19.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.897 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.897 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:19.897 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894606+0000", 2024-04-24T15:33:19.897 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:19.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.897 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.897 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:19.897 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894606+0000", 2024-04-24T15:33:19.897 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:19.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.897 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.898 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:19.898 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894608+0000", 2024-04-24T15:33:19.898 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0880000000000002e-06 2024-04-24T15:33:19.898 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.898 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.898 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:19.898 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894614+0000", 2024-04-24T15:33:19.898 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7830000000000004e-06 2024-04-24T15:33:19.898 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.898 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.898 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:19.898 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894621+0000", 2024-04-24T15:33:19.898 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5809999999999999e-06 2024-04-24T15:33:19.898 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.899 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.899 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:19.899 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933251+0000", 2024-04-24T15:33:19.899 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038629995 2024-04-24T15:33:19.899 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.899 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.899 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:19.899 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933292+0000", 2024-04-24T15:33:19.899 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0380000000000003e-05 2024-04-24T15:33:19.899 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.899 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.899 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:19.899 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933340+0000", 2024-04-24T15:33:19.899 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8662999999999998e-05 2024-04-24T15:33:19.900 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:19.900 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:19.900 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:19.900 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.900 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.900 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14066 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 3325952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:19.900 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.894625+0000", 2024-04-24T15:33:19.900 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 45.931181885000001, 2024-04-24T15:33:19.900 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038825539999999999, 2024-04-24T15:33:19.900 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:19.900 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:19.900 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:19.900 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:19.900 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:19.901 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14066 2024-04-24T15:33:19.901 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.901 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:19.901 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.901 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:19.902 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894625+0000", 2024-04-24T15:33:19.902 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:19.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.902 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.902 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:19.902 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894625+0000", 2024-04-24T15:33:19.902 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:19.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.902 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.902 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:19.902 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894625+0000", 2024-04-24T15:33:19.902 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:19.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.903 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:19.903 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894627+0000", 2024-04-24T15:33:19.903 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.092e-06 2024-04-24T15:33:19.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.903 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:19.903 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894635+0000", 2024-04-24T15:33:19.903 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6660000000000008e-06 2024-04-24T15:33:19.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.903 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:19.903 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894639+0000", 2024-04-24T15:33:19.903 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.799e-06 2024-04-24T15:33:19.904 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.904 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.904 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:19.904 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933360+0000", 2024-04-24T15:33:19.904 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038721569999999997 2024-04-24T15:33:19.904 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.904 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.904 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:19.904 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933403+0000", 2024-04-24T15:33:19.904 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2605999999999998e-05 2024-04-24T15:33:19.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.905 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.905 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:19.905 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933451+0000", 2024-04-24T15:33:19.905 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8149999999999998e-05 2024-04-24T15:33:19.905 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:19.905 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:19.905 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:19.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.905 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.905 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14067 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 3366912~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:19.905 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.894644+0000", 2024-04-24T15:33:19.905 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 45.931163826999999, 2024-04-24T15:33:19.905 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038926068000000001, 2024-04-24T15:33:19.905 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:19.906 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:19.906 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:19.906 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:19.906 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:19.906 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14067 2024-04-24T15:33:19.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.906 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:19.906 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.906 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:19.906 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894644+0000", 2024-04-24T15:33:19.906 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:19.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.906 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.906 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:19.907 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894644+0000", 2024-04-24T15:33:19.907 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:19.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.907 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:19.907 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894643+0000", 2024-04-24T15:33:19.907 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:19.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.907 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:19.907 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894646+0000", 2024-04-24T15:33:19.907 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3379999999999999e-06 2024-04-24T15:33:19.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.907 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:19.907 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894651+0000", 2024-04-24T15:33:19.908 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.485e-06 2024-04-24T15:33:19.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.908 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.908 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:19.908 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894655+0000", 2024-04-24T15:33:19.908 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.709e-06 2024-04-24T15:33:19.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.908 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.908 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:19.908 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933472+0000", 2024-04-24T15:33:19.908 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038816992000000002 2024-04-24T15:33:19.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.908 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.908 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:19.909 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933521+0000", 2024-04-24T15:33:19.909 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8841999999999998e-05 2024-04-24T15:33:19.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.909 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.909 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:19.909 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933570+0000", 2024-04-24T15:33:19.909 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9063000000000001e-05 2024-04-24T15:33:19.909 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:19.909 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:19.909 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:19.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.909 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.909 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:30240 4.0 4:059b194c:::rbd_object_map.114a2e5eebc:head [read 0~43] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-04-24T15:33:19.909 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:47.244473+0000", 2024-04-24T15:33:19.910 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.581334640999998, 2024-04-24T15:33:19.910 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4146627299999999, 2024-04-24T15:33:19.910 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:19.910 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:19.910 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:19.910 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:19.910 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:19.910 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30240 2024-04-24T15:33:19.910 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.910 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:19.910 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.910 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:19.910 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.244473+0000", 2024-04-24T15:33:19.910 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:19.911 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.911 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.911 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:19.911 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.244473+0000", 2024-04-24T15:33:19.911 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:19.911 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.911 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.911 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:19.911 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.244471+0000", 2024-04-24T15:33:19.911 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999986 2024-04-24T15:33:19.911 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.911 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.911 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:19.911 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.244477+0000", 2024-04-24T15:33:19.911 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7819999999999999e-06 2024-04-24T15:33:19.912 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.912 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.912 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:19.912 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.244487+0000", 2024-04-24T15:33:19.912 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0247e-05 2024-04-24T15:33:19.912 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.912 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.912 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:19.912 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.244499+0000", 2024-04-24T15:33:19.912 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1324e-05 2024-04-24T15:33:19.912 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.912 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.912 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:19.912 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.244545+0000", 2024-04-24T15:33:19.912 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6186000000000001e-05 2024-04-24T15:33:19.912 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.913 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.913 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-04-24T15:33:19.913 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.244553+0000", 2024-04-24T15:33:19.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5380000000000002e-06 2024-04-24T15:33:19.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.913 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.913 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:19.913 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.631596+0000", 2024-04-24T15:33:19.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.38704289600000003 2024-04-24T15:33:19.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.913 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.913 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-04-24T15:33:19.913 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.631606+0000", 2024-04-24T15:33:19.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6390000000000004e-06 2024-04-24T15:33:19.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.914 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.914 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:19.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:48.658831+0000", 2024-04-24T15:33:19.914 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0272250540000001 2024-04-24T15:33:19.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.914 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.914 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:19.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:48.658991+0000", 2024-04-24T15:33:19.914 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015994800000000001 2024-04-24T15:33:19.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:19.914 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:19.914 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:19.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:48.659135+0000", 2024-04-24T15:33:19.914 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000144456 2024-04-24T15:33:19.915 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:19.915 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:19.915 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:19.915 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:19.915 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:19.915 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:19.915 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-24T15:33:19.917 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-24T15:33:19.918 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:19.917+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:19.929 INFO:teuthology.orchestra.run.smithi096.stdout:leaving image image_client.0-clone27 intact 2024-04-24T15:33:20.017 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:20.018+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:20.018 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #26, image image_client.0-clone26 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent27 2024-04-24T15:33:20.029 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:20.029 INFO:teuthology.orchestra.run.smithi096.stdout:4941 punch from 0x1b70b09 to 0x1b7bb28, (0xb01f bytes) 2024-04-24T15:33:20.044 INFO:teuthology.orchestra.run.smithi096.stdout:4942 write 0x6a59e8b thru 0x6a66c00 (0xcd76 bytes) 2024-04-24T15:33:20.044 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:20.044 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [ 2024-04-24T15:33:20.044 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.045 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:65484 4.9 4:96deeb01:::rbd_header.114aed92ca6e:head [call rbd.image_group_get in=18b out=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-04-24T15:33:20.045 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:20.045640+0000", 2024-04-24T15:33:20.045 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 0.000199619, 2024-04-24T15:33:20.045 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00020828099999999999, 2024-04-24T15:33:20.045 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:20.045 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:20.045 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:20.045 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:20.045 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:20.045 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 65484 2024-04-24T15:33:20.045 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.045 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:20.045 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.045 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:20.045 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:20.045640+0000", 2024-04-24T15:33:20.046 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.046 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.046 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.046 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:20.046 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:20.045640+0000", 2024-04-24T15:33:20.046 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.046 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.046 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.046 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:20.046 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:20.045639+0000", 2024-04-24T15:33:20.046 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:20.046 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.047 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.047 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:20.047 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:20.045642+0000", 2024-04-24T15:33:20.047 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.695e-06 2024-04-24T15:33:20.047 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.047 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.047 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:20.047 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:20.045648+0000", 2024-04-24T15:33:20.047 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.5450000000000003e-06 2024-04-24T15:33:20.047 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.047 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.047 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:20.047 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:20.045651+0000", 2024-04-24T15:33:20.047 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.9239999999999999e-06 2024-04-24T15:33:20.047 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.048 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.048 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:20.048 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:20.045681+0000", 2024-04-24T15:33:20.048 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.042e-05 2024-04-24T15:33:20.048 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.048 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.048 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:20.048 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:20.045750+0000", 2024-04-24T15:33:20.048 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.9261e-05 2024-04-24T15:33:20.048 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.048 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.048 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:20.048 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:20.045848+0000", 2024-04-24T15:33:20.048 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.7910000000000003e-05 2024-04-24T15:33:20.049 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.049 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:20.049 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.049 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.049 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-24T15:33:20.049 INFO:teuthology.orchestra.run.smithi151.stdout: "num_ops": 1 2024-04-24T15:33:20.049 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:20.050 INFO:teuthology.orchestra.run.smithi096.stdout:4944 read 0x26a79c2 thru 0x26a9044 (0x1683 bytes) 2024-04-24T15:33:20.050 INFO:teuthology.orchestra.run.smithi096.stdout:4945 read 0x3ba80aa thru 0x3bb7808 (0xf75f bytes) 2024-04-24T15:33:20.050 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:20.050 INFO:teuthology.orchestra.run.smithi096.stdout:4949 punch from 0x544bb5c to 0x545848f, (0xc933 bytes) 2024-04-24T15:33:20.050 INFO:teuthology.orchestra.run.smithi096.stdout:4950 read 0x3e0bbfe thru 0x3e11bff (0x6002 bytes) 2024-04-24T15:33:20.050 INFO:teuthology.orchestra.run.smithi096.stdout:4952 write 0x1891245 thru 0x189e182 (0xcf3e bytes) 2024-04-24T15:33:20.055 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-24T15:33:20.060 INFO:teuthology.orchestra.run.smithi096.stdout:4953 read 0x4a5422e thru 0x4a5c76c (0x853f bytes) 2024-04-24T15:33:20.061 INFO:teuthology.orchestra.run.smithi096.stdout:4954 trunc from 0x6a66c01 to 0x60db24c 2024-04-24T15:33:20.070 INFO:teuthology.orchestra.run.smithi096.stdout:4956 write 0x19ad7a6 thru 0x19b595f (0x81ba bytes) 2024-04-24T15:33:20.097 INFO:teuthology.orchestra.run.smithi096.stdout:4957 writesame 0xbb69288 thru 0xbb776b6 data_size 0x7(0xe42f bytes) 2024-04-24T15:33:20.109 INFO:teuthology.orchestra.run.smithi096.stdout:4958 trunc from 0xbb776b7 to 0x2423836 2024-04-24T15:33:20.118 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:33:20.119+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:33:20.147 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:20.147 INFO:teuthology.orchestra.run.smithi096.stdout:4960 read 0x1282eeb thru 0x128646f (0x3585 bytes) 2024-04-24T15:33:20.156 INFO:teuthology.orchestra.run.smithi096.stdout:4961 write 0x5cff4bb thru 0x5d016a8 (0x21ee bytes) 2024-04-24T15:33:20.159 INFO:teuthology.orchestra.run.smithi096.stdout:4962 read 0x5444dd0 thru 0x544dea8 (0x90d9 bytes) 2024-04-24T15:33:20.160 INFO:teuthology.orchestra.run.smithi096.stdout:4963 punch from 0x1285879 to 0x12903c9, (0xab50 bytes) 2024-04-24T15:33:20.160 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:20.160 INFO:teuthology.orchestra.run.smithi096.stdout:4966 read 0x21e6dfe thru 0x21f0bda (0x9ddd bytes) 2024-04-24T15:33:20.165 INFO:teuthology.orchestra.run.smithi096.stdout:4967 trunc from 0x5d016a9 to 0x1c81346 2024-04-24T15:33:20.190 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:20.191 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:33:20.191 INFO:teuthology.orchestra.run.smithi151.stdout: "complaint_time": 30, 2024-04-24T15:33:20.191 INFO:teuthology.orchestra.run.smithi151.stdout: "num_blocked_ops": 0 2024-04-24T15:33:20.191 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:20.194 INFO:teuthology.orchestra.run.smithi096.stdout:4968 writesame 0xabc634c thru 0xabccb76 data_size 0x3(0x682b bytes) 2024-04-24T15:33:20.194 INFO:teuthology.orchestra.run.smithi096.stdout:4970 writesame 0x24f5e82 thru 0x24f86a0 data_size 0x1(0x281f bytes) 2024-04-24T15:33:20.202 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-24T15:33:20.207 INFO:teuthology.orchestra.run.smithi096.stdout:4971 write 0xd98ef85 thru 0xd993edf (0x4f5b bytes) 2024-04-24T15:33:20.209 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:20.209 INFO:teuthology.orchestra.run.smithi096.stdout:4974 read 0x6a0a3b9 thru 0x6a0b732 (0x137a bytes) 2024-04-24T15:33:20.209 INFO:teuthology.orchestra.run.smithi096.stdout:4977 read 0xb3bed9a thru 0xb3ccb31 (0xdd98 bytes) 2024-04-24T15:33:20.209 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:20.213 INFO:teuthology.orchestra.run.smithi096.stdout:4979 trunc from 0xd993ee0 to 0xa7976a4 2024-04-24T15:33:20.228 INFO:teuthology.orchestra.run.smithi096.stdout:4980 read 0x825a43e thru 0x826657b (0xc13e bytes) 2024-04-24T15:33:20.228 INFO:teuthology.orchestra.run.smithi096.stdout:4981 write 0x68645c4 thru 0x686befe (0x793b bytes) 2024-04-24T15:33:20.239 INFO:teuthology.orchestra.run.smithi096.stdout:4982 write 0xdd44569 thru 0xdd4efd8 (0xaa70 bytes) 2024-04-24T15:33:20.245 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:20.246+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:20.255 INFO:teuthology.orchestra.run.smithi096.stdout:4984 trunc from 0xdd4efd9 to 0x27a9f35 2024-04-24T15:33:20.274 INFO:teuthology.orchestra.run.smithi096.stdout:4985 punch from 0x18d2c55 to 0x18dcbf3, (0x9f9e bytes) 2024-04-24T15:33:20.282 INFO:teuthology.orchestra.run.smithi096.stdout:4987 write 0x59c7d45 thru 0x59cc97b (0x4c37 bytes) 2024-04-24T15:33:20.286 INFO:teuthology.orchestra.run.smithi096.stdout:4988 read 0x2836aec thru 0x283d9ce (0x6ee3 bytes) 2024-04-24T15:33:20.286 INFO:teuthology.orchestra.run.smithi096.stdout:4990 punch from 0x441daf3 to 0x442177f, (0x3c8c bytes) 2024-04-24T15:33:20.286 INFO:teuthology.orchestra.run.smithi096.stdout:4991 write 0x437f1c7 thru 0x438249c (0x32d6 bytes) 2024-04-24T15:33:20.298 INFO:teuthology.orchestra.run.smithi096.stdout:4992 write 0x79dc451 thru 0x79dd1b0 (0xd60 bytes) 2024-04-24T15:33:20.301 INFO:teuthology.orchestra.run.smithi096.stdout:4993 write 0x3037774 thru 0x3043156 (0xb9e3 bytes) 2024-04-24T15:33:20.308 INFO:teuthology.orchestra.run.smithi096.stdout:4994 writesame 0x56e757f thru 0x56f070d data_size 0x1(0x918f bytes) 2024-04-24T15:33:20.320 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:20.321+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:20.332 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:20.332 INFO:teuthology.orchestra.run.smithi151.stdout: "size": 20, 2024-04-24T15:33:20.332 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 600, 2024-04-24T15:33:20.333 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [ 2024-04-24T15:33:20.333 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.333 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:39041 4.1 4:82c1a902:::rbd_data.114abe589119.000000000000032c:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e68)", 2024-04-24T15:33:20.333 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:01.760552+0000", 2024-04-24T15:33:20.333 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 18.571807259, 2024-04-24T15:33:20.333 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.87570748300000001, 2024-04-24T15:33:20.333 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:20.333 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "reached pg", 2024-04-24T15:33:20.333 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:20.333 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:20.333 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:20.333 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 39041 2024-04-24T15:33:20.333 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.334 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:20.334 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.334 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:20.334 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760552+0000", 2024-04-24T15:33:20.334 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.334 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.334 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.334 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:20.334 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760552+0000", 2024-04-24T15:33:20.334 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.334 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.334 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.334 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:20.335 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760550+0000", 2024-04-24T15:33:20.335 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999986 2024-04-24T15:33:20.335 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.335 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.335 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:20.335 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760557+0000", 2024-04-24T15:33:20.335 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1420000000000004e-06 2024-04-24T15:33:20.335 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.335 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.335 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:20.335 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760572+0000", 2024-04-24T15:33:20.335 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.483e-05 2024-04-24T15:33:20.335 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.336 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.336 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:20.336 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760578+0000", 2024-04-24T15:33:20.336 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.2820000000000002e-06 2024-04-24T15:33:20.336 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.336 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.336 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:20.336 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760628+0000", 2024-04-24T15:33:20.336 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.0040000000000002e-05 2024-04-24T15:33:20.336 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.336 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.336 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op must wait for map", 2024-04-24T15:33:20.336 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760637+0000", 2024-04-24T15:33:20.336 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.6899999999999998e-06 2024-04-24T15:33:20.336 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.337 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.337 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:20.337 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.770562+0000", 2024-04-24T15:33:20.337 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0099252739999999996 2024-04-24T15:33:20.337 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.337 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.337 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for peered", 2024-04-24T15:33:20.337 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.770572+0000", 2024-04-24T15:33:20.337 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0105999999999999e-05 2024-04-24T15:33:20.337 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.337 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.337 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:20.337 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:02.636170+0000", 2024-04-24T15:33:20.337 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.86559722299999997 2024-04-24T15:33:20.338 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.338 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.338 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:20.338 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:02.636259+0000", 2024-04-24T15:33:20.338 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.9446999999999999e-05 2024-04-24T15:33:20.338 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.338 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:20.338 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.338 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.338 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.338 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48675 4.d 4:b465d49b:::rbd_data.114a858e6641.0000000000000002:17 [read 2768896~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:20.338 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.926403+0000", 2024-04-24T15:33:20.339 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 9.4059556890000007, 2024-04-24T15:33:20.339 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035742807000000001, 2024-04-24T15:33:20.339 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:20.339 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:20.339 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:20.339 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:20.339 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:20.339 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48675 2024-04-24T15:33:20.339 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.339 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:20.339 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.339 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:20.340 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926403+0000", 2024-04-24T15:33:20.340 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.340 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.340 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.340 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:20.340 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926403+0000", 2024-04-24T15:33:20.340 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.340 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.340 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.340 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:20.340 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926403+0000", 2024-04-24T15:33:20.340 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:20.340 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.340 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.341 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:20.341 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926404+0000", 2024-04-24T15:33:20.341 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.424e-06 2024-04-24T15:33:20.341 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.341 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.341 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:20.341 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926413+0000", 2024-04-24T15:33:20.341 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.6850000000000007e-06 2024-04-24T15:33:20.341 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.341 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.341 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:20.341 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926417+0000", 2024-04-24T15:33:20.341 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.7859999999999998e-06 2024-04-24T15:33:20.341 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.341 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.342 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:20.342 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.961986+0000", 2024-04-24T15:33:20.342 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035568862 2024-04-24T15:33:20.342 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.342 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.342 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:20.342 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.962063+0000", 2024-04-24T15:33:20.342 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7257999999999998e-05 2024-04-24T15:33:20.342 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.342 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.342 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:20.342 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.962146+0000", 2024-04-24T15:33:20.342 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3094000000000001e-05 2024-04-24T15:33:20.343 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.343 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:20.343 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.343 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.343 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.343 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48676 4.d 4:b465d49b:::rbd_data.114a858e6641.0000000000000002:17 [read 2834432~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:20.343 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.926421+0000", 2024-04-24T15:33:20.343 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 9.4059381460000004, 2024-04-24T15:33:20.343 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035829002999999998, 2024-04-24T15:33:20.343 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:20.343 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:20.343 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:20.343 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:20.343 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:20.343 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48676 2024-04-24T15:33:20.344 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.344 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:20.344 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.344 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:20.344 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926421+0000", 2024-04-24T15:33:20.344 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.344 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.344 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.344 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:20.344 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926421+0000", 2024-04-24T15:33:20.344 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.344 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.344 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.344 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:20.344 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926420+0000", 2024-04-24T15:33:20.345 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:20.345 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.345 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.345 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:20.345 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926423+0000", 2024-04-24T15:33:20.345 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.356e-06 2024-04-24T15:33:20.345 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.345 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.345 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:20.345 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926428+0000", 2024-04-24T15:33:20.345 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.3340000000000001e-06 2024-04-24T15:33:20.345 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.345 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.345 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:20.346 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926431+0000", 2024-04-24T15:33:20.346 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.2899999999999998e-06 2024-04-24T15:33:20.346 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.346 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.346 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:20.346 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.962156+0000", 2024-04-24T15:33:20.346 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035725159999999999 2024-04-24T15:33:20.346 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.346 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.346 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:20.346 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.962197+0000", 2024-04-24T15:33:20.346 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0457999999999998e-05 2024-04-24T15:33:20.347 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.347 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.347 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:20.347 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.962250+0000", 2024-04-24T15:33:20.347 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.2812000000000002e-05 2024-04-24T15:33:20.347 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.347 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:20.347 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.347 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.347 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.347 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48703 4.d 4:b59d5735:::rbd_data.114a858e6641.0000000000000004:17 [read 2637824~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:20.347 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928531+0000", 2024-04-24T15:33:20.348 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 9.4038275200000001, 2024-04-24T15:33:20.348 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035675198999999998, 2024-04-24T15:33:20.348 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:20.348 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:20.348 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:20.348 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:20.348 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:20.348 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48703 2024-04-24T15:33:20.348 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.348 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:20.348 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.348 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:20.348 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928531+0000", 2024-04-24T15:33:20.348 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.348 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.348 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.349 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:20.349 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928531+0000", 2024-04-24T15:33:20.349 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.349 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.349 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.349 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:20.349 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928531+0000", 2024-04-24T15:33:20.349 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:20.349 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.349 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.349 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:20.349 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928532+0000", 2024-04-24T15:33:20.350 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.33e-06 2024-04-24T15:33:20.350 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.350 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.350 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:20.350 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928536+0000", 2024-04-24T15:33:20.350 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.4419999999999998e-06 2024-04-24T15:33:20.350 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.350 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.350 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:20.350 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928538+0000", 2024-04-24T15:33:20.350 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.0669999999999999e-06 2024-04-24T15:33:20.350 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.350 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.351 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:20.351 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964040+0000", 2024-04-24T15:33:20.351 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035501900000000003 2024-04-24T15:33:20.351 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.351 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.351 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:20.351 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964118+0000", 2024-04-24T15:33:20.351 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.8490000000000002e-05 2024-04-24T15:33:20.351 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.352 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.352 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:20.352 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964207+0000", 2024-04-24T15:33:20.352 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.8202999999999996e-05 2024-04-24T15:33:20.352 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.352 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:20.352 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.352 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.352 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.352 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48704 4.d 4:b59d5735:::rbd_data.114a858e6641.0000000000000004:17 [read 2703360~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:20.353 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928541+0000", 2024-04-24T15:33:20.353 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 9.4038181759999997, 2024-04-24T15:33:20.353 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035763016000000002, 2024-04-24T15:33:20.353 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:20.353 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:20.353 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:20.353 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:20.353 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:20.353 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48704 2024-04-24T15:33:20.353 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.353 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:20.353 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.353 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:20.353 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928541+0000", 2024-04-24T15:33:20.354 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.354 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.354 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.354 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:20.354 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928541+0000", 2024-04-24T15:33:20.354 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.354 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.354 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.354 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:20.354 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928540+0000", 2024-04-24T15:33:20.354 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:20.354 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.354 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.354 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:20.354 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928542+0000", 2024-04-24T15:33:20.355 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.37e-06 2024-04-24T15:33:20.355 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.355 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.355 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:20.355 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928545+0000", 2024-04-24T15:33:20.355 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8229999999999999e-06 2024-04-24T15:33:20.355 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.355 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.355 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:20.355 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928547+0000", 2024-04-24T15:33:20.355 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.9360000000000002e-06 2024-04-24T15:33:20.355 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.355 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.355 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:20.356 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964219+0000", 2024-04-24T15:33:20.356 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035672813999999997 2024-04-24T15:33:20.356 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.356 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.356 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:20.356 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964259+0000", 2024-04-24T15:33:20.356 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0070000000000001e-05 2024-04-24T15:33:20.356 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.356 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.356 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:20.356 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964304+0000", 2024-04-24T15:33:20.356 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4229e-05 2024-04-24T15:33:20.356 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.356 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:20.357 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.357 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.357 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.357 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48719 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 1851392~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:20.357 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928549+0000", 2024-04-24T15:33:20.357 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 9.4038100290000006, 2024-04-24T15:33:20.357 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035890645999999998, 2024-04-24T15:33:20.357 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:20.357 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:20.357 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:20.357 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:20.357 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:20.357 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48719 2024-04-24T15:33:20.357 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.358 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:20.358 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.358 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:20.358 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928549+0000", 2024-04-24T15:33:20.358 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.358 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.358 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.358 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:20.358 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928549+0000", 2024-04-24T15:33:20.358 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.358 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.358 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.358 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:20.358 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928549+0000", 2024-04-24T15:33:20.359 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:20.359 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.359 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.359 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:20.359 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928550+0000", 2024-04-24T15:33:20.359 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3829999999999999e-06 2024-04-24T15:33:20.359 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.359 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.359 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:20.359 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928553+0000", 2024-04-24T15:33:20.359 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7369999999999998e-06 2024-04-24T15:33:20.359 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.359 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.359 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:20.359 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928555+0000", 2024-04-24T15:33:20.360 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.8920000000000001e-06 2024-04-24T15:33:20.360 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.360 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.360 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:20.360 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964240+0000", 2024-04-24T15:33:20.360 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035685537000000003 2024-04-24T15:33:20.360 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.360 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.360 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:20.360 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964318+0000", 2024-04-24T15:33:20.360 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7838999999999998e-05 2024-04-24T15:33:20.360 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.360 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.360 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:20.360 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964439+0000", 2024-04-24T15:33:20.361 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000121564 2024-04-24T15:33:20.361 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.361 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:20.361 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.361 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.361 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.361 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48720 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 1916928~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:20.361 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928557+0000", 2024-04-24T15:33:20.361 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 9.4038021839999999, 2024-04-24T15:33:20.361 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036005219999999998, 2024-04-24T15:33:20.361 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:20.361 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:20.361 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:20.361 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:20.361 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:20.362 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48720 2024-04-24T15:33:20.362 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.362 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:20.362 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.362 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:20.362 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928557+0000", 2024-04-24T15:33:20.362 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.362 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.362 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.362 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:20.362 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928557+0000", 2024-04-24T15:33:20.362 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.362 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.363 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.363 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:20.363 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928556+0000", 2024-04-24T15:33:20.363 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:20.363 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.363 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.363 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:20.363 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928558+0000", 2024-04-24T15:33:20.363 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.13e-06 2024-04-24T15:33:20.363 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.363 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.363 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:20.364 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928561+0000", 2024-04-24T15:33:20.364 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5530000000000001e-06 2024-04-24T15:33:20.364 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.364 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.364 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:20.364 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928563+0000", 2024-04-24T15:33:20.364 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.8810000000000001e-06 2024-04-24T15:33:20.364 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.364 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.364 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:20.364 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964450+0000", 2024-04-24T15:33:20.364 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035887040000000002 2024-04-24T15:33:20.364 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.364 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.365 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:20.365 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964494+0000", 2024-04-24T15:33:20.365 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3699000000000003e-05 2024-04-24T15:33:20.365 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.365 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.365 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:20.365 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964562+0000", 2024-04-24T15:33:20.365 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.8149000000000001e-05 2024-04-24T15:33:20.365 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.365 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:20.365 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.365 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.365 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.365 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48721 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 1982464~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:20.365 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928565+0000", 2024-04-24T15:33:20.366 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 9.4037936599999998, 2024-04-24T15:33:20.366 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036233466999999998, 2024-04-24T15:33:20.366 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:20.366 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:20.366 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:20.366 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:20.366 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:20.366 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48721 2024-04-24T15:33:20.366 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.366 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:20.366 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.366 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:20.366 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928565+0000", 2024-04-24T15:33:20.367 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.367 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.367 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.367 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:20.367 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928565+0000", 2024-04-24T15:33:20.367 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.367 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.367 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.367 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:20.367 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928565+0000", 2024-04-24T15:33:20.367 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:20.367 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.367 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.367 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:20.367 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928566+0000", 2024-04-24T15:33:20.368 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.336e-06 2024-04-24T15:33:20.368 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.368 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.368 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:20.368 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928569+0000", 2024-04-24T15:33:20.368 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.9189999999999999e-06 2024-04-24T15:33:20.368 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.368 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.368 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:20.368 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928571+0000", 2024-04-24T15:33:20.368 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.1909999999999999e-06 2024-04-24T15:33:20.368 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.368 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.368 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:20.368 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964592+0000", 2024-04-24T15:33:20.369 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036020120000000003 2024-04-24T15:33:20.369 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.369 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.369 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:20.369 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964673+0000", 2024-04-24T15:33:20.369 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.1254000000000005e-05 2024-04-24T15:33:20.369 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.369 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.369 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:20.369 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964799+0000", 2024-04-24T15:33:20.369 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00012588700000000001 2024-04-24T15:33:20.369 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.369 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:20.369 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.369 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.370 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.370 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48722 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2048000~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:20.370 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928586+0000", 2024-04-24T15:33:20.370 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 9.4037725349999999, 2024-04-24T15:33:20.370 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036331347999999999, 2024-04-24T15:33:20.370 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:20.370 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:20.370 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:20.370 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:20.370 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:20.370 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48722 2024-04-24T15:33:20.370 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.370 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:20.370 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.371 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:20.371 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928586+0000", 2024-04-24T15:33:20.371 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.371 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.371 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.371 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:20.371 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928586+0000", 2024-04-24T15:33:20.371 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.371 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.371 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.371 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:20.371 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928586+0000", 2024-04-24T15:33:20.371 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:20.371 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.372 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.372 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:20.372 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928587+0000", 2024-04-24T15:33:20.372 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3519999999999999e-06 2024-04-24T15:33:20.372 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.372 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.372 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:20.372 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928590+0000", 2024-04-24T15:33:20.372 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.9699999999999999e-06 2024-04-24T15:33:20.372 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.372 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.372 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:20.373 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928592+0000", 2024-04-24T15:33:20.373 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.097e-06 2024-04-24T15:33:20.373 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.373 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.373 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:20.373 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964810+0000", 2024-04-24T15:33:20.373 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036217249 2024-04-24T15:33:20.373 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.373 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.373 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:20.373 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964851+0000", 2024-04-24T15:33:20.373 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.1151e-05 2024-04-24T15:33:20.373 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.374 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.374 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:20.374 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964918+0000", 2024-04-24T15:33:20.374 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.6898999999999998e-05 2024-04-24T15:33:20.374 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.374 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:20.374 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.374 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.374 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.374 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48723 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2113536~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:20.374 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928598+0000", 2024-04-24T15:33:20.374 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 9.4037611460000008, 2024-04-24T15:33:20.374 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036552125999999997, 2024-04-24T15:33:20.375 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:20.375 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:20.375 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:20.375 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:20.375 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:20.375 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48723 2024-04-24T15:33:20.375 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.375 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:20.375 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.375 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:20.375 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928598+0000", 2024-04-24T15:33:20.375 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.375 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.376 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.376 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:20.376 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928598+0000", 2024-04-24T15:33:20.376 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.376 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.376 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.376 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:20.376 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928597+0000", 2024-04-24T15:33:20.376 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:20.376 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.376 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.376 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:20.376 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928599+0000", 2024-04-24T15:33:20.377 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.621e-06 2024-04-24T15:33:20.377 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.377 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.377 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:20.377 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928602+0000", 2024-04-24T15:33:20.377 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.9629999999999998e-06 2024-04-24T15:33:20.377 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.377 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.377 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:20.377 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928607+0000", 2024-04-24T15:33:20.377 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8940000000000001e-06 2024-04-24T15:33:20.377 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.377 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.377 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:20.378 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964947+0000", 2024-04-24T15:33:20.378 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036340439000000002 2024-04-24T15:33:20.378 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.378 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.378 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:20.378 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965028+0000", 2024-04-24T15:33:20.378 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.1054e-05 2024-04-24T15:33:20.378 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.378 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.378 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:20.378 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965150+0000", 2024-04-24T15:33:20.378 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00012154000000000001 2024-04-24T15:33:20.378 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.378 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:20.379 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.379 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.379 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.379 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48724 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2179072~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:20.379 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928609+0000", 2024-04-24T15:33:20.379 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 9.4037497709999993, 2024-04-24T15:33:20.379 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036679614999999999, 2024-04-24T15:33:20.379 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:20.379 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:20.379 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:20.379 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:20.379 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:20.379 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48724 2024-04-24T15:33:20.379 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.380 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:20.380 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.380 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:20.380 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928609+0000", 2024-04-24T15:33:20.380 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.380 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.380 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.380 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:20.380 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928609+0000", 2024-04-24T15:33:20.380 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.380 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.380 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.380 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:20.380 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928609+0000", 2024-04-24T15:33:20.380 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:20.381 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.381 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.381 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:20.381 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928610+0000", 2024-04-24T15:33:20.381 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3030000000000001e-06 2024-04-24T15:33:20.381 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.381 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.381 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:20.381 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928613+0000", 2024-04-24T15:33:20.381 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.762e-06 2024-04-24T15:33:20.381 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.381 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.382 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:20.382 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928615+0000", 2024-04-24T15:33:20.382 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.832e-06 2024-04-24T15:33:20.382 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.382 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.382 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:20.382 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965174+0000", 2024-04-24T15:33:20.382 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036559633000000001 2024-04-24T15:33:20.382 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.382 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.382 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:20.382 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965221+0000", 2024-04-24T15:33:20.382 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6239999999999998e-05 2024-04-24T15:33:20.383 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.383 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.383 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:20.383 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965289+0000", 2024-04-24T15:33:20.383 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.8053999999999996e-05 2024-04-24T15:33:20.383 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.383 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:20.383 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.383 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.383 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.383 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48725 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2244608~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:20.383 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928617+0000", 2024-04-24T15:33:20.383 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 9.4037420009999995, 2024-04-24T15:33:20.383 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036902659999999997, 2024-04-24T15:33:20.384 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:20.384 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:20.384 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:20.384 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:20.384 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:20.384 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48725 2024-04-24T15:33:20.384 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.384 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:20.384 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.384 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:20.384 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928617+0000", 2024-04-24T15:33:20.384 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.384 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.384 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.385 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:20.385 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928617+0000", 2024-04-24T15:33:20.385 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.385 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.385 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.385 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:20.385 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928617+0000", 2024-04-24T15:33:20.385 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:20.385 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.385 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.385 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:20.386 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928618+0000", 2024-04-24T15:33:20.386 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.206e-06 2024-04-24T15:33:20.386 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.386 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.386 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:20.386 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928621+0000", 2024-04-24T15:33:20.386 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5130000000000001e-06 2024-04-24T15:33:20.386 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.386 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.386 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:20.386 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928623+0000", 2024-04-24T15:33:20.386 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.9640000000000002e-06 2024-04-24T15:33:20.387 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.387 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.387 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:20.387 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965318+0000", 2024-04-24T15:33:20.387 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036694953000000002 2024-04-24T15:33:20.387 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.387 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.387 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:20.387 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965398+0000", 2024-04-24T15:33:20.387 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.9986000000000003e-05 2024-04-24T15:33:20.387 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.387 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.387 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:20.387 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965520+0000", 2024-04-24T15:33:20.387 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000121247 2024-04-24T15:33:20.388 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.388 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:20.388 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.388 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.388 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.388 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48726 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2310144~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:20.388 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928625+0000", 2024-04-24T15:33:20.388 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 9.4037334539999993, 2024-04-24T15:33:20.388 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037015521000000003, 2024-04-24T15:33:20.388 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:20.388 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:20.388 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:20.388 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:20.388 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:20.389 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48726 2024-04-24T15:33:20.389 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.389 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:20.389 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.389 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:20.389 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928625+0000", 2024-04-24T15:33:20.389 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.389 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.389 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.389 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:20.389 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928625+0000", 2024-04-24T15:33:20.389 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.389 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.389 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.390 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:20.390 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928625+0000", 2024-04-24T15:33:20.390 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:20.390 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.390 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.390 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:20.390 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928627+0000", 2024-04-24T15:33:20.390 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3179999999999999e-06 2024-04-24T15:33:20.390 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.390 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.390 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:20.390 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928629+0000", 2024-04-24T15:33:20.390 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.819e-06 2024-04-24T15:33:20.390 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.391 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.391 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:20.391 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928634+0000", 2024-04-24T15:33:20.391 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6999999999999999e-06 2024-04-24T15:33:20.391 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.391 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.391 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:20.391 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965531+0000", 2024-04-24T15:33:20.391 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036897040999999998 2024-04-24T15:33:20.391 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.391 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.391 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:20.392 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965573+0000", 2024-04-24T15:33:20.392 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.189e-05 2024-04-24T15:33:20.392 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.392 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.392 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:20.392 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965641+0000", 2024-04-24T15:33:20.392 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.7959000000000004e-05 2024-04-24T15:33:20.392 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.392 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:20.392 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.392 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.392 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.392 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48727 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2375680~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:20.392 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928637+0000", 2024-04-24T15:33:20.393 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 9.4037220930000007, 2024-04-24T15:33:20.393 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037235129999999998, 2024-04-24T15:33:20.393 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:20.393 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:20.393 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:20.393 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:20.393 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:20.393 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48727 2024-04-24T15:33:20.393 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.393 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:20.393 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.393 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:20.393 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928637+0000", 2024-04-24T15:33:20.393 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.394 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.394 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.394 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:20.394 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928637+0000", 2024-04-24T15:33:20.394 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.394 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.394 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.394 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:20.394 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928637+0000", 2024-04-24T15:33:20.394 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:20.394 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.394 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.394 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:20.395 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928638+0000", 2024-04-24T15:33:20.395 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.4279999999999999e-06 2024-04-24T15:33:20.395 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.395 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.395 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:20.395 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928641+0000", 2024-04-24T15:33:20.395 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.909e-06 2024-04-24T15:33:20.395 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.395 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.395 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:20.395 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928643+0000", 2024-04-24T15:33:20.395 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.9269999999999999e-06 2024-04-24T15:33:20.395 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.396 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.396 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:20.396 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965671+0000", 2024-04-24T15:33:20.396 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037028393 2024-04-24T15:33:20.396 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.396 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.396 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:20.396 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965752+0000", 2024-04-24T15:33:20.396 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0921999999999995e-05 2024-04-24T15:33:20.396 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.396 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.396 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:20.396 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965872+0000", 2024-04-24T15:33:20.397 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000119759 2024-04-24T15:33:20.397 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.397 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:20.397 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.397 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.397 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.397 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48728 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2441216~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:20.397 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928645+0000", 2024-04-24T15:33:20.397 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 9.4037139100000005, 2024-04-24T15:33:20.397 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037346852999999999, 2024-04-24T15:33:20.397 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:20.397 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:20.397 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:20.397 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:20.398 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:20.398 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48728 2024-04-24T15:33:20.398 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.398 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:20.398 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.398 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:20.398 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928645+0000", 2024-04-24T15:33:20.398 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.398 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.398 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.398 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:20.398 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928645+0000", 2024-04-24T15:33:20.399 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.399 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.399 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.399 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:20.399 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928645+0000", 2024-04-24T15:33:20.399 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:20.399 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.399 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.399 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:20.399 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928646+0000", 2024-04-24T15:33:20.399 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.291e-06 2024-04-24T15:33:20.399 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.399 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.399 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:20.400 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928649+0000", 2024-04-24T15:33:20.400 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8509999999999999e-06 2024-04-24T15:33:20.400 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.400 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.400 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:20.400 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928651+0000", 2024-04-24T15:33:20.400 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.809e-06 2024-04-24T15:33:20.400 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.400 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.400 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:20.400 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965882+0000", 2024-04-24T15:33:20.400 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037231715999999998 2024-04-24T15:33:20.400 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.400 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.400 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:20.401 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965925+0000", 2024-04-24T15:33:20.401 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2493999999999997e-05 2024-04-24T15:33:20.401 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.401 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.401 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:20.401 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965992+0000", 2024-04-24T15:33:20.401 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.6907000000000002e-05 2024-04-24T15:33:20.401 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.401 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:20.401 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.401 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.401 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.401 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48729 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2506752~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:20.401 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928653+0000", 2024-04-24T15:33:20.402 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 9.4037060350000008, 2024-04-24T15:33:20.402 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037576904000000001, 2024-04-24T15:33:20.402 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:20.402 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:20.402 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:20.402 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:20.402 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:20.402 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48729 2024-04-24T15:33:20.402 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.402 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:20.402 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.402 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:20.403 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928653+0000", 2024-04-24T15:33:20.403 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.403 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.403 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.403 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:20.403 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928653+0000", 2024-04-24T15:33:20.403 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.403 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.403 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.403 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:20.403 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928653+0000", 2024-04-24T15:33:20.404 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:20.404 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.404 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.404 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:20.404 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928654+0000", 2024-04-24T15:33:20.404 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3209999999999999e-06 2024-04-24T15:33:20.404 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.404 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.404 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:20.404 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928657+0000", 2024-04-24T15:33:20.404 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8559999999999998e-06 2024-04-24T15:33:20.404 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.404 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.404 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:20.405 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928659+0000", 2024-04-24T15:33:20.405 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7080000000000002e-06 2024-04-24T15:33:20.405 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.405 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.405 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:20.405 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966021+0000", 2024-04-24T15:33:20.405 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037361296000000002 2024-04-24T15:33:20.405 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.405 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.405 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:20.405 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966102+0000", 2024-04-24T15:33:20.405 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0939999999999994e-05 2024-04-24T15:33:20.405 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.405 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.405 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:20.406 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966230+0000", 2024-04-24T15:33:20.406 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00012799999999999999 2024-04-24T15:33:20.406 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.406 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:20.406 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.406 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.406 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.406 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48730 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2572288~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:20.406 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928662+0000", 2024-04-24T15:33:20.406 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 9.4036969470000003, 2024-04-24T15:33:20.406 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037687246000000001, 2024-04-24T15:33:20.406 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:20.407 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:20.407 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:20.407 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:20.407 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:20.407 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48730 2024-04-24T15:33:20.407 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.407 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:20.407 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.407 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:20.407 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928662+0000", 2024-04-24T15:33:20.407 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.407 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.407 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.407 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:20.407 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928662+0000", 2024-04-24T15:33:20.407 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.408 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.408 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.408 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:20.408 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928662+0000", 2024-04-24T15:33:20.408 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:20.408 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.408 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.408 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:20.408 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928663+0000", 2024-04-24T15:33:20.408 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.381e-06 2024-04-24T15:33:20.408 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.408 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.408 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:20.408 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928666+0000", 2024-04-24T15:33:20.409 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.796e-06 2024-04-24T15:33:20.409 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.409 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.409 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:20.409 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928668+0000", 2024-04-24T15:33:20.409 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.8920000000000001e-06 2024-04-24T15:33:20.409 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.409 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.409 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:20.409 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966240+0000", 2024-04-24T15:33:20.409 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037572886999999999 2024-04-24T15:33:20.409 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.409 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.409 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:20.409 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966281+0000", 2024-04-24T15:33:20.409 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0450000000000001e-05 2024-04-24T15:33:20.410 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.410 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.410 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:20.410 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966349+0000", 2024-04-24T15:33:20.410 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.8215999999999999e-05 2024-04-24T15:33:20.410 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.410 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:20.410 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.410 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.410 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.410 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48731 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2637824~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:20.410 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928747+0000", 2024-04-24T15:33:20.410 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 9.4036119419999995, 2024-04-24T15:33:20.411 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037833602000000001, 2024-04-24T15:33:20.411 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:20.411 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:20.411 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:20.411 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:20.411 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:20.411 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48731 2024-04-24T15:33:20.411 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.411 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:20.411 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.411 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:20.411 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928747+0000", 2024-04-24T15:33:20.411 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.411 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.411 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.412 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:20.412 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928747+0000", 2024-04-24T15:33:20.412 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.412 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.412 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.412 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:20.412 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928670+0000", 2024-04-24T15:33:20.412 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999228 2024-04-24T15:33:20.412 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.412 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.412 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:20.412 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928750+0000", 2024-04-24T15:33:20.412 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0527999999999999e-05 2024-04-24T15:33:20.412 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.413 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.413 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:20.413 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928757+0000", 2024-04-24T15:33:20.413 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.9419999999999998e-06 2024-04-24T15:33:20.413 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.413 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.413 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:20.413 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928762+0000", 2024-04-24T15:33:20.413 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4240000000000003e-06 2024-04-24T15:33:20.413 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.413 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.413 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:20.413 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966379+0000", 2024-04-24T15:33:20.413 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037617267000000003 2024-04-24T15:33:20.413 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.414 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.414 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:20.414 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966461+0000", 2024-04-24T15:33:20.414 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.2720999999999996e-05 2024-04-24T15:33:20.414 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.414 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.414 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:20.414 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966581+0000", 2024-04-24T15:33:20.414 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000119014 2024-04-24T15:33:20.414 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.414 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:20.414 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.414 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.414 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.415 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48732 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2703360~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:20.415 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928766+0000", 2024-04-24T15:33:20.415 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 9.4035924580000003, 2024-04-24T15:33:20.415 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037930924999999997, 2024-04-24T15:33:20.415 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:20.415 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:20.415 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:20.415 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:20.415 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:20.415 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48732 2024-04-24T15:33:20.415 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.415 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:20.415 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.415 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:20.415 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928766+0000", 2024-04-24T15:33:20.415 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.416 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.416 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.416 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:20.416 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928766+0000", 2024-04-24T15:33:20.416 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.416 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.416 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.416 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:20.416 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928766+0000", 2024-04-24T15:33:20.416 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:20.416 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.416 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.416 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:20.416 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928768+0000", 2024-04-24T15:33:20.416 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.3420000000000002e-06 2024-04-24T15:33:20.417 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.417 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.417 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:20.417 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928783+0000", 2024-04-24T15:33:20.417 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.4813e-05 2024-04-24T15:33:20.417 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.417 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.417 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:20.417 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928787+0000", 2024-04-24T15:33:20.417 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.1999999999999996e-06 2024-04-24T15:33:20.417 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.417 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.417 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:20.417 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966592+0000", 2024-04-24T15:33:20.418 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037804484999999999 2024-04-24T15:33:20.418 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.418 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.418 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:20.418 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966632+0000", 2024-04-24T15:33:20.418 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0014e-05 2024-04-24T15:33:20.418 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.418 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.418 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:20.418 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966697+0000", 2024-04-24T15:33:20.418 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.5489999999999998e-05 2024-04-24T15:33:20.418 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.418 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:20.419 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.419 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.419 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.419 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48813 4.d 4:b39400aa:::rbd_data.114a8b543e91.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:20.419 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.936015+0000", 2024-04-24T15:33:20.419 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 9.3963441240000005, 2024-04-24T15:33:20.419 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.039312294999999997, 2024-04-24T15:33:20.419 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:20.419 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:20.419 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:20.419 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:20.419 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:20.420 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48813 2024-04-24T15:33:20.420 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.420 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:20.420 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.420 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:20.420 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.936015+0000", 2024-04-24T15:33:20.420 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.420 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.420 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.420 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:20.420 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.936015+0000", 2024-04-24T15:33:20.420 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.421 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.421 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.421 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:20.421 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.936014+0000", 2024-04-24T15:33:20.421 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:20.421 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.421 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.421 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:20.421 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.942618+0000", 2024-04-24T15:33:20.421 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0066038499999999997 2024-04-24T15:33:20.421 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.421 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.421 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:20.421 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.942629+0000", 2024-04-24T15:33:20.421 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0606e-05 2024-04-24T15:33:20.422 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.422 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.422 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:20.422 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.942634+0000", 2024-04-24T15:33:20.422 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.2000000000000002e-06 2024-04-24T15:33:20.422 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.422 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.422 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:20.422 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.969006+0000", 2024-04-24T15:33:20.422 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.026371564 2024-04-24T15:33:20.422 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.423 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.423 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:20.423 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.969126+0000", 2024-04-24T15:33:20.423 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000120085 2024-04-24T15:33:20.423 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.423 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.423 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 2", 2024-04-24T15:33:20.423 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.969469+0000", 2024-04-24T15:33:20.423 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00034295400000000002 2024-04-24T15:33:20.423 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.423 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.423 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:33:20.423 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.973118+0000", 2024-04-24T15:33:20.423 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0036489700000000001 2024-04-24T15:33:20.423 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.424 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.424 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:33:20.424 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.975250+0000", 2024-04-24T15:33:20.424 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0021327030000000001 2024-04-24T15:33:20.424 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.424 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.424 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:33:20.424 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.975293+0000", 2024-04-24T15:33:20.424 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2951999999999998e-05 2024-04-24T15:33:20.424 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.424 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.424 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:20.424 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.975327+0000", 2024-04-24T15:33:20.424 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.3840000000000001e-05 2024-04-24T15:33:20.424 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.424 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:20.425 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.425 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.425 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:20.425 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:20.427 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-24T15:33:20.427 INFO:teuthology.orchestra.run.smithi151.stderr:Nothing to do for non-filestore backend 2024-04-24T15:33:20.428 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:33:20.427+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:33:20.428 INFO:teuthology.orchestra.run.smithi096.stdout:4996 trunc from 0x79dd1b1 to 0x2b9d9dd 2024-04-24T15:33:20.428 INFO:teuthology.orchestra.run.smithi096.stdout:4998 punch from 0x168fc37 to 0x1694e73, (0x523c bytes) 2024-04-24T15:33:20.428 INFO:teuthology.orchestra.run.smithi096.stdout:4999 write 0xbb4f4be thru 0xbb52f8c (0x3acf bytes) 2024-04-24T15:33:20.429 INFO:teuthology.orchestra.run.smithi096.stdout:5000 write 0x2ea6be4 thru 0x2eb2e92 (0xc2af bytes) 2024-04-24T15:33:20.429 INFO:teuthology.orchestra.run.smithi096.stdout:5001 writesame 0x14fa73d thru 0x1507945 data_size 0x1(0xd209 bytes) 2024-04-24T15:33:20.429 INFO:teuthology.orchestra.run.smithi096.stdout:5003 read 0x9cd0848 thru 0x9cdffc6 (0xf77f bytes) 2024-04-24T15:33:20.429 INFO:teuthology.orchestra.run.smithi096.stdout:5004 read 0x4d80d3d thru 0x4d81e5c (0x1120 bytes) 2024-04-24T15:33:20.429 INFO:teuthology.orchestra.run.smithi096.stdout:5005 write 0x1bcc276 thru 0x1bcea87 (0x2812 bytes) 2024-04-24T15:33:20.429 INFO:teuthology.orchestra.run.smithi096.stdout:5006 read 0x2b8cf7b thru 0x2b9924a (0xc2d0 bytes) 2024-04-24T15:33:20.429 INFO:teuthology.orchestra.run.smithi096.stdout:5007 writesame 0xd1e1b5d thru 0xd1f0818 data_size 0x8b(0xecbc bytes) 2024-04-24T15:33:20.429 INFO:teuthology.orchestra.run.smithi096.stdout:5008 read 0x2200060 thru 0x220d777 (0xd718 bytes) 2024-04-24T15:33:20.429 INFO:teuthology.orchestra.run.smithi096.stdout:5009 punch from 0x91511e7 to 0x9155d4e, (0x4b67 bytes) 2024-04-24T15:33:20.429 INFO:teuthology.orchestra.run.smithi096.stdout:5010 read 0x9726e94 thru 0x972b7ca (0x4937 bytes) 2024-04-24T15:33:20.429 INFO:teuthology.orchestra.run.smithi096.stdout:5011 write 0x6b8d43 thru 0x6c4543 (0xb801 bytes) 2024-04-24T15:33:20.434 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:20.435 INFO:teuthology.orchestra.run.smithi096.stdout:5014 read 0x1c96957 thru 0x1ca51c2 (0xe86c bytes) 2024-04-24T15:33:20.450 INFO:teuthology.orchestra.run.smithi096.stdout:5015 trunc from 0xd1f0819 to 0x4c34e0 2024-04-24T15:33:20.489 INFO:teuthology.orchestra.run.smithi096.stdout:5016 punch from 0x1bf0bb to 0x1cab65, (0xbaaa bytes) 2024-04-24T15:33:20.506 INFO:teuthology.orchestra.run.smithi096.stdout:5017 write 0x92091e7 thru 0x920fc58 (0x6a72 bytes) 2024-04-24T15:33:20.514 INFO:teuthology.orchestra.run.smithi096.stdout:5018 trunc from 0x920fc59 to 0xde4ed2f 2024-04-24T15:33:20.520 INFO:teuthology.orchestra.run.smithi096.stdout:5020 read 0x75c7685 thru 0x75c9dfa (0x2776 bytes) 2024-04-24T15:33:20.520 INFO:teuthology.orchestra.run.smithi096.stdout:5021 read 0xd2e6b99 thru 0xd2f2658 (0xbac0 bytes) 2024-04-24T15:33:20.521 INFO:teuthology.orchestra.run.smithi096.stdout:5022 writesame 0x3d9939e thru 0x3da83d5 data_size 0x4(0xf038 bytes) 2024-04-24T15:33:20.527 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:33:20.527+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:33:20.528 INFO:teuthology.orchestra.run.smithi096.stdout:5023 read 0xa343fd4 thru 0xa3522e5 (0xe312 bytes) 2024-04-24T15:33:20.528 INFO:teuthology.orchestra.run.smithi096.stdout:5024 read 0x7ecabdb thru 0x7ed9d36 (0xf15c bytes) 2024-04-24T15:33:20.528 INFO:teuthology.orchestra.run.smithi096.stdout:5025 write 0xc1af196 thru 0xc1ba0a0 (0xaf0b bytes) 2024-04-24T15:33:20.531 INFO:teuthology.orchestra.run.smithi096.stdout:5026 writesame 0x4ca506d thru 0x4cb47a9 data_size 0x1(0xf73d bytes) 2024-04-24T15:33:20.551 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:20.556 INFO:teuthology.orchestra.run.smithi096.stdout:5028 trunc from 0xde4ed2f to 0x9543853 2024-04-24T15:33:20.566 INFO:teuthology.orchestra.run.smithi096.stdout:5030 write 0x702742 thru 0x70953e (0x6dfd bytes) 2024-04-24T15:33:20.568 INFO:teuthology.orchestra.run.smithi096.stdout:5032 read 0x72cea2a thru 0x72d733b (0x8912 bytes) 2024-04-24T15:33:20.568 INFO:teuthology.orchestra.run.smithi096.stdout:5033 read 0x2c32276 thru 0x2c370cb (0x4e56 bytes) 2024-04-24T15:33:20.568 INFO:teuthology.orchestra.run.smithi096.stdout:5034 read 0x12a70ea thru 0x12a9681 (0x2598 bytes) 2024-04-24T15:33:20.569 INFO:teuthology.orchestra.run.smithi096.stdout:5035 punch from 0x984eb5 to 0x990ab1, (0xbbfc bytes) 2024-04-24T15:33:20.569 INFO:teuthology.orchestra.run.smithi096.stdout:5036 read 0x8ec079c thru 0x8ec85d4 (0x7e39 bytes) 2024-04-24T15:33:20.569 INFO:teuthology.orchestra.run.smithi096.stdout:5037 writesame 0x199c1bf thru 0x199f66e data_size 0x18(0x34b0 bytes) 2024-04-24T15:33:20.571 INFO:teuthology.orchestra.run.smithi096.stdout:5039 read 0x4b06144 thru 0x4b0d79c (0x7659 bytes) 2024-04-24T15:33:20.571 INFO:teuthology.orchestra.run.smithi096.stdout:5040 writesame 0x2ffbb8b thru 0x3004283 data_size 0x1(0x86f9 bytes) 2024-04-24T15:33:20.587 INFO:teuthology.orchestra.run.smithi096.stdout:5042 writesame 0x36cbbf6 thru 0x36d8f92 data_size 0x47(0xd39d bytes) 2024-04-24T15:33:20.592 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:20.592 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:33:20.592 INFO:teuthology.orchestra.run.smithi151.stdout: "num_ops": 0 2024-04-24T15:33:20.592 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:20.602 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-24T15:33:20.608 INFO:teuthology.orchestra.run.smithi096.stdout:5043 write 0xeaed268 thru 0xeaf6c22 (0x99bb bytes) 2024-04-24T15:33:20.610 INFO:teuthology.orchestra.run.smithi096.stdout:5046 read 0x9c38ea5 thru 0x9c48e25 (0xff81 bytes) 2024-04-24T15:33:20.611 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:20.618 INFO:teuthology.orchestra.run.smithi096.stdout:5049 trunc from 0xeaf6c23 to 0x892ac5e 2024-04-24T15:33:20.627 INFO:teuthology.orchestra.run.smithi096.stdout:5051 read 0x5e54cb1 thru 0x5e62915 (0xdc65 bytes) 2024-04-24T15:33:20.639 INFO:teuthology.orchestra.run.smithi096.stdout:5052 writesame 0xe13ecb6 thru 0xe14ac39 data_size 0x44(0xbf84 bytes) 2024-04-24T15:33:20.639 INFO:teuthology.orchestra.run.smithi096.stdout:5054 read 0x11b17a4 thru 0x11b373a (0x1f97 bytes) 2024-04-24T15:33:20.639 INFO:teuthology.orchestra.run.smithi096.stdout:5055 write 0x325ba5 thru 0x32b92c (0x5d88 bytes) 2024-04-24T15:33:20.640 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:20.640 INFO:teuthology.orchestra.run.smithi096.stdout:5058 punch from 0x7cf79c0 to 0x7cf97ca, (0x1e0a bytes) 2024-04-24T15:33:20.643 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:20.644+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:20.647 INFO:teuthology.orchestra.run.smithi096.stdout:5060 trunc from 0xe14ac3a to 0x7f6b335 2024-04-24T15:33:20.654 INFO:teuthology.orchestra.run.smithi096.stdout:5061 write 0x646fb8 thru 0x648a55 (0x1a9e bytes) 2024-04-24T15:33:20.656 INFO:teuthology.orchestra.run.smithi096.stdout:5062 read 0x4b934aa thru 0x4b971f3 (0x3d4a bytes) 2024-04-24T15:33:20.656 INFO:teuthology.orchestra.run.smithi096.stdout:5063 read 0x2306ffd thru 0x230ef6f (0x7f73 bytes) 2024-04-24T15:33:20.656 INFO:teuthology.orchestra.run.smithi096.stdout:5064 read 0x1e1683e thru 0x1e1719d (0x960 bytes) 2024-04-24T15:33:20.656 INFO:teuthology.orchestra.run.smithi096.stdout:5066 read 0x5062c9 thru 0x515cd7 (0xfa0f bytes) 2024-04-24T15:33:20.660 INFO:teuthology.orchestra.run.smithi096.stdout:5067 trunc from 0x7f6b335 to 0xb5f534f 2024-04-24T15:33:20.665 INFO:teuthology.orchestra.run.smithi096.stdout:5068 write 0x22e009d thru 0x22e0294 (0x1f8 bytes) 2024-04-24T15:33:20.669 INFO:teuthology.orchestra.run.smithi096.stdout:5069 read 0x38f98dc thru 0x39033e5 (0x9b0a bytes) 2024-04-24T15:33:20.669 INFO:teuthology.orchestra.run.smithi096.stdout:5070 read 0x1cac2cb thru 0x1cb5dc0 (0x9af6 bytes) 2024-04-24T15:33:20.670 INFO:teuthology.orchestra.run.smithi096.stdout:5072 read 0x9c38d03 thru 0x9c3d0df (0x43dd bytes) 2024-04-24T15:33:20.674 INFO:teuthology.orchestra.run.smithi096.stdout:5074 trunc from 0xb5f534f to 0xeb8dde9 2024-04-24T15:33:20.681 INFO:teuthology.orchestra.run.smithi096.stdout:5075 writesame 0x7d4c152 thru 0x7d57faa data_size 0x25(0xbe59 bytes) 2024-04-24T15:33:20.684 INFO:teuthology.orchestra.run.smithi096.stdout:5077 read 0x33349f1 thru 0x333867d (0x3c8d bytes) 2024-04-24T15:33:20.685 INFO:teuthology.orchestra.run.smithi096.stdout:5078 writesame 0xaaadf06 thru 0xaab7a39 data_size 0x9a(0x9b34 bytes) 2024-04-24T15:33:20.688 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:20.688 INFO:teuthology.orchestra.run.smithi096.stdout:5082 read 0x28c003e thru 0x28c31ad (0x3170 bytes) 2024-04-24T15:33:20.688 INFO:teuthology.orchestra.run.smithi096.stdout:5083 write 0x695dfdf thru 0x696cb30 (0xeb52 bytes) 2024-04-24T15:33:20.690 INFO:teuthology.orchestra.run.smithi096.stdout:5084 read 0x1cbe16f thru 0x1cc32f4 (0x5186 bytes) 2024-04-24T15:33:20.691 INFO:teuthology.orchestra.run.smithi096.stdout:5085 read 0xb600aa thru 0xb690dc (0x9033 bytes) 2024-04-24T15:33:20.691 INFO:teuthology.orchestra.run.smithi096.stdout:5086 write 0xa784aa1 thru 0xa78518a (0x6ea bytes) 2024-04-24T15:33:20.693 INFO:teuthology.orchestra.run.smithi096.stdout:5088 read 0xa2f122e thru 0xa2fdec8 (0xcc9b bytes) 2024-04-24T15:33:20.694 INFO:teuthology.orchestra.run.smithi096.stdout:5089 write 0x8890248 thru 0x8890d60 (0xb19 bytes) 2024-04-24T15:33:20.695 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:20.695 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:20.695 INFO:teuthology.orchestra.run.smithi096.stdout:5095 read 0x67d6c6a thru 0x67e2763 (0xbafa bytes) 2024-04-24T15:33:20.696 INFO:teuthology.orchestra.run.smithi096.stdout:5096 read 0x80d262d thru 0x80df88c (0xd260 bytes) 2024-04-24T15:33:20.696 INFO:teuthology.orchestra.run.smithi096.stdout:5097 read 0x34799c2 thru 0x347d3e7 (0x3a26 bytes) 2024-04-24T15:33:20.696 INFO:teuthology.orchestra.run.smithi096.stdout:5099 write 0x814fdfd thru 0x815faf0 (0xfcf4 bytes) 2024-04-24T15:33:20.700 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:20.708 INFO:teuthology.orchestra.run.smithi096.stdout:5101 trunc from 0xeb8dde9 to 0x83108f2 2024-04-24T15:33:20.728 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:20.729+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:20.729 INFO:teuthology.orchestra.run.smithi096.stdout:5104 read 0x7c7f169 thru 0x7c88a7c (0x9914 bytes) 2024-04-24T15:33:20.730 INFO:teuthology.orchestra.run.smithi096.stdout:5105 read 0x6678191 thru 0x667cc5d (0x4acd bytes) 2024-04-24T15:33:20.742 INFO:teuthology.orchestra.run.smithi096.stdout:5106 write 0xda86117 thru 0xda93296 (0xd180 bytes) 2024-04-24T15:33:20.755 INFO:teuthology.orchestra.run.smithi096.stdout:5107 trunc from 0xda93297 to 0x5257592 2024-04-24T15:33:20.769 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:20.769 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:33:20.769 INFO:teuthology.orchestra.run.smithi151.stdout: "complaint_time": 30, 2024-04-24T15:33:20.769 INFO:teuthology.orchestra.run.smithi151.stdout: "num_blocked_ops": 0 2024-04-24T15:33:20.769 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:20.779 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-24T15:33:20.788 INFO:teuthology.orchestra.run.smithi096.stdout:5113 write 0xd483d17 thru 0xd487380 (0x366a bytes) 2024-04-24T15:33:20.790 INFO:teuthology.orchestra.run.smithi096.stdout:5114 read 0x75d3ab2 thru 0x75d7253 (0x37a2 bytes) 2024-04-24T15:33:20.790 INFO:teuthology.orchestra.run.smithi096.stdout:5115 write 0xc2e28da thru 0xc2ea4a4 (0x7bcb bytes) 2024-04-24T15:33:20.793 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:20.793 INFO:teuthology.orchestra.run.smithi096.stdout:5117 read 0x97d41a7 thru 0x97dafe6 (0x6e40 bytes) 2024-04-24T15:33:20.793 INFO:teuthology.orchestra.run.smithi096.stdout:5118 read 0x31ebfb8 thru 0x31f5bd0 (0x9c19 bytes) 2024-04-24T15:33:20.793 INFO:teuthology.orchestra.run.smithi096.stdout:5119 punch from 0x3ce9c2f to 0x3cf755a, (0xd92b bytes) 2024-04-24T15:33:20.794 INFO:teuthology.orchestra.run.smithi096.stdout:5121 trunc from 0xd487381 to 0xc4daccb 2024-04-24T15:33:20.806 INFO:teuthology.orchestra.run.smithi096.stdout:5122 write 0x771263a thru 0x771bfeb (0x99b2 bytes) 2024-04-24T15:33:20.808 INFO:teuthology.orchestra.run.smithi096.stdout:5125 write 0x68ff9e5 thru 0x690b1a0 (0xb7bc bytes) 2024-04-24T15:33:20.815 INFO:teuthology.orchestra.run.smithi096.stdout:5127 writesame 0x739e2ce thru 0x73a7969 data_size 0x99(0x969c bytes) 2024-04-24T15:33:20.818 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:20.818 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:20.818 INFO:teuthology.orchestra.run.smithi096.stdout:5133 write 0x47f7ffb thru 0x4806c80 (0xec86 bytes) 2024-04-24T15:33:20.821 INFO:teuthology.orchestra.run.smithi096.stdout:5134 write 0x636f25d thru 0x637430e (0x50b2 bytes) 2024-04-24T15:33:20.824 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:20.824 INFO:teuthology.orchestra.run.smithi096.stdout:5136 write 0xa84f1e7 thru 0xa85eaf5 (0xf90f bytes) 2024-04-24T15:33:20.827 INFO:teuthology.orchestra.run.smithi096.stdout:5138 read 0x8e46f3a thru 0x8e4defb (0x6fc2 bytes) 2024-04-24T15:33:20.827 INFO:teuthology.orchestra.run.smithi096.stdout:5139 read 0x8ff8095 thru 0x8ffb12c (0x3098 bytes) 2024-04-24T15:33:20.829 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:20.830+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:20.834 INFO:teuthology.orchestra.run.smithi096.stdout:5141 trunc from 0xc4daccb to 0x675207b 2024-04-24T15:33:20.846 INFO:teuthology.orchestra.run.smithi096.stdout:5143 read 0x5e84d32 thru 0x5e8c374 (0x7643 bytes) 2024-04-24T15:33:20.860 INFO:teuthology.orchestra.run.smithi096.stderr:nodeep-scrub is set 2024-04-24T15:33:20.864 INFO:teuthology.orchestra.run.smithi096.stdout:5145 writesame 0xe21f746 thru 0xe21fbec data_size 0x1(0x4a7 bytes) 2024-04-24T15:33:20.864 INFO:teuthology.orchestra.run.smithi096.stdout:5147 read 0x6ea88d7 thru 0x6eabd21 (0x344b bytes) 2024-04-24T15:33:20.864 INFO:teuthology.orchestra.run.smithi096.stdout:5148 write 0xda13e5a thru 0xda1d920 (0x9ac7 bytes) 2024-04-24T15:33:20.868 INFO:teuthology.orchestra.run.smithi096.stdout:5149 read 0x95f3863 thru 0x95fa1da (0x6978 bytes) 2024-04-24T15:33:20.868 INFO:teuthology.orchestra.run.smithi096.stdout:5150 write 0xbe6f74 thru 0xbf43a4 (0xd431 bytes) 2024-04-24T15:33:20.869 INFO:teuthology.orchestra.run.smithi096.stdout:5151 writesame 0xd622fd4 thru 0xd623996 data_size 0x15(0x9c3 bytes) 2024-04-24T15:33:20.872 INFO:teuthology.orchestra.run.smithi096.stdout:5152 punch from 0x45fe452 to 0x460a0d7, (0xbc85 bytes) 2024-04-24T15:33:20.872 INFO:teuthology.orchestra.run.smithi096.stdout:5153 write 0xcdeb4d9 thru 0xcdf9c55 (0xe77d bytes) 2024-04-24T15:33:20.876 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:20.892 INFO:teuthology.orchestra.run.smithi096.stdout:5156 trunc from 0xe21fbed to 0x588be8 2024-04-24T15:33:20.905 INFO:teuthology.orchestra.run.smithi096.stdout:5158 read 0x26c67e thru 0x26fe8b (0x380e bytes) 2024-04-24T15:33:20.909 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:20.909 INFO:teuthology.orchestra.run.smithi151.stdout: "size": 20, 2024-04-24T15:33:20.909 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 600, 2024-04-24T15:33:20.910 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [ 2024-04-24T15:33:20.910 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.910 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23613 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 14688256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:20.910 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.792522+0000", 2024-04-24T15:33:20.910 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 46.117177775000002, 2024-04-24T15:33:20.910 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040507356000000001, 2024-04-24T15:33:20.910 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:20.910 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:20.910 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:20.910 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:20.910 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:20.910 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23613 2024-04-24T15:33:20.910 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.911 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:20.911 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.911 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:20.911 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792522+0000", 2024-04-24T15:33:20.911 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.911 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.911 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.911 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:20.911 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792522+0000", 2024-04-24T15:33:20.911 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.911 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.911 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.911 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:20.911 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792522+0000", 2024-04-24T15:33:20.911 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:20.912 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.912 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.912 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:20.912 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792523+0000", 2024-04-24T15:33:20.912 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.576e-06 2024-04-24T15:33:20.912 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.912 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.912 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:20.912 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792526+0000", 2024-04-24T15:33:20.912 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7439999999999999e-06 2024-04-24T15:33:20.912 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.912 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.912 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:20.912 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792528+0000", 2024-04-24T15:33:20.912 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.0600000000000002e-06 2024-04-24T15:33:20.913 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.913 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.913 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:20.913 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.832864+0000", 2024-04-24T15:33:20.913 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040335742000000001 2024-04-24T15:33:20.913 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.913 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.913 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:20.913 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.832953+0000", 2024-04-24T15:33:20.913 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.9281000000000001e-05 2024-04-24T15:33:20.913 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.913 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.913 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:20.913 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.833029+0000", 2024-04-24T15:33:20.913 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.6259000000000003e-05 2024-04-24T15:33:20.914 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.914 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:20.914 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.914 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.914 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.914 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23614 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 14729216~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:20.914 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.792530+0000", 2024-04-24T15:33:20.914 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 46.117169511, 2024-04-24T15:33:20.914 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040633957999999998, 2024-04-24T15:33:20.914 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:20.914 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:20.914 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:20.914 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:20.914 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:20.914 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23614 2024-04-24T15:33:20.915 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.915 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:20.915 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.915 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:20.915 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792530+0000", 2024-04-24T15:33:20.915 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.915 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.915 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.915 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:20.915 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792530+0000", 2024-04-24T15:33:20.915 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.915 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.915 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.915 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:20.915 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792530+0000", 2024-04-24T15:33:20.916 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:20.916 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.916 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.916 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:20.916 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792531+0000", 2024-04-24T15:33:20.916 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.46e-06 2024-04-24T15:33:20.916 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.916 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.916 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:20.916 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792534+0000", 2024-04-24T15:33:20.916 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8959999999999999e-06 2024-04-24T15:33:20.916 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.916 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.916 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:20.916 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792536+0000", 2024-04-24T15:33:20.917 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.841e-06 2024-04-24T15:33:20.917 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.917 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.917 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:20.917 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.833060+0000", 2024-04-24T15:33:20.917 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040524125000000001 2024-04-24T15:33:20.917 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.917 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.917 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:20.917 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.833108+0000", 2024-04-24T15:33:20.917 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.7547999999999999e-05 2024-04-24T15:33:20.917 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.917 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.917 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:20.917 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.833164+0000", 2024-04-24T15:33:20.918 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6297e-05 2024-04-24T15:33:20.918 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.918 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:20.918 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.918 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.918 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.918 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23623 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15097856~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:20.918 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.793887+0000", 2024-04-24T15:33:20.918 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 46.115812550000001, 2024-04-24T15:33:20.918 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040432784999999999, 2024-04-24T15:33:20.918 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:20.918 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:20.918 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:20.918 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:20.919 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:20.919 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23623 2024-04-24T15:33:20.919 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.919 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:20.919 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.919 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:20.919 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793887+0000", 2024-04-24T15:33:20.919 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.919 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.919 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.919 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:20.919 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793887+0000", 2024-04-24T15:33:20.919 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.919 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.919 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.919 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:20.920 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793887+0000", 2024-04-24T15:33:20.920 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:20.920 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.920 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.920 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:20.920 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793888+0000", 2024-04-24T15:33:20.920 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3659999999999999e-06 2024-04-24T15:33:20.920 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.920 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.920 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:20.920 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793891+0000", 2024-04-24T15:33:20.920 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8389999999999998e-06 2024-04-24T15:33:20.920 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.920 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.920 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:20.921 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793893+0000", 2024-04-24T15:33:20.921 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.9549999999999999e-06 2024-04-24T15:33:20.921 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.921 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.921 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:20.921 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834239+0000", 2024-04-24T15:33:20.921 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040345487999999999 2024-04-24T15:33:20.921 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.921 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.921 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:20.921 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834280+0000", 2024-04-24T15:33:20.921 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.1878000000000001e-05 2024-04-24T15:33:20.921 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.921 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.921 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:20.921 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834320+0000", 2024-04-24T15:33:20.922 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9480000000000001e-05 2024-04-24T15:33:20.922 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.922 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:20.922 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.922 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.922 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.922 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23624 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15138816~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:20.922 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.794053+0000", 2024-04-24T15:33:20.922 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 46.115646601999998, 2024-04-24T15:33:20.922 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040370801999999997, 2024-04-24T15:33:20.922 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:20.922 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:20.922 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:20.923 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:20.923 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:20.923 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23624 2024-04-24T15:33:20.923 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.923 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:20.923 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.923 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:20.923 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794053+0000", 2024-04-24T15:33:20.923 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.923 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.923 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.923 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:20.923 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794053+0000", 2024-04-24T15:33:20.923 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.924 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.924 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.924 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:20.924 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793895+0000", 2024-04-24T15:33:20.924 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998422 2024-04-24T15:33:20.924 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.924 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.924 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:20.924 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794057+0000", 2024-04-24T15:33:20.924 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00016149400000000001 2024-04-24T15:33:20.924 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.924 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.924 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:20.924 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794065+0000", 2024-04-24T15:33:20.924 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.5259999999999993e-06 2024-04-24T15:33:20.925 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.925 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.925 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:20.925 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794071+0000", 2024-04-24T15:33:20.925 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6149999999999996e-06 2024-04-24T15:33:20.925 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.925 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.925 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:20.925 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834341+0000", 2024-04-24T15:33:20.925 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040270366000000002 2024-04-24T15:33:20.925 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.925 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.925 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:20.925 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834381+0000", 2024-04-24T15:33:20.926 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0102999999999999e-05 2024-04-24T15:33:20.926 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.926 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.926 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:20.926 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834424+0000", 2024-04-24T15:33:20.926 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2713000000000003e-05 2024-04-24T15:33:20.926 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.926 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:20.926 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.926 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.926 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.926 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23625 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15179776~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:20.926 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.794160+0000", 2024-04-24T15:33:20.926 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 46.115539720000001, 2024-04-24T15:33:20.927 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040413004000000002, 2024-04-24T15:33:20.927 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:20.927 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:20.927 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:20.927 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:20.927 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:20.927 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23625 2024-04-24T15:33:20.927 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.927 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:20.927 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.927 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:20.927 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794160+0000", 2024-04-24T15:33:20.927 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.927 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.927 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.928 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:20.928 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794160+0000", 2024-04-24T15:33:20.928 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.928 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.928 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.928 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:20.928 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794086+0000", 2024-04-24T15:33:20.928 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999256 2024-04-24T15:33:20.928 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.928 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.928 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:20.928 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794163+0000", 2024-04-24T15:33:20.928 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7896000000000001e-05 2024-04-24T15:33:20.928 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.928 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.929 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:20.929 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794171+0000", 2024-04-24T15:33:20.929 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.9500000000000001e-06 2024-04-24T15:33:20.929 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.929 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.929 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:20.929 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794179+0000", 2024-04-24T15:33:20.929 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7970000000000001e-06 2024-04-24T15:33:20.929 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.929 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.929 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:20.929 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834451+0000", 2024-04-24T15:33:20.929 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040271492999999998 2024-04-24T15:33:20.929 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.930 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.930 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:20.930 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834519+0000", 2024-04-24T15:33:20.930 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.7984000000000005e-05 2024-04-24T15:33:20.930 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.930 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.930 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:20.930 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834573+0000", 2024-04-24T15:33:20.930 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.4339999999999998e-05 2024-04-24T15:33:20.930 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.930 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:20.930 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.930 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.930 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.931 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23629 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15343616~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:20.931 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.794748+0000", 2024-04-24T15:33:20.931 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 46.114951918000003, 2024-04-24T15:33:20.931 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040419718, 2024-04-24T15:33:20.931 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:20.931 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:20.931 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:20.931 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:20.931 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:20.931 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23629 2024-04-24T15:33:20.931 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.931 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:20.931 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.931 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:20.931 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794748+0000", 2024-04-24T15:33:20.932 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.932 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.932 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.932 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:20.932 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794748+0000", 2024-04-24T15:33:20.932 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.932 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.932 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.932 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:20.932 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794673+0000", 2024-04-24T15:33:20.932 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999251 2024-04-24T15:33:20.932 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.932 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.932 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:20.933 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794751+0000", 2024-04-24T15:33:20.933 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.8480999999999996e-05 2024-04-24T15:33:20.933 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.933 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.933 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:20.933 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794760+0000", 2024-04-24T15:33:20.933 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3429999999999996e-06 2024-04-24T15:33:20.933 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.933 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.933 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:20.933 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794764+0000", 2024-04-24T15:33:20.933 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5600000000000004e-06 2024-04-24T15:33:20.933 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.933 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.933 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:20.934 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835040+0000", 2024-04-24T15:33:20.934 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040275188000000003 2024-04-24T15:33:20.934 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.934 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.934 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:20.934 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835105+0000", 2024-04-24T15:33:20.934 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.5992000000000001e-05 2024-04-24T15:33:20.934 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.934 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.934 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:20.934 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835167+0000", 2024-04-24T15:33:20.934 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.1981000000000002e-05 2024-04-24T15:33:20.934 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.934 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:20.934 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.935 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.935 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.935 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23630 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15384576~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:20.935 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.794839+0000", 2024-04-24T15:33:20.935 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 46.114860849000003, 2024-04-24T15:33:20.935 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040523894999999997, 2024-04-24T15:33:20.935 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:20.935 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:20.935 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:20.935 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:20.935 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:20.935 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23630 2024-04-24T15:33:20.935 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.935 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:20.936 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.936 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:20.936 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794839+0000", 2024-04-24T15:33:20.936 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.936 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.936 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.936 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:20.936 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794839+0000", 2024-04-24T15:33:20.936 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.936 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.936 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.936 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:20.936 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794768+0000", 2024-04-24T15:33:20.936 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999294 2024-04-24T15:33:20.936 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.937 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.937 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:20.937 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794841+0000", 2024-04-24T15:33:20.937 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2793000000000006e-05 2024-04-24T15:33:20.937 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.937 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.937 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:20.937 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794846+0000", 2024-04-24T15:33:20.937 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.7230000000000004e-06 2024-04-24T15:33:20.937 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.937 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.937 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:20.937 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794848+0000", 2024-04-24T15:33:20.937 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7659999999999999e-06 2024-04-24T15:33:20.938 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.938 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.938 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:20.938 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835223+0000", 2024-04-24T15:33:20.938 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040374275000000001 2024-04-24T15:33:20.938 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.938 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.938 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:20.938 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835295+0000", 2024-04-24T15:33:20.938 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2633999999999996e-05 2024-04-24T15:33:20.938 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.938 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.938 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:20.938 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835363+0000", 2024-04-24T15:33:20.939 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.7355999999999997e-05 2024-04-24T15:33:20.939 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.939 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:20.939 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.939 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.939 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.939 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23631 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15425536~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:20.939 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.794956+0000", 2024-04-24T15:33:20.939 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 46.114744008000002, 2024-04-24T15:33:20.939 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040562383, 2024-04-24T15:33:20.939 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:20.939 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:20.939 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:20.939 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:20.939 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:20.940 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23631 2024-04-24T15:33:20.940 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.940 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:20.940 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.940 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:20.940 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794956+0000", 2024-04-24T15:33:20.940 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.940 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.940 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.940 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:20.940 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794956+0000", 2024-04-24T15:33:20.940 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.940 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.940 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.941 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:20.941 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794859+0000", 2024-04-24T15:33:20.941 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999032 2024-04-24T15:33:20.941 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.941 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.941 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:20.941 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794959+0000", 2024-04-24T15:33:20.941 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000100207 2024-04-24T15:33:20.941 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.941 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.941 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:20.941 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794967+0000", 2024-04-24T15:33:20.941 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7640000000000005e-06 2024-04-24T15:33:20.941 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.941 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.942 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:20.942 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794971+0000", 2024-04-24T15:33:20.942 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.335e-06 2024-04-24T15:33:20.942 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.942 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.942 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:20.942 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835388+0000", 2024-04-24T15:33:20.942 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040417340000000003 2024-04-24T15:33:20.942 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.942 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.942 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:20.942 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835459+0000", 2024-04-24T15:33:20.942 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.0118999999999997e-05 2024-04-24T15:33:20.942 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.943 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.943 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:20.943 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835518+0000", 2024-04-24T15:33:20.943 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.9438999999999998e-05 2024-04-24T15:33:20.943 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.943 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:20.943 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.943 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.943 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.943 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23632 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15466496~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:20.943 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795060+0000", 2024-04-24T15:33:20.943 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 46.114639298, 2024-04-24T15:33:20.943 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040667993, 2024-04-24T15:33:20.943 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:20.943 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:20.944 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:20.944 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:20.944 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:20.944 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23632 2024-04-24T15:33:20.944 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.944 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:20.944 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.944 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:20.944 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795060+0000", 2024-04-24T15:33:20.944 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.944 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.944 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.944 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:20.944 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795060+0000", 2024-04-24T15:33:20.944 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.945 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.945 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.945 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:20.945 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794976+0000", 2024-04-24T15:33:20.945 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999156 2024-04-24T15:33:20.945 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.945 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.945 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:20.945 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795064+0000", 2024-04-24T15:33:20.945 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.7709999999999999e-05 2024-04-24T15:33:20.945 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.945 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.945 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:20.945 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795072+0000", 2024-04-24T15:33:20.945 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3769999999999996e-06 2024-04-24T15:33:20.946 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.946 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.946 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:20.946 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795078+0000", 2024-04-24T15:33:20.946 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.5079999999999996e-06 2024-04-24T15:33:20.946 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.946 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.946 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:20.946 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835541+0000", 2024-04-24T15:33:20.946 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040463440000000003 2024-04-24T15:33:20.946 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.946 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.946 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:20.946 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835657+0000", 2024-04-24T15:33:20.947 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00011572499999999999 2024-04-24T15:33:20.947 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.947 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.947 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:20.947 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835728+0000", 2024-04-24T15:33:20.947 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1402000000000005e-05 2024-04-24T15:33:20.947 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.947 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:20.947 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.947 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.947 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.947 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23633 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15507456~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:20.947 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795134+0000", 2024-04-24T15:33:20.947 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 46.114565642999999, 2024-04-24T15:33:20.948 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040748591000000001, 2024-04-24T15:33:20.948 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:20.948 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:20.948 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:20.948 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:20.948 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:20.948 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23633 2024-04-24T15:33:20.948 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.948 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:20.948 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.948 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:20.948 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795134+0000", 2024-04-24T15:33:20.948 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.948 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.948 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.949 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:20.949 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795134+0000", 2024-04-24T15:33:20.949 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.949 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.949 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.949 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:20.949 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795083+0000", 2024-04-24T15:33:20.949 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999949 2024-04-24T15:33:20.949 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.949 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.949 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:20.949 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795138+0000", 2024-04-24T15:33:20.949 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.4472000000000003e-05 2024-04-24T15:33:20.949 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.950 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.950 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:20.950 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795145+0000", 2024-04-24T15:33:20.950 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2779999999999996e-06 2024-04-24T15:33:20.950 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.950 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.950 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:20.950 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795149+0000", 2024-04-24T15:33:20.950 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.568e-06 2024-04-24T15:33:20.950 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.950 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.950 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:20.950 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835757+0000", 2024-04-24T15:33:20.951 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040608894999999999 2024-04-24T15:33:20.951 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.951 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.951 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:20.951 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835829+0000", 2024-04-24T15:33:20.951 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1879999999999996e-05 2024-04-24T15:33:20.951 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.951 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.951 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:20.951 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835883+0000", 2024-04-24T15:33:20.951 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.3346000000000001e-05 2024-04-24T15:33:20.951 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.951 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:20.951 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.952 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.952 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.952 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23634 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15548416~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:20.952 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795271+0000", 2024-04-24T15:33:20.952 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 46.114428609000001, 2024-04-24T15:33:20.952 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040755845999999998, 2024-04-24T15:33:20.952 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:20.952 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:20.952 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:20.952 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:20.952 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:20.952 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23634 2024-04-24T15:33:20.952 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.952 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:20.953 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.953 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:20.953 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795271+0000", 2024-04-24T15:33:20.953 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.953 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.953 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.953 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:20.953 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795271+0000", 2024-04-24T15:33:20.953 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.953 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.953 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.953 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:20.953 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795152+0000", 2024-04-24T15:33:20.953 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998813 2024-04-24T15:33:20.953 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.954 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.954 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:20.954 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795276+0000", 2024-04-24T15:33:20.954 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00012401799999999999 2024-04-24T15:33:20.954 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.954 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.954 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:20.954 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795281+0000", 2024-04-24T15:33:20.954 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8860000000000003e-06 2024-04-24T15:33:20.954 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.954 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.954 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:20.954 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795284+0000", 2024-04-24T15:33:20.954 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.0450000000000001e-06 2024-04-24T15:33:20.955 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.955 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.955 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:20.955 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835905+0000", 2024-04-24T15:33:20.955 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040621035999999999 2024-04-24T15:33:20.955 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.955 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.955 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:20.955 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835971+0000", 2024-04-24T15:33:20.955 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.5314000000000005e-05 2024-04-24T15:33:20.955 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.955 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.955 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:20.955 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836027+0000", 2024-04-24T15:33:20.956 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6305999999999999e-05 2024-04-24T15:33:20.956 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.956 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:20.956 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.956 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.956 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.956 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23635 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15589376~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:20.956 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795382+0000", 2024-04-24T15:33:20.956 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 46.114317835999998, 2024-04-24T15:33:20.956 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040805363999999997, 2024-04-24T15:33:20.956 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:20.956 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:20.956 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:20.956 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:20.957 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:20.957 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23635 2024-04-24T15:33:20.957 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.957 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:20.957 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.957 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:20.957 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795382+0000", 2024-04-24T15:33:20.957 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.957 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.957 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.957 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:20.957 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795382+0000", 2024-04-24T15:33:20.957 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.957 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.957 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.958 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:20.958 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795288+0000", 2024-04-24T15:33:20.958 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999056 2024-04-24T15:33:20.958 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.958 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.958 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:20.958 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795385+0000", 2024-04-24T15:33:20.958 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.7288999999999996e-05 2024-04-24T15:33:20.958 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.958 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.958 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:20.958 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795392+0000", 2024-04-24T15:33:20.958 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.452e-06 2024-04-24T15:33:20.959 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.959 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.959 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:20.959 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795397+0000", 2024-04-24T15:33:20.959 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2860000000000001e-06 2024-04-24T15:33:20.959 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.959 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.959 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:20.959 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836054+0000", 2024-04-24T15:33:20.959 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040657581999999998 2024-04-24T15:33:20.959 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.959 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.959 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:20.959 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836124+0000", 2024-04-24T15:33:20.959 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.0020999999999999e-05 2024-04-24T15:33:20.960 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.960 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.960 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:20.960 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836187+0000", 2024-04-24T15:33:20.960 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.3009999999999995e-05 2024-04-24T15:33:20.960 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.960 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:20.960 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.960 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.960 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.960 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23636 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15630336~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:20.960 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795610+0000", 2024-04-24T15:33:20.960 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 46.114089477999997, 2024-04-24T15:33:20.960 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040730942999999999, 2024-04-24T15:33:20.960 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:20.961 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:20.961 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:20.961 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:20.961 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:20.961 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23636 2024-04-24T15:33:20.961 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.961 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:20.961 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.961 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:20.961 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795610+0000", 2024-04-24T15:33:20.961 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.961 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.961 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.961 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:20.961 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795610+0000", 2024-04-24T15:33:20.962 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.962 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.962 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.962 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:20.962 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795401+0000", 2024-04-24T15:33:20.962 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9997911 2024-04-24T15:33:20.962 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.962 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.962 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:20.962 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795614+0000", 2024-04-24T15:33:20.962 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00021228600000000001 2024-04-24T15:33:20.962 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.962 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.963 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:20.963 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795622+0000", 2024-04-24T15:33:20.963 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3920000000000003e-06 2024-04-24T15:33:20.963 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.963 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.963 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:20.963 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795627+0000", 2024-04-24T15:33:20.963 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8199999999999996e-06 2024-04-24T15:33:20.963 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.963 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.963 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:20.963 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836211+0000", 2024-04-24T15:33:20.963 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040583803000000002 2024-04-24T15:33:20.963 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.964 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.964 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:20.964 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836284+0000", 2024-04-24T15:33:20.964 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.3617999999999999e-05 2024-04-24T15:33:20.964 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.964 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.964 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:20.964 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836341+0000", 2024-04-24T15:33:20.964 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6959999999999997e-05 2024-04-24T15:33:20.964 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.964 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:20.964 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.964 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.965 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.965 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23637 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15671296~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:20.965 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795699+0000", 2024-04-24T15:33:20.965 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 46.114000310999998, 2024-04-24T15:33:20.965 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040775556999999997, 2024-04-24T15:33:20.965 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:20.965 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:20.965 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:20.965 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:20.965 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:20.965 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23637 2024-04-24T15:33:20.965 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.966 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:20.966 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.966 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:20.966 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795699+0000", 2024-04-24T15:33:20.966 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.966 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.966 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.966 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:20.966 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795699+0000", 2024-04-24T15:33:20.966 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.966 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.966 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.966 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:20.967 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795632+0000", 2024-04-24T15:33:20.967 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999323 2024-04-24T15:33:20.967 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.967 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.967 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:20.967 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795704+0000", 2024-04-24T15:33:20.967 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2428000000000005e-05 2024-04-24T15:33:20.967 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.967 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.967 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:20.967 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795715+0000", 2024-04-24T15:33:20.967 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.1532000000000001e-05 2024-04-24T15:33:20.967 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.967 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.968 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:20.968 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795721+0000", 2024-04-24T15:33:20.968 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.8200000000000002e-06 2024-04-24T15:33:20.968 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.968 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.968 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:20.968 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836364+0000", 2024-04-24T15:33:20.968 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040642997 2024-04-24T15:33:20.968 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.968 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.968 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:20.968 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836435+0000", 2024-04-24T15:33:20.968 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.0821000000000005e-05 2024-04-24T15:33:20.969 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.969 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.969 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:20.969 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836475+0000", 2024-04-24T15:33:20.969 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9796999999999999e-05 2024-04-24T15:33:20.969 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.969 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:20.969 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.969 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.969 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.969 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23638 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15712256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:20.969 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795932+0000", 2024-04-24T15:33:20.969 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 46.113767281000001, 2024-04-24T15:33:20.969 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040668682999999997, 2024-04-24T15:33:20.970 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:20.970 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:20.970 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:20.970 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:20.970 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:20.970 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23638 2024-04-24T15:33:20.970 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.970 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:20.970 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.970 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:20.970 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795932+0000", 2024-04-24T15:33:20.970 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.970 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.970 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.971 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:20.971 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795932+0000", 2024-04-24T15:33:20.971 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.971 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.971 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.971 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:20.971 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795728+0000", 2024-04-24T15:33:20.971 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9997959 2024-04-24T15:33:20.971 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.971 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.971 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:20.971 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795936+0000", 2024-04-24T15:33:20.971 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000208128 2024-04-24T15:33:20.972 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.972 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.972 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:20.972 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795945+0000", 2024-04-24T15:33:20.972 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.7110000000000001e-06 2024-04-24T15:33:20.972 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.972 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.972 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:20.972 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795957+0000", 2024-04-24T15:33:20.972 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.1800999999999999e-05 2024-04-24T15:33:20.972 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.972 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.972 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:20.972 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836505+0000", 2024-04-24T15:33:20.973 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040547622999999998 2024-04-24T15:33:20.973 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.973 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.973 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:20.973 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836561+0000", 2024-04-24T15:33:20.973 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6598999999999998e-05 2024-04-24T15:33:20.973 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.973 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.973 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:20.973 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836601+0000", 2024-04-24T15:33:20.973 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9907000000000003e-05 2024-04-24T15:33:20.973 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.973 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:20.974 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.974 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.974 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.974 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23639 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15753216~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:20.974 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.796071+0000", 2024-04-24T15:33:20.974 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 46.113629019999998, 2024-04-24T15:33:20.974 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040648199000000003, 2024-04-24T15:33:20.974 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:20.974 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:20.974 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:20.974 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:20.974 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:20.974 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23639 2024-04-24T15:33:20.974 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.975 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:20.975 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.975 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:20.975 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796071+0000", 2024-04-24T15:33:20.975 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.975 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.975 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.975 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:20.975 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796071+0000", 2024-04-24T15:33:20.975 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.975 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.975 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.975 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:20.975 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795967+0000", 2024-04-24T15:33:20.976 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998965 2024-04-24T15:33:20.976 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.976 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.976 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:20.976 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796074+0000", 2024-04-24T15:33:20.976 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000107013 2024-04-24T15:33:20.976 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.976 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.976 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:20.976 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796081+0000", 2024-04-24T15:33:20.976 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2590000000000003e-06 2024-04-24T15:33:20.976 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.976 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.976 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:20.976 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796085+0000", 2024-04-24T15:33:20.977 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5269999999999999e-06 2024-04-24T15:33:20.977 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.977 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.977 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:20.977 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836624+0000", 2024-04-24T15:33:20.977 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040539324000000002 2024-04-24T15:33:20.977 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.977 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.977 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:20.977 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836667+0000", 2024-04-24T15:33:20.977 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2562999999999999e-05 2024-04-24T15:33:20.977 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.977 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.977 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:20.977 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836719+0000", 2024-04-24T15:33:20.978 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.2015999999999998e-05 2024-04-24T15:33:20.978 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.978 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:20.978 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.978 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.978 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.978 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23640 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15794176~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:20.978 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.796492+0000", 2024-04-24T15:33:20.978 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 46.113207262000003, 2024-04-24T15:33:20.978 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040420000999999997, 2024-04-24T15:33:20.978 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:20.978 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:20.978 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:20.978 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:20.979 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:20.979 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23640 2024-04-24T15:33:20.979 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.979 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:20.979 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.979 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:20.979 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796492+0000", 2024-04-24T15:33:20.979 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.979 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.979 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.979 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:20.979 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796492+0000", 2024-04-24T15:33:20.979 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.979 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.980 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.980 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:20.980 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796088+0000", 2024-04-24T15:33:20.980 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9995961 2024-04-24T15:33:20.980 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.980 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.980 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:20.980 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796496+0000", 2024-04-24T15:33:20.980 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00040785700000000002 2024-04-24T15:33:20.980 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.980 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.980 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:20.980 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796503+0000", 2024-04-24T15:33:20.980 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1049999999999997e-06 2024-04-24T15:33:20.980 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.981 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.981 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:20.981 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796508+0000", 2024-04-24T15:33:20.981 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.155e-06 2024-04-24T15:33:20.981 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.981 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.981 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:20.981 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836756+0000", 2024-04-24T15:33:20.981 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040248850000000003 2024-04-24T15:33:20.981 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.981 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.981 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:20.981 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836841+0000", 2024-04-24T15:33:20.981 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.4735999999999996e-05 2024-04-24T15:33:20.982 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.982 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.982 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:20.982 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836912+0000", 2024-04-24T15:33:20.982 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1229999999999994e-05 2024-04-24T15:33:20.982 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.982 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:20.982 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.982 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.982 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.982 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23641 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15835136~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:20.982 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.796658+0000", 2024-04-24T15:33:20.982 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 46.113041185999997, 2024-04-24T15:33:20.982 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040420706000000001, 2024-04-24T15:33:20.982 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:20.983 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:20.983 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:20.983 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:20.983 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:20.983 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23641 2024-04-24T15:33:20.983 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.983 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:20.983 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.983 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:20.983 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796658+0000", 2024-04-24T15:33:20.983 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.983 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.983 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.983 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:20.983 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796658+0000", 2024-04-24T15:33:20.984 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.984 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.984 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.984 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:20.984 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796512+0000", 2024-04-24T15:33:20.984 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998531 2024-04-24T15:33:20.984 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.984 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.984 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:20.984 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796669+0000", 2024-04-24T15:33:20.984 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000157577 2024-04-24T15:33:20.984 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.984 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.984 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:20.984 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796677+0000", 2024-04-24T15:33:20.985 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7800000000000001e-06 2024-04-24T15:33:20.985 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.985 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.985 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:20.985 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796681+0000", 2024-04-24T15:33:20.985 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.721e-06 2024-04-24T15:33:20.985 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.985 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.985 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:20.985 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836941+0000", 2024-04-24T15:33:20.985 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040259907999999997 2024-04-24T15:33:20.985 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.985 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.985 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:20.985 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837017+0000", 2024-04-24T15:33:20.985 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.6071999999999999e-05 2024-04-24T15:33:20.986 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.986 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.986 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:20.986 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837079+0000", 2024-04-24T15:33:20.986 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.2358000000000002e-05 2024-04-24T15:33:20.986 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.986 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:20.986 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.986 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.986 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.986 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23645 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15998976~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:20.986 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.797360+0000", 2024-04-24T15:33:20.986 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 46.112339687999999, 2024-04-24T15:33:20.986 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040387645999999999, 2024-04-24T15:33:20.987 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:20.987 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:20.987 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:20.987 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:20.987 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:20.987 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23645 2024-04-24T15:33:20.987 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.987 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:20.987 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.987 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:20.987 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797360+0000", 2024-04-24T15:33:20.987 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.987 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.987 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.987 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:20.988 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797360+0000", 2024-04-24T15:33:20.988 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.988 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.988 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.988 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:20.988 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797288+0000", 2024-04-24T15:33:20.988 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999928 2024-04-24T15:33:20.988 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.988 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.988 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:20.988 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797364+0000", 2024-04-24T15:33:20.988 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.5347999999999998e-05 2024-04-24T15:33:20.988 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.988 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.988 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:20.988 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797372+0000", 2024-04-24T15:33:20.989 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.2409999999999995e-06 2024-04-24T15:33:20.989 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.989 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.989 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:20.989 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797376+0000", 2024-04-24T15:33:20.989 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.515e-06 2024-04-24T15:33:20.989 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.989 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.989 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:20.989 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837586+0000", 2024-04-24T15:33:20.989 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040210047999999998 2024-04-24T15:33:20.989 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.989 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.989 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:20.989 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837672+0000", 2024-04-24T15:33:20.989 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.5997999999999999e-05 2024-04-24T15:33:20.990 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.990 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.990 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:20.990 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837748+0000", 2024-04-24T15:33:20.990 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.5328999999999997e-05 2024-04-24T15:33:20.990 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.990 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:20.990 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.990 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.990 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.990 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23646 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 16039936~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:20.990 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.797498+0000", 2024-04-24T15:33:20.990 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 46.112201587000001, 2024-04-24T15:33:20.990 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040433139, 2024-04-24T15:33:20.991 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:20.991 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:20.991 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:20.991 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:20.991 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:20.991 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23646 2024-04-24T15:33:20.991 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.991 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:20.991 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.991 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:20.991 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797498+0000", 2024-04-24T15:33:20.991 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.991 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.991 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.991 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:20.991 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797498+0000", 2024-04-24T15:33:20.991 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:20.992 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.992 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.992 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:20.992 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797381+0000", 2024-04-24T15:33:20.992 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998832 2024-04-24T15:33:20.992 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.992 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.992 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:20.992 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797501+0000", 2024-04-24T15:33:20.992 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000119759 2024-04-24T15:33:20.992 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.992 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.992 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:20.992 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797510+0000", 2024-04-24T15:33:20.992 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.4770000000000003e-06 2024-04-24T15:33:20.992 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.993 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.993 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:20.993 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797515+0000", 2024-04-24T15:33:20.993 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8330000000000001e-06 2024-04-24T15:33:20.993 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.993 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.993 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:20.993 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837777+0000", 2024-04-24T15:33:20.993 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040262502999999998 2024-04-24T15:33:20.993 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.993 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.993 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:20.993 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837860+0000", 2024-04-24T15:33:20.993 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3002000000000003e-05 2024-04-24T15:33:20.993 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:20.993 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:20.993 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:20.994 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837931+0000", 2024-04-24T15:33:20.994 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1222000000000004e-05 2024-04-24T15:33:20.994 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.994 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:20.994 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.994 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:20.994 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:20.994 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:20.996 DEBUG:teuthology.orchestra.run.smithi151:> CEPH_ARGS='--filestore-merge-threshold 1 --filestore-split-multiple 1' sudo -E ceph-objectstore-tool --no-mon-config --log-file=/var/log/ceph/objectstore_tool.$pid.log --data-path /var/lib/ceph/osd/ceph-6 --journal-path /var/lib/ceph/osd/ceph-6/journal --op apply-layout-settings --pool cache 2024-04-24T15:33:20.997 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-24T15:33:20.997 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:33:20.996+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:33:20.999 INFO:teuthology.orchestra.run.smithi096.stdout:5159 trunc from 0x588be8 to 0xa7f2c30 2024-04-24T15:33:20.999 INFO:teuthology.orchestra.run.smithi096.stdout:5160 writesame 0x7e446e5 thru 0x7e49da8 data_size 0x12(0x56c4 bytes) 2024-04-24T15:33:20.999 INFO:teuthology.orchestra.run.smithi096.stdout:5161 write 0x99f75c5 thru 0x99fe528 (0x6f64 bytes) 2024-04-24T15:33:20.999 INFO:teuthology.orchestra.run.smithi096.stdout:5163 write 0x2da8080 thru 0x2db6817 (0xe798 bytes) 2024-04-24T15:33:20.999 INFO:teuthology.orchestra.run.smithi096.stdout:5164 write 0x3c0b967 thru 0x3c11dcb (0x6465 bytes) 2024-04-24T15:33:20.999 INFO:teuthology.orchestra.run.smithi096.stdout:5165 write 0x89d1898 thru 0x89dcfb7 (0xb720 bytes) 2024-04-24T15:33:20.999 INFO:teuthology.orchestra.run.smithi096.stdout:5166 read 0x68fe369 thru 0x690c848 (0xe4e0 bytes) 2024-04-24T15:33:20.999 INFO:teuthology.orchestra.run.smithi096.stdout:5167 punch from 0x79c63f9 to 0x79d0919, (0xa520 bytes) 2024-04-24T15:33:20.999 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:20.999 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:20.999 INFO:teuthology.orchestra.run.smithi096.stdout:5170 punch from 0x3191bcd to 0x319e124, (0xc557 bytes) 2024-04-24T15:33:20.999 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:20.999 INFO:teuthology.orchestra.run.smithi096.stdout:5172 write 0x877dcdb thru 0x877f662 (0x1988 bytes) 2024-04-24T15:33:20.999 INFO:teuthology.orchestra.run.smithi096.stdout:5173 writesame 0x9697af8 thru 0x96a74ce data_size 0x1(0xf9d7 bytes) 2024-04-24T15:33:21.000 INFO:teuthology.orchestra.run.smithi096.stdout:5174 punch from 0x715eb2f to 0x7169d6d, (0xb23e bytes) 2024-04-24T15:33:21.000 INFO:teuthology.orchestra.run.smithi096.stdout:5176 punch from 0x57c25cf to 0x57c4780, (0x21b1 bytes) 2024-04-24T15:33:21.000 INFO:teuthology.orchestra.run.smithi096.stdout:5177 writesame 0xa3a5606 thru 0xa3b3757 data_size 0x2(0xe152 bytes) 2024-04-24T15:33:21.000 INFO:teuthology.orchestra.run.smithi096.stdout:5179 writesame 0x934b67e thru 0x934c442 data_size 0x19(0xdc5 bytes) 2024-04-24T15:33:21.000 INFO:teuthology.orchestra.run.smithi096.stdout:5180 write 0x19e1953 thru 0x19e6396 (0x4a44 bytes) 2024-04-24T15:33:21.000 INFO:teuthology.orchestra.run.smithi096.stdout:5181 write 0x5110e2b thru 0x511a801 (0x99d7 bytes) 2024-04-24T15:33:21.000 INFO:teuthology.orchestra.run.smithi096.stdout:5182 punch from 0x729525 to 0x736c9f, (0xd77a bytes) 2024-04-24T15:33:21.000 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:21.000 INFO:teuthology.orchestra.run.smithi096.stdout:5184 writesame 0x47dbaf8 thru 0x47e6889 data_size 0xd(0xad92 bytes) 2024-04-24T15:33:21.000 INFO:teuthology.orchestra.run.smithi096.stdout:5185 trunc from 0xa7f2c30 to 0x2204b1b 2024-04-24T15:33:21.017 INFO:teuthology.orchestra.run.smithi096.stdout:5186 read 0xe766d5 thru 0xe7c9c6 (0x62f2 bytes) 2024-04-24T15:33:21.017 INFO:teuthology.orchestra.run.smithi096.stdout:5187 read 0x4e91e thru 0x5d306 (0xe9e9 bytes) 2024-04-24T15:33:21.018 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:21.018 INFO:teuthology.orchestra.run.smithi096.stdout:5189 punch from 0x4293db to 0x431e4d, (0x8a72 bytes) 2024-04-24T15:33:21.018 INFO:teuthology.orchestra.run.smithi096.stdout:5190 read 0x1e64a27 thru 0x1e67d68 (0x3342 bytes) 2024-04-24T15:33:21.018 INFO:teuthology.orchestra.run.smithi096.stdout:5191 read 0x4c6997 thru 0x4d523a (0xe8a4 bytes) 2024-04-24T15:33:21.019 INFO:teuthology.orchestra.run.smithi096.stdout:5192 read 0xa1cb4b thru 0xa2a5a0 (0xda56 bytes) 2024-04-24T15:33:21.019 INFO:teuthology.orchestra.run.smithi096.stdout:5194 read 0x10fe1ae thru 0x110e04d (0xfea0 bytes) 2024-04-24T15:33:21.029 INFO:teuthology.orchestra.run.smithi096.stdout:5197 writesame 0x56c395d thru 0x56d29ec data_size 0x18(0xf090 bytes) 2024-04-24T15:33:21.031 INFO:teuthology.orchestra.run.smithi096.stdout:5198 trunc from 0x56d29ed to 0x7288182 2024-04-24T15:33:21.040 INFO:teuthology.orchestra.run.smithi096.stdout:5199 trunc from 0x7288182 to 0x40c6860 2024-04-24T15:33:21.048 INFO:teuthology.orchestra.run.smithi096.stdout:5200 read 0x1ba4017 thru 0x1badcab (0x9c95 bytes) 2024-04-24T15:33:21.048 INFO:teuthology.orchestra.run.smithi096.stdout:5203 read 0x33b9d5b thru 0x33c969b (0xf941 bytes) 2024-04-24T15:33:21.053 INFO:teuthology.orchestra.run.smithi096.stdout:5204 trunc from 0x40c6860 to 0x84bdcd6 2024-04-24T15:33:21.074 INFO:teuthology.orchestra.run.smithi096.stdout:5205 write 0xe7acd72 thru 0xe7bafa6 (0xe235 bytes) 2024-04-24T15:33:21.077 INFO:teuthology.orchestra.run.smithi096.stdout:5206 punch from 0xce510ed to 0xce55366, (0x4279 bytes) 2024-04-24T15:33:21.081 INFO:teuthology.orchestra.run.smithi096.stdout:5207 trunc from 0xe7bafa7 to 0xaff028b 2024-04-24T15:33:21.093 INFO:teuthology.orchestra.run.smithi096.stdout:5209 read 0x6ca8154 thru 0x6cb55ce (0xd47b bytes) 2024-04-24T15:33:21.093 INFO:teuthology.orchestra.run.smithi096.stdout:5210 write 0x5b356f thru 0x5b90e0 (0x5b72 bytes) 2024-04-24T15:33:21.095 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:21.095 INFO:teuthology.orchestra.run.smithi096.stdout:5213 punch from 0x18e799e to 0x18f06a0, (0x8d02 bytes) 2024-04-24T15:33:21.095 INFO:teuthology.orchestra.run.smithi096.stdout:5214 write 0x5503ad0 thru 0x5506e51 (0x3382 bytes) 2024-04-24T15:33:21.096 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:33:21.097+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:33:21.107 INFO:teuthology.orchestra.run.smithi096.stdout:5215 trunc from 0xaff028b to 0x6348e92 2024-04-24T15:33:21.114 INFO:teuthology.orchestra.run.smithi096.stdout:5216 punch from 0x47cfc00 to 0x47d79cd, (0x7dcd bytes) 2024-04-24T15:33:21.115 INFO:teuthology.orchestra.run.smithi096.stdout:5217 punch from 0x1640aa1 to 0x16482d5, (0x7834 bytes) 2024-04-24T15:33:21.115 INFO:teuthology.orchestra.run.smithi096.stdout:5218 write 0x1b4bf7 thru 0x1b4ede (0x2e8 bytes) 2024-04-24T15:33:21.116 INFO:teuthology.orchestra.run.smithi096.stdout:5219 read 0x11ca0a5 thru 0x11caaf2 (0xa4e bytes) 2024-04-24T15:33:21.116 INFO:teuthology.orchestra.run.smithi096.stdout:5220 write 0x19d6436 thru 0x19e4ff4 (0xebbf bytes) 2024-04-24T15:33:21.118 INFO:teuthology.orchestra.run.smithi096.stdout:5221 writesame 0x310bf62 thru 0x310ce9e data_size 0x1(0xf3d bytes) 2024-04-24T15:33:21.124 INFO:teuthology.orchestra.run.smithi096.stdout:5222 writesame 0x1e7d339 thru 0x1e7dc67 data_size 0x1(0x92f bytes) 2024-04-24T15:33:21.138 INFO:teuthology.orchestra.run.smithi096.stdout:5224 write 0xb9395f8 thru 0xb944ae7 (0xb4f0 bytes) 2024-04-24T15:33:21.143 INFO:teuthology.orchestra.run.smithi096.stdout:5226 read 0x4c74ee8 thru 0x4c78d97 (0x3eb0 bytes) 2024-04-24T15:33:21.146 INFO:teuthology.orchestra.run.smithi096.stdout:5228 trunc from 0xb944ae8 to 0x973c790 2024-04-24T15:33:21.166 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:21.166 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:33:21.166 INFO:teuthology.orchestra.run.smithi151.stdout: "num_ops": 0 2024-04-24T15:33:21.167 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:21.168 INFO:teuthology.orchestra.run.smithi096.stdout:5229 writesame 0xdb03444 thru 0xdb128ad data_size 0x5(0xf46a bytes) 2024-04-24T15:33:21.168 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:21.170 INFO:teuthology.orchestra.run.smithi096.stdout:5232 trunc from 0xdb128ae to 0xbc1c27d 2024-04-24T15:33:21.176 INFO:teuthology.orchestra.run.smithi096.stdout:5233 write 0x985ef28 thru 0x9861972 (0x2a4b bytes) 2024-04-24T15:33:21.177 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-24T15:33:21.178 INFO:teuthology.orchestra.run.smithi096.stdout:5235 read 0x605668e thru 0x6063087 (0xc9fa bytes) 2024-04-24T15:33:21.179 INFO:teuthology.orchestra.run.smithi096.stdout:5236 read 0x376bbd5 thru 0x3779f1a (0xe346 bytes) 2024-04-24T15:33:21.179 INFO:teuthology.orchestra.run.smithi096.stdout:5237 read 0x8cff25a thru 0x8d093e1 (0xa188 bytes) 2024-04-24T15:33:21.180 INFO:teuthology.orchestra.run.smithi096.stdout:5238 punch from 0x520fc9a to 0x521958d, (0x98f3 bytes) 2024-04-24T15:33:21.180 INFO:teuthology.orchestra.run.smithi096.stdout:5239 punch from 0x38b3e23 to 0x38c2cfd, (0xeeda bytes) 2024-04-24T15:33:21.180 INFO:teuthology.orchestra.run.smithi096.stdout:5241 read 0x9bc9baa thru 0x9bd1785 (0x7bdc bytes) 2024-04-24T15:33:21.187 INFO:teuthology.orchestra.run.smithi096.stdout:5242 write 0xd971f43 thru 0xd97f466 (0xd524 bytes) 2024-04-24T15:33:21.196 INFO:teuthology.orchestra.run.smithi096.stdout:5243 trunc from 0xd97f467 to 0x89f1d71 2024-04-24T15:33:21.206 INFO:teuthology.orchestra.run.smithi096.stdout:5244 writesame 0x47ae343 thru 0x47aefb0 data_size 0x2(0xc6e bytes) 2024-04-24T15:33:21.213 INFO:teuthology.orchestra.run.smithi096.stdout:5245 write 0x8e52bdf thru 0x8e59754 (0x6b76 bytes) 2024-04-24T15:33:21.216 INFO:teuthology.orchestra.run.smithi096.stdout:5246 read 0xea6be6 thru 0xeb3ad4 (0xceef bytes) 2024-04-24T15:33:21.216 INFO:teuthology.orchestra.run.smithi096.stdout:5247 read 0x847d8ab thru 0x847eb97 (0x12ed bytes) 2024-04-24T15:33:21.221 INFO:teuthology.orchestra.run.smithi096.stdout:5248 write 0x94ecdf4 thru 0x94ef5d6 (0x27e3 bytes) 2024-04-24T15:33:21.221 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:21.222+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:21.234 INFO:teuthology.orchestra.run.smithi096.stdout:5249 write 0xc763b53 thru 0xc7673ee (0x389c bytes) 2024-04-24T15:33:21.237 INFO:teuthology.orchestra.run.smithi096.stdout:5250 writesame 0x5ed1292 thru 0x5edded5 data_size 0x2(0xcc44 bytes) 2024-04-24T15:33:21.251 INFO:teuthology.orchestra.run.smithi096.stdout:5251 read 0x2d90470 thru 0x2d96e4a (0x69db bytes) 2024-04-24T15:33:21.251 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:21.251 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:21.251 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:21.259 INFO:teuthology.orchestra.run.smithi096.stdout:5260 write 0xe53cb96 thru 0xe5473a0 (0xa80b bytes) 2024-04-24T15:33:21.262 INFO:teuthology.orchestra.run.smithi096.stdout:5261 punch from 0x8600748 to 0x8607106, (0x69be bytes) 2024-04-24T15:33:21.262 INFO:teuthology.orchestra.run.smithi096.stdout:5262 writesame 0x70f0d88 thru 0x70ff67d data_size 0x1(0xe8f6 bytes) 2024-04-24T15:33:21.293 INFO:teuthology.orchestra.run.smithi096.stdout:5263 trunc from 0xe5473a1 to 0x4d92c4e 2024-04-24T15:33:21.297 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:21.299+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:21.304 INFO:teuthology.orchestra.run.smithi096.stdout:5264 punch from 0x1c43c74 to 0x1c4d241, (0x95cd bytes) 2024-04-24T15:33:21.304 INFO:teuthology.orchestra.run.smithi096.stdout:5266 writesame 0x74d8c1 thru 0x758677 data_size 0x1(0xadb7 bytes) 2024-04-24T15:33:21.304 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:21.304 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:33:21.305 INFO:teuthology.orchestra.run.smithi151.stdout: "complaint_time": 30, 2024-04-24T15:33:21.305 INFO:teuthology.orchestra.run.smithi151.stdout: "num_blocked_ops": 0 2024-04-24T15:33:21.305 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:21.315 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-24T15:33:21.337 INFO:teuthology.orchestra.run.smithi096.stdout:5270 trunc from 0x4d92c4e to 0xe9bbc25 2024-04-24T15:33:21.343 INFO:teuthology.orchestra.run.smithi096.stdout:5272 writesame 0x87a0b8c thru 0x87a21c1 data_size 0x1(0x1636 bytes) 2024-04-24T15:33:21.347 INFO:teuthology.orchestra.run.smithi096.stdout:5275 read 0x9ad399a thru 0x9ae2d02 (0xf369 bytes) 2024-04-24T15:33:21.348 INFO:teuthology.orchestra.run.smithi096.stdout:5276 clone 29 order 24 su 131072 sc 15 2024-04-24T15:33:21.399 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:21.400+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:21.413 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-24T15:33:21.493 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:21.493 INFO:teuthology.orchestra.run.smithi151.stdout: "size": 20, 2024-04-24T15:33:21.493 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 600, 2024-04-24T15:33:21.493 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [ 2024-04-24T15:33:21.493 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.494 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18006 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12419072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:21.494 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.207911+0000", 2024-04-24T15:33:21.494 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 47.285460909000001, 2024-04-24T15:33:21.494 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042899496000000002, 2024-04-24T15:33:21.494 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:21.494 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:21.494 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:21.494 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:21.494 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:21.494 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18006 2024-04-24T15:33:21.494 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.494 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:21.494 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.494 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:21.494 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207911+0000", 2024-04-24T15:33:21.495 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:21.495 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.495 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.495 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:21.495 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207911+0000", 2024-04-24T15:33:21.495 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:21.495 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.495 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.495 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:21.495 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207910+0000", 2024-04-24T15:33:21.495 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:21.495 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.495 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.495 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:21.496 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207913+0000", 2024-04-24T15:33:21.496 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.2309999999999999e-06 2024-04-24T15:33:21.496 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.496 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.496 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:21.496 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207918+0000", 2024-04-24T15:33:21.496 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.3399999999999997e-06 2024-04-24T15:33:21.496 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.496 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.496 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:21.496 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207922+0000", 2024-04-24T15:33:21.496 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5439999999999999e-06 2024-04-24T15:33:21.496 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.497 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.497 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:21.497 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250726+0000", 2024-04-24T15:33:21.497 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042804129000000003 2024-04-24T15:33:21.497 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.497 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.497 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:21.497 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250765+0000", 2024-04-24T15:33:21.497 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9554000000000001e-05 2024-04-24T15:33:21.497 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.497 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.497 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:21.497 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250810+0000", 2024-04-24T15:33:21.497 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5027000000000001e-05 2024-04-24T15:33:21.497 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:21.498 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:21.498 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:21.498 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.498 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.498 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18007 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12460032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:21.498 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.207961+0000", 2024-04-24T15:33:21.498 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 47.285411068000002, 2024-04-24T15:33:21.498 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042965470999999998, 2024-04-24T15:33:21.498 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:21.498 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:21.498 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:21.498 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:21.498 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:21.498 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18007 2024-04-24T15:33:21.498 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.499 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:21.499 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.499 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:21.499 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207961+0000", 2024-04-24T15:33:21.499 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:21.499 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.499 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.499 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:21.499 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207961+0000", 2024-04-24T15:33:21.499 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:21.499 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.499 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.499 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:21.499 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207960+0000", 2024-04-24T15:33:21.500 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:21.500 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.500 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.500 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:21.500 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207963+0000", 2024-04-24T15:33:21.500 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.3759999999999999e-06 2024-04-24T15:33:21.500 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.500 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.500 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:21.500 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207970+0000", 2024-04-24T15:33:21.500 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.0489999999999998e-06 2024-04-24T15:33:21.500 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.500 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.500 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:21.501 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207975+0000", 2024-04-24T15:33:21.501 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.1640000000000003e-06 2024-04-24T15:33:21.501 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.501 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.501 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:21.501 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250834+0000", 2024-04-24T15:33:21.501 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042859060999999997 2024-04-24T15:33:21.501 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.501 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.501 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:21.501 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250874+0000", 2024-04-24T15:33:21.501 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0674999999999999e-05 2024-04-24T15:33:21.501 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.501 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.501 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:21.502 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250926+0000", 2024-04-24T15:33:21.502 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.1693000000000001e-05 2024-04-24T15:33:21.502 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:21.502 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:21.502 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:21.502 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.502 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.502 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18008 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12500992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:21.502 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208030+0000", 2024-04-24T15:33:21.502 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 47.285341975000001, 2024-04-24T15:33:21.502 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043003337000000003, 2024-04-24T15:33:21.502 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:21.502 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:21.503 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:21.503 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:21.503 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:21.503 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18008 2024-04-24T15:33:21.503 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.503 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:21.503 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.503 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:21.503 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208030+0000", 2024-04-24T15:33:21.503 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:21.503 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.503 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.503 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:21.504 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208030+0000", 2024-04-24T15:33:21.504 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:21.504 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.504 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.504 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:21.504 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208029+0000", 2024-04-24T15:33:21.504 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:21.504 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.504 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.504 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:21.504 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208032+0000", 2024-04-24T15:33:21.504 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.117e-06 2024-04-24T15:33:21.504 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.504 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.505 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:21.505 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208039+0000", 2024-04-24T15:33:21.505 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.0129999999999996e-06 2024-04-24T15:33:21.505 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.505 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.505 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:21.505 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208043+0000", 2024-04-24T15:33:21.505 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0600000000000001e-06 2024-04-24T15:33:21.505 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.505 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.505 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:21.505 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250948+0000", 2024-04-24T15:33:21.505 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042904577999999999 2024-04-24T15:33:21.505 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.506 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.506 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:21.506 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250989+0000", 2024-04-24T15:33:21.506 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0978e-05 2024-04-24T15:33:21.506 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.506 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.506 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:21.506 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251033+0000", 2024-04-24T15:33:21.506 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4161e-05 2024-04-24T15:33:21.506 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:21.506 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:21.506 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:21.506 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.506 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.507 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18009 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12541952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:21.507 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208093+0000", 2024-04-24T15:33:21.507 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 47.285278228999999, 2024-04-24T15:33:21.507 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043043451000000003, 2024-04-24T15:33:21.507 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:21.507 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:21.507 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:21.507 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:21.507 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:21.507 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18009 2024-04-24T15:33:21.507 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.507 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:21.507 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.507 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:21.507 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208093+0000", 2024-04-24T15:33:21.508 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:21.508 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.508 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.508 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:21.508 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208093+0000", 2024-04-24T15:33:21.508 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:21.508 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.508 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.508 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:21.508 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208093+0000", 2024-04-24T15:33:21.508 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:21.508 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.508 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.508 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:21.508 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208096+0000", 2024-04-24T15:33:21.508 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.439e-06 2024-04-24T15:33:21.509 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.509 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.509 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:21.509 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208103+0000", 2024-04-24T15:33:21.509 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1019999999999999e-06 2024-04-24T15:33:21.509 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.509 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.509 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:21.509 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208107+0000", 2024-04-24T15:33:21.509 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0910000000000003e-06 2024-04-24T15:33:21.509 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.509 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.509 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:21.509 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251054+0000", 2024-04-24T15:33:21.509 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042946826 2024-04-24T15:33:21.510 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.510 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.510 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:21.510 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251092+0000", 2024-04-24T15:33:21.510 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.7941999999999997e-05 2024-04-24T15:33:21.510 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.510 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.510 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:21.510 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251137+0000", 2024-04-24T15:33:21.510 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4638000000000002e-05 2024-04-24T15:33:21.510 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:21.510 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:21.510 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:21.510 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.510 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.511 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18010 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12615680~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:21.511 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208202+0000", 2024-04-24T15:33:21.511 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 47.285169398000001, 2024-04-24T15:33:21.511 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043046549000000003, 2024-04-24T15:33:21.511 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:21.511 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:21.511 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:21.511 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:21.511 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:21.511 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18010 2024-04-24T15:33:21.511 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.511 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:21.511 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.511 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:21.511 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208202+0000", 2024-04-24T15:33:21.512 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:21.512 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.512 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.512 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:21.512 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208202+0000", 2024-04-24T15:33:21.512 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:21.512 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.512 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.512 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:21.512 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208202+0000", 2024-04-24T15:33:21.512 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:21.512 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.512 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.512 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:21.512 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208205+0000", 2024-04-24T15:33:21.512 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.2490000000000001e-06 2024-04-24T15:33:21.513 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.513 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.513 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:21.513 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208215+0000", 2024-04-24T15:33:21.513 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.9070000000000002e-06 2024-04-24T15:33:21.513 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.513 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.513 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:21.513 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208220+0000", 2024-04-24T15:33:21.513 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8849999999999998e-06 2024-04-24T15:33:21.513 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.513 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.513 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:21.513 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251158+0000", 2024-04-24T15:33:21.513 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042938416 2024-04-24T15:33:21.513 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.514 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.514 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:21.514 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251197+0000", 2024-04-24T15:33:21.514 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9341000000000002e-05 2024-04-24T15:33:21.514 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.514 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.514 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:21.514 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251249+0000", 2024-04-24T15:33:21.514 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.1391000000000003e-05 2024-04-24T15:33:21.514 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:21.514 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:21.514 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:21.514 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.514 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.514 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18011 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12656640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:21.515 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208258+0000", 2024-04-24T15:33:21.515 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 47.285113551999999, 2024-04-24T15:33:21.515 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043102708000000003, 2024-04-24T15:33:21.515 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:21.515 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:21.515 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:21.515 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:21.515 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:21.515 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18011 2024-04-24T15:33:21.515 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.515 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:21.515 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.515 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:21.515 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208258+0000", 2024-04-24T15:33:21.516 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:21.516 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.516 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.516 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:21.516 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208258+0000", 2024-04-24T15:33:21.516 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:21.516 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.516 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.516 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:21.516 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208258+0000", 2024-04-24T15:33:21.516 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:21.516 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.516 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.516 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:21.517 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208260+0000", 2024-04-24T15:33:21.517 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7810000000000001e-06 2024-04-24T15:33:21.517 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.517 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.517 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:21.517 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208267+0000", 2024-04-24T15:33:21.517 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.7610000000000002e-06 2024-04-24T15:33:21.517 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.517 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.517 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:21.517 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208271+0000", 2024-04-24T15:33:21.517 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0890000000000002e-06 2024-04-24T15:33:21.517 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.517 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.517 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:21.517 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251271+0000", 2024-04-24T15:33:21.517 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043000159000000003 2024-04-24T15:33:21.518 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.518 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.518 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:21.518 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251315+0000", 2024-04-24T15:33:21.518 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3446999999999999e-05 2024-04-24T15:33:21.518 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.518 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.518 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:21.518 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251361+0000", 2024-04-24T15:33:21.518 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5952000000000003e-05 2024-04-24T15:33:21.518 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:21.518 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:21.518 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:21.518 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.518 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.519 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18012 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12697600~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:21.519 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208289+0000", 2024-04-24T15:33:21.519 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 47.285082508000002, 2024-04-24T15:33:21.519 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043174774999999999, 2024-04-24T15:33:21.519 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:21.519 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:21.519 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:21.519 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:21.519 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:21.519 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18012 2024-04-24T15:33:21.519 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.519 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:21.519 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.519 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:21.519 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208289+0000", 2024-04-24T15:33:21.520 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:21.520 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.520 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.520 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:21.520 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208289+0000", 2024-04-24T15:33:21.520 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:21.520 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.520 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.520 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:21.520 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208289+0000", 2024-04-24T15:33:21.520 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:21.520 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.520 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.520 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:21.520 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208291+0000", 2024-04-24T15:33:21.521 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.542e-06 2024-04-24T15:33:21.521 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.521 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.521 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:21.521 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208298+0000", 2024-04-24T15:33:21.521 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.3899999999999998e-06 2024-04-24T15:33:21.521 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.521 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.521 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:21.521 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208302+0000", 2024-04-24T15:33:21.521 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.019e-06 2024-04-24T15:33:21.521 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.521 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.521 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:21.521 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251382+0000", 2024-04-24T15:33:21.521 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043080790000000001 2024-04-24T15:33:21.521 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.522 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.522 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:21.522 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251421+0000", 2024-04-24T15:33:21.522 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8473000000000002e-05 2024-04-24T15:33:21.522 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.522 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.522 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:21.522 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251464+0000", 2024-04-24T15:33:21.522 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3133999999999997e-05 2024-04-24T15:33:21.522 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:21.522 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:21.522 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:21.522 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.522 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.522 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18013 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12738560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:21.523 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208423+0000", 2024-04-24T15:33:21.523 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 47.284948399999998, 2024-04-24T15:33:21.523 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043145554000000003, 2024-04-24T15:33:21.523 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:21.523 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:21.523 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:21.523 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:21.523 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:21.523 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18013 2024-04-24T15:33:21.523 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.523 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:21.523 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.523 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:21.523 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208423+0000", 2024-04-24T15:33:21.524 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:21.524 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.524 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.524 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:21.524 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208423+0000", 2024-04-24T15:33:21.524 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:21.524 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.524 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.524 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:21.524 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208423+0000", 2024-04-24T15:33:21.524 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:21.524 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.524 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.524 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:21.524 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208426+0000", 2024-04-24T15:33:21.525 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.0539999999999999e-06 2024-04-24T15:33:21.525 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.525 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.525 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:21.525 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208433+0000", 2024-04-24T15:33:21.525 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.43e-06 2024-04-24T15:33:21.525 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.525 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.525 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:21.525 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208437+0000", 2024-04-24T15:33:21.525 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5590000000000001e-06 2024-04-24T15:33:21.525 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.525 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.525 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:21.525 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251485+0000", 2024-04-24T15:33:21.526 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043048259999999998 2024-04-24T15:33:21.526 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.526 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.526 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:21.526 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251525+0000", 2024-04-24T15:33:21.526 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9916999999999997e-05 2024-04-24T15:33:21.526 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.526 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.526 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:21.526 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251569+0000", 2024-04-24T15:33:21.526 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3939000000000001e-05 2024-04-24T15:33:21.526 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:21.526 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:21.526 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:21.527 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.527 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.527 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18015 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12779520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:21.527 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208630+0000", 2024-04-24T15:33:21.527 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 47.284741875000002, 2024-04-24T15:33:21.527 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043047328000000003, 2024-04-24T15:33:21.527 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:21.527 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:21.527 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:21.527 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:21.527 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:21.527 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18015 2024-04-24T15:33:21.527 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.527 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:21.527 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.527 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:21.528 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208630+0000", 2024-04-24T15:33:21.528 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:21.528 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.528 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.528 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:21.528 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208630+0000", 2024-04-24T15:33:21.528 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:21.528 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.528 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.528 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:21.528 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208629+0000", 2024-04-24T15:33:21.528 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999999 2024-04-24T15:33:21.528 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.528 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.528 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:21.528 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208634+0000", 2024-04-24T15:33:21.529 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8960000000000002e-06 2024-04-24T15:33:21.529 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.529 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.529 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:21.529 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208645+0000", 2024-04-24T15:33:21.529 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0939e-05 2024-04-24T15:33:21.529 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.529 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.529 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:21.529 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208651+0000", 2024-04-24T15:33:21.529 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.896e-06 2024-04-24T15:33:21.529 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.529 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.529 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:21.529 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251591+0000", 2024-04-24T15:33:21.530 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042939862000000002 2024-04-24T15:33:21.530 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.530 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.530 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:21.530 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251631+0000", 2024-04-24T15:33:21.530 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0216000000000003e-05 2024-04-24T15:33:21.530 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.530 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.530 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:21.530 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251677+0000", 2024-04-24T15:33:21.530 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6390999999999997e-05 2024-04-24T15:33:21.530 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:21.530 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:21.530 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:21.531 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.531 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.531 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18016 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12820480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:21.531 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208660+0000", 2024-04-24T15:33:21.531 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 47.284712085999999, 2024-04-24T15:33:21.531 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043127510000000001, 2024-04-24T15:33:21.531 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:21.531 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:21.531 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:21.531 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:21.531 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:21.531 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18016 2024-04-24T15:33:21.531 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.531 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:21.532 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.532 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:21.532 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208660+0000", 2024-04-24T15:33:21.532 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:21.532 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.532 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.532 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:21.532 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208660+0000", 2024-04-24T15:33:21.532 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:21.532 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.532 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.532 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:21.532 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208659+0000", 2024-04-24T15:33:21.532 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:21.532 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.533 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.533 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:21.533 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208662+0000", 2024-04-24T15:33:21.533 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.0979999999999998e-06 2024-04-24T15:33:21.533 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.533 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.533 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:21.533 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208669+0000", 2024-04-24T15:33:21.533 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.4470000000000003e-06 2024-04-24T15:33:21.533 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.533 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.533 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:21.533 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208673+0000", 2024-04-24T15:33:21.533 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2810000000000002e-06 2024-04-24T15:33:21.534 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.534 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.534 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:21.534 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251699+0000", 2024-04-24T15:33:21.534 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043025728999999999 2024-04-24T15:33:21.534 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.534 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.534 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:21.534 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251743+0000", 2024-04-24T15:33:21.534 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4469999999999999e-05 2024-04-24T15:33:21.534 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.534 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.534 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:21.535 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251787+0000", 2024-04-24T15:33:21.535 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3949999999999998e-05 2024-04-24T15:33:21.535 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:21.535 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:21.535 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:21.535 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.535 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.535 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18017 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12861440~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:21.535 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208721+0000", 2024-04-24T15:33:21.535 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 47.284650638000002, 2024-04-24T15:33:21.535 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043167749999999998, 2024-04-24T15:33:21.535 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:21.535 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:21.535 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:21.536 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:21.536 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:21.536 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18017 2024-04-24T15:33:21.536 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.536 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:21.536 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.536 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:21.536 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208721+0000", 2024-04-24T15:33:21.536 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:21.536 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.536 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.536 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:21.536 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208721+0000", 2024-04-24T15:33:21.536 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:21.537 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.537 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.537 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:21.537 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208720+0000", 2024-04-24T15:33:21.537 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:21.537 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.537 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.537 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:21.537 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208724+0000", 2024-04-24T15:33:21.537 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.3919999999999999e-06 2024-04-24T15:33:21.537 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.537 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.537 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:21.537 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208729+0000", 2024-04-24T15:33:21.538 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.3480000000000003e-06 2024-04-24T15:33:21.538 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.538 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.538 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:21.538 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208732+0000", 2024-04-24T15:33:21.538 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.9529999999999999e-06 2024-04-24T15:33:21.538 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.538 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.538 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:21.538 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251807+0000", 2024-04-24T15:33:21.538 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043074492999999998 2024-04-24T15:33:21.538 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.538 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.539 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:21.539 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251846+0000", 2024-04-24T15:33:21.539 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8958999999999997e-05 2024-04-24T15:33:21.539 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.539 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.539 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:21.539 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251889+0000", 2024-04-24T15:33:21.539 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3253000000000001e-05 2024-04-24T15:33:21.539 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:21.539 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:21.539 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:21.539 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.539 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.539 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18018 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12902400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:21.540 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208752+0000", 2024-04-24T15:33:21.540 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 47.284620025000002, 2024-04-24T15:33:21.540 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043241012000000002, 2024-04-24T15:33:21.540 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:21.540 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:21.540 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:21.540 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:21.540 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:21.540 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18018 2024-04-24T15:33:21.540 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.540 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:21.540 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.540 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:21.540 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208752+0000", 2024-04-24T15:33:21.541 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:21.541 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.541 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.541 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:21.541 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208752+0000", 2024-04-24T15:33:21.541 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:21.541 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.541 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.541 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:21.541 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208751+0000", 2024-04-24T15:33:21.541 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:21.541 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.541 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.541 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:21.541 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208753+0000", 2024-04-24T15:33:21.542 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.894e-06 2024-04-24T15:33:21.542 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.542 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.542 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:21.542 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208757+0000", 2024-04-24T15:33:21.542 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.856e-06 2024-04-24T15:33:21.542 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.542 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.542 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:21.542 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208760+0000", 2024-04-24T15:33:21.542 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.6749999999999998e-06 2024-04-24T15:33:21.542 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.542 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.542 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:21.543 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251909+0000", 2024-04-24T15:33:21.543 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043149145999999999 2024-04-24T15:33:21.543 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.543 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.543 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:21.543 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251948+0000", 2024-04-24T15:33:21.543 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9159000000000002e-05 2024-04-24T15:33:21.543 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.543 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.543 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:21.543 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251993+0000", 2024-04-24T15:33:21.543 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4638999999999997e-05 2024-04-24T15:33:21.543 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:21.543 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:21.543 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:21.544 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.544 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.544 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18019 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12943360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:21.544 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208938+0000", 2024-04-24T15:33:21.544 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 47.284433733999997, 2024-04-24T15:33:21.544 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043162272000000002, 2024-04-24T15:33:21.544 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:21.544 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:21.544 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:21.544 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:21.544 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:21.544 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18019 2024-04-24T15:33:21.544 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.544 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:21.545 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.545 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:21.545 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208938+0000", 2024-04-24T15:33:21.545 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:21.545 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.545 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.545 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:21.545 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208938+0000", 2024-04-24T15:33:21.545 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:21.545 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.545 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.545 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:21.546 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208762+0000", 2024-04-24T15:33:21.546 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999824 2024-04-24T15:33:21.546 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.546 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.546 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:21.546 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208942+0000", 2024-04-24T15:33:21.546 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000180008 2024-04-24T15:33:21.546 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.546 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.546 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:21.546 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208950+0000", 2024-04-24T15:33:21.546 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.1550000000000007e-06 2024-04-24T15:33:21.546 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.546 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.547 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:21.547 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208956+0000", 2024-04-24T15:33:21.547 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.7910000000000001e-06 2024-04-24T15:33:21.547 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.547 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.547 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:21.547 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252014+0000", 2024-04-24T15:33:21.547 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043058427000000003 2024-04-24T15:33:21.547 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.547 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.547 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:21.547 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252054+0000", 2024-04-24T15:33:21.547 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0071999999999999e-05 2024-04-24T15:33:21.547 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.548 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.548 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:21.548 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252100+0000", 2024-04-24T15:33:21.548 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5917000000000001e-05 2024-04-24T15:33:21.548 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:21.548 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:21.548 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:21.548 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.548 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.548 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18020 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12984320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:21.548 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.209092+0000", 2024-04-24T15:33:21.548 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 47.284279736000002, 2024-04-24T15:33:21.548 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043111295000000001, 2024-04-24T15:33:21.548 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:21.549 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:21.549 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:21.549 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:21.549 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:21.549 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18020 2024-04-24T15:33:21.549 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.549 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:21.549 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.549 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:21.549 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209092+0000", 2024-04-24T15:33:21.549 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:21.549 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.549 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.549 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:21.549 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209092+0000", 2024-04-24T15:33:21.549 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:21.550 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.550 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.550 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:21.550 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208964+0000", 2024-04-24T15:33:21.550 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998722 2024-04-24T15:33:21.550 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.550 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.550 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:21.550 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209095+0000", 2024-04-24T15:33:21.550 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000130648 2024-04-24T15:33:21.550 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.550 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.550 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:21.550 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209103+0000", 2024-04-24T15:33:21.551 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0320000000000003e-06 2024-04-24T15:33:21.551 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.551 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.551 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:21.551 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209108+0000", 2024-04-24T15:33:21.551 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8690000000000003e-06 2024-04-24T15:33:21.551 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.551 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.551 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:21.551 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252121+0000", 2024-04-24T15:33:21.551 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043013323999999999 2024-04-24T15:33:21.551 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.551 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.552 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:21.552 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252160+0000", 2024-04-24T15:33:21.552 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8376e-05 2024-04-24T15:33:21.552 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.552 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.552 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:21.552 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252203+0000", 2024-04-24T15:33:21.552 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.371e-05 2024-04-24T15:33:21.552 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:21.552 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:21.552 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:21.552 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.552 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.552 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18021 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 13025280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:21.552 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.209132+0000", 2024-04-24T15:33:21.553 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 47.284239378000002, 2024-04-24T15:33:21.553 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043183903000000003, 2024-04-24T15:33:21.553 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:21.553 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:21.553 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:21.553 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:21.553 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:21.553 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18021 2024-04-24T15:33:21.553 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.553 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:21.553 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.553 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:21.553 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209132+0000", 2024-04-24T15:33:21.553 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:21.554 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.554 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.554 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:21.554 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209132+0000", 2024-04-24T15:33:21.554 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:21.554 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.554 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.554 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:21.554 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209132+0000", 2024-04-24T15:33:21.554 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:21.554 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.554 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.554 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:21.555 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209134+0000", 2024-04-24T15:33:21.555 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.891e-06 2024-04-24T15:33:21.555 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.555 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.555 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:21.555 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209138+0000", 2024-04-24T15:33:21.555 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0659999999999997e-06 2024-04-24T15:33:21.555 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.555 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.555 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:21.555 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209140+0000", 2024-04-24T15:33:21.555 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.5339999999999998e-06 2024-04-24T15:33:21.555 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.555 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.556 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:21.556 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252232+0000", 2024-04-24T15:33:21.556 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043091142999999998 2024-04-24T15:33:21.556 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.556 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.556 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:21.556 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252270+0000", 2024-04-24T15:33:21.556 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8872000000000003e-05 2024-04-24T15:33:21.556 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.556 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.556 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:21.556 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252316+0000", 2024-04-24T15:33:21.556 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5788000000000003e-05 2024-04-24T15:33:21.556 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:21.556 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:21.557 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:21.557 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.557 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.557 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18022 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 13066240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:21.557 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.209411+0000", 2024-04-24T15:33:21.557 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 47.283960829999998, 2024-04-24T15:33:21.557 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043016616000000001, 2024-04-24T15:33:21.557 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:21.557 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:21.557 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:21.557 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:21.557 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:21.557 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18022 2024-04-24T15:33:21.557 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.557 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:21.558 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.558 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:21.558 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209411+0000", 2024-04-24T15:33:21.558 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:21.558 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.558 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.558 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:21.558 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209411+0000", 2024-04-24T15:33:21.558 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:21.558 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.558 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.558 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:21.558 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209143+0000", 2024-04-24T15:33:21.559 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999732 2024-04-24T15:33:21.559 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.559 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.559 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:21.559 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209415+0000", 2024-04-24T15:33:21.559 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000271873 2024-04-24T15:33:21.559 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.559 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.559 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:21.559 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209423+0000", 2024-04-24T15:33:21.559 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3080000000000008e-06 2024-04-24T15:33:21.559 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.559 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.559 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:21.560 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209428+0000", 2024-04-24T15:33:21.560 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.5040000000000002e-06 2024-04-24T15:33:21.560 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.560 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.560 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:21.560 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252339+0000", 2024-04-24T15:33:21.560 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042910419999999998 2024-04-24T15:33:21.560 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.560 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.560 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:21.560 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252380+0000", 2024-04-24T15:33:21.560 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0991000000000001e-05 2024-04-24T15:33:21.560 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.560 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.561 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:21.561 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252427+0000", 2024-04-24T15:33:21.561 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.7647e-05 2024-04-24T15:33:21.561 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:21.561 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:21.561 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:21.561 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.561 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.561 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18023 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 13107200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:21.561 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.209594+0000", 2024-04-24T15:33:21.561 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 47.283777317000002, 2024-04-24T15:33:21.561 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042938543000000003, 2024-04-24T15:33:21.561 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:21.561 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:21.561 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:21.561 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:21.562 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:21.562 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18023 2024-04-24T15:33:21.562 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.562 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:21.562 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.562 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:21.562 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209594+0000", 2024-04-24T15:33:21.562 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:21.562 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.562 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.562 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:21.562 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209594+0000", 2024-04-24T15:33:21.562 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:21.562 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.562 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.563 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:21.563 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209440+0000", 2024-04-24T15:33:21.563 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999846 2024-04-24T15:33:21.563 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.563 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.563 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:21.563 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209598+0000", 2024-04-24T15:33:21.563 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00015731100000000001 2024-04-24T15:33:21.563 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.563 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.563 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:21.563 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209604+0000", 2024-04-24T15:33:21.563 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.2910000000000004e-06 2024-04-24T15:33:21.563 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.563 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.563 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:21.564 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209607+0000", 2024-04-24T15:33:21.564 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5389999999999999e-06 2024-04-24T15:33:21.564 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.564 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.564 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:21.564 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252449+0000", 2024-04-24T15:33:21.564 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042841833000000003 2024-04-24T15:33:21.564 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.564 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.564 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:21.564 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252489+0000", 2024-04-24T15:33:21.564 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9898000000000003e-05 2024-04-24T15:33:21.564 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.564 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.564 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:21.565 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252533+0000", 2024-04-24T15:33:21.565 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3702999999999998e-05 2024-04-24T15:33:21.565 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:21.565 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:21.565 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:21.565 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.565 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.565 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18025 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 13189120~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:21.565 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.209832+0000", 2024-04-24T15:33:21.565 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 47.283539816000001, 2024-04-24T15:33:21.565 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042913556999999998, 2024-04-24T15:33:21.565 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:21.565 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:21.565 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:21.566 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:21.566 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:21.566 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18025 2024-04-24T15:33:21.566 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.566 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:21.566 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.566 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:21.566 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209832+0000", 2024-04-24T15:33:21.566 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:21.566 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.566 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.566 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:21.566 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209832+0000", 2024-04-24T15:33:21.566 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:21.567 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.567 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.567 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:21.567 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209774+0000", 2024-04-24T15:33:21.567 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999418 2024-04-24T15:33:21.567 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.567 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.567 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:21.567 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209835+0000", 2024-04-24T15:33:21.567 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.1377000000000005e-05 2024-04-24T15:33:21.567 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.567 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.567 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:21.567 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209843+0000", 2024-04-24T15:33:21.567 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.2330000000000006e-06 2024-04-24T15:33:21.567 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.568 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.568 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:21.568 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209848+0000", 2024-04-24T15:33:21.568 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6149999999999999e-06 2024-04-24T15:33:21.568 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.568 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.568 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:21.568 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252658+0000", 2024-04-24T15:33:21.568 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042810002 2024-04-24T15:33:21.568 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.568 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.568 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:21.568 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252697+0000", 2024-04-24T15:33:21.568 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8757999999999997e-05 2024-04-24T15:33:21.568 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.568 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.569 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:21.569 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252745+0000", 2024-04-24T15:33:21.569 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.863e-05 2024-04-24T15:33:21.569 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:21.569 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:21.569 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:21.569 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.569 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.569 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:24749 4.8 4:137f5701:::rbd_header.114ae15b9a20:head [watch reconnect cookie 140672595753696 gen 11] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-04-24T15:33:21.569 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:37.628703+0000", 2024-04-24T15:33:21.569 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 43.864668197, 2024-04-24T15:33:21.569 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0102918700000001, 2024-04-24T15:33:21.569 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:21.569 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:21.570 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:21.570 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:21.570 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:21.570 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 24749 2024-04-24T15:33:21.570 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.570 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:21.570 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.570 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:21.570 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628703+0000", 2024-04-24T15:33:21.570 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:21.570 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.570 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.570 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:21.571 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628703+0000", 2024-04-24T15:33:21.571 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:21.571 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.571 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.571 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:21.571 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628703+0000", 2024-04-24T15:33:21.571 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999999 2024-04-24T15:33:21.571 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.571 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.571 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:21.571 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628708+0000", 2024-04-24T15:33:21.571 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.4589999999999997e-06 2024-04-24T15:33:21.571 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.572 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.572 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:21.572 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628721+0000", 2024-04-24T15:33:21.572 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.2601e-05 2024-04-24T15:33:21.572 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.572 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.572 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:21.572 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628728+0000", 2024-04-24T15:33:21.572 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.9059999999999996e-06 2024-04-24T15:33:21.572 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.572 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.572 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:21.572 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628789+0000", 2024-04-24T15:33:21.572 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.1672000000000002e-05 2024-04-24T15:33:21.573 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.573 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.573 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting_for_map not empty", 2024-04-24T15:33:21.573 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628797+0000", 2024-04-24T15:33:21.573 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.8189999999999992e-06 2024-04-24T15:33:21.573 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.573 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.573 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:21.573 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.631255+0000", 2024-04-24T15:33:21.573 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0024577610000000001 2024-04-24T15:33:21.573 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.573 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.573 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for peered", 2024-04-24T15:33:21.573 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.631263+0000", 2024-04-24T15:33:21.574 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.4109999999999996e-06 2024-04-24T15:33:21.574 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.574 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.574 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:21.574 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:38.637560+0000", 2024-04-24T15:33:21.574 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0062970339999999 2024-04-24T15:33:21.574 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.574 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.574 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:21.574 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:38.637747+0000", 2024-04-24T15:33:21.574 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00018693400000000001 2024-04-24T15:33:21.574 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.575 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.575 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:21.575 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:38.638995+0000", 2024-04-24T15:33:21.575 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0012481300000000001 2024-04-24T15:33:21.575 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:21.575 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:21.575 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:21.575 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.575 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.575 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:24750 4.8 4:137f5701:::rbd_header.114ae15b9a20:head [notify cookie 140672594434416] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-04-24T15:33:21.575 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:37.628736+0000", 2024-04-24T15:33:21.575 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 43.864635937000003, 2024-04-24T15:33:21.575 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0096094309999999, 2024-04-24T15:33:21.575 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:21.575 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:21.576 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:21.576 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:21.576 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:21.576 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 24750 2024-04-24T15:33:21.576 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.576 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:21.576 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.576 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:21.576 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628736+0000", 2024-04-24T15:33:21.576 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:21.576 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.576 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.576 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:21.576 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628736+0000", 2024-04-24T15:33:21.577 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:21.577 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.577 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.577 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:21.577 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628735+0000", 2024-04-24T15:33:21.577 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:21.577 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.577 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.577 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:21.577 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628739+0000", 2024-04-24T15:33:21.578 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5109999999999999e-06 2024-04-24T15:33:21.578 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.578 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.578 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:21.578 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628745+0000", 2024-04-24T15:33:21.578 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.8300000000000001e-06 2024-04-24T15:33:21.578 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.578 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.578 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:21.578 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628749+0000", 2024-04-24T15:33:21.578 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.7110000000000003e-06 2024-04-24T15:33:21.578 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.578 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.579 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:21.579 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628826+0000", 2024-04-24T15:33:21.579 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.6678e-05 2024-04-24T15:33:21.579 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.579 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.579 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting_for_map not empty", 2024-04-24T15:33:21.579 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628834+0000", 2024-04-24T15:33:21.579 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.6190000000000002e-06 2024-04-24T15:33:21.579 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.579 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.579 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:21.579 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.631305+0000", 2024-04-24T15:33:21.579 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0024714400000000001 2024-04-24T15:33:21.579 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.580 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.580 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for peered", 2024-04-24T15:33:21.580 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.631313+0000", 2024-04-24T15:33:21.580 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.6529999999999994e-06 2024-04-24T15:33:21.580 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.580 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.580 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:21.580 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:38.638068+0000", 2024-04-24T15:33:21.580 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0067548369999999 2024-04-24T15:33:21.580 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.580 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.580 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:21.580 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:38.638158+0000", 2024-04-24T15:33:21.580 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.0079000000000003e-05 2024-04-24T15:33:21.581 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:21.581 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:21.581 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:21.581 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:38.638345+0000", 2024-04-24T15:33:21.581 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00018757100000000001 2024-04-24T15:33:21.581 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:21.581 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:21.581 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:21.581 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:21.581 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:21.581 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:21.583 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-04-24T15:33:21.583 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-04-24T15:33:21.583 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:21.583+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:21.584 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-24T15:33:21.587 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-04-24T15:33:21.608 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-04-24T15:33:21.626 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-04-24T15:33:21.645 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-04-24T15:33:21.664 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-04-24T15:33:21.664 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T15:33:21.664 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T15:33:21.664 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T15:33:21.664 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T15:33:21.664 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T15:33:21.665 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T15:33:21.665 INFO:teuthology.orchestra.run.smithi096.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-04-24T15:33:21.665 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T15:33:21.667 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-24T15:33:21.668 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-24T15:33:21.683 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:21.684+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:21.710 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:21.710 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:21.720 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-24T15:33:21.784 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:21.784+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:21.843 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:21.843 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:21.853 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-24T15:33:21.885 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:21.885+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:21.945 INFO:teuthology.orchestra.run.smithi096.stdout:flattening image image_client.0-clone28 2024-04-24T15:33:21.986 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:21.987+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:21.988 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:21.988 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:22.000 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-24T15:33:22.086 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:33:22.087+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:33:22.122 INFO:teuthology.orchestra.run.smithi151.stderr:Nothing to do for non-filestore backend 2024-04-24T15:33:22.127 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #27, image image_client.0-clone27 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent28 2024-04-24T15:33:22.177 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:22.177 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:22.187 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:22.189+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:22.188 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-24T15:33:22.200 INFO:teuthology.orchestra.run.smithi096.stdout:5277 write 0x56a8e45 thru 0x56b343e (0xa5fa bytes) 2024-04-24T15:33:22.214 INFO:teuthology.orchestra.run.smithi096.stdout:5279 read 0x5f1dc16 thru 0x5f27cdf (0xa0ca bytes) 2024-04-24T15:33:22.214 INFO:teuthology.orchestra.run.smithi096.stdout:5281 punch from 0x64f79cc to 0x6505912, (0xdf46 bytes) 2024-04-24T15:33:22.214 INFO:teuthology.orchestra.run.smithi096.stdout:5283 punch from 0x35c9bc4 to 0x35d675d, (0xcb99 bytes) 2024-04-24T15:33:22.223 INFO:teuthology.orchestra.run.smithi096.stdout:5288 trunc from 0xe9bbc25 to 0x6e18d15 2024-04-24T15:33:22.274 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:22.274 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:22.280 INFO:teuthology.orchestra.run.smithi096.stdout:5291 trunc from 0x6e18d15 to 0xc16135b 2024-04-24T15:33:22.285 INFO:teuthology.orchestra.run.smithi096.stdout:5292 writesame 0x96705da thru 0x967fd0a data_size 0x1(0xf731 bytes) 2024-04-24T15:33:22.288 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:33:22.290+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:33:22.327 INFO:teuthology.orchestra.run.smithi096.stdout:5293 trunc from 0xc16135b to 0x2b991de 2024-04-24T15:33:22.336 INFO:teuthology.orchestra.run.smithi096.stdout:5294 read 0x398b61 thru 0x3a6838 (0xdcd8 bytes) 2024-04-24T15:33:22.337 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:22.337 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:22.351 INFO:teuthology.orchestra.run.smithi096.stdout:5297 writesame 0xa5e9b12 thru 0xa5f6e4c data_size 0x4b(0xd33b bytes) 2024-04-24T15:33:22.352 INFO:teuthology.orchestra.run.smithi096.stdout:5299 write 0x29f0a74 thru 0x29ffd88 (0xf315 bytes) 2024-04-24T15:33:22.352 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:22.352 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:22.354 INFO:teuthology.orchestra.run.smithi096.stdout:5302 write 0x1961db1 thru 0x196a749 (0x8999 bytes) 2024-04-24T15:33:22.355 INFO:teuthology.orchestra.run.smithi096.stdout:5304 write 0x59ab69a thru 0x59b8d08 (0xd66f bytes) 2024-04-24T15:33:22.357 INFO:teuthology.orchestra.run.smithi096.stdout:5305 writesame 0x46b8604 thru 0x46bf41a data_size 0x1(0x6e17 bytes) 2024-04-24T15:33:22.363 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-24T15:33:22.368 INFO:teuthology.orchestra.run.smithi096.stdout:5307 punch from 0x6f5b693 to 0x6f6154d, (0x5eba bytes) 2024-04-24T15:33:22.369 INFO:teuthology.orchestra.run.smithi096.stdout:5309 writesame 0x6822c06 thru 0x6828c84 data_size 0x1(0x607f bytes) 2024-04-24T15:33:22.377 INFO:teuthology.orchestra.run.smithi096.stdout:5310 read 0x2402c79 thru 0x240a369 (0x76f1 bytes) 2024-04-24T15:33:22.377 INFO:teuthology.orchestra.run.smithi096.stdout:5311 write 0x299f13f thru 0x29a466d (0x552f bytes) 2024-04-24T15:33:22.379 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:22.379 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:22.379 INFO:teuthology.orchestra.run.smithi096.stdout:5315 write 0x9d2a801 thru 0x9d3a0a0 (0xf8a0 bytes) 2024-04-24T15:33:22.381 INFO:teuthology.orchestra.run.smithi096.stdout:5319 write 0x6347e3 thru 0x638263 (0x3a81 bytes) 2024-04-24T15:33:22.383 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:22.383 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:22.383 INFO:teuthology.orchestra.run.smithi096.stdout:5322 punch from 0x4aa0967 to 0x4aa5a54, (0x50ed bytes) 2024-04-24T15:33:22.383 INFO:teuthology.orchestra.run.smithi096.stdout:5324 write 0x271ccfb thru 0x27206fd (0x3a03 bytes) 2024-04-24T15:33:22.389 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:22.390+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:22.394 INFO:teuthology.orchestra.run.smithi096.stdout:5326 write 0xe5ce8c1 thru 0xe5d1c98 (0x33d8 bytes) 2024-04-24T15:33:22.395 INFO:teuthology.orchestra.run.smithi096.stdout:5328 write 0x831112e thru 0x831d68d (0xc560 bytes) 2024-04-24T15:33:22.397 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:22.405 INFO:teuthology.orchestra.run.smithi096.stdout:5330 trunc from 0xe5d1c99 to 0x7570ecf 2024-04-24T15:33:22.425 INFO:teuthology.orchestra.run.smithi096.stdout:5331 writesame 0xbfa0887 thru 0xbfaf301 data_size 0x6b(0xea7b bytes) 2024-04-24T15:33:22.426 INFO:teuthology.orchestra.run.smithi096.stdout:5332 read 0x9364a69 thru 0x936c3a9 (0x7941 bytes) 2024-04-24T15:33:22.427 INFO:teuthology.orchestra.run.smithi096.stdout:5333 trunc from 0xbfaf302 to 0xbf059b1 2024-04-24T15:33:22.435 INFO:teuthology.orchestra.run.smithi096.stdout:5334 read 0x7985f89 thru 0x798e648 (0x86c0 bytes) 2024-04-24T15:33:22.443 INFO:teuthology.orchestra.run.smithi096.stdout:5335 write 0xd4534c0 thru 0xd459ea6 (0x69e7 bytes) 2024-04-24T15:33:22.450 INFO:teuthology.orchestra.run.smithi096.stdout:5336 writesame 0xe539b73 thru 0xe544c88 data_size 0x13(0xb116 bytes) 2024-04-24T15:33:22.453 INFO:teuthology.orchestra.run.smithi096.stdout:5338 punch from 0x800a018 to 0x8019e14, (0xfdfc bytes) 2024-04-24T15:33:22.453 INFO:teuthology.orchestra.run.smithi096.stdout:5340 write 0x12cc92e thru 0x12d1e08 (0x54db bytes) 2024-04-24T15:33:22.454 INFO:teuthology.orchestra.run.smithi096.stdout:5341 punch from 0xc3b609a to 0xc3beb7e, (0x8ae4 bytes) 2024-04-24T15:33:22.454 INFO:teuthology.orchestra.run.smithi096.stdout:5342 read 0x52603dd thru 0x5267970 (0x7594 bytes) 2024-04-24T15:33:22.454 INFO:teuthology.orchestra.run.smithi096.stdout:5343 write 0xd3afa79 thru 0xd3b547e (0x5a06 bytes) 2024-04-24T15:33:22.456 INFO:teuthology.orchestra.run.smithi096.stdout:5346 read 0x2a14d9d thru 0x2a1d1a6 (0x840a bytes) 2024-04-24T15:33:22.456 INFO:teuthology.orchestra.run.smithi096.stdout:5347 write 0x403e39c thru 0x403fc3b (0x18a0 bytes) 2024-04-24T15:33:22.457 INFO:teuthology.orchestra.run.smithi096.stdout:5348 read 0x54eb2d0 thru 0x54f8769 (0xd49a bytes) 2024-04-24T15:33:22.458 INFO:teuthology.orchestra.run.smithi096.stdout:5349 read 0x517add3 thru 0x517b920 (0xb4e bytes) 2024-04-24T15:33:22.458 INFO:teuthology.orchestra.run.smithi096.stdout:5351 write 0x68bd82f thru 0x68cc4e9 (0xecbb bytes) 2024-04-24T15:33:22.460 INFO:teuthology.orchestra.run.smithi096.stdout:5352 write 0x9802b8 thru 0x98e747 (0xe490 bytes) 2024-04-24T15:33:22.462 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:22.462 INFO:teuthology.orchestra.run.smithi096.stdout:5354 write 0xc33d86f thru 0xc347667 (0x9df9 bytes) 2024-04-24T15:33:22.464 INFO:teuthology.orchestra.run.smithi096.stdout:5355 read 0x1e5f99c thru 0x1e5fd34 (0x399 bytes) 2024-04-24T15:33:22.464 INFO:teuthology.orchestra.run.smithi096.stdout:5356 punch from 0xc991360 to 0xc99ed41, (0xd9e1 bytes) 2024-04-24T15:33:22.465 INFO:teuthology.orchestra.run.smithi096.stdout:5357 write 0xd77b552 thru 0xd780352 (0x4e01 bytes) 2024-04-24T15:33:22.467 INFO:teuthology.orchestra.run.smithi096.stdout:5358 writesame 0xc72a6d0 thru 0xc7365c3 data_size 0x79(0xbef4 bytes) 2024-04-24T15:33:22.469 INFO:teuthology.orchestra.run.smithi096.stdout:5359 write 0x25a249b thru 0x25b00ee (0xdc54 bytes) 2024-04-24T15:33:22.472 INFO:teuthology.orchestra.run.smithi096.stdout:5360 punch from 0x65695cb to 0x656bb03, (0x2538 bytes) 2024-04-24T15:33:22.472 INFO:teuthology.orchestra.run.smithi096.stdout:5361 punch from 0xa6fe2de to 0xa707a4b, (0x976d bytes) 2024-04-24T15:33:22.472 INFO:teuthology.orchestra.run.smithi096.stdout:5362 write 0x45ca4c6 thru 0x45cde3e (0x3979 bytes) 2024-04-24T15:33:22.473 INFO:teuthology.orchestra.run.smithi096.stdout:5364 writesame 0x30394fe thru 0x303f426 data_size 0x1(0x5f29 bytes) 2024-04-24T15:33:22.484 INFO:teuthology.orchestra.run.smithi096.stdout:5365 write 0x96525d thru 0x96ed52 (0x9af6 bytes) 2024-04-24T15:33:22.486 INFO:teuthology.orchestra.run.smithi096.stdout:5366 write 0xc8c1fd1 thru 0xc8d1757 (0xf787 bytes) 2024-04-24T15:33:22.490 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:22.491+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:22.494 INFO:teuthology.orchestra.run.smithi096.stdout:5367 trunc from 0xe544c89 to 0x8fc2190 2024-04-24T15:33:22.509 INFO:teuthology.orchestra.run.smithi096.stdout:5369 writesame 0x9cdee57 thru 0x9ce2865 data_size 0x1(0x3a0f bytes) 2024-04-24T15:33:22.513 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:22.513 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:22.521 INFO:teuthology.orchestra.run.smithi096.stdout:5370 writesame 0xd6c476f thru 0xd6ca412 data_size 0x79(0x5ca4 bytes) 2024-04-24T15:33:22.522 INFO:teuthology.orchestra.run.smithi096.stdout:5375 write 0x2db4d0d thru 0x2dbae1c (0x6110 bytes) 2024-04-24T15:33:22.523 INFO:teuthology.orchestra.run.smithi096.stdout:5377 write 0x1b499fa thru 0x1b4cc03 (0x320a bytes) 2024-04-24T15:33:22.526 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-24T15:33:22.527 INFO:teuthology.orchestra.run.smithi096.stdout:5380 read 0xc7c1c61 thru 0xc7d0243 (0xe5e3 bytes) 2024-04-24T15:33:22.527 INFO:teuthology.orchestra.run.smithi096.stdout:5381 writesame 0x7dcc94 thru 0x7e6d51 data_size 0x19(0xa0be bytes) 2024-04-24T15:33:22.528 INFO:teuthology.orchestra.run.smithi096.stdout:5382 punch from 0x2eb5aa0 to 0x2ec1c4f, (0xc1af bytes) 2024-04-24T15:33:22.542 INFO:teuthology.orchestra.run.smithi096.stdout:5384 trunc from 0xd6ca413 to 0x1a31c2c 2024-04-24T15:33:22.555 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:22.556 INFO:teuthology.orchestra.run.smithi096.stdout:5390 write 0xc22724 thru 0xc31d06 (0xf5e3 bytes) 2024-04-24T15:33:22.569 INFO:teuthology.orchestra.run.smithi096.stdout:5391 write 0x646cdb7 thru 0x6472c99 (0x5ee3 bytes) 2024-04-24T15:33:22.585 INFO:teuthology.orchestra.run.smithi096.stdout:5392 writesame 0xdf50e85 thru 0xdf599a1 data_size 0x3(0x8b1d bytes) 2024-04-24T15:33:22.586 INFO:teuthology.orchestra.run.smithi096.stdout:5394 write 0xac72cc3 thru 0xac79d71 (0x70af bytes) 2024-04-24T15:33:22.588 INFO:teuthology.orchestra.run.smithi096.stdout:5396 writesame 0x275546 thru 0x2760f3 data_size 0x5(0xbae bytes) 2024-04-24T15:33:22.590 INFO:teuthology.orchestra.run.smithi096.stdout:5398 read 0x3990366 thru 0x39978a1 (0x753c bytes) 2024-04-24T15:33:22.591 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:22.592+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:22.601 INFO:teuthology.orchestra.run.smithi096.stdout:5399 trunc from 0xdf599a2 to 0x547c670 2024-04-24T15:33:22.612 INFO:teuthology.orchestra.run.smithi096.stdout:5400 punch from 0xc44812 to 0xc4bce5, (0x74d3 bytes) 2024-04-24T15:33:22.624 INFO:teuthology.orchestra.run.smithi096.stdout:5401 writesame 0xab6c829 thru 0xab721dd data_size 0x5(0x59b5 bytes) 2024-04-24T15:33:22.638 INFO:teuthology.orchestra.run.smithi096.stdout:5402 trunc from 0xab721de to 0x5ec166 2024-04-24T15:33:22.640 DEBUG:teuthology.orchestra.run.smithi151:> CEPH_ARGS='--filestore-merge-threshold 1 --filestore-split-multiple 1' sudo -E ceph-objectstore-tool --no-mon-config --log-file=/var/log/ceph/objectstore_tool.$pid.log --data-path /var/lib/ceph/osd/ceph-6 --journal-path /var/lib/ceph/osd/ceph-6/journal --op apply-layout-settings --pool pool_client.0 2024-04-24T15:33:22.651 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:22.658 INFO:teuthology.orchestra.run.smithi096.stdout:5405 write 0x223733f thru 0x223cbdd (0x589f bytes) 2024-04-24T15:33:22.660 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:22.660 INFO:teuthology.orchestra.run.smithi096.stdout:5408 read 0x9034de thru 0x911885 (0xe3a8 bytes) 2024-04-24T15:33:22.660 INFO:teuthology.orchestra.run.smithi096.stdout:5409 read 0x19d3eb8 thru 0x19d5f31 (0x207a bytes) 2024-04-24T15:33:22.660 INFO:teuthology.orchestra.run.smithi096.stdout:5410 read 0x14efa34 thru 0x14f4eac (0x5479 bytes) 2024-04-24T15:33:22.661 INFO:teuthology.orchestra.run.smithi096.stdout:5411 punch from 0x568f63 to 0x5752e6, (0xc383 bytes) 2024-04-24T15:33:22.661 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:22.677 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:22.678 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:22.680 INFO:teuthology.orchestra.run.smithi096.stdout:5414 writesame 0xe0ed5bd thru 0xe0ee92c data_size 0x2(0x1370 bytes) 2024-04-24T15:33:22.682 INFO:teuthology.orchestra.run.smithi096.stdout:5415 punch from 0x169142 to 0x176591, (0xd44f bytes) 2024-04-24T15:33:22.682 INFO:teuthology.orchestra.run.smithi096.stdout:5416 read 0xbacd3cc thru 0xbad3fe6 (0x6c1b bytes) 2024-04-24T15:33:22.682 INFO:teuthology.orchestra.run.smithi096.stdout:5417 writesame 0xa4fc336 thru 0xa50bff2 data_size 0x75(0xfcbd bytes) 2024-04-24T15:33:22.685 INFO:teuthology.orchestra.run.smithi096.stdout:5419 read 0xae0e796 thru 0xae1ac6a (0xc4d5 bytes) 2024-04-24T15:33:22.685 INFO:teuthology.orchestra.run.smithi096.stdout:5421 writesame 0xd915ef8 thru 0xd918cd0 data_size 0x1(0x2dd9 bytes) 2024-04-24T15:33:22.688 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-24T15:33:22.693 INFO:teuthology.orchestra.run.smithi096.stdout:5422 read 0x2b20549 thru 0x2b227e6 (0x229e bytes) 2024-04-24T15:33:22.693 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:22.694+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:22.693 INFO:teuthology.orchestra.run.smithi096.stdout:5423 write 0x6b4b695 thru 0x6b5adc0 (0xf72c bytes) 2024-04-24T15:33:22.698 INFO:teuthology.orchestra.run.smithi096.stdout:5424 trunc from 0xe0ee92d to 0xb5d8742 2024-04-24T15:33:22.707 INFO:teuthology.orchestra.run.smithi096.stdout:5425 read 0x9e8710b thru 0x9e9386e (0xc764 bytes) 2024-04-24T15:33:22.708 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:22.708 INFO:teuthology.orchestra.run.smithi096.stdout:5428 write 0x974cb00 thru 0x9758e76 (0xc377 bytes) 2024-04-24T15:33:22.721 INFO:teuthology.orchestra.run.smithi096.stdout:5429 trunc from 0xb5d8742 to 0x2025d12 2024-04-24T15:33:22.731 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:22.731 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:22.731 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:22.737 INFO:teuthology.orchestra.run.smithi096.stdout:5434 trunc from 0x2025d12 to 0x72af53c 2024-04-24T15:33:22.743 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:22.743 INFO:teuthology.orchestra.run.smithi096.stdout:5438 read 0x32801d5 thru 0x328cbca (0xc9f6 bytes) 2024-04-24T15:33:22.755 INFO:teuthology.orchestra.run.smithi096.stdout:5439 writesame 0xb60f1b9 thru 0xb61282c data_size 0x29(0x3674 bytes) 2024-04-24T15:33:22.756 INFO:teuthology.orchestra.run.smithi096.stdout:5440 punch from 0x3e9dc52 to 0x3ea32a7, (0x5655 bytes) 2024-04-24T15:33:22.756 INFO:teuthology.orchestra.run.smithi096.stdout:5442 read 0x9e18f27 thru 0x9e1df8a (0x5064 bytes) 2024-04-24T15:33:22.756 INFO:teuthology.orchestra.run.smithi096.stdout:5443 punch from 0x58cf554 to 0x58df139, (0xfbe5 bytes) 2024-04-24T15:33:22.758 INFO:teuthology.orchestra.run.smithi096.stdout:5444 trunc from 0xb61282d to 0xd379542 2024-04-24T15:33:22.776 INFO:teuthology.orchestra.run.smithi096.stdout:5446 trunc from 0xd379542 to 0x38bf421 2024-04-24T15:33:22.784 INFO:teuthology.orchestra.run.smithi096.stdout:5447 read 0xcf661b thru 0xcf7725 (0x110b bytes) 2024-04-24T15:33:22.784 INFO:teuthology.orchestra.run.smithi096.stdout:5449 punch from 0x2312e75 to 0x232146a, (0xe5f5 bytes) 2024-04-24T15:33:22.792 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:33:22.793+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:33:22.793 INFO:teuthology.orchestra.run.smithi096.stdout:5451 write 0x7080de9 thru 0x708b948 (0xab60 bytes) 2024-04-24T15:33:22.805 INFO:teuthology.orchestra.run.smithi096.stdout:5452 writesame 0xb7dd181 thru 0xb7e4d0c data_size 0x2(0x7b8c bytes) 2024-04-24T15:33:22.806 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:22.806 INFO:teuthology.orchestra.run.smithi096.stdout:5454 read 0x87252d4 thru 0x872b6de (0x640b bytes) 2024-04-24T15:33:22.806 INFO:teuthology.orchestra.run.smithi096.stdout:5455 writesame 0x3f7a3f0 thru 0x3f816c1 data_size 0x2e(0x72d2 bytes) 2024-04-24T15:33:22.808 INFO:teuthology.orchestra.run.smithi096.stdout:5456 punch from 0x6ce4762 to 0x6cf42f1, (0xfb8f bytes) 2024-04-24T15:33:22.808 INFO:teuthology.orchestra.run.smithi096.stdout:5457 read 0xaccd0c1 thru 0xacd5aef (0x8a2f bytes) 2024-04-24T15:33:22.808 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:22.810 INFO:teuthology.orchestra.run.smithi096.stdout:5459 trunc from 0xb7e4d0d to 0xc7279ae 2024-04-24T15:33:22.815 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:22.815 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:22.815 INFO:teuthology.orchestra.run.smithi096.stdout:5463 write 0x13ee456 thru 0x13fc5ef (0xe19a bytes) 2024-04-24T15:33:22.825 INFO:teuthology.orchestra.run.smithi096.stdout:5464 trunc from 0xc7279ae to 0x54a7ec7 2024-04-24T15:33:22.838 INFO:teuthology.orchestra.run.smithi096.stdout:5465 trunc from 0x54a7ec7 to 0x91264f0 2024-04-24T15:33:22.843 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:22.843 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:22.848 INFO:teuthology.orchestra.run.smithi096.stdout:5466 write 0x9e88b91 thru 0x9e91363 (0x87d3 bytes) 2024-04-24T15:33:22.858 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-24T15:33:22.860 INFO:teuthology.orchestra.run.smithi096.stdout:5467 write 0xe969f01 thru 0xe974c46 (0xad46 bytes) 2024-04-24T15:33:22.872 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-24T15:33:22.878 INFO:teuthology.orchestra.run.smithi096.stdout:5468 trunc from 0xe974c47 to 0x191bac2 2024-04-24T15:33:22.888 INFO:teuthology.orchestra.run.smithi096.stdout:5471 writesame 0xbca2e2 thru 0xbd3dc8 data_size 0x1(0x9ae7 bytes) 2024-04-24T15:33:22.893 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:22.894+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:22.909 INFO:teuthology.orchestra.run.smithi096.stdout:5472 writesame 0xa2b8c3 thru 0xa2e041 data_size 0x1(0x277f bytes) 2024-04-24T15:33:22.915 INFO:teuthology.orchestra.run.smithi096.stdout:5474 read 0xea44a4 thru 0xead969 (0x94c6 bytes) 2024-04-24T15:33:22.919 INFO:teuthology.orchestra.run.smithi096.stdout:5475 trunc from 0x191bac2 to 0x4821d33 2024-04-24T15:33:22.937 INFO:teuthology.orchestra.run.smithi096.stdout:5476 writesame 0xb0bdf1a thru 0xb0c0794 data_size 0x1(0x287b bytes) 2024-04-24T15:33:22.939 INFO:teuthology.orchestra.run.smithi096.stdout:5477 read 0x4013c9 thru 0x40802d (0x6c65 bytes) 2024-04-24T15:33:22.940 INFO:teuthology.orchestra.run.smithi096.stdout:5478 write 0x7ab74a2 thru 0x7ac5fcf (0xeb2e bytes) 2024-04-24T15:33:22.942 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:22.942 INFO:teuthology.orchestra.run.smithi096.stdout:5481 read 0x8465f15 thru 0x8471023 (0xb10f bytes) 2024-04-24T15:33:22.943 INFO:teuthology.orchestra.run.smithi096.stdout:5482 punch from 0xaa1a2d0 to 0xaa1a54e, (0x27e bytes) 2024-04-24T15:33:22.943 INFO:teuthology.orchestra.run.smithi096.stdout:5483 writesame 0x38eea6f thru 0x38faced data_size 0x7(0xc27f bytes) 2024-04-24T15:33:22.947 INFO:teuthology.orchestra.run.smithi096.stdout:5485 write 0x8b9c2d8 thru 0x8ba7c41 (0xb96a bytes) 2024-04-24T15:33:22.960 INFO:teuthology.orchestra.run.smithi096.stdout:5486 writesame 0xe4547c1 thru 0xe45b2df data_size 0x1(0x6b1f bytes) 2024-04-24T15:33:22.961 INFO:teuthology.orchestra.run.smithi096.stdout:5487 write 0x128685c thru 0x128d11a (0x68bf bytes) 2024-04-24T15:33:22.963 INFO:teuthology.orchestra.run.smithi096.stdout:5488 read 0x15d3562 thru 0x15d6505 (0x2fa4 bytes) 2024-04-24T15:33:22.976 INFO:teuthology.orchestra.run.smithi096.stdout:5489 trunc from 0xe45b2e0 to 0x408d60b 2024-04-24T15:33:22.994 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:22.994+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:23.000 INFO:teuthology.orchestra.run.smithi096.stdout:5490 trunc from 0x408d60b to 0xa297deb 2024-04-24T15:33:23.009 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:23.009 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:23.015 INFO:teuthology.orchestra.run.smithi096.stdout:5492 write 0xda3f418 thru 0xda49227 (0x9e10 bytes) 2024-04-24T15:33:23.019 INFO:teuthology.orchestra.run.smithi096.stdout:5493 read 0xcce73a8 thru 0xccf3a1c (0xc675 bytes) 2024-04-24T15:33:23.020 INFO:teuthology.orchestra.run.smithi096.stdout:5494 writesame 0xce934d9 thru 0xcea1d96 data_size 0x1f(0xe8be bytes) 2024-04-24T15:33:23.023 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-24T15:33:23.023 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:23.035 INFO:teuthology.orchestra.run.smithi096.stdout:5496 trunc from 0xda49228 to 0x3ab36c5 2024-04-24T15:33:23.048 INFO:teuthology.orchestra.run.smithi096.stdout:5498 trunc from 0x3ab36c5 to 0x3f4a14e 2024-04-24T15:33:23.054 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:23.068 INFO:teuthology.orchestra.run.smithi096.stdout:5501 writesame 0xad87662 thru 0xad8f5bd data_size 0x4e(0x7f5c bytes) 2024-04-24T15:33:23.069 INFO:teuthology.orchestra.run.smithi096.stdout:5503 write 0x7e8f3e8 thru 0x7e9d73f (0xe358 bytes) 2024-04-24T15:33:23.070 INFO:teuthology.orchestra.run.smithi096.stdout:5504 read 0x7ababdc thru 0x7abf5ef (0x4a14 bytes) 2024-04-24T15:33:23.071 INFO:teuthology.orchestra.run.smithi096.stdout:5505 read 0xdc98bb thru 0xdce288 (0x49ce bytes) 2024-04-24T15:33:23.071 INFO:teuthology.orchestra.run.smithi096.stdout:5506 writesame 0x56d47f thru 0x57232d data_size 0x1(0x4eaf bytes) 2024-04-24T15:33:23.081 INFO:teuthology.orchestra.run.smithi096.stdout:5509 write 0x181bb4b thru 0x182ac1e (0xf0d4 bytes) 2024-04-24T15:33:23.083 INFO:teuthology.orchestra.run.smithi096.stdout:5510 read 0x4a608a2 thru 0x4a6f93c (0xf09b bytes) 2024-04-24T15:33:23.083 INFO:teuthology.orchestra.run.smithi096.stdout:5511 read 0x12ac51e thru 0x12ba7aa (0xe28d bytes) 2024-04-24T15:33:23.084 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:23.084 INFO:teuthology.orchestra.run.smithi096.stdout:5513 read 0x25e166a thru 0x25ee67b (0xd012 bytes) 2024-04-24T15:33:23.084 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:23.084 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:23.084 INFO:teuthology.orchestra.run.smithi096.stdout:5517 write 0xd6bf06 thru 0xd70804 (0x48ff bytes) 2024-04-24T15:33:23.086 INFO:teuthology.orchestra.run.smithi096.stdout:5520 punch from 0x6d3bb63 to 0x6d41ab6, (0x5f53 bytes) 2024-04-24T15:33:23.086 INFO:teuthology.orchestra.run.smithi096.stdout:5521 read 0x403ff82 thru 0x4042288 (0x2307 bytes) 2024-04-24T15:33:23.086 INFO:teuthology.orchestra.run.smithi096.stdout:5523 punch from 0x2f415cb to 0x2f46fed, (0x5a22 bytes) 2024-04-24T15:33:23.086 INFO:teuthology.orchestra.run.smithi096.stdout:5524 write 0x47b2aea thru 0x47c25e8 (0xfaff bytes) 2024-04-24T15:33:23.090 INFO:teuthology.orchestra.run.smithi096.stdout:5527 write 0x887c65c thru 0x8884d11 (0x86b6 bytes) 2024-04-24T15:33:23.095 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:23.096+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:23.096 INFO:teuthology.orchestra.run.smithi096.stdout:5529 read 0x62cd8ef thru 0x62d5b10 (0x8222 bytes) 2024-04-24T15:33:23.096 INFO:teuthology.orchestra.run.smithi096.stdout:5532 write 0x8f74834 thru 0x8f7c42e (0x7bfb bytes) 2024-04-24T15:33:23.110 INFO:teuthology.orchestra.run.smithi096.stdout:5533 writesame 0xeb9ad8f thru 0xeb9de7f data_size 0x11(0x30f1 bytes) 2024-04-24T15:33:23.111 INFO:teuthology.orchestra.run.smithi096.stdout:5535 writesame 0x29fb0a thru 0x2a2b9f data_size 0x9(0x3096 bytes) 2024-04-24T15:33:23.113 INFO:teuthology.orchestra.run.smithi096.stdout:5536 writesame 0xc8c15ee thru 0xc8c7fa4 data_size 0x5d(0x69b7 bytes) 2024-04-24T15:33:23.115 INFO:teuthology.orchestra.run.smithi096.stdout:5537 writesame 0xbfe1e20 thru 0xbfeacbb data_size 0x2(0x8e9c bytes) 2024-04-24T15:33:23.125 INFO:teuthology.orchestra.run.smithi096.stdout:5540 read 0xbe35250 thru 0xbe398a0 (0x4651 bytes) 2024-04-24T15:33:23.125 INFO:teuthology.orchestra.run.smithi096.stdout:5541 write 0x92c8ebf thru 0x92d7fc5 (0xf107 bytes) 2024-04-24T15:33:23.127 INFO:teuthology.orchestra.run.smithi096.stdout:5542 write 0x838a4dd thru 0x8396f79 (0xca9d bytes) 2024-04-24T15:33:23.143 INFO:teuthology.orchestra.run.smithi096.stdout:5543 trunc from 0xeb9de80 to 0x2abd855 2024-04-24T15:33:23.162 INFO:teuthology.orchestra.run.smithi096.stdout:5544 write 0x4a1e614 thru 0x4a25aec (0x74d9 bytes) 2024-04-24T15:33:23.164 INFO:teuthology.orchestra.run.smithi096.stdout:5545 read 0x199d10f thru 0x19a633f (0x9231 bytes) 2024-04-24T15:33:23.164 INFO:teuthology.orchestra.run.smithi096.stdout:5546 write 0x1508473 thru 0x150a4b5 (0x2043 bytes) 2024-04-24T15:33:23.165 INFO:teuthology.orchestra.run.smithi096.stdout:5547 punch from 0x580418 to 0x588c7a, (0x8862 bytes) 2024-04-24T15:33:23.165 INFO:teuthology.orchestra.run.smithi096.stdout:5548 read 0x289ef84 thru 0x28a5d3e (0x6dbb bytes) 2024-04-24T15:33:23.169 INFO:teuthology.orchestra.run.smithi096.stdout:5549 trunc from 0x4a25aed to 0x206cfb1 2024-04-24T15:33:23.179 INFO:teuthology.orchestra.run.smithi096.stdout:5554 trunc from 0x206cfb1 to 0x10d3282 2024-04-24T15:33:23.184 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:23.184 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:23.192 INFO:teuthology.orchestra.run.smithi096.stdout:5555 trunc from 0x10d3282 to 0x3f39b00 2024-04-24T15:33:23.196 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:33:23.198+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:33:23.201 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-24T15:33:23.201 INFO:teuthology.orchestra.run.smithi096.stdout:5558 trunc from 0x3f39b00 to 0x66a5ba1 2024-04-24T15:33:23.205 INFO:teuthology.orchestra.run.smithi096.stdout:5559 write 0x51c9cfb thru 0x51d8c2e (0xef34 bytes) 2024-04-24T15:33:23.208 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:23.208 INFO:teuthology.orchestra.run.smithi096.stdout:5563 writesame 0x1679b42 thru 0x167d5a7 data_size 0x17(0x3a66 bytes) 2024-04-24T15:33:23.209 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:23.209 INFO:teuthology.orchestra.run.smithi096.stdout:5566 read 0x595b0a6 thru 0x5962eec (0x7e47 bytes) 2024-04-24T15:33:23.209 INFO:teuthology.orchestra.run.smithi096.stdout:5567 read 0x482d4e7 thru 0x4833484 (0x5f9e bytes) 2024-04-24T15:33:23.209 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:23.210 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:23.216 INFO:teuthology.orchestra.run.smithi096.stdout:5571 writesame 0x79d326b thru 0x79dc0ad data_size 0x1(0x8e43 bytes) 2024-04-24T15:33:23.220 INFO:teuthology.orchestra.run.smithi096.stdout:5573 trunc from 0x79dc0ae to 0x560fccb 2024-04-24T15:33:23.231 INFO:teuthology.orchestra.run.smithi096.stdout:5574 read 0x4cb05aa thru 0x4cbad1a (0xa771 bytes) 2024-04-24T15:33:23.246 INFO:teuthology.orchestra.run.smithi096.stdout:5575 write 0xcd3741f thru 0xcd3aefd (0x3adf bytes) 2024-04-24T15:33:23.247 INFO:teuthology.orchestra.run.smithi096.stdout:5576 writesame 0x58750c9 thru 0x588036a data_size 0x55(0xb2a2 bytes) 2024-04-24T15:33:23.249 INFO:teuthology.orchestra.run.smithi096.stdout:5577 read 0x6d2edc2 thru 0x6d3afda (0xc219 bytes) 2024-04-24T15:33:23.249 INFO:teuthology.orchestra.run.smithi096.stdout:5579 read 0x253e3f2 thru 0x254803e (0x9c4d bytes) 2024-04-24T15:33:23.249 INFO:teuthology.orchestra.run.smithi096.stdout:5580 write 0x1e87cfe thru 0x1e95262 (0xd565 bytes) 2024-04-24T15:33:23.257 INFO:teuthology.orchestra.run.smithi096.stdout:5581 write 0xd33f8e9 thru 0xd348680 (0x8d98 bytes) 2024-04-24T15:33:23.259 INFO:teuthology.orchestra.run.smithi096.stdout:5582 writesame 0x8f98e47 thru 0x8f9c3fc data_size 0x19(0x35b6 bytes) 2024-04-24T15:33:23.266 INFO:teuthology.orchestra.run.smithi096.stdout:5583 writesame 0xdb84261 thru 0xdb8aad9 data_size 0x5(0x6879 bytes) 2024-04-24T15:33:23.268 INFO:teuthology.orchestra.run.smithi096.stdout:5584 write 0xa0205bd thru 0xa02c1d5 (0xbc19 bytes) 2024-04-24T15:33:23.269 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:23.269 INFO:teuthology.orchestra.run.smithi096.stdout:5588 read 0xb08afe9 thru 0xb096b75 (0xbb8d bytes) 2024-04-24T15:33:23.275 INFO:teuthology.orchestra.run.smithi096.stdout:5590 trunc from 0xdb8aada to 0x923f016 2024-04-24T15:33:23.284 INFO:teuthology.orchestra.run.smithi096.stdout:5591 clone 30 order 24 su 262144 sc 6 2024-04-24T15:33:23.297 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:23.299+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:23.342 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:23.342 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:23.352 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-24T15:33:23.436 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:23.437+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:23.499 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:23.500+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:23.514 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:23.514 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:23.524 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-24T15:33:23.600 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:33:23.601+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:33:23.649 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:23.650 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:23.659 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-24T15:33:23.669 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-24T15:33:23.713 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:33:23.714+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:33:23.770 INFO:teuthology.orchestra.run.smithi151.stderr:Nothing to do for non-filestore backend 2024-04-24T15:33:23.802 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:23.803+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:23.807 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-04-24T15:33:23.825 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-04-24T15:33:23.827 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:23.827 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:23.837 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-24T15:33:23.842 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-04-24T15:33:23.859 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-04-24T15:33:23.880 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-04-24T15:33:23.880 INFO:teuthology.orchestra.run.smithi096.stderr:noscrub is unset 2024-04-24T15:33:23.903 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:23.903+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:23.903 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-04-24T15:33:23.924 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-04-24T15:33:23.924 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T15:33:23.924 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T15:33:23.924 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T15:33:23.924 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T15:33:23.924 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T15:33:23.924 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T15:33:23.924 INFO:teuthology.orchestra.run.smithi096.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-04-24T15:33:23.924 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T15:33:23.926 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-24T15:33:23.927 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-24T15:33:23.930 INFO:teuthology.orchestra.run.smithi096.stdout:leaving image image_client.0-clone29 intact 2024-04-24T15:33:23.969 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:23.970 INFO:teuthology.orchestra.run.smithi151.stdout: "size": 20, 2024-04-24T15:33:23.970 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 600, 2024-04-24T15:33:23.970 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [ 2024-04-24T15:33:23.970 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:23.970 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:39041 4.1 4:82c1a902:::rbd_data.114abe589119.000000000000032c:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e68)", 2024-04-24T15:33:23.970 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:01.760552+0000", 2024-04-24T15:33:23.970 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 22.207761740999999, 2024-04-24T15:33:23.970 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.87570748300000001, 2024-04-24T15:33:23.970 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:23.971 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "reached pg", 2024-04-24T15:33:23.971 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:23.971 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:23.971 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:23.971 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 39041 2024-04-24T15:33:23.971 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:23.971 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:23.971 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:23.971 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:23.971 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760552+0000", 2024-04-24T15:33:23.971 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:23.972 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:23.972 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:23.972 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:23.972 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760552+0000", 2024-04-24T15:33:23.972 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:23.972 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:23.972 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:23.972 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:23.972 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760550+0000", 2024-04-24T15:33:23.972 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999986 2024-04-24T15:33:23.972 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:23.972 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:23.972 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:23.972 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760557+0000", 2024-04-24T15:33:23.973 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1420000000000004e-06 2024-04-24T15:33:23.973 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:23.973 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:23.973 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:23.973 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760572+0000", 2024-04-24T15:33:23.973 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.483e-05 2024-04-24T15:33:23.973 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:23.973 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:23.973 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:23.973 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760578+0000", 2024-04-24T15:33:23.973 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.2820000000000002e-06 2024-04-24T15:33:23.973 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:23.973 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:23.973 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:23.974 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760628+0000", 2024-04-24T15:33:23.974 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.0040000000000002e-05 2024-04-24T15:33:23.974 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:23.974 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:23.974 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op must wait for map", 2024-04-24T15:33:23.974 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760637+0000", 2024-04-24T15:33:23.974 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.6899999999999998e-06 2024-04-24T15:33:23.974 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:23.974 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:23.974 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:23.974 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.770562+0000", 2024-04-24T15:33:23.974 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0099252739999999996 2024-04-24T15:33:23.975 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:23.975 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:23.975 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for peered", 2024-04-24T15:33:23.975 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.770572+0000", 2024-04-24T15:33:23.975 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0105999999999999e-05 2024-04-24T15:33:23.975 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:23.975 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:23.975 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:23.975 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:02.636170+0000", 2024-04-24T15:33:23.975 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.86559722299999997 2024-04-24T15:33:23.975 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:23.975 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:23.975 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:23.975 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:02.636259+0000", 2024-04-24T15:33:23.976 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.9446999999999999e-05 2024-04-24T15:33:23.976 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:23.976 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:23.976 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:23.976 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:23.976 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:23.976 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48675 4.d 4:b465d49b:::rbd_data.114a858e6641.0000000000000002:17 [read 2768896~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:23.976 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.926403+0000", 2024-04-24T15:33:23.976 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 13.041910171, 2024-04-24T15:33:23.976 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035742807000000001, 2024-04-24T15:33:23.976 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:23.976 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:23.976 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:23.976 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:23.976 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:23.977 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48675 2024-04-24T15:33:23.977 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:23.977 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:23.977 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:23.977 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:23.977 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926403+0000", 2024-04-24T15:33:23.977 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:23.977 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:23.977 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:23.977 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:23.977 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926403+0000", 2024-04-24T15:33:23.977 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:23.977 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:23.977 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:23.977 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:23.978 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926403+0000", 2024-04-24T15:33:23.978 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:23.978 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:23.978 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:23.978 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:23.978 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926404+0000", 2024-04-24T15:33:23.978 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.424e-06 2024-04-24T15:33:23.978 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:23.978 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:23.978 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:23.979 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926413+0000", 2024-04-24T15:33:23.979 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.6850000000000007e-06 2024-04-24T15:33:23.979 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:23.979 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:23.979 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:23.979 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926417+0000", 2024-04-24T15:33:23.979 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.7859999999999998e-06 2024-04-24T15:33:23.979 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:23.979 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:23.979 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:23.979 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.961986+0000", 2024-04-24T15:33:23.979 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035568862 2024-04-24T15:33:23.979 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:23.979 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:23.979 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:23.979 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.962063+0000", 2024-04-24T15:33:23.979 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7257999999999998e-05 2024-04-24T15:33:23.980 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:23.980 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:23.980 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:23.980 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.962146+0000", 2024-04-24T15:33:23.980 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3094000000000001e-05 2024-04-24T15:33:23.980 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:23.980 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:23.980 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:23.980 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:23.980 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:23.980 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48676 4.d 4:b465d49b:::rbd_data.114a858e6641.0000000000000002:17 [read 2834432~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:23.980 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.926421+0000", 2024-04-24T15:33:23.980 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 13.041892627999999, 2024-04-24T15:33:23.980 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035829002999999998, 2024-04-24T15:33:23.980 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:23.981 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:23.981 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:23.981 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:23.981 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:23.981 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48676 2024-04-24T15:33:23.981 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:23.981 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:23.981 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:23.981 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:23.981 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926421+0000", 2024-04-24T15:33:23.981 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:23.981 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:23.981 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:23.981 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:23.982 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926421+0000", 2024-04-24T15:33:23.982 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:23.982 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:23.982 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:23.982 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:23.982 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926420+0000", 2024-04-24T15:33:23.982 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:23.982 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:23.982 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:23.982 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:23.982 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926423+0000", 2024-04-24T15:33:23.982 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.356e-06 2024-04-24T15:33:23.982 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:23.982 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:23.983 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:23.983 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926428+0000", 2024-04-24T15:33:23.983 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.3340000000000001e-06 2024-04-24T15:33:23.983 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:23.983 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:23.983 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:23.983 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926431+0000", 2024-04-24T15:33:23.983 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.2899999999999998e-06 2024-04-24T15:33:23.983 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:23.983 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:23.983 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:23.983 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.962156+0000", 2024-04-24T15:33:23.983 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035725159999999999 2024-04-24T15:33:23.983 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:23.984 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:23.984 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:23.984 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.962197+0000", 2024-04-24T15:33:23.984 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0457999999999998e-05 2024-04-24T15:33:23.984 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:23.984 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:23.984 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:23.984 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.962250+0000", 2024-04-24T15:33:23.984 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.2812000000000002e-05 2024-04-24T15:33:23.984 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:23.984 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:23.984 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:23.984 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:23.984 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:23.984 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48703 4.d 4:b59d5735:::rbd_data.114a858e6641.0000000000000004:17 [read 2637824~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:23.984 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928531+0000", 2024-04-24T15:33:23.985 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 13.039782002000001, 2024-04-24T15:33:23.985 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035675198999999998, 2024-04-24T15:33:23.985 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:23.985 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:23.985 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:23.985 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:23.985 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:23.985 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48703 2024-04-24T15:33:23.985 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:23.985 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:23.985 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:23.985 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:23.985 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928531+0000", 2024-04-24T15:33:23.985 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:23.986 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:23.986 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:23.986 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:23.986 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928531+0000", 2024-04-24T15:33:23.986 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:23.986 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:23.986 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:23.986 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:23.986 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928531+0000", 2024-04-24T15:33:23.986 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:23.986 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:23.986 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:23.986 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:23.986 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928532+0000", 2024-04-24T15:33:23.987 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.33e-06 2024-04-24T15:33:23.987 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:23.987 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:23.987 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:23.987 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928536+0000", 2024-04-24T15:33:23.987 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.4419999999999998e-06 2024-04-24T15:33:23.987 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:23.987 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:23.987 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:23.987 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928538+0000", 2024-04-24T15:33:23.987 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.0669999999999999e-06 2024-04-24T15:33:23.987 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:23.987 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:23.987 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:23.987 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964040+0000", 2024-04-24T15:33:23.987 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035501900000000003 2024-04-24T15:33:23.988 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:23.988 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:23.988 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:23.988 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964118+0000", 2024-04-24T15:33:23.988 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.8490000000000002e-05 2024-04-24T15:33:23.988 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:23.988 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:23.988 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:23.988 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964207+0000", 2024-04-24T15:33:23.988 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.8202999999999996e-05 2024-04-24T15:33:23.988 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:23.988 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:23.988 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:23.988 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:23.988 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:23.989 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48704 4.d 4:b59d5735:::rbd_data.114a858e6641.0000000000000004:17 [read 2703360~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:23.989 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928541+0000", 2024-04-24T15:33:23.989 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 13.039772658, 2024-04-24T15:33:23.989 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035763016000000002, 2024-04-24T15:33:23.989 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:23.989 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:23.989 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:23.989 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:23.989 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:23.989 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48704 2024-04-24T15:33:23.989 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:23.989 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:23.989 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:23.989 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:23.989 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928541+0000", 2024-04-24T15:33:23.989 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:23.990 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:23.990 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:23.990 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:23.990 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928541+0000", 2024-04-24T15:33:23.990 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:23.990 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:23.990 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:23.990 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:23.990 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928540+0000", 2024-04-24T15:33:23.990 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:23.990 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:23.990 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:23.990 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:23.991 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928542+0000", 2024-04-24T15:33:23.991 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.37e-06 2024-04-24T15:33:23.991 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:23.991 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:23.991 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:23.991 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928545+0000", 2024-04-24T15:33:23.991 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8229999999999999e-06 2024-04-24T15:33:23.991 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:23.991 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:23.991 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:23.991 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928547+0000", 2024-04-24T15:33:23.991 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.9360000000000002e-06 2024-04-24T15:33:23.991 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:23.991 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:23.991 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:23.992 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964219+0000", 2024-04-24T15:33:23.992 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035672813999999997 2024-04-24T15:33:23.992 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:23.992 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:23.992 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:23.992 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964259+0000", 2024-04-24T15:33:23.992 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0070000000000001e-05 2024-04-24T15:33:23.992 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:23.992 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:23.992 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:23.992 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964304+0000", 2024-04-24T15:33:23.992 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4229e-05 2024-04-24T15:33:23.992 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:23.992 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:23.992 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:23.993 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:23.993 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:23.993 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48719 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 1851392~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:23.993 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928549+0000", 2024-04-24T15:33:23.993 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 13.039764511, 2024-04-24T15:33:23.993 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035890645999999998, 2024-04-24T15:33:23.993 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:23.993 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:23.993 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:23.993 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:23.993 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:23.993 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48719 2024-04-24T15:33:23.993 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:23.993 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:23.993 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:23.993 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:23.994 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928549+0000", 2024-04-24T15:33:23.994 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:23.994 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:23.994 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:23.994 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:23.994 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928549+0000", 2024-04-24T15:33:23.994 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:23.994 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:23.994 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:23.994 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:23.994 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928549+0000", 2024-04-24T15:33:23.994 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:23.994 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:23.994 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:23.995 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:23.995 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928550+0000", 2024-04-24T15:33:23.995 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3829999999999999e-06 2024-04-24T15:33:23.995 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:23.995 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:23.995 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:23.995 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928553+0000", 2024-04-24T15:33:23.995 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7369999999999998e-06 2024-04-24T15:33:23.995 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:23.995 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:23.995 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:23.995 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928555+0000", 2024-04-24T15:33:23.995 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.8920000000000001e-06 2024-04-24T15:33:23.996 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:23.996 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:23.996 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:23.996 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964240+0000", 2024-04-24T15:33:23.996 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035685537000000003 2024-04-24T15:33:23.996 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:23.996 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:23.996 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:23.996 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964318+0000", 2024-04-24T15:33:23.996 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7838999999999998e-05 2024-04-24T15:33:23.996 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:23.996 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:23.996 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:23.996 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964439+0000", 2024-04-24T15:33:23.996 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000121564 2024-04-24T15:33:23.996 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:23.997 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:23.997 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:23.997 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:23.997 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:23.997 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48720 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 1916928~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:23.997 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928557+0000", 2024-04-24T15:33:23.997 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 13.039756666000001, 2024-04-24T15:33:23.997 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036005219999999998, 2024-04-24T15:33:23.997 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:23.997 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:23.997 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:23.997 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:23.997 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:23.997 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48720 2024-04-24T15:33:23.998 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:23.998 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:23.998 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:23.998 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:23.998 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928557+0000", 2024-04-24T15:33:23.998 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:23.998 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:23.998 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:23.998 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:23.998 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928557+0000", 2024-04-24T15:33:23.998 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:23.998 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:23.998 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:23.998 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:23.998 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928556+0000", 2024-04-24T15:33:23.999 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:23.999 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:23.999 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:23.999 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:23.999 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928558+0000", 2024-04-24T15:33:23.999 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.13e-06 2024-04-24T15:33:23.999 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:23.999 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:23.999 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:23.999 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928561+0000", 2024-04-24T15:33:23.999 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5530000000000001e-06 2024-04-24T15:33:23.999 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:23.999 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:23.999 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:23.999 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928563+0000", 2024-04-24T15:33:23.999 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.8810000000000001e-06 2024-04-24T15:33:24.000 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.000 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.000 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:24.000 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964450+0000", 2024-04-24T15:33:24.000 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035887040000000002 2024-04-24T15:33:24.000 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.000 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.000 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:24.000 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964494+0000", 2024-04-24T15:33:24.000 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3699000000000003e-05 2024-04-24T15:33:24.000 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.000 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.000 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:24.000 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964562+0000", 2024-04-24T15:33:24.000 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.8149000000000001e-05 2024-04-24T15:33:24.000 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:24.001 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:24.001 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:24.001 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.001 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.001 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48721 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 1982464~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:24.001 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928565+0000", 2024-04-24T15:33:24.001 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 13.039748142000001, 2024-04-24T15:33:24.001 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036233466999999998, 2024-04-24T15:33:24.001 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:24.001 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:24.001 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:24.001 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:24.001 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:24.002 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48721 2024-04-24T15:33:24.002 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.002 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:24.002 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.002 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:24.002 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928565+0000", 2024-04-24T15:33:24.002 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:24.002 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.002 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.002 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:24.002 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928565+0000", 2024-04-24T15:33:24.002 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:24.002 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.002 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.002 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:24.003 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928565+0000", 2024-04-24T15:33:24.003 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:24.003 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.003 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.003 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:24.003 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928566+0000", 2024-04-24T15:33:24.003 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.336e-06 2024-04-24T15:33:24.003 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.003 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.003 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:24.003 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928569+0000", 2024-04-24T15:33:24.003 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.9189999999999999e-06 2024-04-24T15:33:24.003 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.003 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.003 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:24.003 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928571+0000", 2024-04-24T15:33:24.004 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.1909999999999999e-06 2024-04-24T15:33:24.004 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.004 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.004 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:24.004 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964592+0000", 2024-04-24T15:33:24.004 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036020120000000003 2024-04-24T15:33:24.004 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.004 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.004 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:24.004 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964673+0000", 2024-04-24T15:33:24.004 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.1254000000000005e-05 2024-04-24T15:33:24.004 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.004 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.004 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:24.004 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964799+0000", 2024-04-24T15:33:24.004 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00012588700000000001 2024-04-24T15:33:24.005 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:24.005 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:24.005 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:24.005 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.005 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.005 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48722 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2048000~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:24.005 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928586+0000", 2024-04-24T15:33:24.005 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 13.039727017000001, 2024-04-24T15:33:24.005 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036331347999999999, 2024-04-24T15:33:24.005 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:24.005 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:24.005 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:24.005 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:24.006 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:24.006 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48722 2024-04-24T15:33:24.006 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.006 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:24.006 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.006 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:24.006 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928586+0000", 2024-04-24T15:33:24.006 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:24.006 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.006 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.006 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:24.006 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928586+0000", 2024-04-24T15:33:24.006 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:24.006 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.007 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.007 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:24.007 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928586+0000", 2024-04-24T15:33:24.007 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:24.007 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.007 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.007 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:24.007 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928587+0000", 2024-04-24T15:33:24.007 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3519999999999999e-06 2024-04-24T15:33:24.007 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.007 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.007 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:24.007 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928590+0000", 2024-04-24T15:33:24.007 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.9699999999999999e-06 2024-04-24T15:33:24.007 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.007 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.008 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:24.008 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928592+0000", 2024-04-24T15:33:24.008 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.097e-06 2024-04-24T15:33:24.008 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.008 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.008 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:24.008 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964810+0000", 2024-04-24T15:33:24.008 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036217249 2024-04-24T15:33:24.008 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.008 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.008 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:24.008 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964851+0000", 2024-04-24T15:33:24.008 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.1151e-05 2024-04-24T15:33:24.008 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.008 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.008 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:24.009 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964918+0000", 2024-04-24T15:33:24.009 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.6898999999999998e-05 2024-04-24T15:33:24.009 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:24.009 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:24.009 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:24.009 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.009 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.009 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48723 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2113536~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:24.009 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928598+0000", 2024-04-24T15:33:24.009 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 13.039715628, 2024-04-24T15:33:24.009 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036552125999999997, 2024-04-24T15:33:24.009 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:24.009 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:24.009 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:24.009 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:24.010 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:24.010 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48723 2024-04-24T15:33:24.010 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.010 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:24.010 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.010 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:24.010 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928598+0000", 2024-04-24T15:33:24.010 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:24.010 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.010 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.010 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:24.010 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928598+0000", 2024-04-24T15:33:24.010 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:24.010 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.011 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.011 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:24.011 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928597+0000", 2024-04-24T15:33:24.011 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:24.011 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.011 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.011 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:24.011 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928599+0000", 2024-04-24T15:33:24.011 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.621e-06 2024-04-24T15:33:24.011 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.011 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.011 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:24.011 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928602+0000", 2024-04-24T15:33:24.011 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.9629999999999998e-06 2024-04-24T15:33:24.011 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.012 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.012 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:24.012 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928607+0000", 2024-04-24T15:33:24.012 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8940000000000001e-06 2024-04-24T15:33:24.012 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.012 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.012 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:24.012 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964947+0000", 2024-04-24T15:33:24.012 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036340439000000002 2024-04-24T15:33:24.012 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.012 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.012 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:24.012 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965028+0000", 2024-04-24T15:33:24.012 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.1054e-05 2024-04-24T15:33:24.012 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.012 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.013 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:24.013 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965150+0000", 2024-04-24T15:33:24.013 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00012154000000000001 2024-04-24T15:33:24.013 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:24.013 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:24.013 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:24.013 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.013 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.013 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48724 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2179072~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:24.013 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928609+0000", 2024-04-24T15:33:24.013 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 13.039704253, 2024-04-24T15:33:24.013 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036679614999999999, 2024-04-24T15:33:24.013 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:24.013 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:24.013 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:24.014 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:24.014 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:24.014 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48724 2024-04-24T15:33:24.014 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.014 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:24.014 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.014 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:24.014 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928609+0000", 2024-04-24T15:33:24.014 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:24.014 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.014 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.014 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:24.014 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928609+0000", 2024-04-24T15:33:24.014 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:24.014 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.015 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.015 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:24.015 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928609+0000", 2024-04-24T15:33:24.015 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:24.015 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.015 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.015 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:24.015 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928610+0000", 2024-04-24T15:33:24.015 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3030000000000001e-06 2024-04-24T15:33:24.015 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.015 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.015 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:24.015 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928613+0000", 2024-04-24T15:33:24.015 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.762e-06 2024-04-24T15:33:24.015 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.015 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.016 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:24.016 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928615+0000", 2024-04-24T15:33:24.016 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.832e-06 2024-04-24T15:33:24.016 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.016 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.016 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:24.016 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965174+0000", 2024-04-24T15:33:24.016 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036559633000000001 2024-04-24T15:33:24.016 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.016 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.016 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:24.016 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965221+0000", 2024-04-24T15:33:24.016 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6239999999999998e-05 2024-04-24T15:33:24.016 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.016 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.017 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:24.017 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965289+0000", 2024-04-24T15:33:24.017 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.8053999999999996e-05 2024-04-24T15:33:24.017 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:24.017 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:24.017 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:24.017 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.017 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.017 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48725 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2244608~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:24.017 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928617+0000", 2024-04-24T15:33:24.017 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 13.039696483, 2024-04-24T15:33:24.017 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036902659999999997, 2024-04-24T15:33:24.017 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:24.017 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:24.017 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:24.017 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:24.018 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:24.018 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48725 2024-04-24T15:33:24.018 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.018 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:24.018 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.018 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:24.018 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928617+0000", 2024-04-24T15:33:24.018 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:24.018 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.018 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.018 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:24.018 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928617+0000", 2024-04-24T15:33:24.018 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:24.019 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.019 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.019 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:24.019 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928617+0000", 2024-04-24T15:33:24.019 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:24.019 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.019 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.019 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:24.019 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928618+0000", 2024-04-24T15:33:24.019 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.206e-06 2024-04-24T15:33:24.019 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.019 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.019 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:24.019 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928621+0000", 2024-04-24T15:33:24.019 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5130000000000001e-06 2024-04-24T15:33:24.020 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.020 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.020 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:24.020 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928623+0000", 2024-04-24T15:33:24.020 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.9640000000000002e-06 2024-04-24T15:33:24.020 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.020 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.020 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:24.020 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965318+0000", 2024-04-24T15:33:24.020 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036694953000000002 2024-04-24T15:33:24.020 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.020 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.020 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:24.020 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965398+0000", 2024-04-24T15:33:24.020 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.9986000000000003e-05 2024-04-24T15:33:24.021 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.021 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.021 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:24.021 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965520+0000", 2024-04-24T15:33:24.021 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000121247 2024-04-24T15:33:24.021 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:24.021 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:24.021 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:24.021 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.021 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.021 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48726 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2310144~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:24.021 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928625+0000", 2024-04-24T15:33:24.021 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 13.039687936, 2024-04-24T15:33:24.021 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037015521000000003, 2024-04-24T15:33:24.021 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:24.022 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:24.022 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:24.022 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:24.022 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:24.022 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48726 2024-04-24T15:33:24.022 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.022 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:24.022 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.022 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:24.022 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928625+0000", 2024-04-24T15:33:24.022 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:24.022 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.022 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.022 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:24.023 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928625+0000", 2024-04-24T15:33:24.023 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:24.023 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.023 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.023 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:24.023 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928625+0000", 2024-04-24T15:33:24.023 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:24.023 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.023 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.023 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:24.023 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928627+0000", 2024-04-24T15:33:24.023 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3179999999999999e-06 2024-04-24T15:33:24.023 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.023 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.023 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:24.024 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928629+0000", 2024-04-24T15:33:24.024 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.819e-06 2024-04-24T15:33:24.024 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.024 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.024 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:24.024 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928634+0000", 2024-04-24T15:33:24.024 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6999999999999999e-06 2024-04-24T15:33:24.024 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.024 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.024 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:24.024 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965531+0000", 2024-04-24T15:33:24.024 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036897040999999998 2024-04-24T15:33:24.024 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.024 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.024 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:24.024 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965573+0000", 2024-04-24T15:33:24.025 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.189e-05 2024-04-24T15:33:24.025 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.025 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.025 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:24.025 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965641+0000", 2024-04-24T15:33:24.025 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.7959000000000004e-05 2024-04-24T15:33:24.025 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:24.025 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:24.025 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:24.025 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.025 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.025 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48727 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2375680~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:24.025 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928637+0000", 2024-04-24T15:33:24.025 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 13.039676575, 2024-04-24T15:33:24.025 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037235129999999998, 2024-04-24T15:33:24.026 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:24.026 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:24.026 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:24.026 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:24.026 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:24.026 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48727 2024-04-24T15:33:24.026 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.026 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:24.026 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.026 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:24.026 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928637+0000", 2024-04-24T15:33:24.026 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:24.026 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.026 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.027 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:24.027 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928637+0000", 2024-04-24T15:33:24.027 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:24.027 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.027 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.027 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:24.027 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928637+0000", 2024-04-24T15:33:24.027 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:24.027 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.027 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.027 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:24.027 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928638+0000", 2024-04-24T15:33:24.027 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.4279999999999999e-06 2024-04-24T15:33:24.027 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.028 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.028 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:24.028 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928641+0000", 2024-04-24T15:33:24.028 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.909e-06 2024-04-24T15:33:24.028 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.028 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.028 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:24.028 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928643+0000", 2024-04-24T15:33:24.028 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.9269999999999999e-06 2024-04-24T15:33:24.028 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.028 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.028 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:24.028 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965671+0000", 2024-04-24T15:33:24.028 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037028393 2024-04-24T15:33:24.028 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.029 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.029 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:24.029 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965752+0000", 2024-04-24T15:33:24.029 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0921999999999995e-05 2024-04-24T15:33:24.029 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.029 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.029 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:24.029 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965872+0000", 2024-04-24T15:33:24.029 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000119759 2024-04-24T15:33:24.029 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:24.029 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:24.029 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:24.029 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.029 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.029 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48728 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2441216~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:24.030 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928645+0000", 2024-04-24T15:33:24.030 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 13.039668391999999, 2024-04-24T15:33:24.030 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037346852999999999, 2024-04-24T15:33:24.030 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:24.030 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:24.030 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:24.030 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:24.030 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:24.030 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48728 2024-04-24T15:33:24.030 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.030 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:24.030 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.030 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:24.030 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928645+0000", 2024-04-24T15:33:24.031 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:24.031 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.031 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.031 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:24.031 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928645+0000", 2024-04-24T15:33:24.031 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:24.031 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.031 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.031 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:24.031 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928645+0000", 2024-04-24T15:33:24.031 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:24.031 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.031 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.031 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:24.031 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928646+0000", 2024-04-24T15:33:24.032 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.291e-06 2024-04-24T15:33:24.032 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.032 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.032 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:24.032 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928649+0000", 2024-04-24T15:33:24.032 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8509999999999999e-06 2024-04-24T15:33:24.032 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.032 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.032 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:24.032 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928651+0000", 2024-04-24T15:33:24.032 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.809e-06 2024-04-24T15:33:24.032 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.032 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.033 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:24.033 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965882+0000", 2024-04-24T15:33:24.033 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037231715999999998 2024-04-24T15:33:24.033 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.033 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.033 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:24.033 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965925+0000", 2024-04-24T15:33:24.033 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2493999999999997e-05 2024-04-24T15:33:24.033 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.033 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.033 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:24.033 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965992+0000", 2024-04-24T15:33:24.033 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.6907000000000002e-05 2024-04-24T15:33:24.033 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:24.033 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:24.034 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:24.034 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.034 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.034 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48729 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2506752~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:24.034 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928653+0000", 2024-04-24T15:33:24.034 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 13.039660517, 2024-04-24T15:33:24.034 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037576904000000001, 2024-04-24T15:33:24.034 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:24.034 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:24.034 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:24.034 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:24.034 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:24.034 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48729 2024-04-24T15:33:24.034 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.034 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:24.035 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.035 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:24.035 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928653+0000", 2024-04-24T15:33:24.035 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:24.035 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.035 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.035 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:24.035 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928653+0000", 2024-04-24T15:33:24.035 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:24.035 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.035 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.035 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:24.035 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928653+0000", 2024-04-24T15:33:24.035 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:24.035 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.035 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.036 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:24.036 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928654+0000", 2024-04-24T15:33:24.036 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3209999999999999e-06 2024-04-24T15:33:24.036 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.036 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.036 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:24.036 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928657+0000", 2024-04-24T15:33:24.036 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8559999999999998e-06 2024-04-24T15:33:24.036 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.036 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.036 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:24.036 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928659+0000", 2024-04-24T15:33:24.036 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7080000000000002e-06 2024-04-24T15:33:24.036 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.036 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.037 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:24.037 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966021+0000", 2024-04-24T15:33:24.037 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037361296000000002 2024-04-24T15:33:24.037 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.037 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.037 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:24.037 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966102+0000", 2024-04-24T15:33:24.037 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0939999999999994e-05 2024-04-24T15:33:24.037 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.037 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.037 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:24.037 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966230+0000", 2024-04-24T15:33:24.037 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00012799999999999999 2024-04-24T15:33:24.037 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:24.037 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:24.037 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:24.038 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.038 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.038 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48730 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2572288~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:24.038 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928662+0000", 2024-04-24T15:33:24.038 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 13.039651428999999, 2024-04-24T15:33:24.038 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037687246000000001, 2024-04-24T15:33:24.038 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:24.038 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:24.038 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:24.038 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:24.038 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:24.038 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48730 2024-04-24T15:33:24.038 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.038 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:24.038 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.039 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:24.039 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928662+0000", 2024-04-24T15:33:24.039 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:24.039 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.039 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.039 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:24.039 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928662+0000", 2024-04-24T15:33:24.039 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:24.039 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.039 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.039 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:24.039 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928662+0000", 2024-04-24T15:33:24.039 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:24.039 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.039 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.040 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:24.040 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928663+0000", 2024-04-24T15:33:24.040 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.381e-06 2024-04-24T15:33:24.040 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.040 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.040 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:24.040 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928666+0000", 2024-04-24T15:33:24.040 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.796e-06 2024-04-24T15:33:24.040 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.040 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.040 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:24.040 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928668+0000", 2024-04-24T15:33:24.040 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.8920000000000001e-06 2024-04-24T15:33:24.040 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.040 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.040 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:24.041 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966240+0000", 2024-04-24T15:33:24.041 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037572886999999999 2024-04-24T15:33:24.041 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.041 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.041 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:24.041 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966281+0000", 2024-04-24T15:33:24.041 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0450000000000001e-05 2024-04-24T15:33:24.041 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.041 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.041 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:24.041 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966349+0000", 2024-04-24T15:33:24.041 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.8215999999999999e-05 2024-04-24T15:33:24.041 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:24.041 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:24.041 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:24.041 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.042 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.042 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48731 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2637824~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:24.042 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928747+0000", 2024-04-24T15:33:24.042 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 13.039566424, 2024-04-24T15:33:24.042 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037833602000000001, 2024-04-24T15:33:24.042 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:24.042 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:24.042 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:24.042 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:24.042 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:24.042 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48731 2024-04-24T15:33:24.042 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.042 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:24.042 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.042 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:24.043 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928747+0000", 2024-04-24T15:33:24.043 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:24.043 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.043 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.043 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:24.043 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928747+0000", 2024-04-24T15:33:24.043 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:24.043 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.044 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.044 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:24.044 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928670+0000", 2024-04-24T15:33:24.044 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999228 2024-04-24T15:33:24.044 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.044 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.044 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:24.044 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928750+0000", 2024-04-24T15:33:24.044 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0527999999999999e-05 2024-04-24T15:33:24.044 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.045 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.045 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:24.045 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928757+0000", 2024-04-24T15:33:24.045 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.9419999999999998e-06 2024-04-24T15:33:24.045 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.045 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.045 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:24.046 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928762+0000", 2024-04-24T15:33:24.046 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4240000000000003e-06 2024-04-24T15:33:24.046 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.046 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.046 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:24.046 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966379+0000", 2024-04-24T15:33:24.047 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037617267000000003 2024-04-24T15:33:24.047 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.047 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.047 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:24.047 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966461+0000", 2024-04-24T15:33:24.047 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.2720999999999996e-05 2024-04-24T15:33:24.048 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.048 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.048 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:24.048 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966581+0000", 2024-04-24T15:33:24.048 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000119014 2024-04-24T15:33:24.048 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:24.049 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:24.049 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:24.049 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.049 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.049 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48732 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2703360~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:24.049 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928766+0000", 2024-04-24T15:33:24.049 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 13.039546939999999, 2024-04-24T15:33:24.049 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037930924999999997, 2024-04-24T15:33:24.049 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:24.049 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:24.049 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:24.049 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:24.049 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:24.049 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48732 2024-04-24T15:33:24.050 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.050 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:24.050 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.050 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:24.050 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928766+0000", 2024-04-24T15:33:24.050 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:24.050 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.050 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.050 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:24.050 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928766+0000", 2024-04-24T15:33:24.050 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:24.050 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.050 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.050 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:24.050 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928766+0000", 2024-04-24T15:33:24.051 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:24.051 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.051 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.051 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:24.051 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928768+0000", 2024-04-24T15:33:24.051 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.3420000000000002e-06 2024-04-24T15:33:24.051 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.051 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.051 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:24.051 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928783+0000", 2024-04-24T15:33:24.051 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.4813e-05 2024-04-24T15:33:24.051 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.051 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.051 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:24.051 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928787+0000", 2024-04-24T15:33:24.051 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.1999999999999996e-06 2024-04-24T15:33:24.052 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.052 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.052 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:24.052 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966592+0000", 2024-04-24T15:33:24.052 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037804484999999999 2024-04-24T15:33:24.052 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.052 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.052 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:24.052 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966632+0000", 2024-04-24T15:33:24.052 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0014e-05 2024-04-24T15:33:24.052 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.052 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.052 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:24.052 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966697+0000", 2024-04-24T15:33:24.052 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.5489999999999998e-05 2024-04-24T15:33:24.053 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:24.053 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:24.053 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:24.053 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.053 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.053 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48813 4.d 4:b39400aa:::rbd_data.114a8b543e91.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:24.053 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.936015+0000", 2024-04-24T15:33:24.053 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 13.032298605999999, 2024-04-24T15:33:24.053 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.039312294999999997, 2024-04-24T15:33:24.053 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:24.053 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:24.053 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:24.053 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:24.053 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:24.054 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48813 2024-04-24T15:33:24.054 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.054 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:24.054 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.054 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:24.054 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.936015+0000", 2024-04-24T15:33:24.054 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:24.054 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.054 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.054 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:24.054 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.936015+0000", 2024-04-24T15:33:24.054 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:24.054 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.054 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.055 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:24.055 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.936014+0000", 2024-04-24T15:33:24.055 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:24.055 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.055 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.055 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:24.055 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.942618+0000", 2024-04-24T15:33:24.055 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0066038499999999997 2024-04-24T15:33:24.055 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.055 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.055 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:24.055 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.942629+0000", 2024-04-24T15:33:24.055 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0606e-05 2024-04-24T15:33:24.055 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.055 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.055 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:24.056 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.942634+0000", 2024-04-24T15:33:24.056 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.2000000000000002e-06 2024-04-24T15:33:24.056 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.056 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.056 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:24.056 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.969006+0000", 2024-04-24T15:33:24.056 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.026371564 2024-04-24T15:33:24.056 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.056 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.056 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:24.056 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.969126+0000", 2024-04-24T15:33:24.056 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000120085 2024-04-24T15:33:24.056 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.056 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.056 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 2", 2024-04-24T15:33:24.056 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.969469+0000", 2024-04-24T15:33:24.057 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00034295400000000002 2024-04-24T15:33:24.057 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.057 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.057 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:33:24.057 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.973118+0000", 2024-04-24T15:33:24.057 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0036489700000000001 2024-04-24T15:33:24.057 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.057 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.057 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:33:24.057 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.975250+0000", 2024-04-24T15:33:24.057 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0021327030000000001 2024-04-24T15:33:24.057 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.057 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.057 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:33:24.058 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.975293+0000", 2024-04-24T15:33:24.058 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2951999999999998e-05 2024-04-24T15:33:24.058 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.058 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.058 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:24.058 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.975327+0000", 2024-04-24T15:33:24.058 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.3840000000000001e-05 2024-04-24T15:33:24.058 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:24.058 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:24.058 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:24.058 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:24.058 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:24.058 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:24.059 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #28, image image_client.0-clone28 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent29 2024-04-24T15:33:24.060 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-24T15:33:24.060 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:24.061+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:24.160 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:24.161+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:24.226 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:24.226 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:33:24.226 INFO:teuthology.orchestra.run.smithi151.stdout: "num_ops": 0 2024-04-24T15:33:24.226 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:24.236 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-24T15:33:24.261 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:24.262+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:24.298 INFO:teuthology.orchestra.run.smithi096.stdout:5592 trunc from 0x923f016 to 0xa1f9a46 2024-04-24T15:33:24.306 INFO:teuthology.orchestra.run.smithi096.stdout:5593 clone 31 order 20 su 524288 sc 5 2024-04-24T15:33:24.360 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:24.360 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:33:24.360 INFO:teuthology.orchestra.run.smithi151.stdout: "complaint_time": 30, 2024-04-24T15:33:24.360 INFO:teuthology.orchestra.run.smithi151.stdout: "num_blocked_ops": 0 2024-04-24T15:33:24.360 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:24.361 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:24.363+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:24.370 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-24T15:33:24.463 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:24.463+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:24.495 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:24.495 INFO:teuthology.orchestra.run.smithi151.stdout: "size": 20, 2024-04-24T15:33:24.495 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 600, 2024-04-24T15:33:24.495 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [ 2024-04-24T15:33:24.495 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.495 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23613 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 14688256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:24.495 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.792522+0000", 2024-04-24T15:33:24.496 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 49.702870329, 2024-04-24T15:33:24.496 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040507356000000001, 2024-04-24T15:33:24.496 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:24.496 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:24.496 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:24.496 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:24.496 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:24.496 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23613 2024-04-24T15:33:24.496 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.496 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:24.496 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.496 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:24.496 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792522+0000", 2024-04-24T15:33:24.496 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:24.496 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.497 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.497 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:24.497 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792522+0000", 2024-04-24T15:33:24.497 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:24.497 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.497 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.497 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:24.497 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792522+0000", 2024-04-24T15:33:24.497 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:24.497 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.497 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.497 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:24.497 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792523+0000", 2024-04-24T15:33:24.497 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.576e-06 2024-04-24T15:33:24.497 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.498 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.498 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:24.498 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792526+0000", 2024-04-24T15:33:24.498 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7439999999999999e-06 2024-04-24T15:33:24.498 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.498 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.498 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:24.498 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792528+0000", 2024-04-24T15:33:24.498 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.0600000000000002e-06 2024-04-24T15:33:24.498 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.498 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.498 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:24.498 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.832864+0000", 2024-04-24T15:33:24.498 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040335742000000001 2024-04-24T15:33:24.498 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.499 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.499 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:24.499 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.832953+0000", 2024-04-24T15:33:24.499 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.9281000000000001e-05 2024-04-24T15:33:24.499 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.499 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.499 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:24.499 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.833029+0000", 2024-04-24T15:33:24.499 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.6259000000000003e-05 2024-04-24T15:33:24.499 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:24.499 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:24.499 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:24.499 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.499 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.499 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23614 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 14729216~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:24.499 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.792530+0000", 2024-04-24T15:33:24.500 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 49.702862064999998, 2024-04-24T15:33:24.500 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040633957999999998, 2024-04-24T15:33:24.500 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:24.500 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:24.500 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:24.500 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:24.500 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:24.500 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23614 2024-04-24T15:33:24.500 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.500 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:24.500 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.500 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:24.500 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792530+0000", 2024-04-24T15:33:24.500 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:24.500 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.500 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.501 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:24.501 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792530+0000", 2024-04-24T15:33:24.501 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:24.501 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.501 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.501 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:24.501 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792530+0000", 2024-04-24T15:33:24.501 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:24.501 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.501 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.501 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:24.501 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792531+0000", 2024-04-24T15:33:24.501 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.46e-06 2024-04-24T15:33:24.501 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.501 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.502 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:24.502 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792534+0000", 2024-04-24T15:33:24.502 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8959999999999999e-06 2024-04-24T15:33:24.502 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.502 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.502 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:24.502 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792536+0000", 2024-04-24T15:33:24.502 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.841e-06 2024-04-24T15:33:24.502 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.502 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.502 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:24.502 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.833060+0000", 2024-04-24T15:33:24.502 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040524125000000001 2024-04-24T15:33:24.503 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.503 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.503 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:24.503 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.833108+0000", 2024-04-24T15:33:24.503 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.7547999999999999e-05 2024-04-24T15:33:24.503 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.503 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.503 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:24.503 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.833164+0000", 2024-04-24T15:33:24.503 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6297e-05 2024-04-24T15:33:24.503 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:24.503 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:24.503 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:24.503 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.503 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.503 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23623 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15097856~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:24.504 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.793887+0000", 2024-04-24T15:33:24.504 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 49.701505103999999, 2024-04-24T15:33:24.504 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040432784999999999, 2024-04-24T15:33:24.504 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:24.504 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:24.504 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:24.504 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:24.504 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:24.504 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23623 2024-04-24T15:33:24.504 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.504 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:24.504 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.504 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:24.504 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793887+0000", 2024-04-24T15:33:24.504 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:24.504 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.505 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.505 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:24.505 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793887+0000", 2024-04-24T15:33:24.505 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:24.505 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.505 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.505 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:24.505 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793887+0000", 2024-04-24T15:33:24.505 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:24.505 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.505 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.505 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:24.505 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793888+0000", 2024-04-24T15:33:24.505 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3659999999999999e-06 2024-04-24T15:33:24.505 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.505 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.506 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:24.506 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793891+0000", 2024-04-24T15:33:24.506 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8389999999999998e-06 2024-04-24T15:33:24.506 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.506 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.506 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:24.506 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793893+0000", 2024-04-24T15:33:24.506 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.9549999999999999e-06 2024-04-24T15:33:24.506 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.506 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.506 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:24.506 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834239+0000", 2024-04-24T15:33:24.506 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040345487999999999 2024-04-24T15:33:24.506 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.507 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.507 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:24.507 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834280+0000", 2024-04-24T15:33:24.507 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.1878000000000001e-05 2024-04-24T15:33:24.507 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.507 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.507 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:24.507 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834320+0000", 2024-04-24T15:33:24.507 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9480000000000001e-05 2024-04-24T15:33:24.507 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:24.507 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:24.507 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:24.507 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.507 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.507 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23624 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15138816~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:24.507 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.794053+0000", 2024-04-24T15:33:24.508 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 49.701339156000003, 2024-04-24T15:33:24.508 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040370801999999997, 2024-04-24T15:33:24.508 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:24.508 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:24.508 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:24.508 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:24.508 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:24.508 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23624 2024-04-24T15:33:24.508 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.508 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:24.508 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.508 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:24.508 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794053+0000", 2024-04-24T15:33:24.508 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:24.508 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.509 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.509 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:24.509 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794053+0000", 2024-04-24T15:33:24.509 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:24.509 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.509 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.509 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:24.509 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793895+0000", 2024-04-24T15:33:24.509 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998422 2024-04-24T15:33:24.509 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.509 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.509 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:24.509 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794057+0000", 2024-04-24T15:33:24.509 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00016149400000000001 2024-04-24T15:33:24.509 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.509 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.510 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:24.510 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794065+0000", 2024-04-24T15:33:24.510 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.5259999999999993e-06 2024-04-24T15:33:24.510 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.510 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.510 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:24.510 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794071+0000", 2024-04-24T15:33:24.510 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6149999999999996e-06 2024-04-24T15:33:24.510 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.510 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.510 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:24.510 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834341+0000", 2024-04-24T15:33:24.510 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040270366000000002 2024-04-24T15:33:24.510 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.511 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.511 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:24.511 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834381+0000", 2024-04-24T15:33:24.511 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0102999999999999e-05 2024-04-24T15:33:24.511 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.511 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.511 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:24.511 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834424+0000", 2024-04-24T15:33:24.511 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2713000000000003e-05 2024-04-24T15:33:24.511 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:24.511 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:24.511 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:24.511 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.511 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.511 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23625 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15179776~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:24.511 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.794160+0000", 2024-04-24T15:33:24.512 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 49.701232273999999, 2024-04-24T15:33:24.512 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040413004000000002, 2024-04-24T15:33:24.512 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:24.512 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:24.512 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:24.512 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:24.512 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:24.512 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23625 2024-04-24T15:33:24.512 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.512 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:24.512 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.512 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:24.512 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794160+0000", 2024-04-24T15:33:24.512 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:24.512 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.513 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.513 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:24.513 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794160+0000", 2024-04-24T15:33:24.513 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:24.513 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.513 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.513 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:24.513 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794086+0000", 2024-04-24T15:33:24.513 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999256 2024-04-24T15:33:24.513 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.513 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.513 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:24.513 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794163+0000", 2024-04-24T15:33:24.513 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7896000000000001e-05 2024-04-24T15:33:24.513 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.514 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.514 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:24.514 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794171+0000", 2024-04-24T15:33:24.514 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.9500000000000001e-06 2024-04-24T15:33:24.514 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.514 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.514 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:24.514 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794179+0000", 2024-04-24T15:33:24.514 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7970000000000001e-06 2024-04-24T15:33:24.514 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.514 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.514 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:24.514 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834451+0000", 2024-04-24T15:33:24.514 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040271492999999998 2024-04-24T15:33:24.514 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.514 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.515 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:24.515 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834519+0000", 2024-04-24T15:33:24.515 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.7984000000000005e-05 2024-04-24T15:33:24.515 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.515 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.515 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:24.515 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834573+0000", 2024-04-24T15:33:24.515 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.4339999999999998e-05 2024-04-24T15:33:24.515 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:24.515 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:24.515 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:24.515 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.515 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.515 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23629 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15343616~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:24.515 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.794748+0000", 2024-04-24T15:33:24.516 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 49.700644472, 2024-04-24T15:33:24.516 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040419718, 2024-04-24T15:33:24.516 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:24.516 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:24.516 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:24.516 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:24.516 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:24.516 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23629 2024-04-24T15:33:24.516 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.516 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:24.516 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.516 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:24.516 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794748+0000", 2024-04-24T15:33:24.516 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:24.517 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.517 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.517 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:24.517 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794748+0000", 2024-04-24T15:33:24.517 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:24.517 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.517 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.517 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:24.517 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794673+0000", 2024-04-24T15:33:24.517 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999251 2024-04-24T15:33:24.517 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.517 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.517 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:24.517 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794751+0000", 2024-04-24T15:33:24.518 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.8480999999999996e-05 2024-04-24T15:33:24.518 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.518 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.518 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:24.518 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794760+0000", 2024-04-24T15:33:24.518 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3429999999999996e-06 2024-04-24T15:33:24.518 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.518 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.518 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:24.518 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794764+0000", 2024-04-24T15:33:24.518 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5600000000000004e-06 2024-04-24T15:33:24.518 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.518 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.518 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:24.518 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835040+0000", 2024-04-24T15:33:24.519 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040275188000000003 2024-04-24T15:33:24.519 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.519 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.519 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:24.519 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835105+0000", 2024-04-24T15:33:24.519 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.5992000000000001e-05 2024-04-24T15:33:24.519 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.519 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.519 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:24.519 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835167+0000", 2024-04-24T15:33:24.519 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.1981000000000002e-05 2024-04-24T15:33:24.519 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:24.519 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:24.519 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:24.519 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.519 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.520 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23630 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15384576~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:24.520 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.794839+0000", 2024-04-24T15:33:24.520 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 49.700553403000001, 2024-04-24T15:33:24.520 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040523894999999997, 2024-04-24T15:33:24.520 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:24.520 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:24.520 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:24.520 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:24.520 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:24.520 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23630 2024-04-24T15:33:24.520 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.520 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:24.520 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.520 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:24.520 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794839+0000", 2024-04-24T15:33:24.521 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:24.521 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.521 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.521 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:24.521 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794839+0000", 2024-04-24T15:33:24.521 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:24.521 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.521 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.521 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:24.521 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794768+0000", 2024-04-24T15:33:24.521 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999294 2024-04-24T15:33:24.521 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.521 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.522 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:24.522 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794841+0000", 2024-04-24T15:33:24.522 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2793000000000006e-05 2024-04-24T15:33:24.522 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.522 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.522 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:24.522 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794846+0000", 2024-04-24T15:33:24.522 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.7230000000000004e-06 2024-04-24T15:33:24.522 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.522 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.522 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:24.522 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794848+0000", 2024-04-24T15:33:24.522 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7659999999999999e-06 2024-04-24T15:33:24.522 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.523 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.523 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:24.523 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835223+0000", 2024-04-24T15:33:24.523 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040374275000000001 2024-04-24T15:33:24.523 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.523 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.523 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:24.523 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835295+0000", 2024-04-24T15:33:24.523 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2633999999999996e-05 2024-04-24T15:33:24.523 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.523 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.523 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:24.523 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835363+0000", 2024-04-24T15:33:24.523 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.7355999999999997e-05 2024-04-24T15:33:24.523 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:24.524 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:24.524 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:24.524 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.524 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.524 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23631 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15425536~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:24.524 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.794956+0000", 2024-04-24T15:33:24.524 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 49.700436562, 2024-04-24T15:33:24.524 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040562383, 2024-04-24T15:33:24.524 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:24.524 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:24.524 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:24.524 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:24.524 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:24.524 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23631 2024-04-24T15:33:24.524 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.525 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:24.525 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.525 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:24.525 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794956+0000", 2024-04-24T15:33:24.525 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:24.525 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.525 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.525 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:24.525 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794956+0000", 2024-04-24T15:33:24.525 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:24.525 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.525 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.525 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:24.525 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794859+0000", 2024-04-24T15:33:24.525 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999032 2024-04-24T15:33:24.525 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.526 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.526 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:24.526 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794959+0000", 2024-04-24T15:33:24.526 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000100207 2024-04-24T15:33:24.526 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.526 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.526 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:24.526 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794967+0000", 2024-04-24T15:33:24.526 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7640000000000005e-06 2024-04-24T15:33:24.526 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.526 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.526 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:24.526 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794971+0000", 2024-04-24T15:33:24.527 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.335e-06 2024-04-24T15:33:24.527 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.527 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.527 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:24.527 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835388+0000", 2024-04-24T15:33:24.527 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040417340000000003 2024-04-24T15:33:24.527 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.527 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.527 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:24.527 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835459+0000", 2024-04-24T15:33:24.527 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.0118999999999997e-05 2024-04-24T15:33:24.527 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.527 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.527 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:24.527 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835518+0000", 2024-04-24T15:33:24.527 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.9438999999999998e-05 2024-04-24T15:33:24.528 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:24.528 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:24.528 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:24.528 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.528 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.528 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23632 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15466496~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:24.528 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795060+0000", 2024-04-24T15:33:24.528 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 49.700331851999998, 2024-04-24T15:33:24.528 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040667993, 2024-04-24T15:33:24.528 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:24.528 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:24.528 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:24.528 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:24.528 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:24.528 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23632 2024-04-24T15:33:24.528 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.529 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:24.529 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.529 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:24.529 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795060+0000", 2024-04-24T15:33:24.529 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:24.529 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.529 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.529 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:24.529 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795060+0000", 2024-04-24T15:33:24.529 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:24.529 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.529 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.529 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:24.529 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794976+0000", 2024-04-24T15:33:24.529 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999156 2024-04-24T15:33:24.530 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.530 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.530 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:24.530 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795064+0000", 2024-04-24T15:33:24.530 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.7709999999999999e-05 2024-04-24T15:33:24.530 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.530 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.530 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:24.530 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795072+0000", 2024-04-24T15:33:24.530 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3769999999999996e-06 2024-04-24T15:33:24.530 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.530 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.530 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:24.530 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795078+0000", 2024-04-24T15:33:24.531 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.5079999999999996e-06 2024-04-24T15:33:24.531 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.531 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.531 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:24.531 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835541+0000", 2024-04-24T15:33:24.531 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040463440000000003 2024-04-24T15:33:24.531 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.531 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.531 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:24.531 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835657+0000", 2024-04-24T15:33:24.531 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00011572499999999999 2024-04-24T15:33:24.531 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.531 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.531 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:24.531 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835728+0000", 2024-04-24T15:33:24.531 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1402000000000005e-05 2024-04-24T15:33:24.532 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:24.532 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:24.532 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:24.532 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.532 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.532 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23633 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15507456~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:24.532 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795134+0000", 2024-04-24T15:33:24.532 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 49.700258196999997, 2024-04-24T15:33:24.532 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040748591000000001, 2024-04-24T15:33:24.532 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:24.532 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:24.532 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:24.532 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:24.532 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:24.533 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23633 2024-04-24T15:33:24.533 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.533 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:24.533 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.533 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:24.533 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795134+0000", 2024-04-24T15:33:24.533 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:24.533 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.533 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.533 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:24.533 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795134+0000", 2024-04-24T15:33:24.533 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:24.533 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.533 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.533 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:24.533 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795083+0000", 2024-04-24T15:33:24.534 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999949 2024-04-24T15:33:24.534 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.534 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.534 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:24.534 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795138+0000", 2024-04-24T15:33:24.534 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.4472000000000003e-05 2024-04-24T15:33:24.534 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.534 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.534 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:24.534 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795145+0000", 2024-04-24T15:33:24.534 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2779999999999996e-06 2024-04-24T15:33:24.534 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.534 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.535 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:24.535 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795149+0000", 2024-04-24T15:33:24.535 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.568e-06 2024-04-24T15:33:24.535 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.535 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.535 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:24.535 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835757+0000", 2024-04-24T15:33:24.535 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040608894999999999 2024-04-24T15:33:24.535 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.535 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.535 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:24.535 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835829+0000", 2024-04-24T15:33:24.535 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1879999999999996e-05 2024-04-24T15:33:24.535 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.535 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.536 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:24.536 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835883+0000", 2024-04-24T15:33:24.536 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.3346000000000001e-05 2024-04-24T15:33:24.536 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:24.536 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:24.536 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:24.536 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.536 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.536 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23634 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15548416~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:24.536 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795271+0000", 2024-04-24T15:33:24.536 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 49.700121162999999, 2024-04-24T15:33:24.536 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040755845999999998, 2024-04-24T15:33:24.536 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:24.536 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:24.537 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:24.537 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:24.537 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:24.537 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23634 2024-04-24T15:33:24.537 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.537 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:24.537 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.537 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:24.537 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795271+0000", 2024-04-24T15:33:24.537 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:24.537 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.537 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.537 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:24.537 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795271+0000", 2024-04-24T15:33:24.537 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:24.538 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.538 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.538 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:24.538 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795152+0000", 2024-04-24T15:33:24.538 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998813 2024-04-24T15:33:24.538 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.538 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.538 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:24.538 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795276+0000", 2024-04-24T15:33:24.538 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00012401799999999999 2024-04-24T15:33:24.538 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.538 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.538 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:24.538 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795281+0000", 2024-04-24T15:33:24.539 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8860000000000003e-06 2024-04-24T15:33:24.539 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.539 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.539 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:24.539 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795284+0000", 2024-04-24T15:33:24.539 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.0450000000000001e-06 2024-04-24T15:33:24.539 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.539 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.539 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:24.539 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835905+0000", 2024-04-24T15:33:24.539 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040621035999999999 2024-04-24T15:33:24.539 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.539 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.539 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:24.539 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835971+0000", 2024-04-24T15:33:24.540 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.5314000000000005e-05 2024-04-24T15:33:24.540 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.540 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.540 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:24.540 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836027+0000", 2024-04-24T15:33:24.540 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6305999999999999e-05 2024-04-24T15:33:24.540 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:24.540 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:24.540 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:24.540 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.540 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.540 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23635 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15589376~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:24.540 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795382+0000", 2024-04-24T15:33:24.540 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 49.700010390000003, 2024-04-24T15:33:24.541 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040805363999999997, 2024-04-24T15:33:24.541 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:24.541 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:24.541 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:24.541 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:24.541 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:24.541 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23635 2024-04-24T15:33:24.541 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.541 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:24.541 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.541 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:24.541 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795382+0000", 2024-04-24T15:33:24.541 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:24.541 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.541 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.542 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:24.542 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795382+0000", 2024-04-24T15:33:24.542 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:24.542 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.542 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.542 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:24.542 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795288+0000", 2024-04-24T15:33:24.542 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999056 2024-04-24T15:33:24.542 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.542 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.542 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:24.542 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795385+0000", 2024-04-24T15:33:24.543 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.7288999999999996e-05 2024-04-24T15:33:24.543 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.543 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.543 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:24.543 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795392+0000", 2024-04-24T15:33:24.543 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.452e-06 2024-04-24T15:33:24.543 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.543 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.543 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:24.543 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795397+0000", 2024-04-24T15:33:24.543 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2860000000000001e-06 2024-04-24T15:33:24.543 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.543 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.543 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:24.544 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836054+0000", 2024-04-24T15:33:24.544 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040657581999999998 2024-04-24T15:33:24.544 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.544 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.544 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:24.544 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836124+0000", 2024-04-24T15:33:24.544 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.0020999999999999e-05 2024-04-24T15:33:24.544 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.544 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.544 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:24.544 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836187+0000", 2024-04-24T15:33:24.544 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.3009999999999995e-05 2024-04-24T15:33:24.544 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:24.544 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:24.545 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:24.545 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.545 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.545 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23636 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15630336~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:24.545 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795610+0000", 2024-04-24T15:33:24.545 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 49.699782032000002, 2024-04-24T15:33:24.545 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040730942999999999, 2024-04-24T15:33:24.545 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:24.545 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:24.545 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:24.545 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:24.545 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:24.545 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23636 2024-04-24T15:33:24.545 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.545 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:24.545 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.546 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:24.546 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795610+0000", 2024-04-24T15:33:24.546 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:24.546 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.546 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.546 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:24.546 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795610+0000", 2024-04-24T15:33:24.546 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:24.546 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.546 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.546 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:24.546 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795401+0000", 2024-04-24T15:33:24.546 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9997911 2024-04-24T15:33:24.547 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.547 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.547 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:24.547 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795614+0000", 2024-04-24T15:33:24.547 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00021228600000000001 2024-04-24T15:33:24.547 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.547 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.547 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:24.547 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795622+0000", 2024-04-24T15:33:24.547 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3920000000000003e-06 2024-04-24T15:33:24.547 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.547 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.547 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:24.547 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795627+0000", 2024-04-24T15:33:24.548 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8199999999999996e-06 2024-04-24T15:33:24.548 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.548 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.548 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:24.548 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836211+0000", 2024-04-24T15:33:24.548 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040583803000000002 2024-04-24T15:33:24.548 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.548 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.548 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:24.548 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836284+0000", 2024-04-24T15:33:24.548 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.3617999999999999e-05 2024-04-24T15:33:24.548 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.548 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.548 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:24.548 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836341+0000", 2024-04-24T15:33:24.549 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6959999999999997e-05 2024-04-24T15:33:24.549 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:24.549 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:24.549 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:24.549 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.549 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.549 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23637 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15671296~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:24.549 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795699+0000", 2024-04-24T15:33:24.549 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 49.699692865000003, 2024-04-24T15:33:24.549 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040775556999999997, 2024-04-24T15:33:24.549 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:24.549 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:24.549 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:24.549 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:24.550 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:24.550 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23637 2024-04-24T15:33:24.550 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.550 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:24.550 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.550 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:24.550 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795699+0000", 2024-04-24T15:33:24.550 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:24.550 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.550 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.550 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:24.550 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795699+0000", 2024-04-24T15:33:24.550 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:24.551 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.551 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.551 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:24.551 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795632+0000", 2024-04-24T15:33:24.551 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999323 2024-04-24T15:33:24.551 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.551 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.551 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:24.551 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795704+0000", 2024-04-24T15:33:24.551 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2428000000000005e-05 2024-04-24T15:33:24.551 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.551 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.551 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:24.551 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795715+0000", 2024-04-24T15:33:24.551 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.1532000000000001e-05 2024-04-24T15:33:24.552 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.552 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.552 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:24.552 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795721+0000", 2024-04-24T15:33:24.552 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.8200000000000002e-06 2024-04-24T15:33:24.552 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.552 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.552 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:24.552 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836364+0000", 2024-04-24T15:33:24.552 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040642997 2024-04-24T15:33:24.552 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.552 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.552 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:24.552 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836435+0000", 2024-04-24T15:33:24.552 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.0821000000000005e-05 2024-04-24T15:33:24.553 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.553 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.553 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:24.553 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836475+0000", 2024-04-24T15:33:24.553 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9796999999999999e-05 2024-04-24T15:33:24.553 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:24.553 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:24.553 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:24.553 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.553 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.553 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23638 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15712256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:24.553 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795932+0000", 2024-04-24T15:33:24.553 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 49.699459834999999, 2024-04-24T15:33:24.553 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040668682999999997, 2024-04-24T15:33:24.553 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:24.553 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:24.554 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:24.554 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:24.554 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:24.554 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23638 2024-04-24T15:33:24.554 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.554 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:24.554 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.554 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:24.554 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795932+0000", 2024-04-24T15:33:24.554 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:24.554 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.554 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.554 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:24.554 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795932+0000", 2024-04-24T15:33:24.555 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:24.555 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.555 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.555 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:24.555 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795728+0000", 2024-04-24T15:33:24.555 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9997959 2024-04-24T15:33:24.555 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.555 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.555 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:24.555 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795936+0000", 2024-04-24T15:33:24.555 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000208128 2024-04-24T15:33:24.555 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.555 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.555 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:24.556 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795945+0000", 2024-04-24T15:33:24.556 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.7110000000000001e-06 2024-04-24T15:33:24.556 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.556 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.556 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:24.556 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795957+0000", 2024-04-24T15:33:24.556 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.1800999999999999e-05 2024-04-24T15:33:24.556 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.556 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.556 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:24.556 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836505+0000", 2024-04-24T15:33:24.556 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040547622999999998 2024-04-24T15:33:24.556 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.556 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.556 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:24.557 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836561+0000", 2024-04-24T15:33:24.557 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6598999999999998e-05 2024-04-24T15:33:24.557 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.557 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.557 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:24.557 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836601+0000", 2024-04-24T15:33:24.557 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9907000000000003e-05 2024-04-24T15:33:24.557 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:24.557 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:24.557 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:24.557 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.557 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.557 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23639 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15753216~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:24.557 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.796071+0000", 2024-04-24T15:33:24.558 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 49.699321574000002, 2024-04-24T15:33:24.558 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040648199000000003, 2024-04-24T15:33:24.558 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:24.558 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:24.558 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:24.558 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:24.558 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:24.558 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23639 2024-04-24T15:33:24.558 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.558 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:24.558 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.558 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:24.558 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796071+0000", 2024-04-24T15:33:24.559 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:24.559 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.559 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.559 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:24.559 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796071+0000", 2024-04-24T15:33:24.559 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:24.559 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.559 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.559 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:24.559 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795967+0000", 2024-04-24T15:33:24.559 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998965 2024-04-24T15:33:24.559 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.559 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.559 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:24.559 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796074+0000", 2024-04-24T15:33:24.560 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000107013 2024-04-24T15:33:24.560 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.560 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.560 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:24.560 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796081+0000", 2024-04-24T15:33:24.560 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2590000000000003e-06 2024-04-24T15:33:24.560 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.560 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.560 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:24.560 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796085+0000", 2024-04-24T15:33:24.560 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5269999999999999e-06 2024-04-24T15:33:24.560 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.560 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.560 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:24.561 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836624+0000", 2024-04-24T15:33:24.561 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040539324000000002 2024-04-24T15:33:24.561 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.561 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.561 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:24.561 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836667+0000", 2024-04-24T15:33:24.561 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2562999999999999e-05 2024-04-24T15:33:24.561 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.561 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.561 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:24.561 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836719+0000", 2024-04-24T15:33:24.561 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.2015999999999998e-05 2024-04-24T15:33:24.561 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:24.561 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:24.561 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:24.562 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.562 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.562 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23640 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15794176~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:24.562 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.796492+0000", 2024-04-24T15:33:24.562 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 49.698899816000001, 2024-04-24T15:33:24.562 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040420000999999997, 2024-04-24T15:33:24.562 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:24.562 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:24.562 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:24.562 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:24.562 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:24.562 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23640 2024-04-24T15:33:24.563 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.563 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:24.563 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.563 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:24.563 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796492+0000", 2024-04-24T15:33:24.563 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:24.563 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.563 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.563 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:24.563 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796492+0000", 2024-04-24T15:33:24.563 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:24.563 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.563 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.563 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:24.564 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796088+0000", 2024-04-24T15:33:24.564 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9995961 2024-04-24T15:33:24.564 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.564 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.564 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:24.564 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796496+0000", 2024-04-24T15:33:24.564 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00040785700000000002 2024-04-24T15:33:24.564 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.564 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.564 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:24.564 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796503+0000", 2024-04-24T15:33:24.564 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1049999999999997e-06 2024-04-24T15:33:24.564 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.565 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.565 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:24.565 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796508+0000", 2024-04-24T15:33:24.565 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.155e-06 2024-04-24T15:33:24.565 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.565 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.565 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:24.565 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836756+0000", 2024-04-24T15:33:24.565 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040248850000000003 2024-04-24T15:33:24.565 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.565 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.565 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:24.565 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836841+0000", 2024-04-24T15:33:24.565 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.4735999999999996e-05 2024-04-24T15:33:24.566 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.566 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.566 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:24.566 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836912+0000", 2024-04-24T15:33:24.566 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1229999999999994e-05 2024-04-24T15:33:24.566 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:24.566 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:24.566 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:24.566 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.566 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.566 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23641 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15835136~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:24.566 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.796658+0000", 2024-04-24T15:33:24.566 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 49.698733740000002, 2024-04-24T15:33:24.567 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040420706000000001, 2024-04-24T15:33:24.567 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:24.567 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:24.567 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:24.567 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:24.567 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:24.567 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23641 2024-04-24T15:33:24.567 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.567 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:24.567 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.567 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:24.567 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796658+0000", 2024-04-24T15:33:24.567 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:24.568 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.568 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.568 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:24.568 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796658+0000", 2024-04-24T15:33:24.568 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:24.568 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.568 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.568 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:24.568 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796512+0000", 2024-04-24T15:33:24.568 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998531 2024-04-24T15:33:24.568 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.568 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.568 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:24.568 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796669+0000", 2024-04-24T15:33:24.568 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000157577 2024-04-24T15:33:24.568 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.569 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.569 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:24.569 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796677+0000", 2024-04-24T15:33:24.569 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7800000000000001e-06 2024-04-24T15:33:24.569 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.569 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.569 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:24.569 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796681+0000", 2024-04-24T15:33:24.569 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.721e-06 2024-04-24T15:33:24.569 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.569 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.569 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:24.569 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836941+0000", 2024-04-24T15:33:24.570 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040259907999999997 2024-04-24T15:33:24.570 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.570 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.570 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:24.570 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837017+0000", 2024-04-24T15:33:24.570 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.6071999999999999e-05 2024-04-24T15:33:24.570 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.570 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.570 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:24.570 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837079+0000", 2024-04-24T15:33:24.570 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.2358000000000002e-05 2024-04-24T15:33:24.570 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:24.570 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:24.571 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:24.571 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.571 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.571 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23645 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15998976~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:24.571 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.797360+0000", 2024-04-24T15:33:24.571 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 49.698032241999996, 2024-04-24T15:33:24.571 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040387645999999999, 2024-04-24T15:33:24.571 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:24.571 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:24.571 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:24.571 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:24.571 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:24.571 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23645 2024-04-24T15:33:24.571 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.572 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:24.572 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.572 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:24.572 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797360+0000", 2024-04-24T15:33:24.572 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:24.572 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.572 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.572 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:24.572 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797360+0000", 2024-04-24T15:33:24.572 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:24.572 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.572 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.572 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:24.572 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797288+0000", 2024-04-24T15:33:24.572 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999928 2024-04-24T15:33:24.572 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.573 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.573 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:24.573 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797364+0000", 2024-04-24T15:33:24.573 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.5347999999999998e-05 2024-04-24T15:33:24.573 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.573 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.573 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:24.573 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797372+0000", 2024-04-24T15:33:24.573 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.2409999999999995e-06 2024-04-24T15:33:24.573 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.573 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.573 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:24.573 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797376+0000", 2024-04-24T15:33:24.573 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.515e-06 2024-04-24T15:33:24.574 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.574 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.574 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:24.574 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837586+0000", 2024-04-24T15:33:24.574 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040210047999999998 2024-04-24T15:33:24.574 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.574 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.574 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:24.574 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837672+0000", 2024-04-24T15:33:24.574 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.5997999999999999e-05 2024-04-24T15:33:24.574 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.574 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.574 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:24.575 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837748+0000", 2024-04-24T15:33:24.575 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.5328999999999997e-05 2024-04-24T15:33:24.575 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:24.575 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:24.575 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:24.575 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.575 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.575 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23646 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 16039936~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:24.575 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.797498+0000", 2024-04-24T15:33:24.575 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 49.697894140999999, 2024-04-24T15:33:24.575 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040433139, 2024-04-24T15:33:24.575 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:24.575 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:24.575 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:24.575 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:24.576 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:24.576 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23646 2024-04-24T15:33:24.576 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.576 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:24.576 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.576 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:24.576 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797498+0000", 2024-04-24T15:33:24.576 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:24.576 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.576 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.576 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:24.576 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797498+0000", 2024-04-24T15:33:24.576 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:24.576 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.577 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.577 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:24.577 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797381+0000", 2024-04-24T15:33:24.577 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998832 2024-04-24T15:33:24.577 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.577 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.577 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:24.577 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797501+0000", 2024-04-24T15:33:24.577 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000119759 2024-04-24T15:33:24.577 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.577 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.577 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:24.577 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797510+0000", 2024-04-24T15:33:24.577 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.4770000000000003e-06 2024-04-24T15:33:24.578 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.578 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.578 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:24.578 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797515+0000", 2024-04-24T15:33:24.578 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8330000000000001e-06 2024-04-24T15:33:24.578 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.578 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.578 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:24.578 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837777+0000", 2024-04-24T15:33:24.578 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040262502999999998 2024-04-24T15:33:24.578 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.578 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.578 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:24.578 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837860+0000", 2024-04-24T15:33:24.579 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3002000000000003e-05 2024-04-24T15:33:24.579 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:24.579 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:24.579 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:24.579 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837931+0000", 2024-04-24T15:33:24.579 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1222000000000004e-05 2024-04-24T15:33:24.579 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:24.579 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:24.579 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:24.579 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:24.579 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:24.579 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:24.582 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:33:24.583+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:33:24.583 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-24T15:33:24.683 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:24.683+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:24.747 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:24.747 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:33:24.747 INFO:teuthology.orchestra.run.smithi151.stdout: "num_ops": 0 2024-04-24T15:33:24.747 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:24.757 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-24T15:33:24.785 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:24.784+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:24.884 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:24.885+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:24.924 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:24.924 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:33:24.924 INFO:teuthology.orchestra.run.smithi151.stdout: "complaint_time": 30, 2024-04-24T15:33:24.924 INFO:teuthology.orchestra.run.smithi151.stdout: "num_blocked_ops": 0 2024-04-24T15:33:24.924 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:24.934 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-24T15:33:24.935 INFO:teuthology.orchestra.run.smithi096.stdout:truncating image image_client.0-clone30 from 0xa1f9a46 (overlap 0x923f016) to 0x0 2024-04-24T15:33:24.984 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:24.986+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:25.050 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #29, image image_client.0-clone29 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent30 2024-04-24T15:33:25.071 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:25.071 INFO:teuthology.orchestra.run.smithi151.stdout: "size": 20, 2024-04-24T15:33:25.071 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 600, 2024-04-24T15:33:25.071 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [ 2024-04-24T15:33:25.071 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.071 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18006 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12419072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:25.071 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.207911+0000", 2024-04-24T15:33:25.071 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 50.862925766000004, 2024-04-24T15:33:25.071 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042899496000000002, 2024-04-24T15:33:25.071 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:25.071 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:25.071 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:25.071 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:25.072 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:25.072 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18006 2024-04-24T15:33:25.072 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.072 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:25.072 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.072 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:25.072 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207911+0000", 2024-04-24T15:33:25.072 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:25.072 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.072 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.072 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:25.072 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207911+0000", 2024-04-24T15:33:25.072 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:25.073 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.073 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.073 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:25.073 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207910+0000", 2024-04-24T15:33:25.073 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:25.073 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.073 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.073 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:25.073 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207913+0000", 2024-04-24T15:33:25.073 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.2309999999999999e-06 2024-04-24T15:33:25.073 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.073 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.074 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:25.074 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207918+0000", 2024-04-24T15:33:25.074 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.3399999999999997e-06 2024-04-24T15:33:25.074 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.074 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.074 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:25.074 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207922+0000", 2024-04-24T15:33:25.074 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5439999999999999e-06 2024-04-24T15:33:25.074 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.074 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.074 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:25.074 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250726+0000", 2024-04-24T15:33:25.074 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042804129000000003 2024-04-24T15:33:25.074 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.075 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.075 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:25.075 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250765+0000", 2024-04-24T15:33:25.075 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9554000000000001e-05 2024-04-24T15:33:25.075 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.075 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.075 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:25.075 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250810+0000", 2024-04-24T15:33:25.075 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5027000000000001e-05 2024-04-24T15:33:25.075 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:25.075 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:25.075 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:25.075 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.076 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.076 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18007 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12460032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:25.076 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.207961+0000", 2024-04-24T15:33:25.076 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 50.862875924999997, 2024-04-24T15:33:25.076 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042965470999999998, 2024-04-24T15:33:25.076 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:25.076 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:25.076 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:25.076 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:25.076 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:25.076 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18007 2024-04-24T15:33:25.076 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.076 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:25.076 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.076 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:25.077 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207961+0000", 2024-04-24T15:33:25.077 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:25.077 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.077 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.077 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:25.077 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207961+0000", 2024-04-24T15:33:25.077 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:25.077 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.077 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.077 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:25.077 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207960+0000", 2024-04-24T15:33:25.077 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:25.077 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.077 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.078 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:25.078 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207963+0000", 2024-04-24T15:33:25.078 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.3759999999999999e-06 2024-04-24T15:33:25.078 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.078 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.078 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:25.078 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207970+0000", 2024-04-24T15:33:25.078 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.0489999999999998e-06 2024-04-24T15:33:25.078 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.078 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.078 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:25.078 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207975+0000", 2024-04-24T15:33:25.078 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.1640000000000003e-06 2024-04-24T15:33:25.078 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.079 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.079 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:25.079 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250834+0000", 2024-04-24T15:33:25.079 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042859060999999997 2024-04-24T15:33:25.079 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.079 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.079 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:25.079 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250874+0000", 2024-04-24T15:33:25.079 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0674999999999999e-05 2024-04-24T15:33:25.079 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.079 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.079 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:25.079 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250926+0000", 2024-04-24T15:33:25.079 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.1693000000000001e-05 2024-04-24T15:33:25.080 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:25.080 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:25.080 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:25.080 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.080 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.080 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18008 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12500992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:25.080 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208030+0000", 2024-04-24T15:33:25.080 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 50.862806831999997, 2024-04-24T15:33:25.080 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043003337000000003, 2024-04-24T15:33:25.080 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:25.080 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:25.080 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:25.080 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:25.080 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:25.081 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18008 2024-04-24T15:33:25.081 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.081 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:25.081 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.081 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:25.081 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208030+0000", 2024-04-24T15:33:25.081 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:25.081 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.081 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.081 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:25.081 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208030+0000", 2024-04-24T15:33:25.081 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:25.081 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.081 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.082 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:25.082 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208029+0000", 2024-04-24T15:33:25.082 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:25.082 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.082 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.082 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:25.082 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208032+0000", 2024-04-24T15:33:25.082 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.117e-06 2024-04-24T15:33:25.082 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.082 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.082 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:25.082 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208039+0000", 2024-04-24T15:33:25.083 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.0129999999999996e-06 2024-04-24T15:33:25.083 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.083 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.083 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:25.083 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208043+0000", 2024-04-24T15:33:25.083 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0600000000000001e-06 2024-04-24T15:33:25.083 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.083 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.083 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:25.083 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250948+0000", 2024-04-24T15:33:25.083 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042904577999999999 2024-04-24T15:33:25.083 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.083 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.083 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:25.083 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250989+0000", 2024-04-24T15:33:25.084 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0978e-05 2024-04-24T15:33:25.084 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.084 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.084 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:25.084 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251033+0000", 2024-04-24T15:33:25.084 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4161e-05 2024-04-24T15:33:25.084 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:25.084 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:25.084 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:25.084 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.084 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.084 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18009 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12541952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:25.084 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208093+0000", 2024-04-24T15:33:25.084 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 50.862743086000002, 2024-04-24T15:33:25.085 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043043451000000003, 2024-04-24T15:33:25.085 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:25.085 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:25.085 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:25.085 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:25.085 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:25.085 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18009 2024-04-24T15:33:25.085 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.085 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:25.085 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.085 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:25.085 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208093+0000", 2024-04-24T15:33:25.085 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:25.085 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.086 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.086 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:25.086 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208093+0000", 2024-04-24T15:33:25.086 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:25.086 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.086 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.086 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:25.086 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208093+0000", 2024-04-24T15:33:25.086 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:25.086 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.086 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.086 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:25.087 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208096+0000", 2024-04-24T15:33:25.087 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.439e-06 2024-04-24T15:33:25.087 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.087 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.087 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:25.087 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208103+0000", 2024-04-24T15:33:25.087 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1019999999999999e-06 2024-04-24T15:33:25.087 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.087 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.087 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:25.087 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208107+0000", 2024-04-24T15:33:25.087 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0910000000000003e-06 2024-04-24T15:33:25.087 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.087 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.087 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:25.088 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251054+0000", 2024-04-24T15:33:25.088 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042946826 2024-04-24T15:33:25.088 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.088 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.088 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:25.088 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251092+0000", 2024-04-24T15:33:25.088 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.7941999999999997e-05 2024-04-24T15:33:25.088 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.088 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.088 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:25.088 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251137+0000", 2024-04-24T15:33:25.088 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4638000000000002e-05 2024-04-24T15:33:25.088 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:25.088 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:25.088 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:25.088 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.089 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.089 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18010 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12615680~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:25.089 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208202+0000", 2024-04-24T15:33:25.089 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 50.862634255000003, 2024-04-24T15:33:25.089 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043046549000000003, 2024-04-24T15:33:25.089 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:25.089 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:25.089 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:25.089 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:25.089 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:25.089 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18010 2024-04-24T15:33:25.089 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.089 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:25.089 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.089 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:25.090 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208202+0000", 2024-04-24T15:33:25.090 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:25.090 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.090 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.090 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:25.090 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208202+0000", 2024-04-24T15:33:25.090 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:25.090 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.090 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.090 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:25.090 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208202+0000", 2024-04-24T15:33:25.090 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:25.090 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.090 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.091 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:25.091 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208205+0000", 2024-04-24T15:33:25.091 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.2490000000000001e-06 2024-04-24T15:33:25.091 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.091 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.091 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:25.091 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208215+0000", 2024-04-24T15:33:25.091 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.9070000000000002e-06 2024-04-24T15:33:25.091 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.091 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.091 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:25.091 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208220+0000", 2024-04-24T15:33:25.091 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8849999999999998e-06 2024-04-24T15:33:25.091 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.093 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.093 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:25.093 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251158+0000", 2024-04-24T15:33:25.093 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042938416 2024-04-24T15:33:25.093 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.093 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.093 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:25.093 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251197+0000", 2024-04-24T15:33:25.093 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9341000000000002e-05 2024-04-24T15:33:25.093 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.094 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.094 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:25.094 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251249+0000", 2024-04-24T15:33:25.094 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.1391000000000003e-05 2024-04-24T15:33:25.094 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:25.094 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:25.094 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:25.094 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.094 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.094 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18011 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12656640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:25.094 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208258+0000", 2024-04-24T15:33:25.094 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 50.862578409000001, 2024-04-24T15:33:25.094 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043102708000000003, 2024-04-24T15:33:25.094 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:25.095 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:25.095 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:25.095 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:25.095 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:25.095 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18011 2024-04-24T15:33:25.095 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.095 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:25.095 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.095 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:25.095 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208258+0000", 2024-04-24T15:33:25.095 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:25.095 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.095 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.095 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:25.095 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208258+0000", 2024-04-24T15:33:25.095 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:25.096 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.096 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.096 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:25.096 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208258+0000", 2024-04-24T15:33:25.096 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:25.096 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.096 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.096 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:25.096 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208260+0000", 2024-04-24T15:33:25.096 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7810000000000001e-06 2024-04-24T15:33:25.096 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.096 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.096 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:25.096 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208267+0000", 2024-04-24T15:33:25.096 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.7610000000000002e-06 2024-04-24T15:33:25.097 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.097 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.097 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:25.097 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208271+0000", 2024-04-24T15:33:25.097 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0890000000000002e-06 2024-04-24T15:33:25.097 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.097 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.097 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:25.097 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251271+0000", 2024-04-24T15:33:25.097 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043000159000000003 2024-04-24T15:33:25.097 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.097 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.097 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:25.097 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251315+0000", 2024-04-24T15:33:25.098 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3446999999999999e-05 2024-04-24T15:33:25.098 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.098 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.098 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:25.098 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251361+0000", 2024-04-24T15:33:25.098 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5952000000000003e-05 2024-04-24T15:33:25.098 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:25.098 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:25.098 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:25.098 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.098 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.098 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18012 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12697600~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:25.098 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208289+0000", 2024-04-24T15:33:25.098 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 50.862547364999998, 2024-04-24T15:33:25.099 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043174774999999999, 2024-04-24T15:33:25.099 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:25.099 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:25.099 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:25.099 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:25.099 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:25.099 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18012 2024-04-24T15:33:25.099 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.099 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:25.099 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.099 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:25.099 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208289+0000", 2024-04-24T15:33:25.099 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:25.099 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.099 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.100 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:25.100 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208289+0000", 2024-04-24T15:33:25.100 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:25.100 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.100 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.100 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:25.100 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208289+0000", 2024-04-24T15:33:25.100 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:25.100 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.100 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.100 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:25.100 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208291+0000", 2024-04-24T15:33:25.100 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.542e-06 2024-04-24T15:33:25.100 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.100 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.101 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:25.101 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208298+0000", 2024-04-24T15:33:25.101 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.3899999999999998e-06 2024-04-24T15:33:25.101 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.101 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.101 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:25.101 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208302+0000", 2024-04-24T15:33:25.101 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.019e-06 2024-04-24T15:33:25.101 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.101 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.101 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:25.101 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251382+0000", 2024-04-24T15:33:25.101 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043080790000000001 2024-04-24T15:33:25.101 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.102 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.102 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:25.102 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251421+0000", 2024-04-24T15:33:25.102 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8473000000000002e-05 2024-04-24T15:33:25.102 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.102 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.102 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:25.102 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251464+0000", 2024-04-24T15:33:25.103 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3133999999999997e-05 2024-04-24T15:33:25.103 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:25.103 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:25.103 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:25.103 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.103 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.103 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18013 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12738560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:25.103 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208423+0000", 2024-04-24T15:33:25.103 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 50.862413257, 2024-04-24T15:33:25.103 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043145554000000003, 2024-04-24T15:33:25.103 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:25.103 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:25.103 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:25.103 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:25.103 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:25.104 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18013 2024-04-24T15:33:25.104 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.104 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:25.104 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.104 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:25.104 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208423+0000", 2024-04-24T15:33:25.104 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:25.104 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.104 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.104 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:25.104 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208423+0000", 2024-04-24T15:33:25.104 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:25.104 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.104 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.104 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:25.105 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208423+0000", 2024-04-24T15:33:25.105 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:25.105 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.105 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.105 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:25.105 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208426+0000", 2024-04-24T15:33:25.105 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.0539999999999999e-06 2024-04-24T15:33:25.105 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.105 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.105 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:25.105 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208433+0000", 2024-04-24T15:33:25.105 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.43e-06 2024-04-24T15:33:25.105 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.105 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.105 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:25.106 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208437+0000", 2024-04-24T15:33:25.106 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5590000000000001e-06 2024-04-24T15:33:25.106 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.106 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.106 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:25.106 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251485+0000", 2024-04-24T15:33:25.106 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043048259999999998 2024-04-24T15:33:25.106 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.106 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.106 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:25.106 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251525+0000", 2024-04-24T15:33:25.106 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9916999999999997e-05 2024-04-24T15:33:25.106 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.107 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.107 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:25.107 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251569+0000", 2024-04-24T15:33:25.107 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3939000000000001e-05 2024-04-24T15:33:25.107 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:25.107 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:25.107 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:25.107 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.107 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.107 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18015 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12779520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:25.107 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208630+0000", 2024-04-24T15:33:25.107 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 50.862206731999997, 2024-04-24T15:33:25.107 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043047328000000003, 2024-04-24T15:33:25.107 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:25.108 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:25.108 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:25.108 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:25.108 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:25.108 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18015 2024-04-24T15:33:25.108 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.108 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:25.108 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.108 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:25.108 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208630+0000", 2024-04-24T15:33:25.108 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:25.108 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.108 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.108 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:25.109 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208630+0000", 2024-04-24T15:33:25.109 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:25.109 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.109 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.109 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:25.109 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208629+0000", 2024-04-24T15:33:25.109 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999999 2024-04-24T15:33:25.109 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.109 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.109 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:25.109 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208634+0000", 2024-04-24T15:33:25.109 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8960000000000002e-06 2024-04-24T15:33:25.109 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.109 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.109 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:25.109 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208645+0000", 2024-04-24T15:33:25.110 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0939e-05 2024-04-24T15:33:25.110 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.110 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.110 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:25.110 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208651+0000", 2024-04-24T15:33:25.110 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.896e-06 2024-04-24T15:33:25.110 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.110 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.110 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:25.110 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251591+0000", 2024-04-24T15:33:25.110 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042939862000000002 2024-04-24T15:33:25.110 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.110 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.110 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:25.111 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251631+0000", 2024-04-24T15:33:25.111 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0216000000000003e-05 2024-04-24T15:33:25.111 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.111 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.111 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:25.111 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251677+0000", 2024-04-24T15:33:25.111 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6390999999999997e-05 2024-04-24T15:33:25.111 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:25.111 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:25.111 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:25.111 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.111 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.111 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18016 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12820480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:25.111 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208660+0000", 2024-04-24T15:33:25.111 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 50.862176943000001, 2024-04-24T15:33:25.111 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043127510000000001, 2024-04-24T15:33:25.112 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:25.112 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:25.112 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:25.112 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:25.112 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:25.112 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18016 2024-04-24T15:33:25.112 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.112 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:25.112 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.112 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:25.112 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208660+0000", 2024-04-24T15:33:25.112 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:25.112 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.112 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.112 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:25.113 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208660+0000", 2024-04-24T15:33:25.113 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:25.113 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.113 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.113 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:25.113 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208659+0000", 2024-04-24T15:33:25.113 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:25.113 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.113 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.113 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:25.113 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208662+0000", 2024-04-24T15:33:25.113 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.0979999999999998e-06 2024-04-24T15:33:25.114 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.114 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.114 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:25.114 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208669+0000", 2024-04-24T15:33:25.114 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.4470000000000003e-06 2024-04-24T15:33:25.114 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.114 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.114 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:25.114 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208673+0000", 2024-04-24T15:33:25.114 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2810000000000002e-06 2024-04-24T15:33:25.114 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.114 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.114 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:25.114 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251699+0000", 2024-04-24T15:33:25.115 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043025728999999999 2024-04-24T15:33:25.115 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.115 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.115 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:25.115 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251743+0000", 2024-04-24T15:33:25.115 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4469999999999999e-05 2024-04-24T15:33:25.115 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.115 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.115 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:25.115 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251787+0000", 2024-04-24T15:33:25.115 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3949999999999998e-05 2024-04-24T15:33:25.115 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:25.115 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:25.115 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:25.115 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.116 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.116 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18017 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12861440~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:25.116 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208721+0000", 2024-04-24T15:33:25.116 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 50.862115494999998, 2024-04-24T15:33:25.116 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043167749999999998, 2024-04-24T15:33:25.116 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:25.116 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:25.116 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:25.116 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:25.116 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:25.116 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18017 2024-04-24T15:33:25.116 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.116 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:25.116 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.117 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:25.117 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208721+0000", 2024-04-24T15:33:25.117 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:25.117 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.117 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.117 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:25.117 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208721+0000", 2024-04-24T15:33:25.117 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:25.117 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.117 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.117 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:25.117 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208720+0000", 2024-04-24T15:33:25.117 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:25.117 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.117 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.118 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:25.118 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208724+0000", 2024-04-24T15:33:25.118 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.3919999999999999e-06 2024-04-24T15:33:25.118 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.118 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.118 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:25.118 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208729+0000", 2024-04-24T15:33:25.118 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.3480000000000003e-06 2024-04-24T15:33:25.118 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.118 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.118 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:25.118 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208732+0000", 2024-04-24T15:33:25.118 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.9529999999999999e-06 2024-04-24T15:33:25.118 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.119 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.119 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:25.119 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251807+0000", 2024-04-24T15:33:25.119 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043074492999999998 2024-04-24T15:33:25.119 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.119 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.119 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:25.119 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251846+0000", 2024-04-24T15:33:25.119 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8958999999999997e-05 2024-04-24T15:33:25.119 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.119 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.119 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:25.119 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251889+0000", 2024-04-24T15:33:25.119 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3253000000000001e-05 2024-04-24T15:33:25.119 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:25.119 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:25.120 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:25.120 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.120 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.120 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18018 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12902400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:25.120 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208752+0000", 2024-04-24T15:33:25.120 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 50.862084881999998, 2024-04-24T15:33:25.120 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043241012000000002, 2024-04-24T15:33:25.120 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:25.120 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:25.120 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:25.120 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:25.120 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:25.120 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18018 2024-04-24T15:33:25.120 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.120 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:25.120 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.121 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:25.121 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208752+0000", 2024-04-24T15:33:25.121 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:25.121 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.121 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.121 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:25.121 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208752+0000", 2024-04-24T15:33:25.121 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:25.121 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.121 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.121 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:25.121 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208751+0000", 2024-04-24T15:33:25.121 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:25.121 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.121 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.122 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:25.122 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208753+0000", 2024-04-24T15:33:25.122 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.894e-06 2024-04-24T15:33:25.122 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.122 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.122 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:25.122 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208757+0000", 2024-04-24T15:33:25.122 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.856e-06 2024-04-24T15:33:25.122 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.122 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.122 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:25.122 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208760+0000", 2024-04-24T15:33:25.122 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.6749999999999998e-06 2024-04-24T15:33:25.123 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.123 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.123 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:25.123 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251909+0000", 2024-04-24T15:33:25.123 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043149145999999999 2024-04-24T15:33:25.123 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.123 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.123 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:25.123 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251948+0000", 2024-04-24T15:33:25.123 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9159000000000002e-05 2024-04-24T15:33:25.123 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.123 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.123 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:25.123 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251993+0000", 2024-04-24T15:33:25.124 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4638999999999997e-05 2024-04-24T15:33:25.124 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:25.124 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:25.124 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:25.124 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.124 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.124 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18019 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12943360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:25.124 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208938+0000", 2024-04-24T15:33:25.124 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 50.861898590999999, 2024-04-24T15:33:25.124 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043162272000000002, 2024-04-24T15:33:25.124 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:25.124 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:25.124 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:25.124 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:25.125 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:25.125 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18019 2024-04-24T15:33:25.125 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.125 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:25.125 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.125 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:25.125 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208938+0000", 2024-04-24T15:33:25.125 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:25.125 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.125 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.125 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:25.125 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208938+0000", 2024-04-24T15:33:25.125 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:25.125 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.126 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.126 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:25.126 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208762+0000", 2024-04-24T15:33:25.126 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999824 2024-04-24T15:33:25.126 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.126 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.126 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:25.126 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208942+0000", 2024-04-24T15:33:25.126 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000180008 2024-04-24T15:33:25.126 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.126 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.126 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:25.126 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208950+0000", 2024-04-24T15:33:25.127 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.1550000000000007e-06 2024-04-24T15:33:25.127 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.127 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.127 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:25.127 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208956+0000", 2024-04-24T15:33:25.127 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.7910000000000001e-06 2024-04-24T15:33:25.127 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.127 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.127 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:25.127 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252014+0000", 2024-04-24T15:33:25.127 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043058427000000003 2024-04-24T15:33:25.127 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.127 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.127 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:25.127 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252054+0000", 2024-04-24T15:33:25.128 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0071999999999999e-05 2024-04-24T15:33:25.128 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.128 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.128 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:25.128 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252100+0000", 2024-04-24T15:33:25.128 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5917000000000001e-05 2024-04-24T15:33:25.128 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:25.128 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:25.128 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:25.128 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.128 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.128 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18020 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12984320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:25.128 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.209092+0000", 2024-04-24T15:33:25.129 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 50.861744592999997, 2024-04-24T15:33:25.129 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043111295000000001, 2024-04-24T15:33:25.129 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:25.129 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:25.129 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:25.129 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:25.129 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:25.129 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18020 2024-04-24T15:33:25.129 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.129 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:25.129 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.129 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:25.129 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209092+0000", 2024-04-24T15:33:25.129 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:25.129 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.130 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.130 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:25.130 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209092+0000", 2024-04-24T15:33:25.130 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:25.130 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.130 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.130 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:25.130 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208964+0000", 2024-04-24T15:33:25.130 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998722 2024-04-24T15:33:25.131 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.131 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.131 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:25.131 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209095+0000", 2024-04-24T15:33:25.131 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000130648 2024-04-24T15:33:25.131 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.131 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.131 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:25.131 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209103+0000", 2024-04-24T15:33:25.131 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0320000000000003e-06 2024-04-24T15:33:25.131 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.131 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.131 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:25.131 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209108+0000", 2024-04-24T15:33:25.131 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8690000000000003e-06 2024-04-24T15:33:25.132 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.132 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.132 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:25.132 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252121+0000", 2024-04-24T15:33:25.132 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043013323999999999 2024-04-24T15:33:25.132 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.132 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.132 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:25.132 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252160+0000", 2024-04-24T15:33:25.132 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8376e-05 2024-04-24T15:33:25.132 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.132 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.132 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:25.132 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252203+0000", 2024-04-24T15:33:25.133 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.371e-05 2024-04-24T15:33:25.133 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:25.133 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:25.133 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:25.133 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.133 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.133 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18021 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 13025280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:25.133 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.209132+0000", 2024-04-24T15:33:25.133 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 50.861704234999998, 2024-04-24T15:33:25.133 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043183903000000003, 2024-04-24T15:33:25.133 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:25.133 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:25.133 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:25.133 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:25.133 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:25.134 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18021 2024-04-24T15:33:25.134 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.134 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:25.134 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.134 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:25.134 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209132+0000", 2024-04-24T15:33:25.134 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:25.134 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.134 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.134 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:25.134 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209132+0000", 2024-04-24T15:33:25.134 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:25.134 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.134 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.135 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:25.135 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209132+0000", 2024-04-24T15:33:25.135 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:25.135 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.135 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.135 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:25.135 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209134+0000", 2024-04-24T15:33:25.135 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.891e-06 2024-04-24T15:33:25.135 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.135 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.135 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:25.135 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209138+0000", 2024-04-24T15:33:25.135 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0659999999999997e-06 2024-04-24T15:33:25.135 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.135 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.136 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:25.136 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209140+0000", 2024-04-24T15:33:25.136 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.5339999999999998e-06 2024-04-24T15:33:25.136 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.136 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.136 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:25.136 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252232+0000", 2024-04-24T15:33:25.136 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043091142999999998 2024-04-24T15:33:25.136 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.136 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.136 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:25.136 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252270+0000", 2024-04-24T15:33:25.136 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8872000000000003e-05 2024-04-24T15:33:25.136 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.136 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.137 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:25.137 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252316+0000", 2024-04-24T15:33:25.137 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5788000000000003e-05 2024-04-24T15:33:25.137 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:25.137 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:25.137 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:25.137 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.137 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.137 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18022 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 13066240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:25.137 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.209411+0000", 2024-04-24T15:33:25.137 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 50.861425687000001, 2024-04-24T15:33:25.137 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043016616000000001, 2024-04-24T15:33:25.137 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:25.137 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:25.137 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:25.138 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:25.138 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:25.138 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18022 2024-04-24T15:33:25.138 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.138 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:25.138 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.138 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:25.138 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209411+0000", 2024-04-24T15:33:25.138 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:25.138 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.138 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.138 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:25.138 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209411+0000", 2024-04-24T15:33:25.138 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:25.139 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.139 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.139 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:25.139 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209143+0000", 2024-04-24T15:33:25.139 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999732 2024-04-24T15:33:25.139 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.139 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.139 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:25.139 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209415+0000", 2024-04-24T15:33:25.139 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000271873 2024-04-24T15:33:25.139 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.139 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.139 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:25.140 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209423+0000", 2024-04-24T15:33:25.140 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3080000000000008e-06 2024-04-24T15:33:25.140 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.140 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.140 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:25.140 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209428+0000", 2024-04-24T15:33:25.140 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.5040000000000002e-06 2024-04-24T15:33:25.140 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.140 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.140 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:25.140 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252339+0000", 2024-04-24T15:33:25.140 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042910419999999998 2024-04-24T15:33:25.140 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.140 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.140 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:25.141 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252380+0000", 2024-04-24T15:33:25.141 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0991000000000001e-05 2024-04-24T15:33:25.141 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.141 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.141 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:25.141 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252427+0000", 2024-04-24T15:33:25.141 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.7647e-05 2024-04-24T15:33:25.141 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:25.141 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:25.141 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:25.141 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.141 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.141 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18023 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 13107200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:25.141 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.209594+0000", 2024-04-24T15:33:25.141 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 50.861242173999997, 2024-04-24T15:33:25.142 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042938543000000003, 2024-04-24T15:33:25.142 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:25.142 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:25.142 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:25.142 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:25.142 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:25.142 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18023 2024-04-24T15:33:25.142 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.142 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:25.142 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.142 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:25.142 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209594+0000", 2024-04-24T15:33:25.142 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:25.143 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.143 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.143 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:25.143 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209594+0000", 2024-04-24T15:33:25.143 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:25.143 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.143 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.143 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:25.143 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209440+0000", 2024-04-24T15:33:25.143 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999846 2024-04-24T15:33:25.143 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.143 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.143 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:25.144 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209598+0000", 2024-04-24T15:33:25.144 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00015731100000000001 2024-04-24T15:33:25.144 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.144 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.144 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:25.144 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209604+0000", 2024-04-24T15:33:25.144 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.2910000000000004e-06 2024-04-24T15:33:25.144 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.144 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.144 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:25.144 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209607+0000", 2024-04-24T15:33:25.144 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5389999999999999e-06 2024-04-24T15:33:25.144 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.145 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.145 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:25.145 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252449+0000", 2024-04-24T15:33:25.145 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042841833000000003 2024-04-24T15:33:25.145 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.145 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.145 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:25.145 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252489+0000", 2024-04-24T15:33:25.145 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9898000000000003e-05 2024-04-24T15:33:25.145 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.145 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.145 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:25.145 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252533+0000", 2024-04-24T15:33:25.145 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3702999999999998e-05 2024-04-24T15:33:25.146 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:25.146 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:25.146 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:25.146 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.146 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.146 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18025 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 13189120~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:25.146 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.209832+0000", 2024-04-24T15:33:25.146 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 50.861004672999997, 2024-04-24T15:33:25.146 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042913556999999998, 2024-04-24T15:33:25.146 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:25.146 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:25.146 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:25.146 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:25.146 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:25.147 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18025 2024-04-24T15:33:25.147 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.147 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:25.147 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.147 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:25.147 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209832+0000", 2024-04-24T15:33:25.147 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:25.147 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.147 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.147 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:25.147 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209832+0000", 2024-04-24T15:33:25.147 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:25.148 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.148 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.148 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:25.148 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209774+0000", 2024-04-24T15:33:25.148 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999418 2024-04-24T15:33:25.148 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.148 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.148 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:25.148 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209835+0000", 2024-04-24T15:33:25.148 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.1377000000000005e-05 2024-04-24T15:33:25.148 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.148 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.148 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:25.148 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209843+0000", 2024-04-24T15:33:25.148 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.2330000000000006e-06 2024-04-24T15:33:25.149 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.149 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.149 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:25.149 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209848+0000", 2024-04-24T15:33:25.149 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6149999999999999e-06 2024-04-24T15:33:25.149 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.149 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.149 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:25.149 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252658+0000", 2024-04-24T15:33:25.149 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042810002 2024-04-24T15:33:25.149 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.149 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.150 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:25.150 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252697+0000", 2024-04-24T15:33:25.150 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8757999999999997e-05 2024-04-24T15:33:25.150 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.150 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.150 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:25.150 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252745+0000", 2024-04-24T15:33:25.150 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.863e-05 2024-04-24T15:33:25.150 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:25.150 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:25.150 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:25.150 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.150 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.150 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:24749 4.8 4:137f5701:::rbd_header.114ae15b9a20:head [watch reconnect cookie 140672595753696 gen 11] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-04-24T15:33:25.151 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:37.628703+0000", 2024-04-24T15:33:25.151 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 47.442133054000003, 2024-04-24T15:33:25.151 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0102918700000001, 2024-04-24T15:33:25.151 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:25.151 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:25.151 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:25.151 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:25.151 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:25.151 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 24749 2024-04-24T15:33:25.151 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.151 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:25.151 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.151 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:25.151 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628703+0000", 2024-04-24T15:33:25.151 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:25.151 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.152 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.152 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:25.152 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628703+0000", 2024-04-24T15:33:25.152 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:25.152 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.152 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.152 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:25.152 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628703+0000", 2024-04-24T15:33:25.152 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999999 2024-04-24T15:33:25.152 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.152 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.152 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:25.152 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628708+0000", 2024-04-24T15:33:25.152 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.4589999999999997e-06 2024-04-24T15:33:25.152 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.152 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.153 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:25.153 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628721+0000", 2024-04-24T15:33:25.153 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.2601e-05 2024-04-24T15:33:25.153 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.153 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.153 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:25.153 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628728+0000", 2024-04-24T15:33:25.153 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.9059999999999996e-06 2024-04-24T15:33:25.153 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.153 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.153 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:25.153 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628789+0000", 2024-04-24T15:33:25.153 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.1672000000000002e-05 2024-04-24T15:33:25.153 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.153 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.153 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting_for_map not empty", 2024-04-24T15:33:25.154 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628797+0000", 2024-04-24T15:33:25.154 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.8189999999999992e-06 2024-04-24T15:33:25.154 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.154 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.154 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:25.154 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.631255+0000", 2024-04-24T15:33:25.154 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0024577610000000001 2024-04-24T15:33:25.154 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.154 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.154 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for peered", 2024-04-24T15:33:25.154 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.631263+0000", 2024-04-24T15:33:25.154 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.4109999999999996e-06 2024-04-24T15:33:25.154 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.154 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.155 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:25.155 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:38.637560+0000", 2024-04-24T15:33:25.155 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0062970339999999 2024-04-24T15:33:25.155 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.155 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.155 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:25.155 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:38.637747+0000", 2024-04-24T15:33:25.155 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00018693400000000001 2024-04-24T15:33:25.155 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.155 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.155 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:25.155 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:38.638995+0000", 2024-04-24T15:33:25.155 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0012481300000000001 2024-04-24T15:33:25.155 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:25.155 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:25.156 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:25.156 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.156 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.156 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:24750 4.8 4:137f5701:::rbd_header.114ae15b9a20:head [notify cookie 140672594434416] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-04-24T15:33:25.156 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:37.628736+0000", 2024-04-24T15:33:25.156 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 47.442100793999998, 2024-04-24T15:33:25.156 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0096094309999999, 2024-04-24T15:33:25.156 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:25.156 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:25.156 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:25.156 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:25.156 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:25.156 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 24750 2024-04-24T15:33:25.156 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.156 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:25.156 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.157 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:25.157 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628736+0000", 2024-04-24T15:33:25.157 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:25.157 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.157 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.157 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:25.157 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628736+0000", 2024-04-24T15:33:25.157 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:25.157 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.157 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.157 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:25.157 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628735+0000", 2024-04-24T15:33:25.157 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:25.157 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.157 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.157 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:25.158 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628739+0000", 2024-04-24T15:33:25.158 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5109999999999999e-06 2024-04-24T15:33:25.158 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.158 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.158 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:25.158 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628745+0000", 2024-04-24T15:33:25.158 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.8300000000000001e-06 2024-04-24T15:33:25.158 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.158 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.158 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:25.158 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628749+0000", 2024-04-24T15:33:25.158 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.7110000000000003e-06 2024-04-24T15:33:25.158 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.158 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.159 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:25.159 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628826+0000", 2024-04-24T15:33:25.159 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.6678e-05 2024-04-24T15:33:25.159 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.159 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.159 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting_for_map not empty", 2024-04-24T15:33:25.159 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628834+0000", 2024-04-24T15:33:25.159 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.6190000000000002e-06 2024-04-24T15:33:25.159 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.159 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.159 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:25.159 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.631305+0000", 2024-04-24T15:33:25.159 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0024714400000000001 2024-04-24T15:33:25.159 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.160 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.160 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for peered", 2024-04-24T15:33:25.160 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.631313+0000", 2024-04-24T15:33:25.160 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.6529999999999994e-06 2024-04-24T15:33:25.160 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.160 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.160 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:25.160 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:38.638068+0000", 2024-04-24T15:33:25.160 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0067548369999999 2024-04-24T15:33:25.160 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.160 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.160 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:25.160 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:38.638158+0000", 2024-04-24T15:33:25.160 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.0079000000000003e-05 2024-04-24T15:33:25.161 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:25.161 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:25.161 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:25.161 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:38.638345+0000", 2024-04-24T15:33:25.161 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00018757100000000001 2024-04-24T15:33:25.161 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:25.161 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:25.161 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:25.161 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:25.161 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:25.161 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:25.163 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:25.163+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:25.163 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-24T15:33:25.217 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:25.228 INFO:teuthology.orchestra.run.smithi096.stdout:5596 trunc from 0xa1f9a46 to 0x124e4cd 2024-04-24T15:33:25.252 INFO:teuthology.orchestra.run.smithi096.stdout:5597 read 0x1121fed thru 0x1123ab4 (0x1ac8 bytes) 2024-04-24T15:33:25.263 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:33:25.265+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:33:25.266 INFO:teuthology.orchestra.run.smithi096.stdout:5599 trunc from 0x124e4cd to 0xbeaccf6 2024-04-24T15:33:25.271 INFO:teuthology.orchestra.run.smithi096.stdout:5600 punch from 0x4c60a0c to 0x4c64d84, (0x4378 bytes) 2024-04-24T15:33:25.278 INFO:teuthology.orchestra.run.smithi096.stdout:5601 trunc from 0xbeaccf6 to 0x6281397 2024-04-24T15:33:25.299 INFO:teuthology.orchestra.run.smithi096.stdout:5602 writesame 0xe7c93ad thru 0xe7d1660 data_size 0x46(0x82b4 bytes) 2024-04-24T15:33:25.299 INFO:teuthology.orchestra.run.smithi096.stdout:5603 write 0xc737fce thru 0xc73a677 (0x26aa bytes) 2024-04-24T15:33:25.302 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:25.302 INFO:teuthology.orchestra.run.smithi096.stdout:5606 read 0x25c557a thru 0x25c7e5b (0x28e2 bytes) 2024-04-24T15:33:25.302 INFO:teuthology.orchestra.run.smithi096.stdout:5607 read 0x8d9f948 thru 0x8da0b3d (0x11f6 bytes) 2024-04-24T15:33:25.302 INFO:teuthology.orchestra.run.smithi096.stdout:5608 read 0x740ad22 thru 0x7417ae6 (0xcdc5 bytes) 2024-04-24T15:33:25.302 INFO:teuthology.orchestra.run.smithi096.stdout:5610 punch from 0x45e9aaa to 0x45f8770, (0xecc6 bytes) 2024-04-24T15:33:25.302 INFO:teuthology.orchestra.run.smithi096.stdout:5611 write 0xbd7cfb5 thru 0xbd842ad (0x72f9 bytes) 2024-04-24T15:33:25.304 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:25.304 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:33:25.304 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-04-24T15:33:25.304 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:25.306 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:25.311 INFO:teuthology.orchestra.run.smithi096.stdout:5613 trunc from 0xe7d1661 to 0xa396cc0 2024-04-24T15:33:25.319 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-24T15:33:25.321 INFO:teuthology.orchestra.run.smithi096.stdout:5615 write 0x2d1232c thru 0x2d21081 (0xed56 bytes) 2024-04-24T15:33:25.326 INFO:teuthology.orchestra.run.smithi096.stdout:5616 write 0x8d8a1b3 thru 0x8d959d9 (0xb827 bytes) 2024-04-24T15:33:25.330 INFO:teuthology.orchestra.run.smithi096.stdout:5618 write 0x806448e thru 0x8069d04 (0x5877 bytes) 2024-04-24T15:33:25.333 INFO:teuthology.orchestra.run.smithi096.stdout:5619 punch from 0x56635b1 to 0x566f5b9, (0xc008 bytes) 2024-04-24T15:33:25.333 INFO:teuthology.orchestra.run.smithi096.stdout:5621 read 0x35d29da thru 0x35d52d4 (0x28fb bytes) 2024-04-24T15:33:25.344 INFO:teuthology.orchestra.run.smithi096.stdout:5623 write 0xe8d577e thru 0xe8e163a (0xbebd bytes) 2024-04-24T15:33:25.348 INFO:teuthology.orchestra.run.smithi096.stdout:5624 write 0xc1f9b37 thru 0xc209148 (0xf612 bytes) 2024-04-24T15:33:25.356 INFO:teuthology.orchestra.run.smithi096.stdout:5625 writesame 0x328ba9b thru 0x32902a3 data_size 0x9(0x4809 bytes) 2024-04-24T15:33:25.361 INFO:teuthology.orchestra.run.smithi096.stdout:5626 read 0x42f9b91 thru 0x4305676 (0xbae6 bytes) 2024-04-24T15:33:25.364 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:33:25.366+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:33:25.366 INFO:teuthology.orchestra.run.smithi096.stdout:5627 trunc from 0xe8e163b to 0xa7cc7b3 2024-04-24T15:33:25.382 INFO:teuthology.orchestra.run.smithi096.stdout:5628 read 0xa591b09 thru 0xa59f3c2 (0xd8ba bytes) 2024-04-24T15:33:25.394 INFO:teuthology.orchestra.run.smithi096.stdout:5630 write 0xe42ee12 thru 0xe438dbb (0x9faa bytes) 2024-04-24T15:33:25.400 INFO:teuthology.orchestra.run.smithi096.stdout:5631 write 0x951ecd8 thru 0x952aa12 (0xbd3b bytes) 2024-04-24T15:33:25.419 INFO:teuthology.orchestra.run.smithi096.stdout:5632 trunc from 0xe438dbc to 0x1f9c8e2 2024-04-24T15:33:25.432 INFO:teuthology.orchestra.run.smithi096.stdout:5633 punch from 0x12dc2de to 0x12e38e1, (0x7603 bytes) 2024-04-24T15:33:25.432 INFO:teuthology.orchestra.run.smithi096.stdout:5634 punch from 0x5780d4 to 0x578cc0, (0xbec bytes) 2024-04-24T15:33:25.432 INFO:teuthology.orchestra.run.smithi096.stdout:5635 punch from 0x198f4c8 to 0x199014e, (0xc86 bytes) 2024-04-24T15:33:25.432 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:25.443 INFO:teuthology.orchestra.run.smithi096.stdout:5639 write 0x6136931 thru 0x6138813 (0x1ee3 bytes) 2024-04-24T15:33:25.454 INFO:teuthology.orchestra.run.smithi096.stdout:5641 write 0x7355093 thru 0x735e0cb (0x9039 bytes) 2024-04-24T15:33:25.457 INFO:teuthology.orchestra.run.smithi096.stdout:5642 read 0x4de633f thru 0x4de86f3 (0x23b5 bytes) 2024-04-24T15:33:25.463 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:25.463 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:33:25.463 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-04-24T15:33:25.463 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-04-24T15:33:25.463 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:25.464 INFO:teuthology.orchestra.run.smithi096.stdout:5643 trunc from 0x735e0cc to 0x15b0c2c 2024-04-24T15:33:25.465 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:25.466+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:25.474 INFO:teuthology.orchestra.run.smithi096.stdout:5644 punch from 0x15abbe to 0x167a63, (0xcea5 bytes) 2024-04-24T15:33:25.474 INFO:teuthology.orchestra.run.smithi096.stdout:5645 read 0x77149 thru 0x80889 (0x9741 bytes) 2024-04-24T15:33:25.475 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-24T15:33:25.476 INFO:teuthology.orchestra.run.smithi096.stdout:5646 read 0x90f91c thru 0x916ee2 (0x75c7 bytes) 2024-04-24T15:33:25.478 INFO:teuthology.orchestra.run.smithi096.stdout:5648 trunc from 0x15b0c2c to 0x412baf6 2024-04-24T15:33:25.485 INFO:teuthology.orchestra.run.smithi096.stdout:5649 read 0x108d168 thru 0x10926a4 (0x553d bytes) 2024-04-24T15:33:25.485 INFO:teuthology.orchestra.run.smithi096.stdout:5650 write 0x18cb7f0 thru 0x18d937a (0xdb8b bytes) 2024-04-24T15:33:25.488 INFO:teuthology.orchestra.run.smithi096.stdout:5651 punch from 0x2a60c9f to 0x2a65469, (0x47ca bytes) 2024-04-24T15:33:25.508 INFO:teuthology.orchestra.run.smithi096.stdout:5653 writesame 0xee417e8 thru 0xee51575 data_size 0x5(0xfd8e bytes) 2024-04-24T15:33:25.508 INFO:teuthology.orchestra.run.smithi096.stdout:5654 punch from 0x2077397 to 0x20782ed, (0xf56 bytes) 2024-04-24T15:33:25.508 INFO:teuthology.orchestra.run.smithi096.stdout:5656 read 0x845dcc4 thru 0x845dcf6 (0x33 bytes) 2024-04-24T15:33:25.508 INFO:teuthology.orchestra.run.smithi096.stdout:5660 read 0xb8dea55 thru 0xb8df061 (0x60d bytes) 2024-04-24T15:33:25.508 INFO:teuthology.orchestra.run.smithi096.stdout:5661 read 0xb7dc558 thru 0xb7e4ca4 (0x874d bytes) 2024-04-24T15:33:25.509 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:25.509 INFO:teuthology.orchestra.run.smithi096.stdout:5664 punch from 0xe12dc0 to 0xe17d2f, (0x4f6f bytes) 2024-04-24T15:33:25.516 INFO:teuthology.orchestra.run.smithi096.stdout:5665 trunc from 0xee51576 to 0x864eb27 2024-04-24T15:33:25.533 INFO:teuthology.orchestra.run.smithi096.stdout:5666 write 0xd181e9f thru 0xd18fa9b (0xdbfd bytes) 2024-04-24T15:33:25.536 INFO:teuthology.orchestra.run.smithi096.stdout:5668 punch from 0x24f1f03 to 0x24f5317, (0x3414 bytes) 2024-04-24T15:33:25.536 INFO:teuthology.orchestra.run.smithi096.stdout:5669 writesame 0x723eec4 thru 0x7246ded data_size 0x29(0x7f2a bytes) 2024-04-24T15:33:25.540 INFO:teuthology.orchestra.run.smithi096.stdout:5671 write 0x5f7ecff thru 0x5f81095 (0x2397 bytes) 2024-04-24T15:33:25.544 INFO:teuthology.orchestra.run.smithi096.stdout:5672 read 0x63f8858 thru 0x640674c (0xdef5 bytes) 2024-04-24T15:33:25.544 INFO:teuthology.orchestra.run.smithi096.stdout:5673 writesame 0x80b32fe thru 0x80b6e6f data_size 0x7(0x3b72 bytes) 2024-04-24T15:33:25.548 INFO:teuthology.orchestra.run.smithi096.stdout:5674 read 0xa761c86 thru 0xa767292 (0x560d bytes) 2024-04-24T15:33:25.548 INFO:teuthology.orchestra.run.smithi096.stdout:5676 read 0x5e77a90 thru 0x5e84f73 (0xd4e4 bytes) 2024-04-24T15:33:25.548 INFO:teuthology.orchestra.run.smithi096.stdout:5677 read 0x4d61b11 thru 0x4d686cc (0x6bbc bytes) 2024-04-24T15:33:25.548 INFO:teuthology.orchestra.run.smithi096.stdout:5678 write 0x672771c thru 0x672bf56 (0x483b bytes) 2024-04-24T15:33:25.561 INFO:teuthology.orchestra.run.smithi096.stdout:5679 trunc from 0xd18fa9c to 0x4a572a5 2024-04-24T15:33:25.566 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:25.566+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:25.571 INFO:teuthology.orchestra.run.smithi096.stdout:5680 punch from 0xe6fc17 to 0xe7ce41, (0xd22a bytes) 2024-04-24T15:33:25.587 INFO:teuthology.orchestra.run.smithi096.stdout:5681 write 0xdc01b41 thru 0xdc06f9a (0x545a bytes) 2024-04-24T15:33:25.590 INFO:teuthology.orchestra.run.smithi096.stdout:5682 read 0x578e501 thru 0x5799a7f (0xb57f bytes) 2024-04-24T15:33:25.590 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:25.590 INFO:teuthology.orchestra.run.smithi096.stdout:5686 write 0x22de9bb thru 0x22e35d5 (0x4c1b bytes) 2024-04-24T15:33:25.593 INFO:teuthology.orchestra.run.smithi096.stdout:5688 writesame 0x34a100c thru 0x34a3001 data_size 0x1(0x1ff6 bytes) 2024-04-24T15:33:25.601 INFO:teuthology.orchestra.run.smithi096.stdout:5689 write 0x46981cc thru 0x4698e34 (0xc69 bytes) 2024-04-24T15:33:25.603 INFO:teuthology.orchestra.run.smithi096.stdout:5691 read 0x5952446 thru 0x595e37d (0xbf38 bytes) 2024-04-24T15:33:25.603 INFO:teuthology.orchestra.run.smithi096.stdout:5692 read 0xd60b413 thru 0xd60f5a6 (0x4194 bytes) 2024-04-24T15:33:25.603 INFO:teuthology.orchestra.run.smithi096.stdout:5693 punch from 0x7fea4dc to 0x7ff1fe3, (0x7b07 bytes) 2024-04-24T15:33:25.603 INFO:teuthology.orchestra.run.smithi096.stdout:5694 write 0x1e42b58 thru 0x1e4566a (0x2b13 bytes) 2024-04-24T15:33:25.614 INFO:teuthology.orchestra.run.smithi096.stdout:5695 trunc from 0xdc06f9b to 0x7deb0d9 2024-04-24T15:33:25.625 INFO:teuthology.orchestra.run.smithi096.stdout:5697 read 0x4154fe4 thru 0x4162b78 (0xdb95 bytes) 2024-04-24T15:33:25.625 INFO:teuthology.orchestra.run.smithi096.stdout:5699 read 0x2e95228 thru 0x2e9d1be (0x7f97 bytes) 2024-04-24T15:33:25.625 INFO:teuthology.orchestra.run.smithi096.stdout:5702 read 0x6d4a3ad thru 0x6d526cc (0x8320 bytes) 2024-04-24T15:33:25.625 INFO:teuthology.orchestra.run.smithi096.stdout:5703 read 0x5967d51 thru 0x596ed6a (0x701a bytes) 2024-04-24T15:33:25.630 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:25.630 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-04-24T15:33:25.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-04-24T15:33:25.630 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-04-24T15:33:25.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.631 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:47384 4.1f 4:fdac2fcc:::rbd_header.114a8b543e91:head [watch reconnect cookie 140672527126976 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-04-24T15:33:25.631 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:09.802333+0000", 2024-04-24T15:33:25.631 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 15.827782778, 2024-04-24T15:33:25.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0093213050000001, 2024-04-24T15:33:25.631 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:25.631 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:25.631 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:25.631 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:25.631 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:25.631 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 47384 2024-04-24T15:33:25.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.631 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:25.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:25.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.802333+0000", 2024-04-24T15:33:25.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:25.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:25.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.802333+0000", 2024-04-24T15:33:25.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:25.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:25.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.802332+0000", 2024-04-24T15:33:25.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999981 2024-04-24T15:33:25.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.633 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:25.633 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.802340+0000", 2024-04-24T15:33:25.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7700000000000007e-06 2024-04-24T15:33:25.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.633 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:25.633 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.802360+0000", 2024-04-24T15:33:25.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9959999999999999e-05 2024-04-24T15:33:25.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.633 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:25.633 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.802371+0000", 2024-04-24T15:33:25.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0473e-05 2024-04-24T15:33:25.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.633 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:25.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.802447+0000", 2024-04-24T15:33:25.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6089999999999998e-05 2024-04-24T15:33:25.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op must wait for map", 2024-04-24T15:33:25.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.802456+0000", 2024-04-24T15:33:25.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0599999999999997e-06 2024-04-24T15:33:25.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:25.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.808378+0000", 2024-04-24T15:33:25.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0059224910000000002 2024-04-24T15:33:25.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-04-24T15:33:25.635 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.808384+0000", 2024-04-24T15:33:25.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7740000000000001e-06 2024-04-24T15:33:25.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:25.635 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.809812+0000", 2024-04-24T15:33:25.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.001428043 2024-04-24T15:33:25.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:25.635 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.810082+0000", 2024-04-24T15:33:25.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00026971800000000001 2024-04-24T15:33:25.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:25.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.811655+0000", 2024-04-24T15:33:25.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015726869999999999 2024-04-24T15:33:25.636 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:25.636 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:25.636 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:25.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.636 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:47385 4.1f 4:fdac2fcc:::rbd_header.114a8b543e91:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-04-24T15:33:25.636 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:09.803953+0000", 2024-04-24T15:33:25.636 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 15.826162864, 2024-04-24T15:33:25.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0080910729999999, 2024-04-24T15:33:25.636 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:25.636 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:25.637 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:25.637 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:25.637 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:25.637 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 47385 2024-04-24T15:33:25.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.637 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:25.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.637 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:25.637 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.803953+0000", 2024-04-24T15:33:25.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:25.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.637 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:25.637 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.803953+0000", 2024-04-24T15:33:25.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:25.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.638 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:25.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.803952+0000", 2024-04-24T15:33:25.638 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999981 2024-04-24T15:33:25.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.638 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:25.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.803961+0000", 2024-04-24T15:33:25.638 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9549999999999998e-06 2024-04-24T15:33:25.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.638 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:25.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.803977+0000", 2024-04-24T15:33:25.638 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6498999999999999e-05 2024-04-24T15:33:25.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.639 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:25.639 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.803985+0000", 2024-04-24T15:33:25.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5000000000000002e-06 2024-04-24T15:33:25.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.639 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:25.639 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.804032+0000", 2024-04-24T15:33:25.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7657999999999997e-05 2024-04-24T15:33:25.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.639 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2024-04-24T15:33:25.639 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.804039+0000", 2024-04-24T15:33:25.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5719999999999999e-06 2024-04-24T15:33:25.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:25.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.808397+0000", 2024-04-24T15:33:25.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0043579860000000003 2024-04-24T15:33:25.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-04-24T15:33:25.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.808401+0000", 2024-04-24T15:33:25.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3089999999999997e-06 2024-04-24T15:33:25.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:25.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.810700+0000", 2024-04-24T15:33:25.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0022984699999999 2024-04-24T15:33:25.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.641 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:25.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.810758+0000", 2024-04-24T15:33:25.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8121000000000001e-05 2024-04-24T15:33:25.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.641 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:33:25.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.811932+0000", 2024-04-24T15:33:25.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001174542 2024-04-24T15:33:25.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.641 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:25.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.811978+0000", 2024-04-24T15:33:25.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5645000000000001e-05 2024-04-24T15:33:25.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.642 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:25.642 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.812044+0000", 2024-04-24T15:33:25.642 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6575999999999994e-05 2024-04-24T15:33:25.642 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:25.642 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:25.642 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:25.642 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.642 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.642 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48590 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2686976~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:25.642 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889219+0000", 2024-04-24T15:33:25.642 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.740896932, 2024-04-24T15:33:25.642 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016760856000000001, 2024-04-24T15:33:25.642 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:25.642 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:25.643 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:25.643 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:25.643 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:25.643 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48590 2024-04-24T15:33:25.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.643 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:25.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.643 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:25.643 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889219+0000", 2024-04-24T15:33:25.643 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:25.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.643 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:25.643 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889219+0000", 2024-04-24T15:33:25.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:25.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:25.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889219+0000", 2024-04-24T15:33:25.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:25.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:25.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889221+0000", 2024-04-24T15:33:25.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4499999999999998e-06 2024-04-24T15:33:25.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:25.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889227+0000", 2024-04-24T15:33:25.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6869999999999999e-06 2024-04-24T15:33:25.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:25.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889231+0000", 2024-04-24T15:33:25.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6320000000000001e-06 2024-04-24T15:33:25.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:25.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.905769+0000", 2024-04-24T15:33:25.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016538850000000001 2024-04-24T15:33:25.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:25.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.905855+0000", 2024-04-24T15:33:25.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5296000000000005e-05 2024-04-24T15:33:25.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.646 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:25.646 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.905980+0000", 2024-04-24T15:33:25.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000125406 2024-04-24T15:33:25.646 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:25.646 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:25.646 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:25.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.646 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48705 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 1851392~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:25.646 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889236+0000", 2024-04-24T15:33:25.646 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.740880324000001, 2024-04-24T15:33:25.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016984302, 2024-04-24T15:33:25.647 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:25.647 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:25.647 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:25.647 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:25.647 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:25.647 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48705 2024-04-24T15:33:25.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.647 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:25.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.647 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:25.647 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889236+0000", 2024-04-24T15:33:25.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:25.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.648 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:25.648 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889236+0000", 2024-04-24T15:33:25.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:25.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.648 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:25.648 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889235+0000", 2024-04-24T15:33:25.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:25.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.648 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:25.648 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889238+0000", 2024-04-24T15:33:25.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.249e-06 2024-04-24T15:33:25.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:25.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889243+0000", 2024-04-24T15:33:25.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6810000000000003e-06 2024-04-24T15:33:25.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:25.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889247+0000", 2024-04-24T15:33:25.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6780000000000002e-06 2024-04-24T15:33:25.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:25.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906008+0000", 2024-04-24T15:33:25.650 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01676099 2024-04-24T15:33:25.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.650 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:25.650 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906090+0000", 2024-04-24T15:33:25.650 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1508999999999995e-05 2024-04-24T15:33:25.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.650 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:25.650 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906220+0000", 2024-04-24T15:33:25.650 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013056999999999999 2024-04-24T15:33:25.650 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:25.650 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:25.651 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:25.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.651 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48706 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 1916928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:25.651 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889251+0000", 2024-04-24T15:33:25.651 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.740864890999999, 2024-04-24T15:33:25.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01718693, 2024-04-24T15:33:25.651 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:25.651 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:25.651 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:25.651 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:25.651 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:25.651 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48706 2024-04-24T15:33:25.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.651 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:25.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:25.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889251+0000", 2024-04-24T15:33:25.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:25.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:25.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889251+0000", 2024-04-24T15:33:25.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:25.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:25.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889251+0000", 2024-04-24T15:33:25.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:25.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:25.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889253+0000", 2024-04-24T15:33:25.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.244e-06 2024-04-24T15:33:25.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:25.654 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889259+0000", 2024-04-24T15:33:25.654 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3640000000000004e-06 2024-04-24T15:33:25.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.654 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:25.654 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889263+0000", 2024-04-24T15:33:25.654 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3720000000000001e-06 2024-04-24T15:33:25.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.654 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:25.654 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906258+0000", 2024-04-24T15:33:25.654 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016995310999999999 2024-04-24T15:33:25.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.655 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:25.655 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906343+0000", 2024-04-24T15:33:25.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4536000000000005e-05 2024-04-24T15:33:25.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.655 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:25.655 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906438+0000", 2024-04-24T15:33:25.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5502999999999997e-05 2024-04-24T15:33:25.655 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:25.655 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:25.655 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:25.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.656 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48707 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 1982464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:25.656 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889267+0000", 2024-04-24T15:33:25.656 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.740849331, 2024-04-24T15:33:25.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017425395, 2024-04-24T15:33:25.656 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:25.656 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:25.656 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:25.656 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:25.656 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:25.656 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48707 2024-04-24T15:33:25.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.656 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:25.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.656 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:25.656 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889267+0000", 2024-04-24T15:33:25.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:25.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:25.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889267+0000", 2024-04-24T15:33:25.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:25.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:25.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889266+0000", 2024-04-24T15:33:25.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:25.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:25.658 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889275+0000", 2024-04-24T15:33:25.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.568e-06 2024-04-24T15:33:25.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:25.658 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889281+0000", 2024-04-24T15:33:25.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3779999999999998e-06 2024-04-24T15:33:25.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:25.658 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889291+0000", 2024-04-24T15:33:25.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2769999999999994e-06 2024-04-24T15:33:25.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.659 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:25.659 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906476+0000", 2024-04-24T15:33:25.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017185277999999998 2024-04-24T15:33:25.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.659 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:25.659 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906567+0000", 2024-04-24T15:33:25.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0902999999999993e-05 2024-04-24T15:33:25.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.659 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:25.659 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906692+0000", 2024-04-24T15:33:25.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012541700000000001 2024-04-24T15:33:25.659 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:25.660 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:25.660 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:25.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.660 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48708 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2048000~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:25.660 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889295+0000", 2024-04-24T15:33:25.660 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.74082084, 2024-04-24T15:33:25.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017534984999999999, 2024-04-24T15:33:25.660 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:25.660 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:25.660 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:25.660 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:25.660 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:25.660 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48708 2024-04-24T15:33:25.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.661 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:25.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:25.661 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889295+0000", 2024-04-24T15:33:25.661 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:25.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:25.661 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889295+0000", 2024-04-24T15:33:25.661 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:25.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:25.661 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889295+0000", 2024-04-24T15:33:25.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:25.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.662 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:25.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889297+0000", 2024-04-24T15:33:25.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3479999999999998e-06 2024-04-24T15:33:25.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.662 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:25.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889304+0000", 2024-04-24T15:33:25.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3720000000000001e-06 2024-04-24T15:33:25.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.663 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:25.663 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889307+0000", 2024-04-24T15:33:25.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7160000000000001e-06 2024-04-24T15:33:25.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.663 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:25.663 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906711+0000", 2024-04-24T15:33:25.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017403452999999999 2024-04-24T15:33:25.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.663 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:25.663 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906765+0000", 2024-04-24T15:33:25.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4219999999999999e-05 2024-04-24T15:33:25.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.664 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:25.664 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906830+0000", 2024-04-24T15:33:25.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5277000000000005e-05 2024-04-24T15:33:25.664 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:25.664 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:25.664 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:25.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.664 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48709 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2113536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:25.664 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889312+0000", 2024-04-24T15:33:25.664 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.740804533, 2024-04-24T15:33:25.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017772013, 2024-04-24T15:33:25.665 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:25.665 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:25.665 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:25.665 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:25.665 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:25.665 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48709 2024-04-24T15:33:25.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.665 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:25.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:25.665 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889312+0000", 2024-04-24T15:33:25.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:25.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:25.666 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889312+0000", 2024-04-24T15:33:25.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:25.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.666 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:25.666 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889311+0000", 2024-04-24T15:33:25.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:25.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.666 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:25.666 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889314+0000", 2024-04-24T15:33:25.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2759999999999999e-06 2024-04-24T15:33:25.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.667 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:25.667 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889322+0000", 2024-04-24T15:33:25.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8589999999999994e-06 2024-04-24T15:33:25.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.667 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:25.667 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889326+0000", 2024-04-24T15:33:25.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6380000000000002e-06 2024-04-24T15:33:25.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.667 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:25.667 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906862+0000", 2024-04-24T15:33:25.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017536403999999998 2024-04-24T15:33:25.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.668 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:25.668 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906949+0000", 2024-04-24T15:33:25.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6217000000000005e-05 2024-04-24T15:33:25.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.668 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:25.668 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907084+0000", 2024-04-24T15:33:25.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013497100000000001 2024-04-24T15:33:25.668 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:25.668 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:25.668 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:25.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.669 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48710 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2179072~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:25.669 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889332+0000", 2024-04-24T15:33:25.669 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.740783719, 2024-04-24T15:33:25.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017944720000000001, 2024-04-24T15:33:25.669 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:25.669 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:25.669 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:25.669 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:25.669 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:25.669 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48710 2024-04-24T15:33:25.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.669 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:25.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:25.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889332+0000", 2024-04-24T15:33:25.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:25.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:25.670 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889332+0000", 2024-04-24T15:33:25.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:25.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:25.670 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889332+0000", 2024-04-24T15:33:25.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:25.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:25.671 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889334+0000", 2024-04-24T15:33:25.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4210000000000002e-06 2024-04-24T15:33:25.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.671 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:25.671 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889340+0000", 2024-04-24T15:33:25.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7209999999999999e-06 2024-04-24T15:33:25.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.671 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:25.671 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889344+0000", 2024-04-24T15:33:25.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.726e-06 2024-04-24T15:33:25.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.671 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:25.671 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907119+0000", 2024-04-24T15:33:25.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017775551000000001 2024-04-24T15:33:25.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.672 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:25.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907190+0000", 2024-04-24T15:33:25.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0634000000000002e-05 2024-04-24T15:33:25.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.672 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:25.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907277+0000", 2024-04-24T15:33:25.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7120999999999995e-05 2024-04-24T15:33:25.672 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:25.672 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:25.672 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:25.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.673 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48711 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2244608~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:25.673 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889348+0000", 2024-04-24T15:33:25.673 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.740768102000001, 2024-04-24T15:33:25.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018188109000000001, 2024-04-24T15:33:25.673 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:25.673 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:25.673 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:25.673 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:25.673 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:25.673 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48711 2024-04-24T15:33:25.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.673 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:25.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.673 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:25.674 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889348+0000", 2024-04-24T15:33:25.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:25.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.674 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:25.674 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889348+0000", 2024-04-24T15:33:25.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:25.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.674 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:25.674 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889348+0000", 2024-04-24T15:33:25.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:25.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:25.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889350+0000", 2024-04-24T15:33:25.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2330000000000001e-06 2024-04-24T15:33:25.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:25.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889356+0000", 2024-04-24T15:33:25.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8769999999999998e-06 2024-04-24T15:33:25.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:25.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889359+0000", 2024-04-24T15:33:25.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3699999999999999e-06 2024-04-24T15:33:25.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:25.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907316+0000", 2024-04-24T15:33:25.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017956857999999999 2024-04-24T15:33:25.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:25.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907402+0000", 2024-04-24T15:33:25.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5927000000000006e-05 2024-04-24T15:33:25.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:25.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907536+0000", 2024-04-24T15:33:25.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000134258 2024-04-24T15:33:25.676 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:25.677 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:25.677 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:25.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.677 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48712 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2310144~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:25.677 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889363+0000", 2024-04-24T15:33:25.677 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.740752783, 2024-04-24T15:33:25.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018314815000000002, 2024-04-24T15:33:25.677 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:25.677 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:25.677 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:25.677 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:25.677 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:25.677 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48712 2024-04-24T15:33:25.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.678 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:25.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:25.678 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889363+0000", 2024-04-24T15:33:25.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:25.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:25.678 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889363+0000", 2024-04-24T15:33:25.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:25.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.679 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:25.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889363+0000", 2024-04-24T15:33:25.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:25.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.679 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:25.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889366+0000", 2024-04-24T15:33:25.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6709999999999999e-06 2024-04-24T15:33:25.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.679 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:25.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889373+0000", 2024-04-24T15:33:25.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1269999999999997e-06 2024-04-24T15:33:25.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.679 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:25.680 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889377+0000", 2024-04-24T15:33:25.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1219999999999997e-06 2024-04-24T15:33:25.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.680 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:25.680 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907560+0000", 2024-04-24T15:33:25.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018183357000000001 2024-04-24T15:33:25.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.680 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:25.680 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907612+0000", 2024-04-24T15:33:25.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2172999999999997e-05 2024-04-24T15:33:25.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.680 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:25.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907678+0000", 2024-04-24T15:33:25.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5736999999999998e-05 2024-04-24T15:33:25.681 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:25.681 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:25.681 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:25.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.681 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48713 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2375680~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:25.681 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889381+0000", 2024-04-24T15:33:25.681 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.740734890000001, 2024-04-24T15:33:25.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018556019, 2024-04-24T15:33:25.681 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:25.681 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:25.681 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:25.682 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:25.682 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:25.682 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48713 2024-04-24T15:33:25.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.682 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:25.682 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.682 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:25.682 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889381+0000", 2024-04-24T15:33:25.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:25.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.682 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.682 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:25.682 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889381+0000", 2024-04-24T15:33:25.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:25.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.683 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:25.683 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889381+0000", 2024-04-24T15:33:25.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:25.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.683 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:25.683 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889388+0000", 2024-04-24T15:33:25.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8809999999999998e-06 2024-04-24T15:33:25.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.683 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:25.683 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889394+0000", 2024-04-24T15:33:25.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0410000000000003e-06 2024-04-24T15:33:25.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:25.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889397+0000", 2024-04-24T15:33:25.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5939999999999998e-06 2024-04-24T15:33:25.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:25.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907711+0000", 2024-04-24T15:33:25.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018313949999999999 2024-04-24T15:33:25.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:25.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907802+0000", 2024-04-24T15:33:25.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0307000000000001e-05 2024-04-24T15:33:25.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:25.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907937+0000", 2024-04-24T15:33:25.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013561899999999999 2024-04-24T15:33:25.685 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:25.685 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:25.685 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:25.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.685 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48714 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2441216~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:25.685 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889435+0000", 2024-04-24T15:33:25.685 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.740681542000001, 2024-04-24T15:33:25.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018636040999999999, 2024-04-24T15:33:25.686 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:25.686 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:25.686 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:25.686 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:25.686 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:25.686 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48714 2024-04-24T15:33:25.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.686 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:25.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:25.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889435+0000", 2024-04-24T15:33:25.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:25.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.687 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:25.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889435+0000", 2024-04-24T15:33:25.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:25.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.687 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:25.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889434+0000", 2024-04-24T15:33:25.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:25.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.687 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:25.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889437+0000", 2024-04-24T15:33:25.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.875e-06 2024-04-24T15:33:25.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:25.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889444+0000", 2024-04-24T15:33:25.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9870000000000002e-06 2024-04-24T15:33:25.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:25.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889448+0000", 2024-04-24T15:33:25.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3030000000000001e-06 2024-04-24T15:33:25.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:25.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907952+0000", 2024-04-24T15:33:25.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018504245999999999 2024-04-24T15:33:25.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:25.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908007+0000", 2024-04-24T15:33:25.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4579e-05 2024-04-24T15:33:25.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:25.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908071+0000", 2024-04-24T15:33:25.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3615999999999995e-05 2024-04-24T15:33:25.689 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:25.689 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:25.689 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:25.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.690 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48715 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2506752~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:25.690 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889453+0000", 2024-04-24T15:33:25.690 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.740663178, 2024-04-24T15:33:25.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018873935000000001, 2024-04-24T15:33:25.690 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:25.690 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:25.690 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:25.690 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:25.690 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:25.690 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48715 2024-04-24T15:33:25.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.690 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:25.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:25.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889453+0000", 2024-04-24T15:33:25.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:25.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:25.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889453+0000", 2024-04-24T15:33:25.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:25.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:25.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889452+0000", 2024-04-24T15:33:25.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:25.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.692 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:25.692 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889459+0000", 2024-04-24T15:33:25.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9269999999999999e-06 2024-04-24T15:33:25.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.692 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:25.692 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889468+0000", 2024-04-24T15:33:25.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2390000000000002e-06 2024-04-24T15:33:25.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.692 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:25.692 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889475+0000", 2024-04-24T15:33:25.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3939999999999998e-06 2024-04-24T15:33:25.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:25.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908103+0000", 2024-04-24T15:33:25.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018628388999999999 2024-04-24T15:33:25.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:25.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908193+0000", 2024-04-24T15:33:25.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9343000000000002e-05 2024-04-24T15:33:25.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:25.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908327+0000", 2024-04-24T15:33:25.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013415899999999999 2024-04-24T15:33:25.694 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:25.694 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:25.694 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:25.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.694 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48716 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2572288~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:25.694 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889480+0000", 2024-04-24T15:33:25.694 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.740636328000001, 2024-04-24T15:33:25.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018992862999999999, 2024-04-24T15:33:25.694 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:25.694 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:25.694 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:25.694 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:25.695 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:25.695 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48716 2024-04-24T15:33:25.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.695 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:25.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:25.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889480+0000", 2024-04-24T15:33:25.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:25.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:25.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889480+0000", 2024-04-24T15:33:25.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:25.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.696 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:25.696 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889479+0000", 2024-04-24T15:33:25.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:25.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.696 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:25.696 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889482+0000", 2024-04-24T15:33:25.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4729999999999999e-06 2024-04-24T15:33:25.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.696 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:25.696 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889488+0000", 2024-04-24T15:33:25.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1750000000000002e-06 2024-04-24T15:33:25.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.697 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:25.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889501+0000", 2024-04-24T15:33:25.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.288e-05 2024-04-24T15:33:25.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.697 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:25.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908345+0000", 2024-04-24T15:33:25.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018843835999999999 2024-04-24T15:33:25.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.697 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:25.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908395+0000", 2024-04-24T15:33:25.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9765999999999998e-05 2024-04-24T15:33:25.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.697 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:25.698 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908473+0000", 2024-04-24T15:33:25.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8159999999999997e-05 2024-04-24T15:33:25.698 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:25.698 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:25.698 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:25.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.698 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48717 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2637824~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:25.698 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889506+0000", 2024-04-24T15:33:25.698 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.740610076999999, 2024-04-24T15:33:25.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.019241864000000001, 2024-04-24T15:33:25.698 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:25.698 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:25.698 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:25.699 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:25.699 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:25.699 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48717 2024-04-24T15:33:25.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.699 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:25.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:25.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889506+0000", 2024-04-24T15:33:25.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:25.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:25.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889506+0000", 2024-04-24T15:33:25.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:25.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:25.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889506+0000", 2024-04-24T15:33:25.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:25.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:25.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889508+0000", 2024-04-24T15:33:25.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6900000000000001e-06 2024-04-24T15:33:25.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:25.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889515+0000", 2024-04-24T15:33:25.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5409999999999997e-06 2024-04-24T15:33:25.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:25.701 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889519+0000", 2024-04-24T15:33:25.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7129999999999999e-06 2024-04-24T15:33:25.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:25.701 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908520+0000", 2024-04-24T15:33:25.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.019001606000000001 2024-04-24T15:33:25.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:25.701 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908612+0000", 2024-04-24T15:33:25.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1909000000000004e-05 2024-04-24T15:33:25.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.702 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:25.702 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908748+0000", 2024-04-24T15:33:25.702 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000135847 2024-04-24T15:33:25.702 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:25.702 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:25.702 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:25.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.702 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48718 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2703360~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:25.702 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889523+0000", 2024-04-24T15:33:25.702 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.740593085, 2024-04-24T15:33:25.702 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.019451725999999999, 2024-04-24T15:33:25.702 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:25.703 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:25.703 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:25.703 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:25.703 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:25.703 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48718 2024-04-24T15:33:25.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.703 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:25.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.703 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:25.703 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889523+0000", 2024-04-24T15:33:25.703 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:25.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.703 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:25.703 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889523+0000", 2024-04-24T15:33:25.703 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:25.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.704 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.704 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:25.704 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889523+0000", 2024-04-24T15:33:25.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:25.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.704 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.704 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:25.704 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889525+0000", 2024-04-24T15:33:25.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9970000000000001e-06 2024-04-24T15:33:25.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.704 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.704 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:25.704 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889530+0000", 2024-04-24T15:33:25.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7610000000000004e-06 2024-04-24T15:33:25.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:25.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889534+0000", 2024-04-24T15:33:25.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6119999999999999e-06 2024-04-24T15:33:25.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:25.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908775+0000", 2024-04-24T15:33:25.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.019240709000000002 2024-04-24T15:33:25.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:25.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908856+0000", 2024-04-24T15:33:25.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0788e-05 2024-04-24T15:33:25.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.706 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:25.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908975+0000", 2024-04-24T15:33:25.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011927800000000001 2024-04-24T15:33:25.706 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:25.706 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:25.706 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:25.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.706 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:49872 4.1f 4:fe48f4c1:::rbd_data.114ab402c616.0000000000000014:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-04-24T15:33:25.706 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:12.416730+0000", 2024-04-24T15:33:25.706 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.213386107, 2024-04-24T15:33:25.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.020522327999999999, 2024-04-24T15:33:25.706 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:25.707 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:25.707 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:25.707 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:25.707 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:25.707 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 49872 2024-04-24T15:33:25.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.707 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:25.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.707 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:25.707 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.416730+0000", 2024-04-24T15:33:25.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:25.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.707 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:25.707 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.416730+0000", 2024-04-24T15:33:25.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:25.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:25.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.416729+0000", 2024-04-24T15:33:25.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:33:25.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:25.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.416734+0000", 2024-04-24T15:33:25.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0180000000000001e-06 2024-04-24T15:33:25.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:25.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.416744+0000", 2024-04-24T15:33:25.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0069e-05 2024-04-24T15:33:25.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:25.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.416749+0000", 2024-04-24T15:33:25.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7190000000000001e-06 2024-04-24T15:33:25.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:25.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.416777+0000", 2024-04-24T15:33:25.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7532999999999999e-05 2024-04-24T15:33:25.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:25.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.416824+0000", 2024-04-24T15:33:25.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7738000000000003e-05 2024-04-24T15:33:25.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:33:25.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.437145+0000", 2024-04-24T15:33:25.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.020320416000000001 2024-04-24T15:33:25.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:25.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.437189+0000", 2024-04-24T15:33:25.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4493000000000002e-05 2024-04-24T15:33:25.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:25.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.437252+0000", 2024-04-24T15:33:25.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3110999999999999e-05 2024-04-24T15:33:25.711 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:25.711 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:25.711 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:25.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.711 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:52545 4.a 4:5331820a:::rbd_data.114ab867d54d.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-04-24T15:33:25.711 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:14.399696+0000", 2024-04-24T15:33:25.711 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 11.230419703999999, 2024-04-24T15:33:25.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016877893000000001, 2024-04-24T15:33:25.711 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:25.711 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:25.711 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:25.711 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:25.712 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:25.712 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 52545 2024-04-24T15:33:25.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.712 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:25.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:25.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.399696+0000", 2024-04-24T15:33:25.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:25.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:25.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.399696+0000", 2024-04-24T15:33:25.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:25.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.713 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:25.713 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.399695+0000", 2024-04-24T15:33:25.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999981 2024-04-24T15:33:25.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.713 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:25.713 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.399704+0000", 2024-04-24T15:33:25.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.098e-06 2024-04-24T15:33:25.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.713 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:25.713 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.399721+0000", 2024-04-24T15:33:25.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7088999999999999e-05 2024-04-24T15:33:25.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:25.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.399728+0000", 2024-04-24T15:33:25.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5699999999999998e-06 2024-04-24T15:33:25.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:25.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.399775+0000", 2024-04-24T15:33:25.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7445999999999999e-05 2024-04-24T15:33:25.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:25.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.399840+0000", 2024-04-24T15:33:25.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5492000000000002e-05 2024-04-24T15:33:25.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:33:25.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.416468+0000", 2024-04-24T15:33:25.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016627527 2024-04-24T15:33:25.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:25.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.416521+0000", 2024-04-24T15:33:25.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2988000000000002e-05 2024-04-24T15:33:25.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:25.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.416574+0000", 2024-04-24T15:33:25.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3363999999999999e-05 2024-04-24T15:33:25.715 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:25.716 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:25.716 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:25.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.716 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:66966 4.1f 4:fb3b69af:::rbd_data.114aed92ca6e.000000000000001a:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-04-24T15:33:25.716 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:21.265525+0000", 2024-04-24T15:33:25.716 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.3645911630000001, 2024-04-24T15:33:25.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018862027999999999, 2024-04-24T15:33:25.716 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:25.716 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:25.716 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:25.716 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:25.716 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:25.716 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 66966 2024-04-24T15:33:25.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.717 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:25.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:25.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:21.265525+0000", 2024-04-24T15:33:25.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:25.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:25.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:21.265525+0000", 2024-04-24T15:33:25.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:25.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:25.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:21.265523+0000", 2024-04-24T15:33:25.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999981 2024-04-24T15:33:25.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:25.718 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:21.265536+0000", 2024-04-24T15:33:25.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2489e-05 2024-04-24T15:33:25.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:25.718 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:21.265550+0000", 2024-04-24T15:33:25.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4218e-05 2024-04-24T15:33:25.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:25.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:21.265556+0000", 2024-04-24T15:33:25.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.241e-06 2024-04-24T15:33:25.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:25.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:21.265606+0000", 2024-04-24T15:33:25.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9972000000000002e-05 2024-04-24T15:33:25.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:25.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:21.265670+0000", 2024-04-24T15:33:25.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3425999999999999e-05 2024-04-24T15:33:25.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:33:25.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:21.284297+0000", 2024-04-24T15:33:25.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018627517999999999 2024-04-24T15:33:25.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:25.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:21.284339+0000", 2024-04-24T15:33:25.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1628000000000002e-05 2024-04-24T15:33:25.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:25.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:25.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:25.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:21.284387+0000", 2024-04-24T15:33:25.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8242000000000003e-05 2024-04-24T15:33:25.720 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:25.720 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:25.720 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:25.721 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:25.721 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:25.721 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:25.722 INFO:teuthology.orchestra.run.smithi096.stdout:5704 write 0xa2046f6 thru 0xa20a455 (0x5d60 bytes) 2024-04-24T15:33:25.722 INFO:teuthology.orchestra.run.smithi096.stdout:5705 write 0x5baf48f thru 0x5bb5132 (0x5ca4 bytes) 2024-04-24T15:33:25.722 INFO:teuthology.orchestra.run.smithi096.stdout:5706 read 0x6539967 thru 0x653b556 (0x1bf0 bytes) 2024-04-24T15:33:25.723 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-24T15:33:25.725 INFO:teuthology.orchestra.run.smithi096.stdout:5707 read 0x335423b thru 0x3359984 (0x574a bytes) 2024-04-24T15:33:25.725 INFO:teuthology.orchestra.run.smithi096.stdout:5708 writesame 0x37af37e thru 0x37b4866 data_size 0x1(0x54e9 bytes) 2024-04-24T15:33:25.726 INFO:teuthology.orchestra.run.smithi096.stdout:5712 writesame 0x91ad32e thru 0x91afb15 data_size 0x4(0x27e8 bytes) 2024-04-24T15:33:25.726 INFO:teuthology.orchestra.run.smithi096.stdout:5713 read 0x2f6685c thru 0x2f7327a (0xca1f bytes) 2024-04-24T15:33:25.726 INFO:teuthology.orchestra.run.smithi096.stdout:5714 read 0xd0fa97 thru 0xd1c905 (0xce6f bytes) 2024-04-24T15:33:25.726 INFO:teuthology.orchestra.run.smithi096.stdout:5715 read 0x3c22925 thru 0x3c2a153 (0x782f bytes) 2024-04-24T15:33:25.726 INFO:teuthology.orchestra.run.smithi096.stdout:5716 write 0x1366cd thru 0x1398b6 (0x31ea bytes) 2024-04-24T15:33:25.726 INFO:teuthology.orchestra.run.smithi096.stdout:5717 write 0x9dafe6d thru 0x9dba7ea (0xa97e bytes) 2024-04-24T15:33:25.726 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:25.726 INFO:teuthology.orchestra.run.smithi096.stdout:5720 read 0x72b93fe thru 0x72c03dc (0x6fdf bytes) 2024-04-24T15:33:25.726 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:25.726 INFO:teuthology.orchestra.run.smithi096.stdout:5722 trunc from 0xa20a456 to 0xecd7885 2024-04-24T15:33:25.726 INFO:teuthology.orchestra.run.smithi096.stdout:5723 punch from 0x3d45d5d to 0x3d49673, (0x3916 bytes) 2024-04-24T15:33:25.726 INFO:teuthology.orchestra.run.smithi096.stdout:5724 read 0x1b52cf7 thru 0x1b56a6b (0x3d75 bytes) 2024-04-24T15:33:25.726 INFO:teuthology.orchestra.run.smithi096.stdout:5725 read 0xba77677 thru 0xba7ae6a (0x37f4 bytes) 2024-04-24T15:33:25.727 INFO:teuthology.orchestra.run.smithi096.stdout:5726 read 0x6c855fc thru 0x6c90b85 (0xb58a bytes) 2024-04-24T15:33:25.727 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:25.727 INFO:teuthology.orchestra.run.smithi096.stdout:5731 read 0x9ea8b03 thru 0x9eaa9c4 (0x1ec2 bytes) 2024-04-24T15:33:25.727 INFO:teuthology.orchestra.run.smithi096.stdout:5733 trunc from 0xecd7885 to 0x5796180 2024-04-24T15:33:25.727 INFO:teuthology.orchestra.run.smithi096.stdout:5734 read 0x1ecba89 thru 0x1ed4e71 (0x93e9 bytes) 2024-04-24T15:33:25.727 INFO:teuthology.orchestra.run.smithi096.stdout:5736 punch from 0x2384f42 to 0x238d0e7, (0x81a5 bytes) 2024-04-24T15:33:25.727 INFO:teuthology.orchestra.run.smithi096.stdout:5737 read 0x355c10d thru 0x3562333 (0x6227 bytes) 2024-04-24T15:33:25.727 INFO:teuthology.orchestra.run.smithi096.stdout:5738 writesame 0x114961 thru 0x11ab30 data_size 0x28(0x61d0 bytes) 2024-04-24T15:33:25.727 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:25.727 INFO:teuthology.orchestra.run.smithi096.stdout:5741 write 0xd810f4 thru 0xd886fd (0x760a bytes) 2024-04-24T15:33:25.727 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:33:25.723+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:33:25.732 INFO:teuthology.orchestra.run.smithi096.stdout:5743 writesame 0xcd0a528 thru 0xcd181c5 data_size 0x3(0xdc9e bytes) 2024-04-24T15:33:25.732 INFO:teuthology.orchestra.run.smithi096.stdout:5744 read 0xccc10d2 thru 0xccc5859 (0x4788 bytes) 2024-04-24T15:33:25.734 INFO:teuthology.orchestra.run.smithi096.stdout:5746 trunc from 0xcd181c6 to 0xae3048f 2024-04-24T15:33:25.740 INFO:teuthology.orchestra.run.smithi096.stdout:5748 read 0x8444c0d thru 0x844b056 (0x644a bytes) 2024-04-24T15:33:25.740 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:25.750 INFO:teuthology.orchestra.run.smithi096.stdout:5751 trunc from 0xae3048f to 0x2529df6 2024-04-24T15:33:25.762 INFO:teuthology.orchestra.run.smithi096.stdout:5753 punch from 0x1b97239 to 0x1b9cdac, (0x5b73 bytes) 2024-04-24T15:33:25.762 INFO:teuthology.orchestra.run.smithi096.stdout:5754 read 0x1477b0b thru 0x148300c (0xb502 bytes) 2024-04-24T15:33:25.763 INFO:teuthology.orchestra.run.smithi096.stdout:5755 read 0x201fa18 thru 0x2028ae7 (0x90d0 bytes) 2024-04-24T15:33:25.763 INFO:teuthology.orchestra.run.smithi096.stdout:5758 read 0x232ba07 thru 0x232c3b7 (0x9b1 bytes) 2024-04-24T15:33:25.776 INFO:teuthology.orchestra.run.smithi096.stdout:5761 writesame 0x7fa9cc2 thru 0x7fb9853 data_size 0xd(0xfb92 bytes) 2024-04-24T15:33:25.783 INFO:teuthology.orchestra.run.smithi096.stdout:5762 write 0x9080dc9 thru 0x90867bb (0x59f3 bytes) 2024-04-24T15:33:25.793 INFO:teuthology.orchestra.run.smithi096.stdout:5763 trunc from 0x90867bc to 0xb4a656f 2024-04-24T15:33:25.798 INFO:teuthology.orchestra.run.smithi096.stdout:5764 read 0x661afb5 thru 0x66201ef (0x523b bytes) 2024-04-24T15:33:25.798 INFO:teuthology.orchestra.run.smithi096.stdout:5765 punch from 0x375598b to 0x37572ea, (0x195f bytes) 2024-04-24T15:33:25.798 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:25.798 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:25.798 INFO:teuthology.orchestra.run.smithi096.stdout:5769 read 0x2ae1064 thru 0x2ae5f5b (0x4ef8 bytes) 2024-04-24T15:33:25.806 INFO:teuthology.orchestra.run.smithi096.stdout:5772 write 0xd4e997c thru 0xd4ed0d3 (0x3758 bytes) 2024-04-24T15:33:25.810 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:25.810 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:25.810 INFO:teuthology.orchestra.run.smithi096.stdout:5776 punch from 0x87216e5 to 0x87274d8, (0x5df3 bytes) 2024-04-24T15:33:25.810 INFO:teuthology.orchestra.run.smithi096.stdout:5777 writesame 0x70a4adb thru 0x70a8b3a data_size 0xa(0x4060 bytes) 2024-04-24T15:33:25.821 INFO:teuthology.orchestra.run.smithi096.stdout:5778 trunc from 0xd4ed0d4 to 0x6ebcea0 2024-04-24T15:33:25.823 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:25.823+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:25.833 INFO:teuthology.orchestra.run.smithi096.stdout:5781 trunc from 0x6ebcea0 to 0x6f1de88 2024-04-24T15:33:25.840 INFO:teuthology.orchestra.run.smithi096.stdout:5782 trunc from 0x6f1de88 to 0x76c431a 2024-04-24T15:33:25.858 INFO:teuthology.orchestra.run.smithi096.stdout:5783 writesame 0xa9f108e thru 0xa9f33c9 data_size 0x37(0x233c bytes) 2024-04-24T15:33:25.858 INFO:teuthology.orchestra.run.smithi096.stdout:5784 read 0x87b1f1 thru 0x885178 (0x9f88 bytes) 2024-04-24T15:33:25.858 INFO:teuthology.orchestra.run.smithi096.stdout:5785 write 0x5b8f255 thru 0x5b906f2 (0x149e bytes) 2024-04-24T15:33:25.861 INFO:teuthology.orchestra.run.smithi096.stdout:5788 read 0x586bc12 thru 0x5873196 (0x7585 bytes) 2024-04-24T15:33:25.861 INFO:teuthology.orchestra.run.smithi096.stdout:5789 read 0x921f63 thru 0x924302 (0x23a0 bytes) 2024-04-24T15:33:25.861 INFO:teuthology.orchestra.run.smithi096.stdout:5791 write 0x29b0a36 thru 0x29b7c70 (0x723b bytes) 2024-04-24T15:33:25.864 INFO:teuthology.orchestra.run.smithi096.stdout:5792 punch from 0x12bc9fb to 0x12be4c5, (0x1aca bytes) 2024-04-24T15:33:25.864 INFO:teuthology.orchestra.run.smithi096.stdout:5793 write 0x8116530 thru 0x8116868 (0x339 bytes) 2024-04-24T15:33:25.876 INFO:teuthology.orchestra.run.smithi096.stdout:5795 write 0xb96db8c thru 0xb977e2f (0xa2a4 bytes) 2024-04-24T15:33:25.880 INFO:teuthology.orchestra.run.smithi096.stdout:5796 read 0x801410d thru 0x80206e4 (0xc5d8 bytes) 2024-04-24T15:33:25.880 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:25.880 INFO:teuthology.orchestra.run.smithi096.stdout:5798 punch from 0x5ff089e to 0x5ff50b4, (0x4816 bytes) 2024-04-24T15:33:25.880 INFO:teuthology.orchestra.run.smithi096.stdout:5799 punch from 0x813e84c to 0x8145fa4, (0x7758 bytes) 2024-04-24T15:33:25.880 INFO:teuthology.orchestra.run.smithi096.stdout:5800 punch from 0x93eca98 to 0x93ef4cc, (0x2a34 bytes) 2024-04-24T15:33:25.880 INFO:teuthology.orchestra.run.smithi096.stdout:5801 read 0x86fe368 thru 0x870921c (0xaeb5 bytes) 2024-04-24T15:33:25.880 INFO:teuthology.orchestra.run.smithi096.stdout:5802 read 0x9e8585f thru 0x9e8fa50 (0xa1f2 bytes) 2024-04-24T15:33:25.880 INFO:teuthology.orchestra.run.smithi096.stdout:5803 writesame 0x7300fe7 thru 0x7304319 data_size 0x3(0x3333 bytes) 2024-04-24T15:33:25.881 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:25.881 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:33:25.881 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-04-24T15:33:25.881 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:25.891 INFO:teuthology.orchestra.run.smithi096.stdout:5806 write 0xc682b8c thru 0xc687572 (0x49e7 bytes) 2024-04-24T15:33:25.894 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-24T15:33:25.894 INFO:teuthology.orchestra.run.smithi096.stdout:5807 read 0xb53d878 thru 0xb5470ff (0x9888 bytes) 2024-04-24T15:33:25.895 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-24T15:33:25.896 INFO:teuthology.orchestra.run.smithi096.stdout:5809 read 0x960d3fa thru 0x961bc8b (0xe892 bytes) 2024-04-24T15:33:25.896 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:25.896 INFO:teuthology.orchestra.run.smithi096.stdout:5812 writesame 0xc316693 thru 0xc3238ff data_size 0x1(0xd26d bytes) 2024-04-24T15:33:25.913 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:25.913 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:25.914 INFO:teuthology.orchestra.run.smithi096.stdout:5815 write 0x19faa96 thru 0x1a035b7 (0x8b22 bytes) 2024-04-24T15:33:25.922 INFO:teuthology.orchestra.run.smithi096.stdout:5816 write 0x4ac44a thru 0x4ae5c7 (0x217e bytes) 2024-04-24T15:33:25.928 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-24T15:33:25.928 INFO:teuthology.orchestra.run.smithi096.stdout:5817 punch from 0x58ff43a to 0x590774e, (0x8314 bytes) 2024-04-24T15:33:25.929 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:25.930+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:25.938 INFO:teuthology.orchestra.run.smithi096.stdout:5819 write 0xd827f24 thru 0xd82af60 (0x303d bytes) 2024-04-24T15:33:25.944 INFO:teuthology.orchestra.run.smithi096.stdout:5820 read 0x589376a thru 0x58a248c (0xed23 bytes) 2024-04-24T15:33:25.944 INFO:teuthology.orchestra.run.smithi096.stdout:5821 write 0x7fd39 thru 0x864f8 (0x67c0 bytes) 2024-04-24T15:33:25.960 INFO:teuthology.orchestra.run.smithi096.stdout:5822 write 0x134e59d thru 0x1354587 (0x5feb bytes) 2024-04-24T15:33:25.962 INFO:teuthology.orchestra.run.smithi096.stdout:5823 read 0xa2c4d68 thru 0xa2d22b5 (0xd54e bytes) 2024-04-24T15:33:25.962 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:25.962 INFO:teuthology.orchestra.run.smithi096.stdout:5825 write 0xd438a7 thru 0xd5036c (0xcac6 bytes) 2024-04-24T15:33:25.968 INFO:teuthology.orchestra.run.smithi096.stdout:5826 read 0xd271d0d thru 0xd27f852 (0xdb46 bytes) 2024-04-24T15:33:25.968 INFO:teuthology.orchestra.run.smithi096.stdout:5828 read 0x45f5ed3 thru 0x4605613 (0xf741 bytes) 2024-04-24T15:33:25.968 INFO:teuthology.orchestra.run.smithi096.stdout:5829 read 0x1dfc352 thru 0x1e0a5f8 (0xe2a7 bytes) 2024-04-24T15:33:25.973 INFO:teuthology.orchestra.run.smithi096.stdout:5831 trunc from 0xd82af61 to 0x956bc50 2024-04-24T15:33:25.989 INFO:teuthology.orchestra.run.smithi096.stdout:5832 trunc from 0x956bc50 to 0x6819fc2 2024-04-24T15:33:26.006 INFO:teuthology.orchestra.run.smithi096.stdout:5834 read 0x4daf112 thru 0x4daf16f (0x5e bytes) 2024-04-24T15:33:26.006 INFO:teuthology.orchestra.run.smithi096.stdout:5835 writesame 0x400aa38 thru 0x400e31e data_size 0x1(0x38e7 bytes) 2024-04-24T15:33:26.019 INFO:teuthology.orchestra.run.smithi096.stdout:5837 read 0x31c3d5b thru 0x31d14e8 (0xd78e bytes) 2024-04-24T15:33:26.019 INFO:teuthology.orchestra.run.smithi096.stdout:5838 read 0xec6c0e thru 0xec8f1e (0x2311 bytes) 2024-04-24T15:33:26.019 INFO:teuthology.orchestra.run.smithi096.stdout:5839 write 0x3995a44 thru 0x399eb6c (0x9129 bytes) 2024-04-24T15:33:26.023 INFO:teuthology.orchestra.run.smithi096.stdout:5842 write 0x26e0bf3 thru 0x26edbbc (0xcfca bytes) 2024-04-24T15:33:26.024 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:26.024+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:26.028 INFO:teuthology.orchestra.run.smithi096.stdout:5844 read 0x567b63a thru 0x5681b20 (0x64e7 bytes) 2024-04-24T15:33:26.028 INFO:teuthology.orchestra.run.smithi096.stdout:5845 punch from 0x524fbae to 0x5257eb8, (0x830a bytes) 2024-04-24T15:33:26.029 INFO:teuthology.orchestra.run.smithi096.stdout:5846 punch from 0x458ab60 to 0x4590ba5, (0x6045 bytes) 2024-04-24T15:33:26.032 INFO:teuthology.orchestra.run.smithi096.stdout:5847 trunc from 0x6819fc2 to 0x99914a4 2024-04-24T15:33:26.039 INFO:teuthology.orchestra.run.smithi096.stdout:5851 punch from 0x736bbcf to 0x736d3b3, (0x17e4 bytes) 2024-04-24T15:33:26.039 INFO:teuthology.orchestra.run.smithi096.stdout:5853 read 0x710d5b thru 0x716fbe (0x6264 bytes) 2024-04-24T15:33:26.039 INFO:teuthology.orchestra.run.smithi096.stdout:5854 read 0x1284daa thru 0x1289073 (0x42ca bytes) 2024-04-24T15:33:26.047 INFO:teuthology.orchestra.run.smithi096.stdout:5856 write 0xb0f36f8 thru 0xb0f59d0 (0x22d9 bytes) 2024-04-24T15:33:26.054 INFO:teuthology.orchestra.run.smithi096.stdout:5857 punch from 0x63bce60 to 0x63c523a, (0x83da bytes) 2024-04-24T15:33:26.055 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:26.055 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:26.055 INFO:teuthology.orchestra.run.smithi096.stdout:5864 punch from 0x36a397 to 0x376374, (0xbfdd bytes) 2024-04-24T15:33:26.055 INFO:teuthology.orchestra.run.smithi096.stdout:5865 write 0x5bb3036 thru 0x5bc10c0 (0xe08b bytes) 2024-04-24T15:33:26.056 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:26.064 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:26.065 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:33:26.065 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-04-24T15:33:26.065 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-04-24T15:33:26.065 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:26.066 INFO:teuthology.orchestra.run.smithi096.stdout:5867 write 0xbf3039c thru 0xbf3a43d (0xa0a2 bytes) 2024-04-24T15:33:26.071 INFO:teuthology.orchestra.run.smithi096.stdout:5868 write 0x41f056c thru 0x41f7c1e (0x76b3 bytes) 2024-04-24T15:33:26.077 INFO:teuthology.orchestra.run.smithi096.stdout:5869 read 0x19f838e thru 0x19f8c9d (0x910 bytes) 2024-04-24T15:33:26.078 INFO:teuthology.orchestra.run.smithi096.stdout:5870 read 0x6c01376 thru 0x6c0e27c (0xcf07 bytes) 2024-04-24T15:33:26.078 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:26.080 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-24T15:33:26.084 INFO:teuthology.orchestra.run.smithi096.stdout:5873 write 0xc2cb5b0 thru 0xc2cbf2b (0x97c bytes) 2024-04-24T15:33:26.087 INFO:teuthology.orchestra.run.smithi096.stdout:5874 clone 32 order 24 su 131072 sc 10 2024-04-24T15:33:26.098 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-04-24T15:33:26.119 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-04-24T15:33:26.125 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:26.125+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:26.137 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-04-24T15:33:26.155 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-04-24T15:33:26.174 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-04-24T15:33:26.192 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-04-24T15:33:26.210 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:26.210 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:26.213 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-04-24T15:33:26.213 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T15:33:26.213 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T15:33:26.213 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T15:33:26.213 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T15:33:26.213 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T15:33:26.214 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T15:33:26.214 INFO:teuthology.orchestra.run.smithi096.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-04-24T15:33:26.214 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T15:33:26.216 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-24T15:33:26.216 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-24T15:33:26.222 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-24T15:33:26.226 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:26.227+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:26.327 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:26.327+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:26.386 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:26.386 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:26.396 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-24T15:33:26.437 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:26.437+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:26.529 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:26.529+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:26.560 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:26.560 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:26.570 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-24T15:33:26.630 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:26.631+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:26.731 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:26.731+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:26.732 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:26.732 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:26.743 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-24T15:33:26.832 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:26.832+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:26.866 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:26.866 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:26.876 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-24T15:33:26.893 INFO:teuthology.orchestra.run.smithi096.stderr:nodeep-scrub is unset 2024-04-24T15:33:26.932 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:26.933+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:26.964 INFO:teuthology.orchestra.run.smithi096.stdout:truncating image image_client.0-clone31 from 0xc2cbf2c (overlap 0x124e4cd) to 0x0 2024-04-24T15:33:27.034 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:27.035+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:27.039 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:27.040 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:27.051 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-24T15:33:27.057 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #30, image image_client.0-clone30 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent31 2024-04-24T15:33:27.059 INFO:teuthology.orchestra.run.smithi096.stdout:5877 punch from 0x19902a5 to 0x1992190, (0x1eeb bytes) 2024-04-24T15:33:27.062 INFO:teuthology.orchestra.run.smithi096.stdout:5878 read 0x9c07f24 thru 0x9c15284 (0xd361 bytes) 2024-04-24T15:33:27.062 INFO:teuthology.orchestra.run.smithi096.stdout:5879 read 0x5c9494c thru 0x5ca0bc8 (0xc27d bytes) 2024-04-24T15:33:27.071 INFO:teuthology.orchestra.run.smithi096.stdout:5880 trunc from 0xc2cbf2c to 0x53c3ff6 2024-04-24T15:33:27.108 INFO:teuthology.orchestra.run.smithi096.stdout:5881 read 0x38b0df6 thru 0x38be015 (0xd220 bytes) 2024-04-24T15:33:27.117 INFO:teuthology.orchestra.run.smithi096.stdout:5882 writesame 0x82e3dc0 thru 0x82ef9e1 data_size 0x45(0xbc22 bytes) 2024-04-24T15:33:27.119 INFO:teuthology.orchestra.run.smithi096.stdout:5883 punch from 0x64c1b3e to 0x64c4537, (0x29f9 bytes) 2024-04-24T15:33:27.119 INFO:teuthology.orchestra.run.smithi096.stdout:5884 punch from 0x610d2da to 0x6116461, (0x9187 bytes) 2024-04-24T15:33:27.119 INFO:teuthology.orchestra.run.smithi096.stdout:5885 write 0x452962 thru 0x45aa96 (0x8135 bytes) 2024-04-24T15:33:27.120 INFO:teuthology.orchestra.run.smithi096.stdout:5886 write 0x78fd4ff thru 0x7904cc8 (0x77ca bytes) 2024-04-24T15:33:27.124 INFO:teuthology.orchestra.run.smithi096.stdout:5887 write 0x3ea51e0 thru 0x3ea7ceb (0x2b0c bytes) 2024-04-24T15:33:27.126 INFO:teuthology.orchestra.run.smithi096.stdout:5888 punch from 0x66a93a3 to 0x66af473, (0x60d0 bytes) 2024-04-24T15:33:27.127 INFO:teuthology.orchestra.run.smithi096.stdout:5889 read 0x5437d6 thru 0x55344b (0xfc76 bytes) 2024-04-24T15:33:27.127 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:27.127 INFO:teuthology.orchestra.run.smithi096.stdout:5891 clone 33 order 23 su 131072 sc 14 2024-04-24T15:33:27.135 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:27.136+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:27.188 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:27.188 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:27.198 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-24T15:33:27.239 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:33:27.239+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:33:27.322 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:27.322 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:27.332 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-24T15:33:27.376 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:33:27.377+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:33:27.438 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:33:27.438+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:33:27.455 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:27.455 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:27.465 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-24T15:33:27.539 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:33:27.539+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:33:27.629 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:27.630 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:27.640 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-24T15:33:27.640 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:27.640+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:27.741 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:27.741+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:27.804 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:27.804 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:27.815 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-24T15:33:27.842 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:27.843+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:27.943 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:27.943 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:27.943 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:27.943+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:27.954 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-24T15:33:27.971 INFO:teuthology.orchestra.run.smithi096.stdout:truncating image image_client.0-clone32 from 0x82ef9e2 (overlap 0x53c3ff6) to 0x860665 2024-04-24T15:33:28.044 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:33:28.044+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:33:28.048 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #31, image image_client.0-clone31 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent32 2024-04-24T15:33:28.049 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:28.049 INFO:teuthology.orchestra.run.smithi096.stdout:5893 punch from 0xb1df8d to 0xb26961, (0x89d4 bytes) 2024-04-24T15:33:28.055 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:28.056 INFO:teuthology.orchestra.run.smithi096.stdout:5895 read 0x7de775e thru 0x7df6712 (0xefb5 bytes) 2024-04-24T15:33:28.056 INFO:teuthology.orchestra.run.smithi096.stdout:5896 read 0x56962db thru 0x569f8b3 (0x95d9 bytes) 2024-04-24T15:33:28.056 INFO:teuthology.orchestra.run.smithi096.stdout:5898 read 0x234cb8b thru 0x2357c52 (0xb0c8 bytes) 2024-04-24T15:33:28.056 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:28.056 INFO:teuthology.orchestra.run.smithi096.stdout:5901 read 0x4941b9c thru 0x494c199 (0xa5fe bytes) 2024-04-24T15:33:28.064 INFO:teuthology.orchestra.run.smithi096.stdout:5902 write 0x8ce96da thru 0x8cf92c7 (0xfbee bytes) 2024-04-24T15:33:28.069 INFO:teuthology.orchestra.run.smithi096.stdout:5903 read 0x1905b97 thru 0x190bfe7 (0x6451 bytes) 2024-04-24T15:33:28.081 INFO:teuthology.orchestra.run.smithi096.stdout:5907 writesame 0xc99d92b thru 0xc99dda8 data_size 0x17(0x47e bytes) 2024-04-24T15:33:28.083 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:28.083 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:28.085 INFO:teuthology.orchestra.run.smithi096.stdout:5908 write 0x5769849 thru 0x5772114 (0x88cc bytes) 2024-04-24T15:33:28.094 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-24T15:33:28.102 INFO:teuthology.orchestra.run.smithi096.stdout:5911 trunc from 0xc99dda9 to 0x52c040c 2024-04-24T15:33:28.178 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:33:28.179+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:33:28.190 INFO:teuthology.orchestra.run.smithi096.stdout:5912 write 0x35eee61 thru 0x35f73c1 (0x8561 bytes) 2024-04-24T15:33:28.200 INFO:teuthology.orchestra.run.smithi096.stdout:5913 trunc from 0x52c040c to 0xc5b3976 2024-04-24T15:33:28.205 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:28.205 INFO:teuthology.orchestra.run.smithi096.stdout:5915 write 0xb3e8faa thru 0xb3ef6ea (0x6741 bytes) 2024-04-24T15:33:28.211 INFO:teuthology.orchestra.run.smithi096.stdout:5916 punch from 0x49f53b0 to 0x49fb0e2, (0x5d32 bytes) 2024-04-24T15:33:28.212 INFO:teuthology.orchestra.run.smithi096.stdout:5917 write 0x49d47d6 thru 0x49dab15 (0x6340 bytes) 2024-04-24T15:33:28.215 INFO:teuthology.orchestra.run.smithi096.stdout:5918 trunc from 0xc5b3976 to 0xe20c5b0 2024-04-24T15:33:28.217 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-24T15:33:28.223 INFO:teuthology.orchestra.run.smithi096.stdout:5922 read 0x345ddd9 thru 0x3469bfe (0xbe26 bytes) 2024-04-24T15:33:28.223 INFO:teuthology.orchestra.run.smithi096.stdout:5924 read 0x63b6734 thru 0x63b7758 (0x1025 bytes) 2024-04-24T15:33:28.223 INFO:teuthology.orchestra.run.smithi096.stdout:5925 writesame 0x3bb53e2 thru 0x3bbe63a data_size 0x1(0x9259 bytes) 2024-04-24T15:33:28.243 INFO:teuthology.orchestra.run.smithi096.stdout:5926 punch from 0x52b320e to 0x52c0d64, (0xdb56 bytes) 2024-04-24T15:33:28.243 INFO:teuthology.orchestra.run.smithi096.stdout:5927 write 0x9481813 thru 0x948a7b8 (0x8fa6 bytes) 2024-04-24T15:33:28.245 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:28.246+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:28.247 INFO:teuthology.orchestra.run.smithi096.stdout:5928 writesame 0xd34e393 thru 0xd35a21b data_size 0x3(0xbe89 bytes) 2024-04-24T15:33:28.257 INFO:teuthology.orchestra.run.smithi096.stdout:5929 read 0x7a31b28 thru 0x7a3f091 (0xd56a bytes) 2024-04-24T15:33:28.258 INFO:teuthology.orchestra.run.smithi096.stdout:5931 read 0x1c68d5 thru 0x1d06d3 (0x9dff bytes) 2024-04-24T15:33:28.258 INFO:teuthology.orchestra.run.smithi096.stdout:5932 punch from 0x85bb164 to 0x85c0c7c, (0x5b18 bytes) 2024-04-24T15:33:28.259 INFO:teuthology.orchestra.run.smithi096.stdout:5933 write 0x771031f thru 0x77107ec (0x4ce bytes) 2024-04-24T15:33:28.263 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:28.263 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:28.274 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-24T15:33:28.277 INFO:teuthology.orchestra.run.smithi096.stdout:5934 trunc from 0xe20c5b0 to 0x156b3ec 2024-04-24T15:33:28.310 INFO:teuthology.orchestra.run.smithi096.stdout:5937 write 0x5dd9c42 thru 0x5de63d0 (0xc78f bytes) 2024-04-24T15:33:28.313 INFO:teuthology.orchestra.run.smithi096.stdout:5938 write 0x5d87f35 thru 0x5d8bbea (0x3cb6 bytes) 2024-04-24T15:33:28.315 INFO:teuthology.orchestra.run.smithi096.stdout:5939 write 0x17f72f5 thru 0x180345b (0xc167 bytes) 2024-04-24T15:33:28.327 INFO:teuthology.orchestra.run.smithi096.stdout:5940 write 0x83ed75e thru 0x83f0cf0 (0x3593 bytes) 2024-04-24T15:33:28.331 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:28.331 INFO:teuthology.orchestra.run.smithi096.stdout:5944 punch from 0x6a0252f to 0x6a0a27d, (0x7d4e bytes) 2024-04-24T15:33:28.331 INFO:teuthology.orchestra.run.smithi096.stdout:5945 punch from 0x8177aea to 0x8185496, (0xd9ac bytes) 2024-04-24T15:33:28.331 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:28.335 INFO:teuthology.orchestra.run.smithi096.stdout:5949 trunc from 0x83f0cf1 to 0xb58f181 2024-04-24T15:33:28.341 INFO:teuthology.orchestra.run.smithi096.stdout:5950 write 0x957de96 thru 0x95805a5 (0x2710 bytes) 2024-04-24T15:33:28.347 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:28.347+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:28.348 INFO:teuthology.orchestra.run.smithi096.stdout:5951 write 0x45f0c40 thru 0x45fff90 (0xf351 bytes) 2024-04-24T15:33:28.350 INFO:teuthology.orchestra.run.smithi096.stdout:5953 read 0xb4a05a0 thru 0xb4add82 (0xd7e3 bytes) 2024-04-24T15:33:28.350 INFO:teuthology.orchestra.run.smithi096.stdout:5955 write 0x50403a5 thru 0x5049017 (0x8c73 bytes) 2024-04-24T15:33:28.351 INFO:teuthology.orchestra.run.smithi096.stdout:5956 read 0x5f912f9 thru 0x5fa0b55 (0xf85d bytes) 2024-04-24T15:33:28.351 INFO:teuthology.orchestra.run.smithi096.stdout:5957 writesame 0xa656416 thru 0xa65f66b data_size 0x1(0x9256 bytes) 2024-04-24T15:33:28.371 INFO:teuthology.orchestra.run.smithi096.stdout:5958 read 0x75e11af thru 0x75eceec (0xbd3e bytes) 2024-04-24T15:33:28.372 INFO:teuthology.orchestra.run.smithi096.stdout:5959 write 0x1490107 thru 0x1493794 (0x368e bytes) 2024-04-24T15:33:28.374 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-04-24T15:33:28.383 INFO:teuthology.orchestra.run.smithi096.stdout:5960 write 0xba0adc6 thru 0xba1a329 (0xf564 bytes) 2024-04-24T15:33:28.392 INFO:teuthology.orchestra.run.smithi096.stdout:5961 write 0xc6aa5ef thru 0xc6b954c (0xef5e bytes) 2024-04-24T15:33:28.394 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-04-24T15:33:28.396 INFO:teuthology.orchestra.run.smithi096.stdout:5962 read 0x699a90c thru 0x699fc76 (0x536b bytes) 2024-04-24T15:33:28.396 INFO:teuthology.orchestra.run.smithi096.stdout:5964 write 0xc50309f thru 0xc50f22a (0xc18c bytes) 2024-04-24T15:33:28.400 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:28.400 INFO:teuthology.orchestra.run.smithi096.stdout:5967 read 0x3205074 thru 0x320b274 (0x6201 bytes) 2024-04-24T15:33:28.401 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:28.401 INFO:teuthology.orchestra.run.smithi096.stdout:5971 read 0x61dee39 thru 0x61e9d63 (0xaf2b bytes) 2024-04-24T15:33:28.401 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:28.401 INFO:teuthology.orchestra.run.smithi096.stdout:5974 writesame 0x5837999 thru 0x5839c20 data_size 0x5(0x2288 bytes) 2024-04-24T15:33:28.403 INFO:teuthology.orchestra.run.smithi096.stdout:5976 read 0x8d0a0c6 thru 0x8d14aaf (0xa9ea bytes) 2024-04-24T15:33:28.404 INFO:teuthology.orchestra.run.smithi096.stdout:5977 punch from 0x57ee66 to 0x589d64, (0xaefe bytes) 2024-04-24T15:33:28.404 INFO:teuthology.orchestra.run.smithi096.stdout:5978 write 0x37d580b thru 0x37e0f6e (0xb764 bytes) 2024-04-24T15:33:28.406 INFO:teuthology.orchestra.run.smithi096.stdout:5979 read 0x94921f0 thru 0x949a16d (0x7f7e bytes) 2024-04-24T15:33:28.406 INFO:teuthology.orchestra.run.smithi096.stdout:5980 punch from 0x882f951 to 0x883c8e3, (0xcf92 bytes) 2024-04-24T15:33:28.406 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:28.406 INFO:teuthology.orchestra.run.smithi096.stdout:5983 punch from 0x5b3e765 to 0x5b4cf99, (0xe834 bytes) 2024-04-24T15:33:28.406 INFO:teuthology.orchestra.run.smithi096.stdout:5985 writesame 0xa2c1ba5 thru 0xa2c2e0d data_size 0x1(0x1269 bytes) 2024-04-24T15:33:28.408 INFO:teuthology.orchestra.run.smithi096.stdout:5986 writesame 0x6c45f4d thru 0x6c4615e data_size 0x5(0x212 bytes) 2024-04-24T15:33:28.410 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:28.410 INFO:teuthology.orchestra.run.smithi096.stdout:5989 punch from 0x413487e to 0x41425be, (0xdd40 bytes) 2024-04-24T15:33:28.410 INFO:teuthology.orchestra.run.smithi096.stdout:5991 write 0xb048350 thru 0xb05044e (0x80ff bytes) 2024-04-24T15:33:28.415 INFO:teuthology.orchestra.run.smithi096.stdout:5992 read 0xb8bddf thru 0xb970b4 (0xb2d6 bytes) 2024-04-24T15:33:28.415 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:28.415 INFO:teuthology.orchestra.run.smithi096.stdout:5994 write 0x3b8f95b thru 0x3b906b2 (0xd58 bytes) 2024-04-24T15:33:28.416 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-04-24T15:33:28.419 INFO:teuthology.orchestra.run.smithi096.stdout:5995 writesame 0xade8a7c thru 0xadf5a4b data_size 0x4c(0xcfd0 bytes) 2024-04-24T15:33:28.421 INFO:teuthology.orchestra.run.smithi096.stdout:5997 writesame 0x4735078 thru 0x4741e29 data_size 0x71(0xcdb2 bytes) 2024-04-24T15:33:28.426 INFO:teuthology.orchestra.run.smithi096.stdout:5998 punch from 0x860b404 to 0x860d22c, (0x1e28 bytes) 2024-04-24T15:33:28.426 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-04-24T15:33:28.437 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-04-24T15:33:28.448 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:28.448+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:28.448 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:28.448 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:28.457 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-04-24T15:33:28.459 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-24T15:33:28.464 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #32, image image_client.0-clone32 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent33 2024-04-24T15:33:28.477 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-04-24T15:33:28.498 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-04-24T15:33:28.498 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T15:33:28.499 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T15:33:28.499 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T15:33:28.499 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T15:33:28.499 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T15:33:28.499 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T15:33:28.499 INFO:teuthology.orchestra.run.smithi096.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-04-24T15:33:28.500 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T15:33:28.501 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-24T15:33:28.501 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-24T15:33:28.549 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:28.549+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:28.593 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:28.593 INFO:teuthology.orchestra.run.smithi151.stdout: "size": 20, 2024-04-24T15:33:28.594 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 600, 2024-04-24T15:33:28.594 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [ 2024-04-24T15:33:28.594 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.594 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18006 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12419072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:28.594 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.207911+0000", 2024-04-24T15:33:28.594 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 54.384866637999998, 2024-04-24T15:33:28.594 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042899496000000002, 2024-04-24T15:33:28.594 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:28.594 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:28.594 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:28.594 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:28.594 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:28.594 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18006 2024-04-24T15:33:28.595 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.595 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:28.595 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.595 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:28.595 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207911+0000", 2024-04-24T15:33:28.595 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:28.595 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.595 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.595 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:28.595 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207911+0000", 2024-04-24T15:33:28.595 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:28.595 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.595 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.595 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:28.595 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207910+0000", 2024-04-24T15:33:28.596 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:28.596 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.596 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.596 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:28.596 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207913+0000", 2024-04-24T15:33:28.596 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.2309999999999999e-06 2024-04-24T15:33:28.596 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.596 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.596 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:28.596 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207918+0000", 2024-04-24T15:33:28.596 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.3399999999999997e-06 2024-04-24T15:33:28.596 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.596 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.596 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:28.596 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207922+0000", 2024-04-24T15:33:28.597 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5439999999999999e-06 2024-04-24T15:33:28.597 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.597 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.597 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:28.597 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250726+0000", 2024-04-24T15:33:28.597 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042804129000000003 2024-04-24T15:33:28.597 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.597 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.597 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:28.597 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250765+0000", 2024-04-24T15:33:28.597 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9554000000000001e-05 2024-04-24T15:33:28.597 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.597 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.597 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:28.597 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250810+0000", 2024-04-24T15:33:28.597 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5027000000000001e-05 2024-04-24T15:33:28.598 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:28.598 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:28.598 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:28.598 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.598 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.598 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18007 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12460032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:28.598 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.207961+0000", 2024-04-24T15:33:28.598 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 54.384816796999999, 2024-04-24T15:33:28.598 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042965470999999998, 2024-04-24T15:33:28.598 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:28.598 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:28.598 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:28.598 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:28.599 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:28.599 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18007 2024-04-24T15:33:28.599 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.599 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:28.599 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.599 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:28.599 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207961+0000", 2024-04-24T15:33:28.599 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:28.599 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.599 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.599 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:28.599 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207961+0000", 2024-04-24T15:33:28.599 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:28.599 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.599 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.599 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:28.600 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207960+0000", 2024-04-24T15:33:28.600 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:28.600 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.600 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.600 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:28.600 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207963+0000", 2024-04-24T15:33:28.600 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.3759999999999999e-06 2024-04-24T15:33:28.600 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.600 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.600 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:28.600 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207970+0000", 2024-04-24T15:33:28.600 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.0489999999999998e-06 2024-04-24T15:33:28.600 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.600 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.600 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:28.600 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207975+0000", 2024-04-24T15:33:28.601 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.1640000000000003e-06 2024-04-24T15:33:28.601 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.601 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.601 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:28.601 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250834+0000", 2024-04-24T15:33:28.601 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042859060999999997 2024-04-24T15:33:28.601 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.601 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.601 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:28.601 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250874+0000", 2024-04-24T15:33:28.601 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0674999999999999e-05 2024-04-24T15:33:28.601 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.601 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.601 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:28.601 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250926+0000", 2024-04-24T15:33:28.601 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.1693000000000001e-05 2024-04-24T15:33:28.602 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:28.602 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:28.602 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:28.602 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.602 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.602 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18008 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12500992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:28.602 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208030+0000", 2024-04-24T15:33:28.602 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 54.384747703999999, 2024-04-24T15:33:28.602 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043003337000000003, 2024-04-24T15:33:28.602 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:28.602 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:28.602 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:28.602 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:28.603 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:28.603 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18008 2024-04-24T15:33:28.603 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.603 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:28.603 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.603 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:28.603 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208030+0000", 2024-04-24T15:33:28.603 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:28.603 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.603 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.603 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:28.603 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208030+0000", 2024-04-24T15:33:28.603 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:28.603 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.603 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.603 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:28.604 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208029+0000", 2024-04-24T15:33:28.604 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:28.604 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.604 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.604 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:28.604 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208032+0000", 2024-04-24T15:33:28.604 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.117e-06 2024-04-24T15:33:28.604 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.604 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.604 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:28.604 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208039+0000", 2024-04-24T15:33:28.604 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.0129999999999996e-06 2024-04-24T15:33:28.604 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.604 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.605 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:28.605 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208043+0000", 2024-04-24T15:33:28.605 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0600000000000001e-06 2024-04-24T15:33:28.605 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.605 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.605 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:28.605 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250948+0000", 2024-04-24T15:33:28.605 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042904577999999999 2024-04-24T15:33:28.605 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.605 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.605 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:28.605 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250989+0000", 2024-04-24T15:33:28.605 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0978e-05 2024-04-24T15:33:28.605 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.605 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.606 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:28.606 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251033+0000", 2024-04-24T15:33:28.606 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4161e-05 2024-04-24T15:33:28.606 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:28.606 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:28.606 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:28.606 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.606 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.606 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18009 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12541952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:28.606 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208093+0000", 2024-04-24T15:33:28.606 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 54.384683957999997, 2024-04-24T15:33:28.606 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043043451000000003, 2024-04-24T15:33:28.606 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:28.607 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:28.607 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:28.607 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:28.607 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:28.607 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18009 2024-04-24T15:33:28.607 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.607 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:28.607 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.607 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:28.607 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208093+0000", 2024-04-24T15:33:28.607 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:28.607 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.607 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.607 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:28.607 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208093+0000", 2024-04-24T15:33:28.608 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:28.608 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.608 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.608 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:28.608 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208093+0000", 2024-04-24T15:33:28.608 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:28.608 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.608 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.608 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:28.608 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208096+0000", 2024-04-24T15:33:28.608 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.439e-06 2024-04-24T15:33:28.608 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.608 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.608 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:28.608 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208103+0000", 2024-04-24T15:33:28.609 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1019999999999999e-06 2024-04-24T15:33:28.609 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.609 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.609 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:28.609 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208107+0000", 2024-04-24T15:33:28.609 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0910000000000003e-06 2024-04-24T15:33:28.609 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.609 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.609 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:28.609 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251054+0000", 2024-04-24T15:33:28.609 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042946826 2024-04-24T15:33:28.609 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.609 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.609 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:28.609 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251092+0000", 2024-04-24T15:33:28.610 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.7941999999999997e-05 2024-04-24T15:33:28.610 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.610 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.610 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:28.610 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251137+0000", 2024-04-24T15:33:28.610 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4638000000000002e-05 2024-04-24T15:33:28.610 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:28.610 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:28.610 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:28.610 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.610 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.610 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18010 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12615680~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:28.610 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208202+0000", 2024-04-24T15:33:28.611 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 54.384575126999998, 2024-04-24T15:33:28.611 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043046549000000003, 2024-04-24T15:33:28.611 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:28.611 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:28.611 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:28.611 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:28.611 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:28.611 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18010 2024-04-24T15:33:28.611 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.611 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:28.611 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.611 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:28.611 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208202+0000", 2024-04-24T15:33:28.611 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:28.611 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.612 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.612 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:28.612 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208202+0000", 2024-04-24T15:33:28.612 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:28.612 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.612 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.612 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:28.612 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208202+0000", 2024-04-24T15:33:28.612 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:28.612 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.612 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.612 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:28.612 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208205+0000", 2024-04-24T15:33:28.612 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.2490000000000001e-06 2024-04-24T15:33:28.612 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.612 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.613 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:28.613 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208215+0000", 2024-04-24T15:33:28.613 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.9070000000000002e-06 2024-04-24T15:33:28.613 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.613 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.613 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:28.613 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208220+0000", 2024-04-24T15:33:28.613 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8849999999999998e-06 2024-04-24T15:33:28.613 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.613 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.613 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:28.613 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251158+0000", 2024-04-24T15:33:28.613 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042938416 2024-04-24T15:33:28.613 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.613 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.614 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:28.614 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251197+0000", 2024-04-24T15:33:28.614 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9341000000000002e-05 2024-04-24T15:33:28.614 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.614 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.614 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:28.614 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251249+0000", 2024-04-24T15:33:28.614 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.1391000000000003e-05 2024-04-24T15:33:28.614 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:28.614 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:28.614 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:28.614 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.614 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.614 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18011 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12656640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:28.614 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208258+0000", 2024-04-24T15:33:28.615 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 54.384519281000003, 2024-04-24T15:33:28.615 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043102708000000003, 2024-04-24T15:33:28.615 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:28.615 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:28.615 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:28.615 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:28.615 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:28.615 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18011 2024-04-24T15:33:28.615 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.615 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:28.615 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.615 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:28.615 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208258+0000", 2024-04-24T15:33:28.615 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:28.615 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.615 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.616 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:28.616 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208258+0000", 2024-04-24T15:33:28.616 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:28.616 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.616 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.616 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:28.616 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208258+0000", 2024-04-24T15:33:28.616 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:28.616 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.616 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.616 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:28.616 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208260+0000", 2024-04-24T15:33:28.616 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7810000000000001e-06 2024-04-24T15:33:28.616 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.616 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.616 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:28.617 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208267+0000", 2024-04-24T15:33:28.617 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.7610000000000002e-06 2024-04-24T15:33:28.617 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.617 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.617 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:28.617 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208271+0000", 2024-04-24T15:33:28.617 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0890000000000002e-06 2024-04-24T15:33:28.617 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.617 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.617 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:28.617 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251271+0000", 2024-04-24T15:33:28.618 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043000159000000003 2024-04-24T15:33:28.618 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.618 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.618 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:28.618 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251315+0000", 2024-04-24T15:33:28.618 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3446999999999999e-05 2024-04-24T15:33:28.618 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.618 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.618 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:28.618 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251361+0000", 2024-04-24T15:33:28.618 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5952000000000003e-05 2024-04-24T15:33:28.618 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:28.618 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:28.619 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:28.619 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.619 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.619 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18012 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12697600~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:28.619 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208289+0000", 2024-04-24T15:33:28.619 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 54.384488236999999, 2024-04-24T15:33:28.619 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043174774999999999, 2024-04-24T15:33:28.619 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:28.619 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:28.619 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:28.619 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:28.619 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:28.619 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18012 2024-04-24T15:33:28.619 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.619 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:28.620 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.620 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:28.620 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208289+0000", 2024-04-24T15:33:28.620 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:28.620 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.620 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.620 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:28.620 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208289+0000", 2024-04-24T15:33:28.620 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:28.620 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.620 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.620 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:28.620 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208289+0000", 2024-04-24T15:33:28.620 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:28.621 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.621 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.621 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:28.621 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208291+0000", 2024-04-24T15:33:28.621 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.542e-06 2024-04-24T15:33:28.621 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.621 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.621 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:28.621 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208298+0000", 2024-04-24T15:33:28.621 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.3899999999999998e-06 2024-04-24T15:33:28.621 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.621 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.621 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:28.621 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208302+0000", 2024-04-24T15:33:28.621 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.019e-06 2024-04-24T15:33:28.621 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.622 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.622 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:28.622 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251382+0000", 2024-04-24T15:33:28.622 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043080790000000001 2024-04-24T15:33:28.622 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.622 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.622 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:28.622 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251421+0000", 2024-04-24T15:33:28.622 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8473000000000002e-05 2024-04-24T15:33:28.622 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.622 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.622 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:28.622 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251464+0000", 2024-04-24T15:33:28.622 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3133999999999997e-05 2024-04-24T15:33:28.623 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:28.623 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:28.623 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:28.623 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.623 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.623 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18013 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12738560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:28.623 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208423+0000", 2024-04-24T15:33:28.623 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 54.384354129000002, 2024-04-24T15:33:28.623 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043145554000000003, 2024-04-24T15:33:28.623 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:28.623 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:28.623 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:28.624 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:28.624 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:28.624 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18013 2024-04-24T15:33:28.624 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.624 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:28.624 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.624 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:28.624 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208423+0000", 2024-04-24T15:33:28.624 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:28.624 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.624 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.624 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:28.624 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208423+0000", 2024-04-24T15:33:28.624 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:28.624 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.624 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.625 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:28.625 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208423+0000", 2024-04-24T15:33:28.625 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:28.625 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.625 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.625 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:28.625 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208426+0000", 2024-04-24T15:33:28.625 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.0539999999999999e-06 2024-04-24T15:33:28.625 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.625 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.625 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:28.625 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208433+0000", 2024-04-24T15:33:28.625 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.43e-06 2024-04-24T15:33:28.625 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.625 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.625 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:28.626 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208437+0000", 2024-04-24T15:33:28.626 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5590000000000001e-06 2024-04-24T15:33:28.626 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.626 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.626 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:28.626 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251485+0000", 2024-04-24T15:33:28.626 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043048259999999998 2024-04-24T15:33:28.626 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.626 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.626 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:28.626 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251525+0000", 2024-04-24T15:33:28.626 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9916999999999997e-05 2024-04-24T15:33:28.626 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.626 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.627 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:28.627 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251569+0000", 2024-04-24T15:33:28.627 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3939000000000001e-05 2024-04-24T15:33:28.627 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:28.627 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:28.627 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:28.627 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.627 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.627 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18015 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12779520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:28.627 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208630+0000", 2024-04-24T15:33:28.627 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 54.384147603999999, 2024-04-24T15:33:28.627 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043047328000000003, 2024-04-24T15:33:28.627 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:28.627 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:28.627 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:28.627 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:28.628 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:28.628 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18015 2024-04-24T15:33:28.628 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.628 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:28.628 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.628 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:28.628 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208630+0000", 2024-04-24T15:33:28.628 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:28.628 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.628 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.628 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:28.628 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208630+0000", 2024-04-24T15:33:28.628 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:28.628 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.628 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.628 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:28.629 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208629+0000", 2024-04-24T15:33:28.629 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999999 2024-04-24T15:33:28.629 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.629 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.629 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:28.629 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208634+0000", 2024-04-24T15:33:28.629 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8960000000000002e-06 2024-04-24T15:33:28.629 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.629 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.629 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:28.629 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208645+0000", 2024-04-24T15:33:28.629 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0939e-05 2024-04-24T15:33:28.629 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.629 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.629 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:28.629 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208651+0000", 2024-04-24T15:33:28.630 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.896e-06 2024-04-24T15:33:28.630 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.630 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.630 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:28.630 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251591+0000", 2024-04-24T15:33:28.630 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042939862000000002 2024-04-24T15:33:28.630 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.630 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.630 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:28.630 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251631+0000", 2024-04-24T15:33:28.630 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0216000000000003e-05 2024-04-24T15:33:28.630 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.630 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.630 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:28.630 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251677+0000", 2024-04-24T15:33:28.631 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6390999999999997e-05 2024-04-24T15:33:28.631 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:28.631 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:28.631 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:28.631 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.631 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.631 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18016 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12820480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:28.631 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208660+0000", 2024-04-24T15:33:28.631 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 54.384117815000003, 2024-04-24T15:33:28.631 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043127510000000001, 2024-04-24T15:33:28.631 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:28.631 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:28.631 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:28.631 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:28.631 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:28.631 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18016 2024-04-24T15:33:28.632 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.632 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:28.632 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.632 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:28.632 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208660+0000", 2024-04-24T15:33:28.632 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:28.632 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.632 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.632 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:28.632 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208660+0000", 2024-04-24T15:33:28.632 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:28.632 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.632 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.632 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:28.632 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208659+0000", 2024-04-24T15:33:28.632 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:28.633 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.633 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.633 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:28.633 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208662+0000", 2024-04-24T15:33:28.633 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.0979999999999998e-06 2024-04-24T15:33:28.633 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.633 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.633 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:28.633 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208669+0000", 2024-04-24T15:33:28.633 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.4470000000000003e-06 2024-04-24T15:33:28.633 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.633 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.633 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:28.633 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208673+0000", 2024-04-24T15:33:28.633 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2810000000000002e-06 2024-04-24T15:33:28.634 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.634 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.634 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:28.634 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251699+0000", 2024-04-24T15:33:28.634 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043025728999999999 2024-04-24T15:33:28.634 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.634 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.634 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:28.634 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251743+0000", 2024-04-24T15:33:28.634 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4469999999999999e-05 2024-04-24T15:33:28.634 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.634 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.634 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:28.634 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251787+0000", 2024-04-24T15:33:28.634 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3949999999999998e-05 2024-04-24T15:33:28.635 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:28.635 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:28.635 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:28.635 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.635 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.635 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18017 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12861440~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:28.635 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208721+0000", 2024-04-24T15:33:28.635 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 54.384056366999999, 2024-04-24T15:33:28.635 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043167749999999998, 2024-04-24T15:33:28.635 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:28.635 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:28.635 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:28.635 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:28.635 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:28.635 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18017 2024-04-24T15:33:28.635 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.636 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:28.636 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.636 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:28.636 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208721+0000", 2024-04-24T15:33:28.636 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:28.636 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.636 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.636 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:28.636 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208721+0000", 2024-04-24T15:33:28.636 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:28.636 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.636 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.636 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:28.636 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208720+0000", 2024-04-24T15:33:28.637 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:28.637 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.637 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.637 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:28.637 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208724+0000", 2024-04-24T15:33:28.637 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.3919999999999999e-06 2024-04-24T15:33:28.637 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.637 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.637 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:28.637 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208729+0000", 2024-04-24T15:33:28.637 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.3480000000000003e-06 2024-04-24T15:33:28.637 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.637 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.637 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:28.637 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208732+0000", 2024-04-24T15:33:28.637 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.9529999999999999e-06 2024-04-24T15:33:28.638 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.638 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.638 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:28.638 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251807+0000", 2024-04-24T15:33:28.638 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043074492999999998 2024-04-24T15:33:28.638 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.638 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.638 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:28.638 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251846+0000", 2024-04-24T15:33:28.638 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8958999999999997e-05 2024-04-24T15:33:28.638 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.638 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.638 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:28.638 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251889+0000", 2024-04-24T15:33:28.638 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3253000000000001e-05 2024-04-24T15:33:28.639 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:28.639 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:28.639 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:28.639 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.639 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.639 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18018 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12902400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:28.639 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208752+0000", 2024-04-24T15:33:28.639 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 54.384025754, 2024-04-24T15:33:28.639 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043241012000000002, 2024-04-24T15:33:28.639 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:28.639 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:28.639 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:28.639 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:28.639 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:28.639 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18018 2024-04-24T15:33:28.640 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.640 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:28.640 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.640 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:28.640 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208752+0000", 2024-04-24T15:33:28.640 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:28.640 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.640 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.640 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:28.640 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208752+0000", 2024-04-24T15:33:28.640 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:28.640 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.640 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.640 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:28.640 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208751+0000", 2024-04-24T15:33:28.640 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:28.641 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.641 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.641 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:28.641 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208753+0000", 2024-04-24T15:33:28.641 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.894e-06 2024-04-24T15:33:28.641 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.641 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.641 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:28.641 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208757+0000", 2024-04-24T15:33:28.641 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.856e-06 2024-04-24T15:33:28.641 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.641 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.641 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:28.641 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208760+0000", 2024-04-24T15:33:28.641 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.6749999999999998e-06 2024-04-24T15:33:28.641 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.642 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.642 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:28.642 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251909+0000", 2024-04-24T15:33:28.642 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043149145999999999 2024-04-24T15:33:28.642 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.642 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.642 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:28.642 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251948+0000", 2024-04-24T15:33:28.642 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9159000000000002e-05 2024-04-24T15:33:28.642 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.642 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.642 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:28.642 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251993+0000", 2024-04-24T15:33:28.642 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4638999999999997e-05 2024-04-24T15:33:28.643 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:28.643 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:28.643 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:28.643 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.643 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.643 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18019 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12943360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:28.643 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208938+0000", 2024-04-24T15:33:28.643 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 54.383839463000001, 2024-04-24T15:33:28.643 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043162272000000002, 2024-04-24T15:33:28.643 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:28.643 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:28.643 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:28.643 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:28.643 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:28.643 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18019 2024-04-24T15:33:28.643 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.644 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:28.644 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.644 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:28.644 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208938+0000", 2024-04-24T15:33:28.644 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:28.644 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.644 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.644 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:28.644 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208938+0000", 2024-04-24T15:33:28.644 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:28.644 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.644 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.644 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:28.644 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208762+0000", 2024-04-24T15:33:28.644 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999824 2024-04-24T15:33:28.644 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.645 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.645 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:28.645 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208942+0000", 2024-04-24T15:33:28.645 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000180008 2024-04-24T15:33:28.645 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.645 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.645 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:28.645 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208950+0000", 2024-04-24T15:33:28.645 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.1550000000000007e-06 2024-04-24T15:33:28.645 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.645 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.645 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:28.645 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208956+0000", 2024-04-24T15:33:28.645 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.7910000000000001e-06 2024-04-24T15:33:28.645 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.645 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.646 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:28.646 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252014+0000", 2024-04-24T15:33:28.646 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043058427000000003 2024-04-24T15:33:28.646 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.646 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.646 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:28.646 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252054+0000", 2024-04-24T15:33:28.646 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0071999999999999e-05 2024-04-24T15:33:28.646 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.646 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.646 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:28.646 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252100+0000", 2024-04-24T15:33:28.646 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5917000000000001e-05 2024-04-24T15:33:28.646 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:28.647 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:28.647 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:28.647 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.647 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.647 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18020 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12984320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:28.647 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.209092+0000", 2024-04-24T15:33:28.647 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 54.383685464999999, 2024-04-24T15:33:28.647 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043111295000000001, 2024-04-24T15:33:28.647 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:28.647 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:28.647 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:28.647 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:28.647 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:28.647 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18020 2024-04-24T15:33:28.647 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.648 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:28.648 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.648 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:28.648 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209092+0000", 2024-04-24T15:33:28.648 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:28.648 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.648 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.648 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:28.648 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209092+0000", 2024-04-24T15:33:28.648 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:28.648 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.648 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.648 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:28.648 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208964+0000", 2024-04-24T15:33:28.648 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998722 2024-04-24T15:33:28.648 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.649 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.649 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:28.649 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209095+0000", 2024-04-24T15:33:28.649 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000130648 2024-04-24T15:33:28.649 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.649 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.649 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:28.649 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209103+0000", 2024-04-24T15:33:28.649 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0320000000000003e-06 2024-04-24T15:33:28.649 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.649 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.649 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:28.649 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209108+0000", 2024-04-24T15:33:28.649 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8690000000000003e-06 2024-04-24T15:33:28.649 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.650 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.650 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:28.650 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252121+0000", 2024-04-24T15:33:28.650 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043013323999999999 2024-04-24T15:33:28.650 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.650 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.650 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:28.650 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252160+0000", 2024-04-24T15:33:28.650 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8376e-05 2024-04-24T15:33:28.650 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.650 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.650 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:28.650 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252203+0000", 2024-04-24T15:33:28.651 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.371e-05 2024-04-24T15:33:28.651 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:28.651 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:28.651 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:28.651 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.651 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.651 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18021 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 13025280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:28.651 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.209132+0000", 2024-04-24T15:33:28.651 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 54.383645107, 2024-04-24T15:33:28.651 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043183903000000003, 2024-04-24T15:33:28.651 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:28.651 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:28.651 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:28.651 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:28.651 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:28.651 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18021 2024-04-24T15:33:28.652 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.652 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:28.652 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.652 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:28.652 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209132+0000", 2024-04-24T15:33:28.652 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:28.652 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.652 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.652 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:28.652 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209132+0000", 2024-04-24T15:33:28.652 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:28.652 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.652 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.652 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:28.652 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209132+0000", 2024-04-24T15:33:28.653 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:28.653 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.653 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.653 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:28.653 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209134+0000", 2024-04-24T15:33:28.653 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.891e-06 2024-04-24T15:33:28.653 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.653 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.653 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:28.653 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209138+0000", 2024-04-24T15:33:28.653 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0659999999999997e-06 2024-04-24T15:33:28.653 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.653 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.653 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:28.653 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209140+0000", 2024-04-24T15:33:28.653 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.5339999999999998e-06 2024-04-24T15:33:28.654 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.654 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.654 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:28.654 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252232+0000", 2024-04-24T15:33:28.654 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043091142999999998 2024-04-24T15:33:28.654 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.654 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.654 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:28.654 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252270+0000", 2024-04-24T15:33:28.654 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8872000000000003e-05 2024-04-24T15:33:28.654 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.654 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.654 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:28.654 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252316+0000", 2024-04-24T15:33:28.655 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5788000000000003e-05 2024-04-24T15:33:28.655 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:28.655 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:28.655 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:28.655 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.655 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.655 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18022 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 13066240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:28.655 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.209411+0000", 2024-04-24T15:33:28.655 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 54.383366559000002, 2024-04-24T15:33:28.655 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043016616000000001, 2024-04-24T15:33:28.655 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:28.655 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:28.655 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:28.655 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:28.655 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:28.656 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18022 2024-04-24T15:33:28.656 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.656 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:28.656 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.656 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:28.656 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209411+0000", 2024-04-24T15:33:28.656 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:28.656 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.656 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.656 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:28.656 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209411+0000", 2024-04-24T15:33:28.656 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:28.656 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.656 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.656 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:28.656 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209143+0000", 2024-04-24T15:33:28.657 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999732 2024-04-24T15:33:28.657 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.657 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.657 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:28.657 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209415+0000", 2024-04-24T15:33:28.657 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000271873 2024-04-24T15:33:28.657 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.657 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.657 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:28.657 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209423+0000", 2024-04-24T15:33:28.657 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3080000000000008e-06 2024-04-24T15:33:28.657 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.657 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.657 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:28.657 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209428+0000", 2024-04-24T15:33:28.657 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.5040000000000002e-06 2024-04-24T15:33:28.658 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.658 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.658 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:28.658 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252339+0000", 2024-04-24T15:33:28.658 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042910419999999998 2024-04-24T15:33:28.658 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.658 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.658 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:28.658 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252380+0000", 2024-04-24T15:33:28.658 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0991000000000001e-05 2024-04-24T15:33:28.658 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.658 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.658 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:28.658 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252427+0000", 2024-04-24T15:33:28.659 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.7647e-05 2024-04-24T15:33:28.659 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:28.659 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:28.659 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:28.659 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.659 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.659 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18023 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 13107200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:28.659 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.209594+0000", 2024-04-24T15:33:28.659 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 54.383183045999999, 2024-04-24T15:33:28.659 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042938543000000003, 2024-04-24T15:33:28.659 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:28.659 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:28.659 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:28.659 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:28.659 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:28.659 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18023 2024-04-24T15:33:28.660 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.660 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:28.660 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.660 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:28.660 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209594+0000", 2024-04-24T15:33:28.660 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:28.660 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.660 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.660 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:28.660 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209594+0000", 2024-04-24T15:33:28.660 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:28.660 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.660 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.660 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:28.660 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209440+0000", 2024-04-24T15:33:28.661 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999846 2024-04-24T15:33:28.661 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.661 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.661 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:28.661 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209598+0000", 2024-04-24T15:33:28.661 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00015731100000000001 2024-04-24T15:33:28.661 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.661 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.661 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:28.661 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209604+0000", 2024-04-24T15:33:28.661 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.2910000000000004e-06 2024-04-24T15:33:28.661 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.661 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.661 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:28.661 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209607+0000", 2024-04-24T15:33:28.661 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5389999999999999e-06 2024-04-24T15:33:28.662 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.662 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.662 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:28.662 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252449+0000", 2024-04-24T15:33:28.662 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042841833000000003 2024-04-24T15:33:28.662 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.662 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.662 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:28.662 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252489+0000", 2024-04-24T15:33:28.662 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9898000000000003e-05 2024-04-24T15:33:28.662 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.662 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.662 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:28.662 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252533+0000", 2024-04-24T15:33:28.662 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3702999999999998e-05 2024-04-24T15:33:28.663 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:28.663 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:28.663 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:28.663 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.663 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.663 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18025 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 13189120~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:28.663 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.209832+0000", 2024-04-24T15:33:28.663 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 54.382945544999998, 2024-04-24T15:33:28.663 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042913556999999998, 2024-04-24T15:33:28.665 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:28.665 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:28.665 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:28.665 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:28.665 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:28.665 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18025 2024-04-24T15:33:28.665 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.665 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:28.665 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.665 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:28.665 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209832+0000", 2024-04-24T15:33:28.665 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:28.665 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.666 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.666 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:28.666 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209832+0000", 2024-04-24T15:33:28.666 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:28.666 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.666 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.666 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:28.666 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209774+0000", 2024-04-24T15:33:28.666 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999418 2024-04-24T15:33:28.666 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.666 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.666 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:28.666 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209835+0000", 2024-04-24T15:33:28.666 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.1377000000000005e-05 2024-04-24T15:33:28.667 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.667 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.667 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:28.667 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209843+0000", 2024-04-24T15:33:28.667 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.2330000000000006e-06 2024-04-24T15:33:28.667 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.667 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.667 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:28.667 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209848+0000", 2024-04-24T15:33:28.667 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6149999999999999e-06 2024-04-24T15:33:28.667 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.667 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.667 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:28.667 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252658+0000", 2024-04-24T15:33:28.667 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042810002 2024-04-24T15:33:28.668 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.668 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.668 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:28.668 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252697+0000", 2024-04-24T15:33:28.668 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8757999999999997e-05 2024-04-24T15:33:28.668 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.668 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.668 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:28.668 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252745+0000", 2024-04-24T15:33:28.668 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.863e-05 2024-04-24T15:33:28.668 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:28.668 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:28.668 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:28.669 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.669 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.669 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:24749 4.8 4:137f5701:::rbd_header.114ae15b9a20:head [watch reconnect cookie 140672595753696 gen 11] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-04-24T15:33:28.669 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:37.628703+0000", 2024-04-24T15:33:28.669 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 50.964073925999998, 2024-04-24T15:33:28.669 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0102918700000001, 2024-04-24T15:33:28.669 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:28.669 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:28.669 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:28.669 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:28.669 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:28.669 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 24749 2024-04-24T15:33:28.670 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.670 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:28.670 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.670 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:28.670 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628703+0000", 2024-04-24T15:33:28.670 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:28.670 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.670 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.670 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:28.670 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628703+0000", 2024-04-24T15:33:28.670 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:28.670 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.670 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.670 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:28.671 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628703+0000", 2024-04-24T15:33:28.671 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999999 2024-04-24T15:33:28.671 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.671 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.671 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:28.671 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628708+0000", 2024-04-24T15:33:28.671 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.4589999999999997e-06 2024-04-24T15:33:28.671 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.671 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.671 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:28.671 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628721+0000", 2024-04-24T15:33:28.671 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.2601e-05 2024-04-24T15:33:28.671 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.671 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.672 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:28.672 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628728+0000", 2024-04-24T15:33:28.672 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.9059999999999996e-06 2024-04-24T15:33:28.672 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.672 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.672 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:28.672 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628789+0000", 2024-04-24T15:33:28.672 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.1672000000000002e-05 2024-04-24T15:33:28.672 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.672 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.672 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting_for_map not empty", 2024-04-24T15:33:28.673 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628797+0000", 2024-04-24T15:33:28.673 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.8189999999999992e-06 2024-04-24T15:33:28.673 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.673 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.673 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:28.673 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.631255+0000", 2024-04-24T15:33:28.673 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0024577610000000001 2024-04-24T15:33:28.673 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.673 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.673 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for peered", 2024-04-24T15:33:28.673 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.631263+0000", 2024-04-24T15:33:28.673 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.4109999999999996e-06 2024-04-24T15:33:28.673 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.674 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.674 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:28.674 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:38.637560+0000", 2024-04-24T15:33:28.674 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0062970339999999 2024-04-24T15:33:28.674 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.674 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.674 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:28.674 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:38.637747+0000", 2024-04-24T15:33:28.674 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00018693400000000001 2024-04-24T15:33:28.674 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.674 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.674 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:28.674 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:38.638995+0000", 2024-04-24T15:33:28.674 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0012481300000000001 2024-04-24T15:33:28.674 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:28.675 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:28.675 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:28.675 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.675 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.675 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:24750 4.8 4:137f5701:::rbd_header.114ae15b9a20:head [notify cookie 140672594434416] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-04-24T15:33:28.675 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:37.628736+0000", 2024-04-24T15:33:28.675 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 50.964041666, 2024-04-24T15:33:28.675 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0096094309999999, 2024-04-24T15:33:28.675 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:28.675 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:28.675 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:28.675 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:28.675 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:28.675 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 24750 2024-04-24T15:33:28.676 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.676 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:28.676 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.676 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:28.676 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628736+0000", 2024-04-24T15:33:28.676 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:28.676 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.676 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.676 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:28.676 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628736+0000", 2024-04-24T15:33:28.676 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:28.676 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.676 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.676 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:28.677 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628735+0000", 2024-04-24T15:33:28.677 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:28.677 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.677 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.677 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:28.677 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628739+0000", 2024-04-24T15:33:28.677 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5109999999999999e-06 2024-04-24T15:33:28.677 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.677 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.677 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:28.677 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628745+0000", 2024-04-24T15:33:28.677 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.8300000000000001e-06 2024-04-24T15:33:28.677 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.677 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.677 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:28.678 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628749+0000", 2024-04-24T15:33:28.678 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.7110000000000003e-06 2024-04-24T15:33:28.678 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.678 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.678 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:28.678 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628826+0000", 2024-04-24T15:33:28.678 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.6678e-05 2024-04-24T15:33:28.678 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.678 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.678 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting_for_map not empty", 2024-04-24T15:33:28.678 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628834+0000", 2024-04-24T15:33:28.678 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.6190000000000002e-06 2024-04-24T15:33:28.678 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.678 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.679 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:28.679 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.631305+0000", 2024-04-24T15:33:28.679 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0024714400000000001 2024-04-24T15:33:28.679 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.679 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.679 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for peered", 2024-04-24T15:33:28.679 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.631313+0000", 2024-04-24T15:33:28.679 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.6529999999999994e-06 2024-04-24T15:33:28.679 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.679 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.679 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:28.679 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:38.638068+0000", 2024-04-24T15:33:28.679 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0067548369999999 2024-04-24T15:33:28.680 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.680 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.680 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:28.680 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:38.638158+0000", 2024-04-24T15:33:28.680 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.0079000000000003e-05 2024-04-24T15:33:28.680 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:28.680 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:28.680 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:28.680 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:38.638345+0000", 2024-04-24T15:33:28.680 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00018757100000000001 2024-04-24T15:33:28.680 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:28.680 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:28.680 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:28.680 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:28.680 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:28.680 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:28.682 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-24T15:33:28.682 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:28.682+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:28.782 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:28.782+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:28.805 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:28.805 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:33:28.805 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-04-24T15:33:28.805 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:28.815 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-24T15:33:28.898 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:28.899+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:28.909 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-24T15:33:28.983 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:33:28.984+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:33:28.995 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:28.995 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:33:28.995 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-04-24T15:33:28.995 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-04-24T15:33:28.995 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:29.007 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-24T15:33:29.085 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:33:29.084+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:33:29.166 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:29.166 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-04-24T15:33:29.166 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-04-24T15:33:29.166 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-04-24T15:33:29.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.167 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:47384 4.1f 4:fdac2fcc:::rbd_header.114a8b543e91:head [watch reconnect cookie 140672527126976 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-04-24T15:33:29.167 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:09.802333+0000", 2024-04-24T15:33:29.167 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.363768644, 2024-04-24T15:33:29.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0093213050000001, 2024-04-24T15:33:29.167 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:29.167 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:29.167 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:29.167 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:29.167 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:29.168 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 47384 2024-04-24T15:33:29.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.168 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:29.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.168 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:29.168 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.802333+0000", 2024-04-24T15:33:29.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:29.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.168 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:29.168 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.802333+0000", 2024-04-24T15:33:29.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:29.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.169 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.169 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:29.169 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.802332+0000", 2024-04-24T15:33:29.169 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999981 2024-04-24T15:33:29.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.169 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.169 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:29.169 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.802340+0000", 2024-04-24T15:33:29.169 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7700000000000007e-06 2024-04-24T15:33:29.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.169 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.169 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:29.169 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.802360+0000", 2024-04-24T15:33:29.169 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9959999999999999e-05 2024-04-24T15:33:29.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.170 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:29.170 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.802371+0000", 2024-04-24T15:33:29.170 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0473e-05 2024-04-24T15:33:29.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.170 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:29.170 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.802447+0000", 2024-04-24T15:33:29.170 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6089999999999998e-05 2024-04-24T15:33:29.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.170 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op must wait for map", 2024-04-24T15:33:29.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.802456+0000", 2024-04-24T15:33:29.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0599999999999997e-06 2024-04-24T15:33:29.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:29.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.808378+0000", 2024-04-24T15:33:29.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0059224910000000002 2024-04-24T15:33:29.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-04-24T15:33:29.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.808384+0000", 2024-04-24T15:33:29.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7740000000000001e-06 2024-04-24T15:33:29.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:29.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.809812+0000", 2024-04-24T15:33:29.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.001428043 2024-04-24T15:33:29.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:29.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.810082+0000", 2024-04-24T15:33:29.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00026971800000000001 2024-04-24T15:33:29.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.173 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:29.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.811655+0000", 2024-04-24T15:33:29.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015726869999999999 2024-04-24T15:33:29.173 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.173 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:29.173 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.173 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:47385 4.1f 4:fdac2fcc:::rbd_header.114a8b543e91:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-04-24T15:33:29.174 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:09.803953+0000", 2024-04-24T15:33:29.174 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.362148730000001, 2024-04-24T15:33:29.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0080910729999999, 2024-04-24T15:33:29.174 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:29.174 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:29.174 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:29.174 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:29.174 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:29.174 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 47385 2024-04-24T15:33:29.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.174 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:29.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.174 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:29.174 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.803953+0000", 2024-04-24T15:33:29.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:29.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:29.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.803953+0000", 2024-04-24T15:33:29.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:29.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:29.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.803952+0000", 2024-04-24T15:33:29.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999981 2024-04-24T15:33:29.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:29.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.803961+0000", 2024-04-24T15:33:29.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9549999999999998e-06 2024-04-24T15:33:29.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:29.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.803977+0000", 2024-04-24T15:33:29.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6498999999999999e-05 2024-04-24T15:33:29.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:29.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.803985+0000", 2024-04-24T15:33:29.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5000000000000002e-06 2024-04-24T15:33:29.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:29.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.804032+0000", 2024-04-24T15:33:29.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7657999999999997e-05 2024-04-24T15:33:29.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2024-04-24T15:33:29.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.804039+0000", 2024-04-24T15:33:29.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5719999999999999e-06 2024-04-24T15:33:29.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:29.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.808397+0000", 2024-04-24T15:33:29.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0043579860000000003 2024-04-24T15:33:29.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-04-24T15:33:29.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.808401+0000", 2024-04-24T15:33:29.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3089999999999997e-06 2024-04-24T15:33:29.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:29.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.810700+0000", 2024-04-24T15:33:29.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0022984699999999 2024-04-24T15:33:29.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.179 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:29.179 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.810758+0000", 2024-04-24T15:33:29.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8121000000000001e-05 2024-04-24T15:33:29.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.179 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:33:29.179 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.811932+0000", 2024-04-24T15:33:29.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001174542 2024-04-24T15:33:29.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.179 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:29.179 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.811978+0000", 2024-04-24T15:33:29.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5645000000000001e-05 2024-04-24T15:33:29.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:29.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.812044+0000", 2024-04-24T15:33:29.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6575999999999994e-05 2024-04-24T15:33:29.180 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.180 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:29.180 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.180 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48590 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2686976~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:29.180 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889219+0000", 2024-04-24T15:33:29.180 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.276882797999999, 2024-04-24T15:33:29.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016760856000000001, 2024-04-24T15:33:29.181 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:29.181 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:29.181 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:29.181 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:29.181 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:29.181 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48590 2024-04-24T15:33:29.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.181 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:29.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:29.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889219+0000", 2024-04-24T15:33:29.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:29.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:29.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889219+0000", 2024-04-24T15:33:29.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:29.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:29.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889219+0000", 2024-04-24T15:33:29.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:29.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:29.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889221+0000", 2024-04-24T15:33:29.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4499999999999998e-06 2024-04-24T15:33:29.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:29.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889227+0000", 2024-04-24T15:33:29.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6869999999999999e-06 2024-04-24T15:33:29.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:29.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889231+0000", 2024-04-24T15:33:29.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6320000000000001e-06 2024-04-24T15:33:29.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.184 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:29.184 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.905769+0000", 2024-04-24T15:33:29.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016538850000000001 2024-04-24T15:33:29.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.184 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:29.184 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.905855+0000", 2024-04-24T15:33:29.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5296000000000005e-05 2024-04-24T15:33:29.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.185 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:29.185 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.905980+0000", 2024-04-24T15:33:29.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000125406 2024-04-24T15:33:29.185 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.185 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:29.185 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.185 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48705 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 1851392~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:29.185 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889236+0000", 2024-04-24T15:33:29.185 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.27686619, 2024-04-24T15:33:29.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016984302, 2024-04-24T15:33:29.185 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:29.185 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:29.186 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:29.186 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:29.186 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:29.186 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48705 2024-04-24T15:33:29.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.186 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:29.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:29.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889236+0000", 2024-04-24T15:33:29.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:29.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:29.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889236+0000", 2024-04-24T15:33:29.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:29.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:29.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889235+0000", 2024-04-24T15:33:29.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:29.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:29.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889238+0000", 2024-04-24T15:33:29.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.249e-06 2024-04-24T15:33:29.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:29.188 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889243+0000", 2024-04-24T15:33:29.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6810000000000003e-06 2024-04-24T15:33:29.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:29.188 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889247+0000", 2024-04-24T15:33:29.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6780000000000002e-06 2024-04-24T15:33:29.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:29.188 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906008+0000", 2024-04-24T15:33:29.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01676099 2024-04-24T15:33:29.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:29.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906090+0000", 2024-04-24T15:33:29.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1508999999999995e-05 2024-04-24T15:33:29.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:29.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906220+0000", 2024-04-24T15:33:29.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013056999999999999 2024-04-24T15:33:29.189 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.189 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:29.189 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.189 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48706 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 1916928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:29.189 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889251+0000", 2024-04-24T15:33:29.190 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.276850756999998, 2024-04-24T15:33:29.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01718693, 2024-04-24T15:33:29.190 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:29.190 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:29.190 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:29.190 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:29.190 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:29.190 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48706 2024-04-24T15:33:29.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.190 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:29.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:29.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889251+0000", 2024-04-24T15:33:29.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:29.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:29.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889251+0000", 2024-04-24T15:33:29.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:29.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:29.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889251+0000", 2024-04-24T15:33:29.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:29.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:29.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889253+0000", 2024-04-24T15:33:29.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.244e-06 2024-04-24T15:33:29.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:29.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889259+0000", 2024-04-24T15:33:29.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3640000000000004e-06 2024-04-24T15:33:29.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:29.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889263+0000", 2024-04-24T15:33:29.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3720000000000001e-06 2024-04-24T15:33:29.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:29.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906258+0000", 2024-04-24T15:33:29.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016995310999999999 2024-04-24T15:33:29.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:29.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906343+0000", 2024-04-24T15:33:29.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4536000000000005e-05 2024-04-24T15:33:29.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:29.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906438+0000", 2024-04-24T15:33:29.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5502999999999997e-05 2024-04-24T15:33:29.194 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.194 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:29.194 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.194 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48707 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 1982464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:29.194 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889267+0000", 2024-04-24T15:33:29.194 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.276835197, 2024-04-24T15:33:29.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017425395, 2024-04-24T15:33:29.194 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:29.194 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:29.194 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:29.194 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:29.195 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:29.195 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48707 2024-04-24T15:33:29.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.195 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:29.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:29.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889267+0000", 2024-04-24T15:33:29.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:29.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:29.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889267+0000", 2024-04-24T15:33:29.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:29.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:29.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889266+0000", 2024-04-24T15:33:29.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:29.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:29.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889275+0000", 2024-04-24T15:33:29.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.568e-06 2024-04-24T15:33:29.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:29.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889281+0000", 2024-04-24T15:33:29.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3779999999999998e-06 2024-04-24T15:33:29.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:29.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889291+0000", 2024-04-24T15:33:29.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2769999999999994e-06 2024-04-24T15:33:29.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:29.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906476+0000", 2024-04-24T15:33:29.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017185277999999998 2024-04-24T15:33:29.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:29.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906567+0000", 2024-04-24T15:33:29.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0902999999999993e-05 2024-04-24T15:33:29.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.198 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:29.198 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906692+0000", 2024-04-24T15:33:29.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012541700000000001 2024-04-24T15:33:29.198 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.198 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:29.198 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.198 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48708 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2048000~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:29.198 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889295+0000", 2024-04-24T15:33:29.198 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.276806705999999, 2024-04-24T15:33:29.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017534984999999999, 2024-04-24T15:33:29.198 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:29.198 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:29.198 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:29.199 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:29.199 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:29.199 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48708 2024-04-24T15:33:29.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.199 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:29.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:29.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889295+0000", 2024-04-24T15:33:29.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:29.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:29.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889295+0000", 2024-04-24T15:33:29.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:29.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:29.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889295+0000", 2024-04-24T15:33:29.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:29.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:29.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889297+0000", 2024-04-24T15:33:29.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3479999999999998e-06 2024-04-24T15:33:29.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:29.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889304+0000", 2024-04-24T15:33:29.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3720000000000001e-06 2024-04-24T15:33:29.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:29.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889307+0000", 2024-04-24T15:33:29.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7160000000000001e-06 2024-04-24T15:33:29.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:29.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906711+0000", 2024-04-24T15:33:29.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017403452999999999 2024-04-24T15:33:29.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:29.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906765+0000", 2024-04-24T15:33:29.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4219999999999999e-05 2024-04-24T15:33:29.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:29.202 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906830+0000", 2024-04-24T15:33:29.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5277000000000005e-05 2024-04-24T15:33:29.202 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.202 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:29.202 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.202 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48709 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2113536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:29.202 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889312+0000", 2024-04-24T15:33:29.202 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.276790398999999, 2024-04-24T15:33:29.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017772013, 2024-04-24T15:33:29.202 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:29.202 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:29.203 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:29.203 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:29.203 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:29.203 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48709 2024-04-24T15:33:29.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.203 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:29.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:29.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889312+0000", 2024-04-24T15:33:29.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:29.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:29.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889312+0000", 2024-04-24T15:33:29.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:29.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:29.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889311+0000", 2024-04-24T15:33:29.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:29.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:29.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889314+0000", 2024-04-24T15:33:29.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2759999999999999e-06 2024-04-24T15:33:29.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.205 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:29.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889322+0000", 2024-04-24T15:33:29.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8589999999999994e-06 2024-04-24T15:33:29.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.205 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:29.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889326+0000", 2024-04-24T15:33:29.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6380000000000002e-06 2024-04-24T15:33:29.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.205 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:29.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906862+0000", 2024-04-24T15:33:29.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017536403999999998 2024-04-24T15:33:29.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.206 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.206 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:29.206 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906949+0000", 2024-04-24T15:33:29.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6217000000000005e-05 2024-04-24T15:33:29.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.206 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.206 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:29.206 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907084+0000", 2024-04-24T15:33:29.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013497100000000001 2024-04-24T15:33:29.206 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.206 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:29.206 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.207 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48710 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2179072~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:29.207 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889332+0000", 2024-04-24T15:33:29.207 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.276769585, 2024-04-24T15:33:29.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017944720000000001, 2024-04-24T15:33:29.207 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:29.207 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:29.207 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:29.207 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:29.207 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:29.207 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48710 2024-04-24T15:33:29.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.208 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:29.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:29.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889332+0000", 2024-04-24T15:33:29.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:29.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:29.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889332+0000", 2024-04-24T15:33:29.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:29.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:29.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889332+0000", 2024-04-24T15:33:29.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:29.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:29.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889334+0000", 2024-04-24T15:33:29.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4210000000000002e-06 2024-04-24T15:33:29.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:29.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889340+0000", 2024-04-24T15:33:29.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7209999999999999e-06 2024-04-24T15:33:29.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:29.210 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889344+0000", 2024-04-24T15:33:29.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.726e-06 2024-04-24T15:33:29.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.210 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:29.210 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907119+0000", 2024-04-24T15:33:29.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017775551000000001 2024-04-24T15:33:29.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.210 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:29.210 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907190+0000", 2024-04-24T15:33:29.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0634000000000002e-05 2024-04-24T15:33:29.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.211 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:29.211 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907277+0000", 2024-04-24T15:33:29.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7120999999999995e-05 2024-04-24T15:33:29.211 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.211 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:29.211 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.211 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48711 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2244608~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:29.211 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889348+0000", 2024-04-24T15:33:29.211 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.276753968000001, 2024-04-24T15:33:29.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018188109000000001, 2024-04-24T15:33:29.212 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:29.212 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:29.212 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:29.212 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:29.212 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:29.212 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48711 2024-04-24T15:33:29.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.212 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:29.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:29.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889348+0000", 2024-04-24T15:33:29.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:29.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:29.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889348+0000", 2024-04-24T15:33:29.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:29.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:29.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889348+0000", 2024-04-24T15:33:29.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:29.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:29.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889350+0000", 2024-04-24T15:33:29.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2330000000000001e-06 2024-04-24T15:33:29.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:29.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889356+0000", 2024-04-24T15:33:29.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8769999999999998e-06 2024-04-24T15:33:29.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:29.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889359+0000", 2024-04-24T15:33:29.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3699999999999999e-06 2024-04-24T15:33:29.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:29.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907316+0000", 2024-04-24T15:33:29.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017956857999999999 2024-04-24T15:33:29.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:29.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907402+0000", 2024-04-24T15:33:29.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5927000000000006e-05 2024-04-24T15:33:29.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:29.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907536+0000", 2024-04-24T15:33:29.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000134258 2024-04-24T15:33:29.215 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.215 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:29.215 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.215 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48712 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2310144~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:29.216 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889363+0000", 2024-04-24T15:33:29.216 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.276738648999999, 2024-04-24T15:33:29.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018314815000000002, 2024-04-24T15:33:29.216 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:29.216 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:29.216 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:29.216 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:29.216 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:29.216 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48712 2024-04-24T15:33:29.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.216 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:29.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:29.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889363+0000", 2024-04-24T15:33:29.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:29.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:29.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889363+0000", 2024-04-24T15:33:29.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:29.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:29.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889363+0000", 2024-04-24T15:33:29.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:29.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:29.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889366+0000", 2024-04-24T15:33:29.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6709999999999999e-06 2024-04-24T15:33:29.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:29.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889373+0000", 2024-04-24T15:33:29.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1269999999999997e-06 2024-04-24T15:33:29.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:29.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889377+0000", 2024-04-24T15:33:29.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1219999999999997e-06 2024-04-24T15:33:29.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:29.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907560+0000", 2024-04-24T15:33:29.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018183357000000001 2024-04-24T15:33:29.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:29.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907612+0000", 2024-04-24T15:33:29.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2172999999999997e-05 2024-04-24T15:33:29.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:29.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907678+0000", 2024-04-24T15:33:29.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5736999999999998e-05 2024-04-24T15:33:29.220 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.220 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:29.220 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.220 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48713 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2375680~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:29.220 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889381+0000", 2024-04-24T15:33:29.220 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.276720756, 2024-04-24T15:33:29.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018556019, 2024-04-24T15:33:29.220 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:29.220 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:29.220 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:29.220 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:29.221 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:29.221 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48713 2024-04-24T15:33:29.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.221 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:29.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:29.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889381+0000", 2024-04-24T15:33:29.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:29.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:29.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889381+0000", 2024-04-24T15:33:29.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:29.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:29.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889381+0000", 2024-04-24T15:33:29.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:29.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:29.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889388+0000", 2024-04-24T15:33:29.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8809999999999998e-06 2024-04-24T15:33:29.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:29.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889394+0000", 2024-04-24T15:33:29.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0410000000000003e-06 2024-04-24T15:33:29.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:29.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889397+0000", 2024-04-24T15:33:29.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5939999999999998e-06 2024-04-24T15:33:29.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:29.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907711+0000", 2024-04-24T15:33:29.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018313949999999999 2024-04-24T15:33:29.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:29.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907802+0000", 2024-04-24T15:33:29.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0307000000000001e-05 2024-04-24T15:33:29.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:29.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907937+0000", 2024-04-24T15:33:29.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013561899999999999 2024-04-24T15:33:29.224 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.224 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:29.224 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.224 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48714 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2441216~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:29.224 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889435+0000", 2024-04-24T15:33:29.224 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.276667408000002, 2024-04-24T15:33:29.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018636040999999999, 2024-04-24T15:33:29.225 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:29.225 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:29.225 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:29.225 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:29.225 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:29.225 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48714 2024-04-24T15:33:29.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.225 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:29.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:29.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889435+0000", 2024-04-24T15:33:29.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:29.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:29.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889435+0000", 2024-04-24T15:33:29.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:29.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:29.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889434+0000", 2024-04-24T15:33:29.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:29.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:29.227 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889437+0000", 2024-04-24T15:33:29.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.875e-06 2024-04-24T15:33:29.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:29.227 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889444+0000", 2024-04-24T15:33:29.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9870000000000002e-06 2024-04-24T15:33:29.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:29.227 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889448+0000", 2024-04-24T15:33:29.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3030000000000001e-06 2024-04-24T15:33:29.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.228 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:29.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907952+0000", 2024-04-24T15:33:29.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018504245999999999 2024-04-24T15:33:29.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:29.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908007+0000", 2024-04-24T15:33:29.229 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4579e-05 2024-04-24T15:33:29.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:29.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908071+0000", 2024-04-24T15:33:29.229 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3615999999999995e-05 2024-04-24T15:33:29.229 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.229 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:29.230 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.230 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48715 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2506752~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:29.230 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889453+0000", 2024-04-24T15:33:29.230 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.276649043999999, 2024-04-24T15:33:29.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018873935000000001, 2024-04-24T15:33:29.230 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:29.230 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:29.230 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:29.230 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:29.230 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:29.230 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48715 2024-04-24T15:33:29.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.231 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:29.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.231 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:29.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889453+0000", 2024-04-24T15:33:29.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:29.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.231 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:29.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889453+0000", 2024-04-24T15:33:29.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:29.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.231 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:29.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889452+0000", 2024-04-24T15:33:29.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:29.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.232 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:29.232 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889459+0000", 2024-04-24T15:33:29.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9269999999999999e-06 2024-04-24T15:33:29.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.232 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:29.232 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889468+0000", 2024-04-24T15:33:29.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2390000000000002e-06 2024-04-24T15:33:29.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.232 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:29.232 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889475+0000", 2024-04-24T15:33:29.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3939999999999998e-06 2024-04-24T15:33:29.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:29.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908103+0000", 2024-04-24T15:33:29.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018628388999999999 2024-04-24T15:33:29.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:29.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908193+0000", 2024-04-24T15:33:29.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9343000000000002e-05 2024-04-24T15:33:29.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:29.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908327+0000", 2024-04-24T15:33:29.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013415899999999999 2024-04-24T15:33:29.233 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.234 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:29.234 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.234 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48716 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2572288~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:29.234 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889480+0000", 2024-04-24T15:33:29.234 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.276622194000002, 2024-04-24T15:33:29.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018992862999999999, 2024-04-24T15:33:29.234 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:29.234 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:29.234 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:29.234 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:29.234 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:29.235 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48716 2024-04-24T15:33:29.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.235 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:29.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:29.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889480+0000", 2024-04-24T15:33:29.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:29.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:29.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889480+0000", 2024-04-24T15:33:29.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:29.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:29.236 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889479+0000", 2024-04-24T15:33:29.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:29.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.236 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:29.236 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889482+0000", 2024-04-24T15:33:29.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4729999999999999e-06 2024-04-24T15:33:29.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.236 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:29.236 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889488+0000", 2024-04-24T15:33:29.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1750000000000002e-06 2024-04-24T15:33:29.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.236 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:29.236 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889501+0000", 2024-04-24T15:33:29.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.288e-05 2024-04-24T15:33:29.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:29.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908345+0000", 2024-04-24T15:33:29.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018843835999999999 2024-04-24T15:33:29.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:29.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908395+0000", 2024-04-24T15:33:29.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9765999999999998e-05 2024-04-24T15:33:29.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.238 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:29.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908473+0000", 2024-04-24T15:33:29.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8159999999999997e-05 2024-04-24T15:33:29.238 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.238 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:29.238 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.238 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48717 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2637824~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:29.238 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889506+0000", 2024-04-24T15:33:29.238 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.276595943, 2024-04-24T15:33:29.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.019241864000000001, 2024-04-24T15:33:29.239 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:29.239 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:29.239 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:29.239 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:29.239 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:29.239 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48717 2024-04-24T15:33:29.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.239 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:29.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:29.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889506+0000", 2024-04-24T15:33:29.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:29.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:29.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889506+0000", 2024-04-24T15:33:29.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:29.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:29.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889506+0000", 2024-04-24T15:33:29.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:29.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:29.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889508+0000", 2024-04-24T15:33:29.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6900000000000001e-06 2024-04-24T15:33:29.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:29.241 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889515+0000", 2024-04-24T15:33:29.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5409999999999997e-06 2024-04-24T15:33:29.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:29.241 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889519+0000", 2024-04-24T15:33:29.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7129999999999999e-06 2024-04-24T15:33:29.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:29.241 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908520+0000", 2024-04-24T15:33:29.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.019001606000000001 2024-04-24T15:33:29.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:29.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908612+0000", 2024-04-24T15:33:29.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1909000000000004e-05 2024-04-24T15:33:29.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:29.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908748+0000", 2024-04-24T15:33:29.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000135847 2024-04-24T15:33:29.242 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.242 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:29.242 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.243 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48718 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2703360~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:29.243 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889523+0000", 2024-04-24T15:33:29.243 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.276578951000001, 2024-04-24T15:33:29.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.019451725999999999, 2024-04-24T15:33:29.243 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:29.243 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:29.243 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:29.243 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:29.243 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:29.243 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48718 2024-04-24T15:33:29.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.243 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:29.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:29.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889523+0000", 2024-04-24T15:33:29.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:29.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:29.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889523+0000", 2024-04-24T15:33:29.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:29.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:29.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889523+0000", 2024-04-24T15:33:29.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:29.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:29.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889525+0000", 2024-04-24T15:33:29.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9970000000000001e-06 2024-04-24T15:33:29.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:29.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889530+0000", 2024-04-24T15:33:29.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7610000000000004e-06 2024-04-24T15:33:29.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:29.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889534+0000", 2024-04-24T15:33:29.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6119999999999999e-06 2024-04-24T15:33:29.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.246 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:29.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908775+0000", 2024-04-24T15:33:29.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.019240709000000002 2024-04-24T15:33:29.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.246 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:29.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908856+0000", 2024-04-24T15:33:29.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0788e-05 2024-04-24T15:33:29.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.246 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:29.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908975+0000", 2024-04-24T15:33:29.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011927800000000001 2024-04-24T15:33:29.247 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.247 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:29.247 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.247 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:49872 4.1f 4:fe48f4c1:::rbd_data.114ab402c616.0000000000000014:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-04-24T15:33:29.247 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:12.416730+0000", 2024-04-24T15:33:29.247 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 16.749371972999999, 2024-04-24T15:33:29.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.020522327999999999, 2024-04-24T15:33:29.247 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:29.247 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:29.247 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:29.247 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:29.248 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:29.248 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 49872 2024-04-24T15:33:29.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.248 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:29.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:29.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.416730+0000", 2024-04-24T15:33:29.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:29.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:29.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.416730+0000", 2024-04-24T15:33:29.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:29.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:29.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.416729+0000", 2024-04-24T15:33:29.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:33:29.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:29.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.416734+0000", 2024-04-24T15:33:29.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0180000000000001e-06 2024-04-24T15:33:29.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:29.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.416744+0000", 2024-04-24T15:33:29.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0069e-05 2024-04-24T15:33:29.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:29.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.416749+0000", 2024-04-24T15:33:29.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7190000000000001e-06 2024-04-24T15:33:29.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:29.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.416777+0000", 2024-04-24T15:33:29.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7532999999999999e-05 2024-04-24T15:33:29.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:29.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.416824+0000", 2024-04-24T15:33:29.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7738000000000003e-05 2024-04-24T15:33:29.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:33:29.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.437145+0000", 2024-04-24T15:33:29.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.020320416000000001 2024-04-24T15:33:29.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:29.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.437189+0000", 2024-04-24T15:33:29.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4493000000000002e-05 2024-04-24T15:33:29.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:29.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.437252+0000", 2024-04-24T15:33:29.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3110999999999999e-05 2024-04-24T15:33:29.252 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.252 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:29.252 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.252 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:52545 4.a 4:5331820a:::rbd_data.114ab867d54d.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-04-24T15:33:29.252 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:14.399696+0000", 2024-04-24T15:33:29.252 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.76640557, 2024-04-24T15:33:29.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016877893000000001, 2024-04-24T15:33:29.252 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:29.252 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:29.252 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:29.253 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:29.253 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:29.253 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 52545 2024-04-24T15:33:29.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.253 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:29.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:29.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.399696+0000", 2024-04-24T15:33:29.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:29.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:29.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.399696+0000", 2024-04-24T15:33:29.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:29.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:29.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.399695+0000", 2024-04-24T15:33:29.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999981 2024-04-24T15:33:29.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:29.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.399704+0000", 2024-04-24T15:33:29.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.098e-06 2024-04-24T15:33:29.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:29.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.399721+0000", 2024-04-24T15:33:29.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7088999999999999e-05 2024-04-24T15:33:29.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:29.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.399728+0000", 2024-04-24T15:33:29.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5699999999999998e-06 2024-04-24T15:33:29.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:29.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.399775+0000", 2024-04-24T15:33:29.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7445999999999999e-05 2024-04-24T15:33:29.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:29.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.399840+0000", 2024-04-24T15:33:29.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5492000000000002e-05 2024-04-24T15:33:29.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:33:29.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.416468+0000", 2024-04-24T15:33:29.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016627527 2024-04-24T15:33:29.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:29.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.416521+0000", 2024-04-24T15:33:29.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2988000000000002e-05 2024-04-24T15:33:29.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:29.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.416574+0000", 2024-04-24T15:33:29.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3363999999999999e-05 2024-04-24T15:33:29.257 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.257 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:29.257 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.257 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:66966 4.1f 4:fb3b69af:::rbd_data.114aed92ca6e.000000000000001a:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-04-24T15:33:29.257 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:21.265525+0000", 2024-04-24T15:33:29.257 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 7.9005770289999999, 2024-04-24T15:33:29.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018862027999999999, 2024-04-24T15:33:29.258 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:29.258 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:29.258 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:29.258 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:29.258 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:29.258 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 66966 2024-04-24T15:33:29.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.258 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:29.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:29.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:21.265525+0000", 2024-04-24T15:33:29.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:29.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.259 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:29.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:21.265525+0000", 2024-04-24T15:33:29.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:29.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.259 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:29.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:21.265523+0000", 2024-04-24T15:33:29.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999981 2024-04-24T15:33:29.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.259 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:29.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:21.265536+0000", 2024-04-24T15:33:29.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2489e-05 2024-04-24T15:33:29.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:29.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:21.265550+0000", 2024-04-24T15:33:29.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4218e-05 2024-04-24T15:33:29.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:29.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:21.265556+0000", 2024-04-24T15:33:29.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.241e-06 2024-04-24T15:33:29.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:29.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:21.265606+0000", 2024-04-24T15:33:29.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9972000000000002e-05 2024-04-24T15:33:29.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:29.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:21.265670+0000", 2024-04-24T15:33:29.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3425999999999999e-05 2024-04-24T15:33:29.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:33:29.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:21.284297+0000", 2024-04-24T15:33:29.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018627517999999999 2024-04-24T15:33:29.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:29.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:21.284339+0000", 2024-04-24T15:33:29.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1628000000000002e-05 2024-04-24T15:33:29.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:29.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:21.284387+0000", 2024-04-24T15:33:29.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8242000000000003e-05 2024-04-24T15:33:29.262 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.262 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:29.262 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.262 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.262 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:29.263 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:29.264 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-24T15:33:29.265 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:29.264+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:29.291 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 6, 4] out_osds: [1, 5, 7] dead_osds: [6] live_osds: [0, 1, 2, 3, 4, 5, 7] 2024-04-24T15:33:29.291 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-24T15:33:29.291 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-04-24T15:33:29.291 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-04-24T15:33:29.365 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:29.365+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:29.435 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:29.435 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:33:29.435 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-04-24T15:33:29.435 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:29.446 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-24T15:33:29.466 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:29.467+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:29.567 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:29.568+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:29.610 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:29.611 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:33:29.611 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-04-24T15:33:29.611 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-04-24T15:33:29.611 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:29.621 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-24T15:33:29.634 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:33:29.634 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:29.634 INFO:teuthology.orchestra.run.smithi096.stdout: "epoch": 91, 2024-04-24T15:33:29.634 INFO:teuthology.orchestra.run.smithi096.stdout: "fsid": "253d426c-62cb-4194-8514-6cd970fd85c7", 2024-04-24T15:33:29.634 INFO:teuthology.orchestra.run.smithi096.stdout: "created": "2024-04-24T15:31:47.870327+0000", 2024-04-24T15:33:29.634 INFO:teuthology.orchestra.run.smithi096.stdout: "modified": "2024-04-24T15:33:28.894249+0000", 2024-04-24T15:33:29.634 INFO:teuthology.orchestra.run.smithi096.stdout: "last_up_change": "2024-04-24T15:33:09.796895+0000", 2024-04-24T15:33:29.634 INFO:teuthology.orchestra.run.smithi096.stdout: "last_in_change": "2024-04-24T15:32:58.722778+0000", 2024-04-24T15:33:29.634 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-24T15:33:29.634 INFO:teuthology.orchestra.run.smithi096.stdout: "flags_num": 5799936, 2024-04-24T15:33:29.634 INFO:teuthology.orchestra.run.smithi096.stdout: "flags_set": [ 2024-04-24T15:33:29.634 INFO:teuthology.orchestra.run.smithi096.stdout: "pglog_hardlimit", 2024-04-24T15:33:29.635 INFO:teuthology.orchestra.run.smithi096.stdout: "purged_snapdirs", 2024-04-24T15:33:29.635 INFO:teuthology.orchestra.run.smithi096.stdout: "recovery_deletes", 2024-04-24T15:33:29.635 INFO:teuthology.orchestra.run.smithi096.stdout: "sortbitwise" 2024-04-24T15:33:29.635 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-24T15:33:29.635 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_version": 4, 2024-04-24T15:33:29.635 INFO:teuthology.orchestra.run.smithi096.stdout: "full_ratio": 0.94999998807907104, 2024-04-24T15:33:29.635 INFO:teuthology.orchestra.run.smithi096.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-04-24T15:33:29.635 INFO:teuthology.orchestra.run.smithi096.stdout: "nearfull_ratio": 0.85000002384185791, 2024-04-24T15:33:29.635 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_snapshot": "", 2024-04-24T15:33:29.635 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_max": 4, 2024-04-24T15:33:29.635 INFO:teuthology.orchestra.run.smithi096.stdout: "max_osd": 8, 2024-04-24T15:33:29.635 INFO:teuthology.orchestra.run.smithi096.stdout: "require_min_compat_client": "luminous", 2024-04-24T15:33:29.635 INFO:teuthology.orchestra.run.smithi096.stdout: "min_compat_client": "jewel", 2024-04-24T15:33:29.635 INFO:teuthology.orchestra.run.smithi096.stdout: "require_osd_release": "quincy", 2024-04-24T15:33:29.635 INFO:teuthology.orchestra.run.smithi096.stdout: "pools": [ 2024-04-24T15:33:29.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.636 INFO:teuthology.orchestra.run.smithi096.stdout: "pool": 1, 2024-04-24T15:33:29.636 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_name": ".mgr", 2024-04-24T15:33:29.636 INFO:teuthology.orchestra.run.smithi096.stdout: "create_time": "2024-04-24T15:31:59.589762+0000", 2024-04-24T15:33:29.636 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-24T15:33:29.636 INFO:teuthology.orchestra.run.smithi096.stdout: "flags_names": "hashpspool", 2024-04-24T15:33:29.636 INFO:teuthology.orchestra.run.smithi096.stdout: "type": 1, 2024-04-24T15:33:29.636 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 2, 2024-04-24T15:33:29.636 INFO:teuthology.orchestra.run.smithi096.stdout: "min_size": 1, 2024-04-24T15:33:29.636 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_rule": 0, 2024-04-24T15:33:29.636 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_count": 0, 2024-04-24T15:33:29.636 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_target": 0, 2024-04-24T15:33:29.637 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T15:33:29.637 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T15:33:29.637 INFO:teuthology.orchestra.run.smithi096.stdout: "object_hash": 2, 2024-04-24T15:33:29.637 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_autoscale_mode": "off", 2024-04-24T15:33:29.637 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num": 1, 2024-04-24T15:33:29.637 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_placement_num": 1, 2024-04-24T15:33:29.637 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_placement_num_target": 1, 2024-04-24T15:33:29.637 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num_target": 1, 2024-04-24T15:33:29.637 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num_pending": 1, 2024-04-24T15:33:29.637 INFO:teuthology.orchestra.run.smithi096.stdout: "last_pg_merge_meta": { 2024-04-24T15:33:29.637 INFO:teuthology.orchestra.run.smithi096.stdout: "source_pgid": "0.0", 2024-04-24T15:33:29.637 INFO:teuthology.orchestra.run.smithi096.stdout: "ready_epoch": 0, 2024-04-24T15:33:29.638 INFO:teuthology.orchestra.run.smithi096.stdout: "last_epoch_started": 0, 2024-04-24T15:33:29.638 INFO:teuthology.orchestra.run.smithi096.stdout: "last_epoch_clean": 0, 2024-04-24T15:33:29.638 INFO:teuthology.orchestra.run.smithi096.stdout: "source_version": "0'0", 2024-04-24T15:33:29.638 INFO:teuthology.orchestra.run.smithi096.stdout: "target_version": "0'0" 2024-04-24T15:33:29.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.638 INFO:teuthology.orchestra.run.smithi096.stdout: "last_change": "16", 2024-04-24T15:33:29.638 INFO:teuthology.orchestra.run.smithi096.stdout: "last_force_op_resend": "0", 2024-04-24T15:33:29.638 INFO:teuthology.orchestra.run.smithi096.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T15:33:29.638 INFO:teuthology.orchestra.run.smithi096.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T15:33:29.638 INFO:teuthology.orchestra.run.smithi096.stdout: "auid": 0, 2024-04-24T15:33:29.638 INFO:teuthology.orchestra.run.smithi096.stdout: "snap_mode": "selfmanaged", 2024-04-24T15:33:29.638 INFO:teuthology.orchestra.run.smithi096.stdout: "snap_seq": 0, 2024-04-24T15:33:29.638 INFO:teuthology.orchestra.run.smithi096.stdout: "snap_epoch": 0, 2024-04-24T15:33:29.638 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_snaps": [], 2024-04-24T15:33:29.639 INFO:teuthology.orchestra.run.smithi096.stdout: "removed_snaps": "[]", 2024-04-24T15:33:29.639 INFO:teuthology.orchestra.run.smithi096.stdout: "quota_max_bytes": 0, 2024-04-24T15:33:29.639 INFO:teuthology.orchestra.run.smithi096.stdout: "quota_max_objects": 0, 2024-04-24T15:33:29.639 INFO:teuthology.orchestra.run.smithi096.stdout: "tiers": [], 2024-04-24T15:33:29.639 INFO:teuthology.orchestra.run.smithi096.stdout: "tier_of": -1, 2024-04-24T15:33:29.639 INFO:teuthology.orchestra.run.smithi096.stdout: "read_tier": -1, 2024-04-24T15:33:29.639 INFO:teuthology.orchestra.run.smithi096.stdout: "write_tier": -1, 2024-04-24T15:33:29.639 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_mode": "none", 2024-04-24T15:33:29.639 INFO:teuthology.orchestra.run.smithi096.stdout: "target_max_bytes": 0, 2024-04-24T15:33:29.639 INFO:teuthology.orchestra.run.smithi096.stdout: "target_max_objects": 0, 2024-04-24T15:33:29.639 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T15:33:29.639 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T15:33:29.639 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T15:33:29.639 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_min_flush_age": 0, 2024-04-24T15:33:29.640 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_min_evict_age": 0, 2024-04-24T15:33:29.640 INFO:teuthology.orchestra.run.smithi096.stdout: "erasure_code_profile": "", 2024-04-24T15:33:29.640 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_params": { 2024-04-24T15:33:29.640 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "none" 2024-04-24T15:33:29.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.640 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_period": 0, 2024-04-24T15:33:29.640 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_count": 0, 2024-04-24T15:33:29.640 INFO:teuthology.orchestra.run.smithi096.stdout: "use_gmt_hitset": true, 2024-04-24T15:33:29.640 INFO:teuthology.orchestra.run.smithi096.stdout: "min_read_recency_for_promote": 0, 2024-04-24T15:33:29.640 INFO:teuthology.orchestra.run.smithi096.stdout: "min_write_recency_for_promote": 0, 2024-04-24T15:33:29.640 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T15:33:29.641 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_search_last_n": 0, 2024-04-24T15:33:29.641 INFO:teuthology.orchestra.run.smithi096.stdout: "grade_table": [], 2024-04-24T15:33:29.641 INFO:teuthology.orchestra.run.smithi096.stdout: "stripe_width": 0, 2024-04-24T15:33:29.641 INFO:teuthology.orchestra.run.smithi096.stdout: "expected_num_objects": 0, 2024-04-24T15:33:29.641 INFO:teuthology.orchestra.run.smithi096.stdout: "fast_read": false, 2024-04-24T15:33:29.641 INFO:teuthology.orchestra.run.smithi096.stdout: "options": { 2024-04-24T15:33:29.641 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num_max": 32, 2024-04-24T15:33:29.641 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num_min": 1 2024-04-24T15:33:29.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.641 INFO:teuthology.orchestra.run.smithi096.stdout: "application_metadata": { 2024-04-24T15:33:29.641 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr": {} 2024-04-24T15:33:29.641 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.641 INFO:teuthology.orchestra.run.smithi096.stdout: "pool": 2, 2024-04-24T15:33:29.642 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_name": "rbd", 2024-04-24T15:33:29.642 INFO:teuthology.orchestra.run.smithi096.stdout: "create_time": "2024-04-24T15:31:59.971601+0000", 2024-04-24T15:33:29.642 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-24T15:33:29.642 INFO:teuthology.orchestra.run.smithi096.stdout: "flags_names": "hashpspool", 2024-04-24T15:33:29.642 INFO:teuthology.orchestra.run.smithi096.stdout: "type": 1, 2024-04-24T15:33:29.642 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 2, 2024-04-24T15:33:29.642 INFO:teuthology.orchestra.run.smithi096.stdout: "min_size": 1, 2024-04-24T15:33:29.642 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_rule": 0, 2024-04-24T15:33:29.642 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_count": 0, 2024-04-24T15:33:29.642 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_target": 0, 2024-04-24T15:33:29.642 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T15:33:29.642 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T15:33:29.643 INFO:teuthology.orchestra.run.smithi096.stdout: "object_hash": 2, 2024-04-24T15:33:29.643 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_autoscale_mode": "off", 2024-04-24T15:33:29.643 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num": 8, 2024-04-24T15:33:29.643 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_placement_num": 8, 2024-04-24T15:33:29.643 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_placement_num_target": 8, 2024-04-24T15:33:29.643 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num_target": 8, 2024-04-24T15:33:29.643 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num_pending": 8, 2024-04-24T15:33:29.643 INFO:teuthology.orchestra.run.smithi096.stdout: "last_pg_merge_meta": { 2024-04-24T15:33:29.643 INFO:teuthology.orchestra.run.smithi096.stdout: "source_pgid": "0.0", 2024-04-24T15:33:29.643 INFO:teuthology.orchestra.run.smithi096.stdout: "ready_epoch": 0, 2024-04-24T15:33:29.643 INFO:teuthology.orchestra.run.smithi096.stdout: "last_epoch_started": 0, 2024-04-24T15:33:29.644 INFO:teuthology.orchestra.run.smithi096.stdout: "last_epoch_clean": 0, 2024-04-24T15:33:29.644 INFO:teuthology.orchestra.run.smithi096.stdout: "source_version": "0'0", 2024-04-24T15:33:29.644 INFO:teuthology.orchestra.run.smithi096.stdout: "target_version": "0'0" 2024-04-24T15:33:29.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.644 INFO:teuthology.orchestra.run.smithi096.stdout: "last_change": "21", 2024-04-24T15:33:29.644 INFO:teuthology.orchestra.run.smithi096.stdout: "last_force_op_resend": "21", 2024-04-24T15:33:29.644 INFO:teuthology.orchestra.run.smithi096.stdout: "last_force_op_resend_prenautilus": "21", 2024-04-24T15:33:29.644 INFO:teuthology.orchestra.run.smithi096.stdout: "last_force_op_resend_preluminous": "21", 2024-04-24T15:33:29.644 INFO:teuthology.orchestra.run.smithi096.stdout: "auid": 0, 2024-04-24T15:33:29.644 INFO:teuthology.orchestra.run.smithi096.stdout: "snap_mode": "selfmanaged", 2024-04-24T15:33:29.644 INFO:teuthology.orchestra.run.smithi096.stdout: "snap_seq": 0, 2024-04-24T15:33:29.644 INFO:teuthology.orchestra.run.smithi096.stdout: "snap_epoch": 19, 2024-04-24T15:33:29.644 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_snaps": [], 2024-04-24T15:33:29.644 INFO:teuthology.orchestra.run.smithi096.stdout: "removed_snaps": "[]", 2024-04-24T15:33:29.644 INFO:teuthology.orchestra.run.smithi096.stdout: "quota_max_bytes": 0, 2024-04-24T15:33:29.644 INFO:teuthology.orchestra.run.smithi096.stdout: "quota_max_objects": 0, 2024-04-24T15:33:29.645 INFO:teuthology.orchestra.run.smithi096.stdout: "tiers": [ 2024-04-24T15:33:29.645 INFO:teuthology.orchestra.run.smithi096.stdout: 3 2024-04-24T15:33:29.645 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-24T15:33:29.645 INFO:teuthology.orchestra.run.smithi096.stdout: "tier_of": -1, 2024-04-24T15:33:29.645 INFO:teuthology.orchestra.run.smithi096.stdout: "read_tier": 3, 2024-04-24T15:33:29.645 INFO:teuthology.orchestra.run.smithi096.stdout: "write_tier": 3, 2024-04-24T15:33:29.645 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_mode": "none", 2024-04-24T15:33:29.645 INFO:teuthology.orchestra.run.smithi096.stdout: "target_max_bytes": 0, 2024-04-24T15:33:29.645 INFO:teuthology.orchestra.run.smithi096.stdout: "target_max_objects": 0, 2024-04-24T15:33:29.645 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T15:33:29.645 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T15:33:29.645 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T15:33:29.645 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_min_flush_age": 0, 2024-04-24T15:33:29.645 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_min_evict_age": 0, 2024-04-24T15:33:29.645 INFO:teuthology.orchestra.run.smithi096.stdout: "erasure_code_profile": "", 2024-04-24T15:33:29.646 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_params": { 2024-04-24T15:33:29.646 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "none" 2024-04-24T15:33:29.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.646 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_period": 0, 2024-04-24T15:33:29.646 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_count": 0, 2024-04-24T15:33:29.646 INFO:teuthology.orchestra.run.smithi096.stdout: "use_gmt_hitset": true, 2024-04-24T15:33:29.646 INFO:teuthology.orchestra.run.smithi096.stdout: "min_read_recency_for_promote": 0, 2024-04-24T15:33:29.646 INFO:teuthology.orchestra.run.smithi096.stdout: "min_write_recency_for_promote": 0, 2024-04-24T15:33:29.646 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T15:33:29.646 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_search_last_n": 0, 2024-04-24T15:33:29.646 INFO:teuthology.orchestra.run.smithi096.stdout: "grade_table": [], 2024-04-24T15:33:29.646 INFO:teuthology.orchestra.run.smithi096.stdout: "stripe_width": 0, 2024-04-24T15:33:29.646 INFO:teuthology.orchestra.run.smithi096.stdout: "expected_num_objects": 0, 2024-04-24T15:33:29.647 INFO:teuthology.orchestra.run.smithi096.stdout: "fast_read": false, 2024-04-24T15:33:29.647 INFO:teuthology.orchestra.run.smithi096.stdout: "options": {}, 2024-04-24T15:33:29.647 INFO:teuthology.orchestra.run.smithi096.stdout: "application_metadata": { 2024-04-24T15:33:29.647 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd": {} 2024-04-24T15:33:29.647 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.647 INFO:teuthology.orchestra.run.smithi096.stdout: "pool": 3, 2024-04-24T15:33:29.647 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_name": "cache", 2024-04-24T15:33:29.647 INFO:teuthology.orchestra.run.smithi096.stdout: "create_time": "2024-04-24T15:32:06.914463+0000", 2024-04-24T15:33:29.647 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 9, 2024-04-24T15:33:29.647 INFO:teuthology.orchestra.run.smithi096.stdout: "flags_names": "hashpspool,incomplete_clones", 2024-04-24T15:33:29.647 INFO:teuthology.orchestra.run.smithi096.stdout: "type": 1, 2024-04-24T15:33:29.647 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 2, 2024-04-24T15:33:29.647 INFO:teuthology.orchestra.run.smithi096.stdout: "min_size": 1, 2024-04-24T15:33:29.648 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_rule": 0, 2024-04-24T15:33:29.648 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_count": 0, 2024-04-24T15:33:29.648 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_target": 0, 2024-04-24T15:33:29.648 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T15:33:29.648 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T15:33:29.648 INFO:teuthology.orchestra.run.smithi096.stdout: "object_hash": 2, 2024-04-24T15:33:29.648 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_autoscale_mode": "off", 2024-04-24T15:33:29.648 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num": 4, 2024-04-24T15:33:29.648 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_placement_num": 4, 2024-04-24T15:33:29.648 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_placement_num_target": 4, 2024-04-24T15:33:29.648 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num_target": 4, 2024-04-24T15:33:29.648 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num_pending": 4, 2024-04-24T15:33:29.648 INFO:teuthology.orchestra.run.smithi096.stdout: "last_pg_merge_meta": { 2024-04-24T15:33:29.648 INFO:teuthology.orchestra.run.smithi096.stdout: "source_pgid": "0.0", 2024-04-24T15:33:29.648 INFO:teuthology.orchestra.run.smithi096.stdout: "ready_epoch": 0, 2024-04-24T15:33:29.648 INFO:teuthology.orchestra.run.smithi096.stdout: "last_epoch_started": 0, 2024-04-24T15:33:29.649 INFO:teuthology.orchestra.run.smithi096.stdout: "last_epoch_clean": 0, 2024-04-24T15:33:29.649 INFO:teuthology.orchestra.run.smithi096.stdout: "source_version": "0'0", 2024-04-24T15:33:29.649 INFO:teuthology.orchestra.run.smithi096.stdout: "target_version": "0'0" 2024-04-24T15:33:29.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.649 INFO:teuthology.orchestra.run.smithi096.stdout: "last_change": "25", 2024-04-24T15:33:29.649 INFO:teuthology.orchestra.run.smithi096.stdout: "last_force_op_resend": "21", 2024-04-24T15:33:29.649 INFO:teuthology.orchestra.run.smithi096.stdout: "last_force_op_resend_prenautilus": "21", 2024-04-24T15:33:29.649 INFO:teuthology.orchestra.run.smithi096.stdout: "last_force_op_resend_preluminous": "21", 2024-04-24T15:33:29.649 INFO:teuthology.orchestra.run.smithi096.stdout: "auid": 0, 2024-04-24T15:33:29.649 INFO:teuthology.orchestra.run.smithi096.stdout: "snap_mode": "selfmanaged", 2024-04-24T15:33:29.649 INFO:teuthology.orchestra.run.smithi096.stdout: "snap_seq": 0, 2024-04-24T15:33:29.649 INFO:teuthology.orchestra.run.smithi096.stdout: "snap_epoch": 19, 2024-04-24T15:33:29.649 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_snaps": [], 2024-04-24T15:33:29.649 INFO:teuthology.orchestra.run.smithi096.stdout: "removed_snaps": "[]", 2024-04-24T15:33:29.649 INFO:teuthology.orchestra.run.smithi096.stdout: "quota_max_bytes": 0, 2024-04-24T15:33:29.650 INFO:teuthology.orchestra.run.smithi096.stdout: "quota_max_objects": 0, 2024-04-24T15:33:29.650 INFO:teuthology.orchestra.run.smithi096.stdout: "tiers": [], 2024-04-24T15:33:29.650 INFO:teuthology.orchestra.run.smithi096.stdout: "tier_of": 2, 2024-04-24T15:33:29.650 INFO:teuthology.orchestra.run.smithi096.stdout: "read_tier": -1, 2024-04-24T15:33:29.650 INFO:teuthology.orchestra.run.smithi096.stdout: "write_tier": -1, 2024-04-24T15:33:29.650 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_mode": "writeback", 2024-04-24T15:33:29.650 INFO:teuthology.orchestra.run.smithi096.stdout: "target_max_bytes": 0, 2024-04-24T15:33:29.650 INFO:teuthology.orchestra.run.smithi096.stdout: "target_max_objects": 250, 2024-04-24T15:33:29.650 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T15:33:29.650 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T15:33:29.650 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T15:33:29.650 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_min_flush_age": 0, 2024-04-24T15:33:29.650 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_min_evict_age": 0, 2024-04-24T15:33:29.651 INFO:teuthology.orchestra.run.smithi096.stdout: "erasure_code_profile": "", 2024-04-24T15:33:29.651 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_params": { 2024-04-24T15:33:29.651 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bloom", 2024-04-24T15:33:29.651 INFO:teuthology.orchestra.run.smithi096.stdout: "false_positive_probability": 0.050000000000000003, 2024-04-24T15:33:29.651 INFO:teuthology.orchestra.run.smithi096.stdout: "target_size": 0, 2024-04-24T15:33:29.651 INFO:teuthology.orchestra.run.smithi096.stdout: "seed": 0 2024-04-24T15:33:29.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.651 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_period": 60, 2024-04-24T15:33:29.651 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_count": 8, 2024-04-24T15:33:29.651 INFO:teuthology.orchestra.run.smithi096.stdout: "use_gmt_hitset": true, 2024-04-24T15:33:29.651 INFO:teuthology.orchestra.run.smithi096.stdout: "min_read_recency_for_promote": 0, 2024-04-24T15:33:29.651 INFO:teuthology.orchestra.run.smithi096.stdout: "min_write_recency_for_promote": 0, 2024-04-24T15:33:29.651 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T15:33:29.651 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_search_last_n": 0, 2024-04-24T15:33:29.652 INFO:teuthology.orchestra.run.smithi096.stdout: "grade_table": [ 2024-04-24T15:33:29.652 INFO:teuthology.orchestra.run.smithi096.stdout: 1000000, 2024-04-24T15:33:29.652 INFO:teuthology.orchestra.run.smithi096.stdout: 1000000, 2024-04-24T15:33:29.652 INFO:teuthology.orchestra.run.smithi096.stdout: 1000000, 2024-04-24T15:33:29.652 INFO:teuthology.orchestra.run.smithi096.stdout: 1000000, 2024-04-24T15:33:29.652 INFO:teuthology.orchestra.run.smithi096.stdout: 1000000, 2024-04-24T15:33:29.652 INFO:teuthology.orchestra.run.smithi096.stdout: 1000000, 2024-04-24T15:33:29.652 INFO:teuthology.orchestra.run.smithi096.stdout: 1000000, 2024-04-24T15:33:29.652 INFO:teuthology.orchestra.run.smithi096.stdout: 1000000 2024-04-24T15:33:29.652 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-24T15:33:29.652 INFO:teuthology.orchestra.run.smithi096.stdout: "stripe_width": 0, 2024-04-24T15:33:29.652 INFO:teuthology.orchestra.run.smithi096.stdout: "expected_num_objects": 0, 2024-04-24T15:33:29.652 INFO:teuthology.orchestra.run.smithi096.stdout: "fast_read": false, 2024-04-24T15:33:29.652 INFO:teuthology.orchestra.run.smithi096.stdout: "options": {}, 2024-04-24T15:33:29.652 INFO:teuthology.orchestra.run.smithi096.stdout: "application_metadata": { 2024-04-24T15:33:29.653 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd": {} 2024-04-24T15:33:29.653 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.653 INFO:teuthology.orchestra.run.smithi096.stdout: "pool": 4, 2024-04-24T15:33:29.653 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_name": "pool_client.0", 2024-04-24T15:33:29.653 INFO:teuthology.orchestra.run.smithi096.stdout: "create_time": "2024-04-24T15:32:15.929114+0000", 2024-04-24T15:33:29.653 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 8193, 2024-04-24T15:33:29.653 INFO:teuthology.orchestra.run.smithi096.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-04-24T15:33:29.653 INFO:teuthology.orchestra.run.smithi096.stdout: "type": 1, 2024-04-24T15:33:29.653 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 2, 2024-04-24T15:33:29.654 INFO:teuthology.orchestra.run.smithi096.stdout: "min_size": 1, 2024-04-24T15:33:29.654 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_rule": 0, 2024-04-24T15:33:29.654 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_count": 0, 2024-04-24T15:33:29.654 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_target": 0, 2024-04-24T15:33:29.654 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T15:33:29.654 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T15:33:29.654 INFO:teuthology.orchestra.run.smithi096.stdout: "object_hash": 2, 2024-04-24T15:33:29.654 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_autoscale_mode": "off", 2024-04-24T15:33:29.654 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num": 32, 2024-04-24T15:33:29.654 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_placement_num": 32, 2024-04-24T15:33:29.655 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_placement_num_target": 32, 2024-04-24T15:33:29.655 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num_target": 32, 2024-04-24T15:33:29.655 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num_pending": 32, 2024-04-24T15:33:29.655 INFO:teuthology.orchestra.run.smithi096.stdout: "last_pg_merge_meta": { 2024-04-24T15:33:29.655 INFO:teuthology.orchestra.run.smithi096.stdout: "source_pgid": "0.0", 2024-04-24T15:33:29.655 INFO:teuthology.orchestra.run.smithi096.stdout: "ready_epoch": 0, 2024-04-24T15:33:29.655 INFO:teuthology.orchestra.run.smithi096.stdout: "last_epoch_started": 0, 2024-04-24T15:33:29.655 INFO:teuthology.orchestra.run.smithi096.stdout: "last_epoch_clean": 0, 2024-04-24T15:33:29.655 INFO:teuthology.orchestra.run.smithi096.stdout: "source_version": "0'0", 2024-04-24T15:33:29.655 INFO:teuthology.orchestra.run.smithi096.stdout: "target_version": "0'0" 2024-04-24T15:33:29.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.655 INFO:teuthology.orchestra.run.smithi096.stdout: "last_change": "91", 2024-04-24T15:33:29.655 INFO:teuthology.orchestra.run.smithi096.stdout: "last_force_op_resend": "0", 2024-04-24T15:33:29.655 INFO:teuthology.orchestra.run.smithi096.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T15:33:29.655 INFO:teuthology.orchestra.run.smithi096.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T15:33:29.656 INFO:teuthology.orchestra.run.smithi096.stdout: "auid": 0, 2024-04-24T15:33:29.656 INFO:teuthology.orchestra.run.smithi096.stdout: "snap_mode": "selfmanaged", 2024-04-24T15:33:29.656 INFO:teuthology.orchestra.run.smithi096.stdout: "snap_seq": 37, 2024-04-24T15:33:29.656 INFO:teuthology.orchestra.run.smithi096.stdout: "snap_epoch": 91, 2024-04-24T15:33:29.656 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_snaps": [], 2024-04-24T15:33:29.656 INFO:teuthology.orchestra.run.smithi096.stdout: "removed_snaps": "[]", 2024-04-24T15:33:29.656 INFO:teuthology.orchestra.run.smithi096.stdout: "quota_max_bytes": 0, 2024-04-24T15:33:29.656 INFO:teuthology.orchestra.run.smithi096.stdout: "quota_max_objects": 0, 2024-04-24T15:33:29.656 INFO:teuthology.orchestra.run.smithi096.stdout: "tiers": [], 2024-04-24T15:33:29.656 INFO:teuthology.orchestra.run.smithi096.stdout: "tier_of": -1, 2024-04-24T15:33:29.656 INFO:teuthology.orchestra.run.smithi096.stdout: "read_tier": -1, 2024-04-24T15:33:29.656 INFO:teuthology.orchestra.run.smithi096.stdout: "write_tier": -1, 2024-04-24T15:33:29.656 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_mode": "none", 2024-04-24T15:33:29.656 INFO:teuthology.orchestra.run.smithi096.stdout: "target_max_bytes": 0, 2024-04-24T15:33:29.656 INFO:teuthology.orchestra.run.smithi096.stdout: "target_max_objects": 0, 2024-04-24T15:33:29.656 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T15:33:29.657 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T15:33:29.657 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T15:33:29.657 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_min_flush_age": 0, 2024-04-24T15:33:29.657 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_min_evict_age": 0, 2024-04-24T15:33:29.657 INFO:teuthology.orchestra.run.smithi096.stdout: "erasure_code_profile": "", 2024-04-24T15:33:29.657 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_params": { 2024-04-24T15:33:29.657 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "none" 2024-04-24T15:33:29.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.657 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_period": 0, 2024-04-24T15:33:29.657 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_count": 0, 2024-04-24T15:33:29.657 INFO:teuthology.orchestra.run.smithi096.stdout: "use_gmt_hitset": true, 2024-04-24T15:33:29.657 INFO:teuthology.orchestra.run.smithi096.stdout: "min_read_recency_for_promote": 0, 2024-04-24T15:33:29.657 INFO:teuthology.orchestra.run.smithi096.stdout: "min_write_recency_for_promote": 0, 2024-04-24T15:33:29.657 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T15:33:29.657 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_search_last_n": 0, 2024-04-24T15:33:29.658 INFO:teuthology.orchestra.run.smithi096.stdout: "grade_table": [], 2024-04-24T15:33:29.658 INFO:teuthology.orchestra.run.smithi096.stdout: "stripe_width": 0, 2024-04-24T15:33:29.658 INFO:teuthology.orchestra.run.smithi096.stdout: "expected_num_objects": 0, 2024-04-24T15:33:29.658 INFO:teuthology.orchestra.run.smithi096.stdout: "fast_read": false, 2024-04-24T15:33:29.658 INFO:teuthology.orchestra.run.smithi096.stdout: "options": {}, 2024-04-24T15:33:29.658 INFO:teuthology.orchestra.run.smithi096.stdout: "application_metadata": { 2024-04-24T15:33:29.658 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd": {} 2024-04-24T15:33:29.658 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.658 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.658 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-24T15:33:29.658 INFO:teuthology.orchestra.run.smithi096.stdout: "osds": [ 2024-04-24T15:33:29.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.658 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 0, 2024-04-24T15:33:29.659 INFO:teuthology.orchestra.run.smithi096.stdout: "uuid": "92941fb9-96ee-4b2b-8147-3c1d61192f94", 2024-04-24T15:33:29.659 INFO:teuthology.orchestra.run.smithi096.stdout: "up": 1, 2024-04-24T15:33:29.659 INFO:teuthology.orchestra.run.smithi096.stdout: "in": 1, 2024-04-24T15:33:29.659 INFO:teuthology.orchestra.run.smithi096.stdout: "weight": 1, 2024-04-24T15:33:29.659 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity": 0, 2024-04-24T15:33:29.659 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_begin": 0, 2024-04-24T15:33:29.659 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_end": 0, 2024-04-24T15:33:29.659 INFO:teuthology.orchestra.run.smithi096.stdout: "up_from": 13, 2024-04-24T15:33:29.659 INFO:teuthology.orchestra.run.smithi096.stdout: "up_thru": 76, 2024-04-24T15:33:29.659 INFO:teuthology.orchestra.run.smithi096.stdout: "down_at": 0, 2024-04-24T15:33:29.659 INFO:teuthology.orchestra.run.smithi096.stdout: "lost_at": 0, 2024-04-24T15:33:29.659 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addrs": { 2024-04-24T15:33:29.659 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-04-24T15:33:29.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.659 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-04-24T15:33:29.660 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6816", 2024-04-24T15:33:29.660 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 44106 2024-04-24T15:33:29.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.660 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-04-24T15:33:29.660 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6817", 2024-04-24T15:33:29.660 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 44106 2024-04-24T15:33:29.660 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.660 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:29.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.660 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addrs": { 2024-04-24T15:33:29.660 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-04-24T15:33:29.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.660 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-04-24T15:33:29.661 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6818", 2024-04-24T15:33:29.661 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 44106 2024-04-24T15:33:29.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.661 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-04-24T15:33:29.661 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6819", 2024-04-24T15:33:29.661 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 44106 2024-04-24T15:33:29.661 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.661 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:29.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.662 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addrs": { 2024-04-24T15:33:29.662 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-04-24T15:33:29.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.662 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-04-24T15:33:29.662 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6822", 2024-04-24T15:33:29.662 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 44106 2024-04-24T15:33:29.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.662 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-04-24T15:33:29.662 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6823", 2024-04-24T15:33:29.662 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 44106 2024-04-24T15:33:29.663 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.663 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:29.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.663 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addrs": { 2024-04-24T15:33:29.663 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-04-24T15:33:29.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.663 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-04-24T15:33:29.663 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6820", 2024-04-24T15:33:29.663 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 44106 2024-04-24T15:33:29.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.663 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-04-24T15:33:29.663 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6821", 2024-04-24T15:33:29.663 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 44106 2024-04-24T15:33:29.664 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.664 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:29.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.664 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addr": "172.21.15.96:6817/44106", 2024-04-24T15:33:29.664 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addr": "172.21.15.96:6819/44106", 2024-04-24T15:33:29.664 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addr": "172.21.15.96:6823/44106", 2024-04-24T15:33:29.664 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addr": "172.21.15.96:6821/44106", 2024-04-24T15:33:29.664 INFO:teuthology.orchestra.run.smithi096.stdout: "state": [ 2024-04-24T15:33:29.665 INFO:teuthology.orchestra.run.smithi096.stdout: "exists", 2024-04-24T15:33:29.665 INFO:teuthology.orchestra.run.smithi096.stdout: "up" 2024-04-24T15:33:29.665 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:29.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.665 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 1, 2024-04-24T15:33:29.665 INFO:teuthology.orchestra.run.smithi096.stdout: "uuid": "628e868a-9884-42e6-9331-caef97cb6967", 2024-04-24T15:33:29.665 INFO:teuthology.orchestra.run.smithi096.stdout: "up": 1, 2024-04-24T15:33:29.665 INFO:teuthology.orchestra.run.smithi096.stdout: "in": 0, 2024-04-24T15:33:29.665 INFO:teuthology.orchestra.run.smithi096.stdout: "weight": 0, 2024-04-24T15:33:29.665 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity": 1, 2024-04-24T15:33:29.665 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_begin": 0, 2024-04-24T15:33:29.665 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_end": 0, 2024-04-24T15:33:29.665 INFO:teuthology.orchestra.run.smithi096.stdout: "up_from": 13, 2024-04-24T15:33:29.666 INFO:teuthology.orchestra.run.smithi096.stdout: "up_thru": 26, 2024-04-24T15:33:29.666 INFO:teuthology.orchestra.run.smithi096.stdout: "down_at": 0, 2024-04-24T15:33:29.666 INFO:teuthology.orchestra.run.smithi096.stdout: "lost_at": 0, 2024-04-24T15:33:29.666 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addrs": { 2024-04-24T15:33:29.666 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-04-24T15:33:29.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.666 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-04-24T15:33:29.666 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6800", 2024-04-24T15:33:29.666 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 44089 2024-04-24T15:33:29.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.667 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-04-24T15:33:29.667 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6801", 2024-04-24T15:33:29.667 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 44089 2024-04-24T15:33:29.667 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.667 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:29.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.667 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addrs": { 2024-04-24T15:33:29.667 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-04-24T15:33:29.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.667 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-04-24T15:33:29.667 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6802", 2024-04-24T15:33:29.667 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 44089 2024-04-24T15:33:29.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.668 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-04-24T15:33:29.668 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6803", 2024-04-24T15:33:29.668 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 44089 2024-04-24T15:33:29.668 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.668 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:29.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.668 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addrs": { 2024-04-24T15:33:29.668 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-04-24T15:33:29.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.668 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-04-24T15:33:29.668 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6806", 2024-04-24T15:33:29.668 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 44089 2024-04-24T15:33:29.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.669 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-04-24T15:33:29.669 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6807", 2024-04-24T15:33:29.669 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 44089 2024-04-24T15:33:29.669 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.669 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:29.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.669 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addrs": { 2024-04-24T15:33:29.669 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-04-24T15:33:29.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.670 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-04-24T15:33:29.670 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6804", 2024-04-24T15:33:29.670 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 44089 2024-04-24T15:33:29.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.670 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-04-24T15:33:29.670 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6805", 2024-04-24T15:33:29.670 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 44089 2024-04-24T15:33:29.670 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.670 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:29.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.670 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addr": "172.21.15.96:6801/44089", 2024-04-24T15:33:29.671 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addr": "172.21.15.96:6803/44089", 2024-04-24T15:33:29.671 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addr": "172.21.15.96:6807/44089", 2024-04-24T15:33:29.671 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addr": "172.21.15.96:6805/44089", 2024-04-24T15:33:29.671 INFO:teuthology.orchestra.run.smithi096.stdout: "state": [ 2024-04-24T15:33:29.671 INFO:teuthology.orchestra.run.smithi096.stdout: "exists", 2024-04-24T15:33:29.671 INFO:teuthology.orchestra.run.smithi096.stdout: "up" 2024-04-24T15:33:29.671 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:29.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.671 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 2, 2024-04-24T15:33:29.671 INFO:teuthology.orchestra.run.smithi096.stdout: "uuid": "46de3f65-87a8-460c-b57a-4468d9c89a57", 2024-04-24T15:33:29.671 INFO:teuthology.orchestra.run.smithi096.stdout: "up": 1, 2024-04-24T15:33:29.671 INFO:teuthology.orchestra.run.smithi096.stdout: "in": 1, 2024-04-24T15:33:29.671 INFO:teuthology.orchestra.run.smithi096.stdout: "weight": 1, 2024-04-24T15:33:29.672 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity": 0.3755035400390625, 2024-04-24T15:33:29.672 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_begin": 0, 2024-04-24T15:33:29.672 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_end": 0, 2024-04-24T15:33:29.672 INFO:teuthology.orchestra.run.smithi096.stdout: "up_from": 13, 2024-04-24T15:33:29.672 INFO:teuthology.orchestra.run.smithi096.stdout: "up_thru": 76, 2024-04-24T15:33:29.672 INFO:teuthology.orchestra.run.smithi096.stdout: "down_at": 0, 2024-04-24T15:33:29.672 INFO:teuthology.orchestra.run.smithi096.stdout: "lost_at": 0, 2024-04-24T15:33:29.672 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addrs": { 2024-04-24T15:33:29.672 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-04-24T15:33:29.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.672 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-04-24T15:33:29.672 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6808", 2024-04-24T15:33:29.673 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 44090 2024-04-24T15:33:29.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.673 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-04-24T15:33:29.673 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6809", 2024-04-24T15:33:29.673 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 44090 2024-04-24T15:33:29.673 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.673 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:29.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.673 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addrs": { 2024-04-24T15:33:29.673 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-04-24T15:33:29.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.673 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-04-24T15:33:29.674 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6810", 2024-04-24T15:33:29.674 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 44090 2024-04-24T15:33:29.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.674 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-04-24T15:33:29.674 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6811", 2024-04-24T15:33:29.674 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 44090 2024-04-24T15:33:29.674 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.674 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:29.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.674 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addrs": { 2024-04-24T15:33:29.674 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-04-24T15:33:29.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.675 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-04-24T15:33:29.675 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6814", 2024-04-24T15:33:29.675 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 44090 2024-04-24T15:33:29.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.675 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-04-24T15:33:29.675 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6815", 2024-04-24T15:33:29.675 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 44090 2024-04-24T15:33:29.676 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.676 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:29.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.676 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addrs": { 2024-04-24T15:33:29.676 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-04-24T15:33:29.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.676 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-04-24T15:33:29.676 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6812", 2024-04-24T15:33:29.676 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 44090 2024-04-24T15:33:29.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.676 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-04-24T15:33:29.676 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6813", 2024-04-24T15:33:29.676 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 44090 2024-04-24T15:33:29.677 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.677 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:29.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.677 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addr": "172.21.15.96:6809/44090", 2024-04-24T15:33:29.677 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addr": "172.21.15.96:6811/44090", 2024-04-24T15:33:29.677 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addr": "172.21.15.96:6815/44090", 2024-04-24T15:33:29.677 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addr": "172.21.15.96:6813/44090", 2024-04-24T15:33:29.677 INFO:teuthology.orchestra.run.smithi096.stdout: "state": [ 2024-04-24T15:33:29.677 INFO:teuthology.orchestra.run.smithi096.stdout: "exists", 2024-04-24T15:33:29.677 INFO:teuthology.orchestra.run.smithi096.stdout: "up" 2024-04-24T15:33:29.677 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:29.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.678 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 3, 2024-04-24T15:33:29.678 INFO:teuthology.orchestra.run.smithi096.stdout: "uuid": "5507e063-10bc-4e32-b1ec-00aa3e47d993", 2024-04-24T15:33:29.678 INFO:teuthology.orchestra.run.smithi096.stdout: "up": 1, 2024-04-24T15:33:29.678 INFO:teuthology.orchestra.run.smithi096.stdout: "in": 1, 2024-04-24T15:33:29.678 INFO:teuthology.orchestra.run.smithi096.stdout: "weight": 1, 2024-04-24T15:33:29.678 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity": 1, 2024-04-24T15:33:29.678 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_begin": 0, 2024-04-24T15:33:29.678 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_end": 0, 2024-04-24T15:33:29.678 INFO:teuthology.orchestra.run.smithi096.stdout: "up_from": 13, 2024-04-24T15:33:29.678 INFO:teuthology.orchestra.run.smithi096.stdout: "up_thru": 76, 2024-04-24T15:33:29.678 INFO:teuthology.orchestra.run.smithi096.stdout: "down_at": 0, 2024-04-24T15:33:29.678 INFO:teuthology.orchestra.run.smithi096.stdout: "lost_at": 0, 2024-04-24T15:33:29.678 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addrs": { 2024-04-24T15:33:29.679 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-04-24T15:33:29.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.679 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-04-24T15:33:29.679 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6824", 2024-04-24T15:33:29.679 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 44121 2024-04-24T15:33:29.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.679 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-04-24T15:33:29.679 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6825", 2024-04-24T15:33:29.679 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 44121 2024-04-24T15:33:29.679 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.680 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:29.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.680 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addrs": { 2024-04-24T15:33:29.680 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-04-24T15:33:29.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.680 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-04-24T15:33:29.680 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6826", 2024-04-24T15:33:29.680 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 44121 2024-04-24T15:33:29.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.680 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-04-24T15:33:29.680 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6827", 2024-04-24T15:33:29.680 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 44121 2024-04-24T15:33:29.681 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.681 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:29.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.681 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addrs": { 2024-04-24T15:33:29.681 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-04-24T15:33:29.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.681 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-04-24T15:33:29.681 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6830", 2024-04-24T15:33:29.681 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 44121 2024-04-24T15:33:29.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.682 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.682 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-04-24T15:33:29.682 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6831", 2024-04-24T15:33:29.682 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 44121 2024-04-24T15:33:29.682 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.682 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:29.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.682 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addrs": { 2024-04-24T15:33:29.682 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-04-24T15:33:29.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.683 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-04-24T15:33:29.683 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6828", 2024-04-24T15:33:29.683 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 44121 2024-04-24T15:33:29.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.683 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-04-24T15:33:29.683 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6829", 2024-04-24T15:33:29.683 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 44121 2024-04-24T15:33:29.683 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.684 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:29.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.684 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addr": "172.21.15.96:6825/44121", 2024-04-24T15:33:29.684 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addr": "172.21.15.96:6827/44121", 2024-04-24T15:33:29.684 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addr": "172.21.15.96:6831/44121", 2024-04-24T15:33:29.684 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addr": "172.21.15.96:6829/44121", 2024-04-24T15:33:29.684 INFO:teuthology.orchestra.run.smithi096.stdout: "state": [ 2024-04-24T15:33:29.684 INFO:teuthology.orchestra.run.smithi096.stdout: "exists", 2024-04-24T15:33:29.684 INFO:teuthology.orchestra.run.smithi096.stdout: "up" 2024-04-24T15:33:29.684 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:29.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.685 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 4, 2024-04-24T15:33:29.685 INFO:teuthology.orchestra.run.smithi096.stdout: "uuid": "6a74d453-1048-44ef-a5b0-4277583d76c2", 2024-04-24T15:33:29.685 INFO:teuthology.orchestra.run.smithi096.stdout: "up": 1, 2024-04-24T15:33:29.685 INFO:teuthology.orchestra.run.smithi096.stdout: "in": 1, 2024-04-24T15:33:29.685 INFO:teuthology.orchestra.run.smithi096.stdout: "weight": 1, 2024-04-24T15:33:29.686 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity": 1, 2024-04-24T15:33:29.686 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_begin": 0, 2024-04-24T15:33:29.686 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_end": 0, 2024-04-24T15:33:29.686 INFO:teuthology.orchestra.run.smithi096.stdout: "up_from": 13, 2024-04-24T15:33:29.686 INFO:teuthology.orchestra.run.smithi096.stdout: "up_thru": 76, 2024-04-24T15:33:29.686 INFO:teuthology.orchestra.run.smithi096.stdout: "down_at": 0, 2024-04-24T15:33:29.686 INFO:teuthology.orchestra.run.smithi096.stdout: "lost_at": 0, 2024-04-24T15:33:29.686 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addrs": { 2024-04-24T15:33:29.686 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-04-24T15:33:29.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.687 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-04-24T15:33:29.687 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.151:6816", 2024-04-24T15:33:29.687 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 44025 2024-04-24T15:33:29.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.687 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-04-24T15:33:29.687 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.151:6817", 2024-04-24T15:33:29.687 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 44025 2024-04-24T15:33:29.687 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.687 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:29.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.687 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addrs": { 2024-04-24T15:33:29.688 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-04-24T15:33:29.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.688 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-04-24T15:33:29.688 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.151:6818", 2024-04-24T15:33:29.688 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 44025 2024-04-24T15:33:29.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.688 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-04-24T15:33:29.688 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.151:6819", 2024-04-24T15:33:29.688 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 44025 2024-04-24T15:33:29.688 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.688 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:29.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.688 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addrs": { 2024-04-24T15:33:29.689 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-04-24T15:33:29.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.689 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-04-24T15:33:29.689 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.151:6822", 2024-04-24T15:33:29.689 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 44025 2024-04-24T15:33:29.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.689 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-04-24T15:33:29.689 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.151:6823", 2024-04-24T15:33:29.689 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 44025 2024-04-24T15:33:29.689 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.689 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:29.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.689 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addrs": { 2024-04-24T15:33:29.690 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-04-24T15:33:29.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.690 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-04-24T15:33:29.690 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.151:6820", 2024-04-24T15:33:29.690 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 44025 2024-04-24T15:33:29.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.690 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-04-24T15:33:29.690 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.151:6821", 2024-04-24T15:33:29.690 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 44025 2024-04-24T15:33:29.690 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.690 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:29.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.690 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addr": "172.21.15.151:6817/44025", 2024-04-24T15:33:29.691 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addr": "172.21.15.151:6819/44025", 2024-04-24T15:33:29.691 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addr": "172.21.15.151:6823/44025", 2024-04-24T15:33:29.691 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addr": "172.21.15.151:6821/44025", 2024-04-24T15:33:29.691 INFO:teuthology.orchestra.run.smithi096.stdout: "state": [ 2024-04-24T15:33:29.691 INFO:teuthology.orchestra.run.smithi096.stdout: "exists", 2024-04-24T15:33:29.691 INFO:teuthology.orchestra.run.smithi096.stdout: "up" 2024-04-24T15:33:29.691 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:29.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.691 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 5, 2024-04-24T15:33:29.691 INFO:teuthology.orchestra.run.smithi096.stdout: "uuid": "3645ef18-5574-49f8-ab81-b40651208309", 2024-04-24T15:33:29.691 INFO:teuthology.orchestra.run.smithi096.stdout: "up": 1, 2024-04-24T15:33:29.691 INFO:teuthology.orchestra.run.smithi096.stdout: "in": 0, 2024-04-24T15:33:29.692 INFO:teuthology.orchestra.run.smithi096.stdout: "weight": 0, 2024-04-24T15:33:29.692 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity": 1, 2024-04-24T15:33:29.692 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_begin": 0, 2024-04-24T15:33:29.692 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_end": 0, 2024-04-24T15:33:29.692 INFO:teuthology.orchestra.run.smithi096.stdout: "up_from": 13, 2024-04-24T15:33:29.692 INFO:teuthology.orchestra.run.smithi096.stdout: "up_thru": 38, 2024-04-24T15:33:29.692 INFO:teuthology.orchestra.run.smithi096.stdout: "down_at": 0, 2024-04-24T15:33:29.692 INFO:teuthology.orchestra.run.smithi096.stdout: "lost_at": 0, 2024-04-24T15:33:29.692 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addrs": { 2024-04-24T15:33:29.692 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-04-24T15:33:29.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.692 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-04-24T15:33:29.692 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.151:6800", 2024-04-24T15:33:29.692 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 44023 2024-04-24T15:33:29.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.693 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-04-24T15:33:29.693 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.151:6801", 2024-04-24T15:33:29.693 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 44023 2024-04-24T15:33:29.693 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.693 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:29.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.693 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addrs": { 2024-04-24T15:33:29.693 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-04-24T15:33:29.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.693 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-04-24T15:33:29.693 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.151:6802", 2024-04-24T15:33:29.693 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 44023 2024-04-24T15:33:29.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.694 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-04-24T15:33:29.694 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.151:6803", 2024-04-24T15:33:29.694 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 44023 2024-04-24T15:33:29.694 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.694 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:29.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.694 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addrs": { 2024-04-24T15:33:29.694 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-04-24T15:33:29.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.694 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-04-24T15:33:29.695 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.151:6806", 2024-04-24T15:33:29.695 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 44023 2024-04-24T15:33:29.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.695 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-04-24T15:33:29.695 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.151:6807", 2024-04-24T15:33:29.695 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 44023 2024-04-24T15:33:29.695 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.695 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:29.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.695 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addrs": { 2024-04-24T15:33:29.696 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-04-24T15:33:29.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.696 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-04-24T15:33:29.696 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.151:6804", 2024-04-24T15:33:29.696 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 44023 2024-04-24T15:33:29.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.696 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-04-24T15:33:29.696 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.151:6805", 2024-04-24T15:33:29.696 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 44023 2024-04-24T15:33:29.696 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.696 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:29.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.697 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addr": "172.21.15.151:6801/44023", 2024-04-24T15:33:29.697 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addr": "172.21.15.151:6803/44023", 2024-04-24T15:33:29.697 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addr": "172.21.15.151:6807/44023", 2024-04-24T15:33:29.697 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addr": "172.21.15.151:6805/44023", 2024-04-24T15:33:29.697 INFO:teuthology.orchestra.run.smithi096.stdout: "state": [ 2024-04-24T15:33:29.697 INFO:teuthology.orchestra.run.smithi096.stdout: "exists", 2024-04-24T15:33:29.697 INFO:teuthology.orchestra.run.smithi096.stdout: "up" 2024-04-24T15:33:29.697 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:29.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.697 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 6, 2024-04-24T15:33:29.697 INFO:teuthology.orchestra.run.smithi096.stdout: "uuid": "6b8b0b36-9246-4241-a197-c69fa48a6202", 2024-04-24T15:33:29.698 INFO:teuthology.orchestra.run.smithi096.stdout: "up": 0, 2024-04-24T15:33:29.698 INFO:teuthology.orchestra.run.smithi096.stdout: "in": 1, 2024-04-24T15:33:29.698 INFO:teuthology.orchestra.run.smithi096.stdout: "weight": 1, 2024-04-24T15:33:29.698 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity": 1, 2024-04-24T15:33:29.698 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_begin": 0, 2024-04-24T15:33:29.698 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_end": 0, 2024-04-24T15:33:29.698 INFO:teuthology.orchestra.run.smithi096.stdout: "up_from": 13, 2024-04-24T15:33:29.698 INFO:teuthology.orchestra.run.smithi096.stdout: "up_thru": 68, 2024-04-24T15:33:29.698 INFO:teuthology.orchestra.run.smithi096.stdout: "down_at": 76, 2024-04-24T15:33:29.698 INFO:teuthology.orchestra.run.smithi096.stdout: "lost_at": 0, 2024-04-24T15:33:29.698 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addrs": { 2024-04-24T15:33:29.698 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-04-24T15:33:29.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.699 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-04-24T15:33:29.699 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.151:6824", 2024-04-24T15:33:29.699 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 44057 2024-04-24T15:33:29.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.699 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-04-24T15:33:29.699 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.151:6825", 2024-04-24T15:33:29.699 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 44057 2024-04-24T15:33:29.699 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.699 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:29.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.699 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addrs": { 2024-04-24T15:33:29.699 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-04-24T15:33:29.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.700 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-04-24T15:33:29.700 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.151:6826", 2024-04-24T15:33:29.700 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 44057 2024-04-24T15:33:29.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.700 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-04-24T15:33:29.700 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.151:6827", 2024-04-24T15:33:29.700 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 44057 2024-04-24T15:33:29.700 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.700 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:29.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.700 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addrs": { 2024-04-24T15:33:29.700 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-04-24T15:33:29.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.701 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-04-24T15:33:29.701 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.151:6830", 2024-04-24T15:33:29.701 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 44057 2024-04-24T15:33:29.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.701 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-04-24T15:33:29.701 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.151:6831", 2024-04-24T15:33:29.701 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 44057 2024-04-24T15:33:29.701 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.701 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:29.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.701 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addrs": { 2024-04-24T15:33:29.701 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-04-24T15:33:29.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.702 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-04-24T15:33:29.702 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.151:6828", 2024-04-24T15:33:29.702 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 44057 2024-04-24T15:33:29.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.702 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-04-24T15:33:29.702 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.151:6829", 2024-04-24T15:33:29.702 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 44057 2024-04-24T15:33:29.702 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.702 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:29.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.702 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addr": "172.21.15.151:6825/44057", 2024-04-24T15:33:29.703 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addr": "172.21.15.151:6827/44057", 2024-04-24T15:33:29.703 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addr": "172.21.15.151:6831/44057", 2024-04-24T15:33:29.703 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addr": "172.21.15.151:6829/44057", 2024-04-24T15:33:29.703 INFO:teuthology.orchestra.run.smithi096.stdout: "state": [ 2024-04-24T15:33:29.703 INFO:teuthology.orchestra.run.smithi096.stdout: "exists" 2024-04-24T15:33:29.703 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:29.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.703 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 7, 2024-04-24T15:33:29.703 INFO:teuthology.orchestra.run.smithi096.stdout: "uuid": "4501cd9f-e437-4ddd-af3d-129fdf1ee61a", 2024-04-24T15:33:29.703 INFO:teuthology.orchestra.run.smithi096.stdout: "up": 1, 2024-04-24T15:33:29.703 INFO:teuthology.orchestra.run.smithi096.stdout: "in": 0, 2024-04-24T15:33:29.703 INFO:teuthology.orchestra.run.smithi096.stdout: "weight": 0, 2024-04-24T15:33:29.704 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity": 1, 2024-04-24T15:33:29.704 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_begin": 0, 2024-04-24T15:33:29.704 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_end": 0, 2024-04-24T15:33:29.704 INFO:teuthology.orchestra.run.smithi096.stdout: "up_from": 13, 2024-04-24T15:33:29.704 INFO:teuthology.orchestra.run.smithi096.stdout: "up_thru": 46, 2024-04-24T15:33:29.704 INFO:teuthology.orchestra.run.smithi096.stdout: "down_at": 0, 2024-04-24T15:33:29.704 INFO:teuthology.orchestra.run.smithi096.stdout: "lost_at": 0, 2024-04-24T15:33:29.704 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addrs": { 2024-04-24T15:33:29.704 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-04-24T15:33:29.704 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.704 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-04-24T15:33:29.704 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.151:6808", 2024-04-24T15:33:29.704 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 44024 2024-04-24T15:33:29.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.705 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-04-24T15:33:29.705 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.151:6809", 2024-04-24T15:33:29.705 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 44024 2024-04-24T15:33:29.705 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.705 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:29.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.705 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addrs": { 2024-04-24T15:33:29.705 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-04-24T15:33:29.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.705 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-04-24T15:33:29.705 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.151:6810", 2024-04-24T15:33:29.705 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 44024 2024-04-24T15:33:29.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.706 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-04-24T15:33:29.706 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.151:6811", 2024-04-24T15:33:29.706 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 44024 2024-04-24T15:33:29.706 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.706 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:29.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.706 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addrs": { 2024-04-24T15:33:29.706 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-04-24T15:33:29.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.706 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-04-24T15:33:29.706 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.151:6814", 2024-04-24T15:33:29.706 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 44024 2024-04-24T15:33:29.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.707 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-04-24T15:33:29.707 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.151:6815", 2024-04-24T15:33:29.707 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 44024 2024-04-24T15:33:29.707 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.707 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:29.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.707 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addrs": { 2024-04-24T15:33:29.707 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-04-24T15:33:29.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.707 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-04-24T15:33:29.707 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.151:6812", 2024-04-24T15:33:29.707 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 44024 2024-04-24T15:33:29.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.708 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-04-24T15:33:29.708 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.151:6813", 2024-04-24T15:33:29.708 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 44024 2024-04-24T15:33:29.708 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.708 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:29.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.708 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addr": "172.21.15.151:6809/44024", 2024-04-24T15:33:29.708 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addr": "172.21.15.151:6811/44024", 2024-04-24T15:33:29.708 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addr": "172.21.15.151:6815/44024", 2024-04-24T15:33:29.708 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addr": "172.21.15.151:6813/44024", 2024-04-24T15:33:29.708 INFO:teuthology.orchestra.run.smithi096.stdout: "state": [ 2024-04-24T15:33:29.708 INFO:teuthology.orchestra.run.smithi096.stdout: "exists", 2024-04-24T15:33:29.708 INFO:teuthology.orchestra.run.smithi096.stdout: "up" 2024-04-24T15:33:29.708 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:29.708 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.709 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-24T15:33:29.709 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_xinfo": [ 2024-04-24T15:33:29.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.709 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 0, 2024-04-24T15:33:29.709 INFO:teuthology.orchestra.run.smithi096.stdout: "down_stamp": "0.000000", 2024-04-24T15:33:29.709 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_probability": 0, 2024-04-24T15:33:29.709 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_interval": 0, 2024-04-24T15:33:29.709 INFO:teuthology.orchestra.run.smithi096.stdout: "features": 4540138320759226367, 2024-04-24T15:33:29.709 INFO:teuthology.orchestra.run.smithi096.stdout: "old_weight": 0, 2024-04-24T15:33:29.709 INFO:teuthology.orchestra.run.smithi096.stdout: "last_purged_snaps_scrub": "2024-04-24T15:31:56.300910+0000", 2024-04-24T15:33:29.709 INFO:teuthology.orchestra.run.smithi096.stdout: "dead_epoch": 0 2024-04-24T15:33:29.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.709 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 1, 2024-04-24T15:33:29.709 INFO:teuthology.orchestra.run.smithi096.stdout: "down_stamp": "0.000000", 2024-04-24T15:33:29.710 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_probability": 0, 2024-04-24T15:33:29.710 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_interval": 0, 2024-04-24T15:33:29.710 INFO:teuthology.orchestra.run.smithi096.stdout: "features": 4540138320759226367, 2024-04-24T15:33:29.710 INFO:teuthology.orchestra.run.smithi096.stdout: "old_weight": 65536, 2024-04-24T15:33:29.710 INFO:teuthology.orchestra.run.smithi096.stdout: "last_purged_snaps_scrub": "2024-04-24T15:31:56.202064+0000", 2024-04-24T15:33:29.710 INFO:teuthology.orchestra.run.smithi096.stdout: "dead_epoch": 0 2024-04-24T15:33:29.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.710 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 2, 2024-04-24T15:33:29.710 INFO:teuthology.orchestra.run.smithi096.stdout: "down_stamp": "0.000000", 2024-04-24T15:33:29.710 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_probability": 0, 2024-04-24T15:33:29.711 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_interval": 0, 2024-04-24T15:33:29.711 INFO:teuthology.orchestra.run.smithi096.stdout: "features": 4540138320759226367, 2024-04-24T15:33:29.711 INFO:teuthology.orchestra.run.smithi096.stdout: "old_weight": 0, 2024-04-24T15:33:29.711 INFO:teuthology.orchestra.run.smithi096.stdout: "last_purged_snaps_scrub": "2024-04-24T15:31:56.198961+0000", 2024-04-24T15:33:29.711 INFO:teuthology.orchestra.run.smithi096.stdout: "dead_epoch": 0 2024-04-24T15:33:29.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.711 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 3, 2024-04-24T15:33:29.712 INFO:teuthology.orchestra.run.smithi096.stdout: "down_stamp": "0.000000", 2024-04-24T15:33:29.712 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_probability": 0, 2024-04-24T15:33:29.712 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_interval": 0, 2024-04-24T15:33:29.712 INFO:teuthology.orchestra.run.smithi096.stdout: "features": 4540138320759226367, 2024-04-24T15:33:29.712 INFO:teuthology.orchestra.run.smithi096.stdout: "old_weight": 0, 2024-04-24T15:33:29.712 INFO:teuthology.orchestra.run.smithi096.stdout: "last_purged_snaps_scrub": "2024-04-24T15:31:56.346777+0000", 2024-04-24T15:33:29.712 INFO:teuthology.orchestra.run.smithi096.stdout: "dead_epoch": 0 2024-04-24T15:33:29.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.713 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 4, 2024-04-24T15:33:29.713 INFO:teuthology.orchestra.run.smithi096.stdout: "down_stamp": "0.000000", 2024-04-24T15:33:29.713 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_probability": 0, 2024-04-24T15:33:29.713 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_interval": 0, 2024-04-24T15:33:29.713 INFO:teuthology.orchestra.run.smithi096.stdout: "features": 4540138320759226367, 2024-04-24T15:33:29.713 INFO:teuthology.orchestra.run.smithi096.stdout: "old_weight": 0, 2024-04-24T15:33:29.713 INFO:teuthology.orchestra.run.smithi096.stdout: "last_purged_snaps_scrub": "2024-04-24T15:31:56.282585+0000", 2024-04-24T15:33:29.713 INFO:teuthology.orchestra.run.smithi096.stdout: "dead_epoch": 0 2024-04-24T15:33:29.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.714 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 5, 2024-04-24T15:33:29.714 INFO:teuthology.orchestra.run.smithi096.stdout: "down_stamp": "0.000000", 2024-04-24T15:33:29.714 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_probability": 0, 2024-04-24T15:33:29.714 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_interval": 0, 2024-04-24T15:33:29.714 INFO:teuthology.orchestra.run.smithi096.stdout: "features": 4540138320759226367, 2024-04-24T15:33:29.714 INFO:teuthology.orchestra.run.smithi096.stdout: "old_weight": 65536, 2024-04-24T15:33:29.714 INFO:teuthology.orchestra.run.smithi096.stdout: "last_purged_snaps_scrub": "2024-04-24T15:31:56.257943+0000", 2024-04-24T15:33:29.714 INFO:teuthology.orchestra.run.smithi096.stdout: "dead_epoch": 0 2024-04-24T15:33:29.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.715 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 6, 2024-04-24T15:33:29.715 INFO:teuthology.orchestra.run.smithi096.stdout: "down_stamp": "0.000000", 2024-04-24T15:33:29.715 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_probability": 0, 2024-04-24T15:33:29.715 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_interval": 0, 2024-04-24T15:33:29.715 INFO:teuthology.orchestra.run.smithi096.stdout: "features": 4540138320759226367, 2024-04-24T15:33:29.715 INFO:teuthology.orchestra.run.smithi096.stdout: "old_weight": 0, 2024-04-24T15:33:29.715 INFO:teuthology.orchestra.run.smithi096.stdout: "last_purged_snaps_scrub": "2024-04-24T15:31:56.260479+0000", 2024-04-24T15:33:29.715 INFO:teuthology.orchestra.run.smithi096.stdout: "dead_epoch": 76 2024-04-24T15:33:29.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.715 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 7, 2024-04-24T15:33:29.715 INFO:teuthology.orchestra.run.smithi096.stdout: "down_stamp": "0.000000", 2024-04-24T15:33:29.716 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_probability": 0, 2024-04-24T15:33:29.716 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_interval": 0, 2024-04-24T15:33:29.716 INFO:teuthology.orchestra.run.smithi096.stdout: "features": 4540138320759226367, 2024-04-24T15:33:29.716 INFO:teuthology.orchestra.run.smithi096.stdout: "old_weight": 65536, 2024-04-24T15:33:29.716 INFO:teuthology.orchestra.run.smithi096.stdout: "last_purged_snaps_scrub": "2024-04-24T15:31:56.242093+0000", 2024-04-24T15:33:29.716 INFO:teuthology.orchestra.run.smithi096.stdout: "dead_epoch": 0 2024-04-24T15:33:29.716 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.716 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-24T15:33:29.716 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_upmap": [], 2024-04-24T15:33:29.716 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_upmap_items": [], 2024-04-24T15:33:29.716 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_temp": [], 2024-04-24T15:33:29.716 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_temp": [], 2024-04-24T15:33:29.716 INFO:teuthology.orchestra.run.smithi096.stdout: "blocklist": {}, 2024-04-24T15:33:29.716 INFO:teuthology.orchestra.run.smithi096.stdout: "range_blocklist": {}, 2024-04-24T15:33:29.716 INFO:teuthology.orchestra.run.smithi096.stdout: "erasure_code_profiles": { 2024-04-24T15:33:29.717 INFO:teuthology.orchestra.run.smithi096.stdout: "default": { 2024-04-24T15:33:29.717 INFO:teuthology.orchestra.run.smithi096.stdout: "crush-failure-domain": "osd", 2024-04-24T15:33:29.717 INFO:teuthology.orchestra.run.smithi096.stdout: "k": "2", 2024-04-24T15:33:29.717 INFO:teuthology.orchestra.run.smithi096.stdout: "m": "1", 2024-04-24T15:33:29.717 INFO:teuthology.orchestra.run.smithi096.stdout: "plugin": "jerasure", 2024-04-24T15:33:29.717 INFO:teuthology.orchestra.run.smithi096.stdout: "technique": "reed_sol_van" 2024-04-24T15:33:29.717 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:29.717 INFO:teuthology.orchestra.run.smithi096.stdout: "removed_snaps_queue": [ 2024-04-24T15:33:29.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.717 INFO:teuthology.orchestra.run.smithi096.stdout: "pool": 4, 2024-04-24T15:33:29.717 INFO:teuthology.orchestra.run.smithi096.stdout: "snaps": [ 2024-04-24T15:33:29.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.717 INFO:teuthology.orchestra.run.smithi096.stdout: "begin": 36, 2024-04-24T15:33:29.717 INFO:teuthology.orchestra.run.smithi096.stdout: "length": 1 2024-04-24T15:33:29.717 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.718 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:29.718 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.718 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-24T15:33:29.718 INFO:teuthology.orchestra.run.smithi096.stdout: "new_removed_snaps": [ 2024-04-24T15:33:29.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.718 INFO:teuthology.orchestra.run.smithi096.stdout: "pool": 4, 2024-04-24T15:33:29.718 INFO:teuthology.orchestra.run.smithi096.stdout: "snaps": [ 2024-04-24T15:33:29.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:29.718 INFO:teuthology.orchestra.run.smithi096.stdout: "begin": 36, 2024-04-24T15:33:29.718 INFO:teuthology.orchestra.run.smithi096.stdout: "length": 1 2024-04-24T15:33:29.718 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.718 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:29.718 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.719 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-24T15:33:29.719 INFO:teuthology.orchestra.run.smithi096.stdout: "new_purged_snaps": [], 2024-04-24T15:33:29.719 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_node_flags": {}, 2024-04-24T15:33:29.719 INFO:teuthology.orchestra.run.smithi096.stdout: "device_class_flags": {}, 2024-04-24T15:33:29.719 INFO:teuthology.orchestra.run.smithi096.stdout: "stretch_mode": { 2024-04-24T15:33:29.719 INFO:teuthology.orchestra.run.smithi096.stdout: "stretch_mode_enabled": false, 2024-04-24T15:33:29.719 INFO:teuthology.orchestra.run.smithi096.stdout: "stretch_bucket_count": 0, 2024-04-24T15:33:29.719 INFO:teuthology.orchestra.run.smithi096.stdout: "degraded_stretch_mode": 0, 2024-04-24T15:33:29.719 INFO:teuthology.orchestra.run.smithi096.stdout: "recovering_stretch_mode": 0, 2024-04-24T15:33:29.719 INFO:teuthology.orchestra.run.smithi096.stdout: "stretch_mode_bucket": 0 2024-04-24T15:33:29.719 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:29.719 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:29.722 INFO:tasks.thrashosds.thrasher:j is {'epoch': 91, 'fsid': '253d426c-62cb-4194-8514-6cd970fd85c7', 'created': '2024-04-24T15:31:47.870327+0000', 'modified': '2024-04-24T15:33:28.894249+0000', 'last_up_change': '2024-04-24T15:33:09.796895+0000', 'last_in_change': '2024-04-24T15:32:58.722778+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 4, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'quincy', 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T15:31:59.589762+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-04-24T15:31:59.971601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '21', 'last_force_op_resend_prenautilus': '21', 'last_force_op_resend_preluminous': '21', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 19, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [3], 'tier_of': -1, 'read_tier': 3, 'write_tier': 3, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cache', 'create_time': '2024-04-24T15:32:06.914463+0000', 'flags': 9, 'flags_names': 'hashpspool,incomplete_clones', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 4, 'pg_placement_num': 4, 'pg_placement_num_target': 4, 'pg_num_target': 4, 'pg_num_pending': 4, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '21', 'last_force_op_resend_prenautilus': '21', 'last_force_op_resend_preluminous': '21', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 19, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': 2, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'writeback', 'target_max_bytes': 0, 'target_max_objects': 250, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'bloom', 'false_positive_probability': 0.05, 'target_size': 0, 'seed': 0}, 'hit_set_period': 60, 'hit_set_count': 8, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [1000000, 1000000, 1000000, 1000000, 1000000, 1000000, 1000000, 1000000], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 4, 'pool_name': 'pool_client.0', 'create_time': '2024-04-24T15:32:15.929114+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 37, 'snap_epoch': 91, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}], 'osds': [{'osd': 0, 'uuid': '92941fb9-96ee-4b2b-8147-3c1d61192f94', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 76, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6816', 'nonce': 44106}, {'type': 'v1', 'addr': '172.21.15.96:6817', 'nonce': 44106}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6818', 'nonce': 44106}, {'type': 'v1', 'addr': '172.21.15.96:6819', 'nonce': 44106}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6822', 'nonce': 44106}, {'type': 'v1', 'addr': '172.21.15.96:6823', 'nonce': 44106}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6820', 'nonce': 44106}, {'type': 'v1', 'addr': '172.21.15.96:6821', 'nonce': 44106}]}, 'public_addr': '172.21.15.96:6817/44106', 'cluster_addr': '172.21.15.96:6819/44106', 'heartbeat_back_addr': '172.21.15.96:6823/44106', 'heartbeat_front_addr': '172.21.15.96:6821/44106', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '628e868a-9884-42e6-9331-caef97cb6967', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 26, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6800', 'nonce': 44089}, {'type': 'v1', 'addr': '172.21.15.96:6801', 'nonce': 44089}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6802', 'nonce': 44089}, {'type': 'v1', 'addr': '172.21.15.96:6803', 'nonce': 44089}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6806', 'nonce': 44089}, {'type': 'v1', 'addr': '172.21.15.96:6807', 'nonce': 44089}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6804', 'nonce': 44089}, {'type': 'v1', 'addr': '172.21.15.96:6805', 'nonce': 44089}]}, 'public_addr': '172.21.15.96:6801/44089', 'cluster_addr': '172.21.15.96:6803/44089', 'heartbeat_back_addr': '172.21.15.96:6807/44089', 'heartbeat_front_addr': '172.21.15.96:6805/44089', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '46de3f65-87a8-460c-b57a-4468d9c89a57', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.3755035400390625, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 76, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6808', 'nonce': 44090}, {'type': 'v1', 'addr': '172.21.15.96:6809', 'nonce': 44090}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6810', 'nonce': 44090}, {'type': 'v1', 'addr': '172.21.15.96:6811', 'nonce': 44090}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6814', 'nonce': 44090}, {'type': 'v1', 'addr': '172.21.15.96:6815', 'nonce': 44090}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6812', 'nonce': 44090}, {'type': 'v1', 'addr': '172.21.15.96:6813', 'nonce': 44090}]}, 'public_addr': '172.21.15.96:6809/44090', 'cluster_addr': '172.21.15.96:6811/44090', 'heartbeat_back_addr': '172.21.15.96:6815/44090', 'heartbeat_front_addr': '172.21.15.96:6813/44090', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '5507e063-10bc-4e32-b1ec-00aa3e47d993', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 76, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6824', 'nonce': 44121}, {'type': 'v1', 'addr': '172.21.15.96:6825', 'nonce': 44121}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6826', 'nonce': 44121}, {'type': 'v1', 'addr': '172.21.15.96:6827', 'nonce': 44121}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6830', 'nonce': 44121}, {'type': 'v1', 'addr': '172.21.15.96:6831', 'nonce': 44121}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6828', 'nonce': 44121}, {'type': 'v1', 'addr': '172.21.15.96:6829', 'nonce': 44121}]}, 'public_addr': '172.21.15.96:6825/44121', 'cluster_addr': '172.21.15.96:6827/44121', 'heartbeat_back_addr': '172.21.15.96:6831/44121', 'heartbeat_front_addr': '172.21.15.96:6829/44121', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '6a74d453-1048-44ef-a5b0-4277583d76c2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 76, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6816', 'nonce': 44025}, {'type': 'v1', 'addr': '172.21.15.151:6817', 'nonce': 44025}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6818', 'nonce': 44025}, {'type': 'v1', 'addr': '172.21.15.151:6819', 'nonce': 44025}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6822', 'nonce': 44025}, {'type': 'v1', 'addr': '172.21.15.151:6823', 'nonce': 44025}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6820', 'nonce': 44025}, {'type': 'v1', 'addr': '172.21.15.151:6821', 'nonce': 44025}]}, 'public_addr': '172.21.15.151:6817/44025', 'cluster_addr': '172.21.15.151:6819/44025', 'heartbeat_back_addr': '172.21.15.151:6823/44025', 'heartbeat_front_addr': '172.21.15.151:6821/44025', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '3645ef18-5574-49f8-ab81-b40651208309', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 38, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6800', 'nonce': 44023}, {'type': 'v1', 'addr': '172.21.15.151:6801', 'nonce': 44023}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6802', 'nonce': 44023}, {'type': 'v1', 'addr': '172.21.15.151:6803', 'nonce': 44023}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6806', 'nonce': 44023}, {'type': 'v1', 'addr': '172.21.15.151:6807', 'nonce': 44023}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6804', 'nonce': 44023}, {'type': 'v1', 'addr': '172.21.15.151:6805', 'nonce': 44023}]}, 'public_addr': '172.21.15.151:6801/44023', 'cluster_addr': '172.21.15.151:6803/44023', 'heartbeat_back_addr': '172.21.15.151:6807/44023', 'heartbeat_front_addr': '172.21.15.151:6805/44023', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '6b8b0b36-9246-4241-a197-c69fa48a6202', 'up': 0, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 68, 'down_at': 76, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6824', 'nonce': 44057}, {'type': 'v1', 'addr': '172.21.15.151:6825', 'nonce': 44057}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6826', 'nonce': 44057}, {'type': 'v1', 'addr': '172.21.15.151:6827', 'nonce': 44057}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6830', 'nonce': 44057}, {'type': 'v1', 'addr': '172.21.15.151:6831', 'nonce': 44057}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6828', 'nonce': 44057}, {'type': 'v1', 'addr': '172.21.15.151:6829', 'nonce': 44057}]}, 'public_addr': '172.21.15.151:6825/44057', 'cluster_addr': '172.21.15.151:6827/44057', 'heartbeat_back_addr': '172.21.15.151:6831/44057', 'heartbeat_front_addr': '172.21.15.151:6829/44057', 'state': ['exists']}, {'osd': 7, 'uuid': '4501cd9f-e437-4ddd-af3d-129fdf1ee61a', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 46, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6808', 'nonce': 44024}, {'type': 'v1', 'addr': '172.21.15.151:6809', 'nonce': 44024}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6810', 'nonce': 44024}, {'type': 'v1', 'addr': '172.21.15.151:6811', 'nonce': 44024}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6814', 'nonce': 44024}, {'type': 'v1', 'addr': '172.21.15.151:6815', 'nonce': 44024}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6812', 'nonce': 44024}, {'type': 'v1', 'addr': '172.21.15.151:6813', 'nonce': 44024}]}, 'public_addr': '172.21.15.151:6809/44024', 'cluster_addr': '172.21.15.151:6811/44024', 'heartbeat_back_addr': '172.21.15.151:6815/44024', 'heartbeat_front_addr': '172.21.15.151:6813/44024', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-24T15:31:56.300910+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-04-24T15:31:56.202064+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-24T15:31:56.198961+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-24T15:31:56.346777+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-24T15:31:56.282585+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-04-24T15:31:56.257943+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-24T15:31:56.260479+0000', 'dead_epoch': 76}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-04-24T15:31:56.242093+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 4, 'snaps': [{'begin': 36, 'length': 1}]}], 'new_removed_snaps': [{'pool': 4, 'snaps': [{'begin': 36, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-04-24T15:33:29.722 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T15:33:29.722 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:33:29.721+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:33:29.744 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:29.744 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-04-24T15:33:29.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-04-24T15:33:29.744 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [] 2024-04-24T15:33:29.745 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:29.755 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-24T15:33:29.839 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:33:29.839+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:33:29.921 INFO:teuthology.orchestra.run.smithi096.stderr:noscrub is set 2024-04-24T15:33:29.922 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:29.922+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:29.945 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:29.945 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:33:29.945 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-04-24T15:33:29.945 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:29.956 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-24T15:33:30.022 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:30.023+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:30.106 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:33:30.107 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-04-24T15:33:30.119 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":131,"stamp":"2024-04-24T15:33:29.906606+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1491180203,"num_objects":1107,"num_object_clones":308,"num_object_copies":2214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1104,"num_whiteouts":285,"num_read":106667,"num_read_kb":720583,"num_write":18826,"num_write_kb":358339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1803,"num_bytes_recovered":56550993,"num_keys_recovered":162,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14986,"ondisk_log_size":14986,"up":72,"acting":72,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":471859200,"kb_used":1562468,"kb_used_data":55116,"kb_used_omap":8,"kb_used_meta":1507191,"kb_avail":470296732,"statfs":{"total":483183820800,"available":481583853568,"internally_reserved":0,"allocated":56438784,"data_stored":549440651,"data_compressed":20898200,"data_compressed_allocated":43016192,"data_compressed_original":542225335,"omap_allocated":8520,"internal_metadata":1543364280},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":111757552,"num_objects":105,"num_object_clones":39,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":47,"num_read":7785,"num_read_kb":52779,"num_write":944,"num_write_kb":32650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.339697"},"pg_stats":[{"pgid":"4.1f","version":"91'1419","reported_seq":2009,"reported_epoch":91,"state":"active+undersized+degraded","last_fresh":"2024-04-24T15:33:28.908021+0000","last_change":"2024-04-24T15:33:10.808767+0000","last_active":"2024-04-24T15:33:28.908021+0000","last_peered":"2024-04-24T15:33:28.908021+0000","last_clean":"2024-04-24T15:33:09.532714+0000","last_became_active":"2024-04-24T15:33:10.808767+0000","last_became_peered":"2024-04-24T15:33:10.808767+0000","last_unstale":"2024-04-24T15:33:28.908021+0000","last_undegraded":"2024-04-24T15:33:10.808143+0000","last_fullsized":"2024-04-24T15:33:10.808068+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":1419,"log_dups_size":0,"ondisk_log_size":1419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:52:13.964900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000483979,"stat_sum":{"num_bytes":26732043,"num_objects":25,"num_object_clones":7,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":6,"num_read":7832,"num_read_kb":14462,"num_write":2524,"num_write_kb":16062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":3834120,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":25}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1e","version":"89'433","reported_seq":6037,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T15:33:28.915524+0000","last_change":"2024-04-24T15:33:28.915897+0000","last_active":"2024-04-24T15:33:28.915524+0000","last_peered":"2024-04-24T15:33:28.915524+0000","last_clean":"2024-04-24T15:33:28.915524+0000","last_became_active":"2024-04-24T15:32:50.654760+0000","last_became_peered":"2024-04-24T15:32:50.654760+0000","last_unstale":"2024-04-24T15:33:28.915524+0000","last_undegraded":"2024-04-24T15:33:28.915524+0000","last_fullsized":"2024-04-24T15:33:28.915524+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:50:53.605739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0076013269999999997,"stat_sum":{"num_bytes":69111898,"num_objects":31,"num_object_clones":7,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":7,"num_read":6945,"num_read_kb":38833,"num_write":503,"num_write_kb":6578,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":2247966,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"24","length":"1"}]},{"pgid":"4.1d","version":"91'437","reported_seq":3107,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T15:33:28.914813+0000","last_change":"2024-04-24T15:33:28.915071+0000","last_active":"2024-04-24T15:33:28.914813+0000","last_peered":"2024-04-24T15:33:28.914813+0000","last_clean":"2024-04-24T15:33:28.914813+0000","last_became_active":"2024-04-24T15:33:03.637986+0000","last_became_peered":"2024-04-24T15:33:03.637986+0000","last_unstale":"2024-04-24T15:33:28.914813+0000","last_undegraded":"2024-04-24T15:33:28.914813+0000","last_fullsized":"2024-04-24T15:33:28.914813+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":437,"log_dups_size":0,"ondisk_log_size":437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:32:29.354937+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0073037290000000001,"stat_sum":{"num_bytes":58051145,"num_objects":38,"num_object_clones":9,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":8,"num_read":4054,"num_read_kb":51576,"num_write":538,"num_write_kb":18743,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":580671,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"24","length":"1"}]},{"pgid":"4.1c","version":"90'416","reported_seq":1957,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T15:33:28.908361+0000","last_change":"2024-04-24T15:33:28.908361+0000","last_active":"2024-04-24T15:33:28.908361+0000","last_peered":"2024-04-24T15:33:28.908361+0000","last_clean":"2024-04-24T15:33:28.908361+0000","last_became_active":"2024-04-24T15:32:42.207673+0000","last_became_peered":"2024-04-24T15:32:42.207673+0000","last_unstale":"2024-04-24T15:33:28.908361+0000","last_undegraded":"2024-04-24T15:33:28.908361+0000","last_fullsized":"2024-04-24T15:33:28.908361+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":416,"log_dups_size":0,"ondisk_log_size":416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:35:35.261035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065481700000000003,"stat_sum":{"num_bytes":17385062,"num_objects":26,"num_object_clones":4,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":5,"num_read":3146,"num_read_kb":9561,"num_write":420,"num_write_kb":2871,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"24","length":"1"}]},{"pgid":"4.1b","version":"90'326","reported_seq":2049,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T15:33:28.904706+0000","last_change":"2024-04-24T15:33:28.905320+0000","last_active":"2024-04-24T15:33:28.904706+0000","last_peered":"2024-04-24T15:33:28.904706+0000","last_clean":"2024-04-24T15:33:28.904706+0000","last_became_active":"2024-04-24T15:33:02.638431+0000","last_became_peered":"2024-04-24T15:33:02.638431+0000","last_unstale":"2024-04-24T15:33:28.904706+0000","last_undegraded":"2024-04-24T15:33:28.904706+0000","last_fullsized":"2024-04-24T15:33:28.904706+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":326,"log_dups_size":0,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:44:40.561360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015312279999999999,"stat_sum":{"num_bytes":71028081,"num_objects":37,"num_object_clones":14,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":13,"num_read":2163,"num_read_kb":25256,"num_write":303,"num_write_kb":6112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":79,"num_bytes_recovered":2143880,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"24","length":"1"}]},{"pgid":"4.1a","version":"90'406","reported_seq":6229,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T15:33:28.903976+0000","last_change":"2024-04-24T15:33:28.903976+0000","last_active":"2024-04-24T15:33:28.903976+0000","last_peered":"2024-04-24T15:33:28.903976+0000","last_clean":"2024-04-24T15:33:28.903976+0000","last_became_active":"2024-04-24T15:33:02.638273+0000","last_became_peered":"2024-04-24T15:33:02.638273+0000","last_unstale":"2024-04-24T15:33:28.903976+0000","last_undegraded":"2024-04-24T15:33:28.903976+0000","last_fullsized":"2024-04-24T15:33:28.903976+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":406,"log_dups_size":0,"ondisk_log_size":406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:35:38.376100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045967099999999998,"stat_sum":{"num_bytes":46054910,"num_objects":33,"num_object_clones":11,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":11,"num_read":6159,"num_read_kb":19607,"num_write":453,"num_write_kb":10189,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":89,"num_bytes_recovered":1137673,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"24","length":"1"}]},{"pgid":"4.19","version":"90'455","reported_seq":2247,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T15:33:28.906728+0000","last_change":"2024-04-24T15:33:28.906979+0000","last_active":"2024-04-24T15:33:28.906728+0000","last_peered":"2024-04-24T15:33:28.906728+0000","last_clean":"2024-04-24T15:33:28.906728+0000","last_became_active":"2024-04-24T15:32:17.047842+0000","last_became_peered":"2024-04-24T15:32:17.047842+0000","last_unstale":"2024-04-24T15:33:28.906728+0000","last_undegraded":"2024-04-24T15:33:28.906728+0000","last_fullsized":"2024-04-24T15:33:28.906728+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:09:52.065772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037732599999999999,"stat_sum":{"num_bytes":52348551,"num_objects":38,"num_object_clones":10,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":9,"num_read":3730,"num_read_kb":22365,"num_write":476,"num_write_kb":11181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"24","length":"1"}]},{"pgid":"4.18","version":"88'398","reported_seq":1677,"reported_epoch":90,"state":"active+undersized+degraded","last_fresh":"2024-04-24T15:33:27.896066+0000","last_change":"2024-04-24T15:33:10.809148+0000","last_active":"2024-04-24T15:33:27.896066+0000","last_peered":"2024-04-24T15:33:27.896066+0000","last_clean":"2024-04-24T15:33:09.476266+0000","last_became_active":"2024-04-24T15:33:10.809148+0000","last_became_peered":"2024-04-24T15:33:10.809148+0000","last_unstale":"2024-04-24T15:33:27.896066+0000","last_undegraded":"2024-04-24T15:33:10.808507+0000","last_fullsized":"2024-04-24T15:33:10.808416+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":398,"log_dups_size":0,"ondisk_log_size":398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:57:39.383168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033230299999999999,"stat_sum":{"num_bytes":23299660,"num_objects":39,"num_object_clones":10,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":12,"num_read":1863,"num_read_kb":15390,"num_write":522,"num_write_kb":8155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":92,"num_bytes_recovered":1436100,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[{"shards":"4","objects":39}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"90'427","reported_seq":1788,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T15:33:28.915136+0000","last_change":"2024-04-24T15:33:28.915430+0000","last_active":"2024-04-24T15:33:28.915136+0000","last_peered":"2024-04-24T15:33:28.915136+0000","last_clean":"2024-04-24T15:33:28.915136+0000","last_became_active":"2024-04-24T15:32:50.657240+0000","last_became_peered":"2024-04-24T15:32:50.657240+0000","last_unstale":"2024-04-24T15:33:28.915136+0000","last_undegraded":"2024-04-24T15:33:28.915136+0000","last_fullsized":"2024-04-24T15:33:28.915136+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":427,"log_dups_size":0,"ondisk_log_size":427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:48:26.843070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0075889399999999997,"stat_sum":{"num_bytes":16270400,"num_objects":33,"num_object_clones":5,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":6,"num_read":2871,"num_read_kb":15851,"num_write":514,"num_write_kb":12731,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":45056,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"24","length":"1"}]},{"pgid":"4.1","version":"90'280","reported_seq":2127,"reported_epoch":90,"state":"active+undersized+degraded","last_fresh":"2024-04-24T15:33:28.586832+0000","last_change":"2024-04-24T15:33:10.810673+0000","last_active":"2024-04-24T15:33:28.586832+0000","last_peered":"2024-04-24T15:33:28.586832+0000","last_clean":"2024-04-24T15:33:09.555555+0000","last_became_active":"2024-04-24T15:33:10.810673+0000","last_became_peered":"2024-04-24T15:33:10.810673+0000","last_unstale":"2024-04-24T15:33:28.586832+0000","last_undegraded":"2024-04-24T15:33:10.808669+0000","last_fullsized":"2024-04-24T15:33:10.808594+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":280,"log_dups_size":0,"ondisk_log_size":280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:19:13.802747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000468851,"stat_sum":{"num_bytes":65192187,"num_objects":34,"num_object_clones":10,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":8,"num_read":1629,"num_read_kb":26734,"num_write":287,"num_write_kb":13822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":6125156,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[{"shards":"4","objects":34}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":70,"reported_epoch":90,"state":"active+undersized","last_fresh":"2024-04-24T15:33:27.896151+0000","last_change":"2024-04-24T15:33:10.813024+0000","last_active":"2024-04-24T15:33:27.896151+0000","last_peered":"2024-04-24T15:33:27.896151+0000","last_clean":"2024-04-24T15:32:45.222047+0000","last_became_active":"2024-04-24T15:33:10.813024+0000","last_became_peered":"2024-04-24T15:33:10.813024+0000","last_unstale":"2024-04-24T15:33:27.896151+0000","last_undegraded":"2024-04-24T15:33:27.896151+0000","last_fullsized":"2024-04-24T15:33:10.812398+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:11:02.813956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"90'776","reported_seq":2602,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T15:33:28.906682+0000","last_change":"2024-04-24T15:33:28.907043+0000","last_active":"2024-04-24T15:33:28.906682+0000","last_peered":"2024-04-24T15:33:28.906682+0000","last_clean":"2024-04-24T15:33:28.906682+0000","last_became_active":"2024-04-24T15:32:50.654697+0000","last_became_peered":"2024-04-24T15:32:50.654697+0000","last_unstale":"2024-04-24T15:33:28.906682+0000","last_undegraded":"2024-04-24T15:33:28.906682+0000","last_fullsized":"2024-04-24T15:33:28.906682+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":776,"log_dups_size":0,"ondisk_log_size":776,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:15:13.259423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045239400000000001,"stat_sum":{"num_bytes":54756218,"num_objects":54,"num_object_clones":15,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":16,"num_read":4333,"num_read_kb":23570,"num_write":993,"num_write_kb":13082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":2375192,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"24","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":83,"reported_epoch":90,"state":"active+undersized","last_fresh":"2024-04-24T15:33:27.896522+0000","last_change":"2024-04-24T15:33:10.810646+0000","last_active":"2024-04-24T15:33:27.896522+0000","last_peered":"2024-04-24T15:33:27.896522+0000","last_clean":"2024-04-24T15:32:57.724923+0000","last_became_active":"2024-04-24T15:33:10.810646+0000","last_became_peered":"2024-04-24T15:33:10.810646+0000","last_unstale":"2024-04-24T15:33:27.896522+0000","last_undegraded":"2024-04-24T15:33:27.896522+0000","last_fullsized":"2024-04-24T15:33:10.808606+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:49:34.555065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T15:33:27.896354+0000","last_change":"2024-04-24T15:32:59.750351+0000","last_active":"2024-04-24T15:33:27.896354+0000","last_peered":"2024-04-24T15:33:27.896354+0000","last_clean":"2024-04-24T15:33:27.896354+0000","last_became_active":"2024-04-24T15:32:59.750060+0000","last_became_peered":"2024-04-24T15:32:59.750060+0000","last_unstale":"2024-04-24T15:33:27.896354+0000","last_undegraded":"2024-04-24T15:33:27.896354+0000","last_fullsized":"2024-04-24T15:33:27.896354+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:30:23.191594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"90'402","reported_seq":4434,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T15:33:28.904295+0000","last_change":"2024-04-24T15:33:28.904668+0000","last_active":"2024-04-24T15:33:28.904295+0000","last_peered":"2024-04-24T15:33:28.904295+0000","last_clean":"2024-04-24T15:33:28.904295+0000","last_became_active":"2024-04-24T15:32:50.651725+0000","last_became_peered":"2024-04-24T15:32:50.651725+0000","last_unstale":"2024-04-24T15:33:28.904295+0000","last_undegraded":"2024-04-24T15:33:28.904295+0000","last_fullsized":"2024-04-24T15:33:28.904295+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":402,"log_dups_size":0,"ondisk_log_size":402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:34:39.675295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061216799999999996,"stat_sum":{"num_bytes":13989939,"num_objects":18,"num_object_clones":4,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":3,"num_read":3188,"num_read_kb":14526,"num_write":427,"num_write_kb":11249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":11799,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"24","length":"1"}]},{"pgid":"3.1","version":"60'1","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T15:33:27.896450+0000","last_change":"2024-04-24T15:32:59.752573+0000","last_active":"2024-04-24T15:33:27.896450+0000","last_peered":"2024-04-24T15:33:27.896450+0000","last_clean":"2024-04-24T15:33:27.896450+0000","last_became_active":"2024-04-24T15:32:59.750157+0000","last_became_peered":"2024-04-24T15:32:59.750157+0000","last_unstale":"2024-04-24T15:33:27.896450+0000","last_undegraded":"2024-04-24T15:33:27.896450+0000","last_fullsized":"2024-04-24T15:33:27.896450+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_clean_scrub_stamp":"2024-04-24T15:32:06.972928+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:30:52.191826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T15:33:27.909796+0000","last_change":"2024-04-24T15:32:47.635254+0000","last_active":"2024-04-24T15:33:27.909796+0000","last_peered":"2024-04-24T15:33:27.909796+0000","last_clean":"2024-04-24T15:33:27.909796+0000","last_became_active":"2024-04-24T15:32:47.634970+0000","last_became_peered":"2024-04-24T15:32:47.634970+0000","last_unstale":"2024-04-24T15:33:27.909796+0000","last_undegraded":"2024-04-24T15:33:27.909796+0000","last_fullsized":"2024-04-24T15:33:27.909796+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:23:36.303998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"90'507","reported_seq":1793,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T15:33:28.905142+0000","last_change":"2024-04-24T15:33:28.905442+0000","last_active":"2024-04-24T15:33:28.905142+0000","last_peered":"2024-04-24T15:33:28.905142+0000","last_clean":"2024-04-24T15:33:28.905142+0000","last_became_active":"2024-04-24T15:32:24.073408+0000","last_became_peered":"2024-04-24T15:32:24.073408+0000","last_unstale":"2024-04-24T15:33:28.905142+0000","last_undegraded":"2024-04-24T15:33:28.905142+0000","last_fullsized":"2024-04-24T15:33:28.905142+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":507,"log_dups_size":0,"ondisk_log_size":507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:04:54.522860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071767700000000001,"stat_sum":{"num_bytes":48258261,"num_objects":29,"num_object_clones":8,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":6,"num_read":2747,"num_read_kb":19598,"num_write":641,"num_write_kb":5098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"24","length":"1"}]},{"pgid":"3.2","version":"90'2","reported_seq":96,"reported_epoch":90,"state":"active+undersized+degraded","last_fresh":"2024-04-24T15:33:28.645363+0000","last_change":"2024-04-24T15:33:10.810542+0000","last_active":"2024-04-24T15:33:28.645363+0000","last_peered":"2024-04-24T15:33:28.645363+0000","last_clean":"2024-04-24T15:33:08.799835+0000","last_became_active":"2024-04-24T15:33:10.810542+0000","last_became_peered":"2024-04-24T15:33:10.810542+0000","last_unstale":"2024-04-24T15:33:28.645363+0000","last_undegraded":"2024-04-24T15:33:10.809321+0000","last_fullsized":"2024-04-24T15:33:10.809248+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_clean_scrub_stamp":"2024-04-24T15:32:06.972928+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:18:06.113145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[{"shards":"3","objects":2}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":195,"reported_epoch":90,"state":"active+undersized+degraded","last_fresh":"2024-04-24T15:33:27.899292+0000","last_change":"2024-04-24T15:33:10.808473+0000","last_active":"2024-04-24T15:33:27.899292+0000","last_peered":"2024-04-24T15:33:27.899292+0000","last_clean":"2024-04-24T15:32:01.949368+0000","last_became_active":"2024-04-24T15:33:10.808473+0000","last_became_peered":"2024-04-24T15:33:10.808473+0000","last_unstale":"2024-04-24T15:33:27.899292+0000","last_undegraded":"2024-04-24T15:33:10.808028+0000","last_fullsized":"2024-04-24T15:33:10.807933+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:31:59.935287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:31:59.935287+0000","last_clean_scrub_stamp":"2024-04-24T15:31:59.935287+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:03:14.127621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":68,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T15:33:27.895836+0000","last_change":"2024-04-24T15:32:47.639623+0000","last_active":"2024-04-24T15:33:27.895836+0000","last_peered":"2024-04-24T15:33:27.895836+0000","last_clean":"2024-04-24T15:33:27.895836+0000","last_became_active":"2024-04-24T15:32:47.639356+0000","last_became_peered":"2024-04-24T15:32:47.639356+0000","last_unstale":"2024-04-24T15:33:27.895836+0000","last_undegraded":"2024-04-24T15:33:27.895836+0000","last_fullsized":"2024-04-24T15:33:27.895836+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:10:46.402999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"88'524","reported_seq":2098,"reported_epoch":90,"state":"active+undersized+degraded","last_fresh":"2024-04-24T15:33:27.896246+0000","last_change":"2024-04-24T15:33:10.810666+0000","last_active":"2024-04-24T15:33:27.896246+0000","last_peered":"2024-04-24T15:33:27.896246+0000","last_clean":"2024-04-24T15:33:09.203505+0000","last_became_active":"2024-04-24T15:33:10.810666+0000","last_became_peered":"2024-04-24T15:33:10.810666+0000","last_unstale":"2024-04-24T15:33:27.896246+0000","last_undegraded":"2024-04-24T15:33:10.808818+0000","last_fullsized":"2024-04-24T15:33:10.808742+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":524,"log_dups_size":0,"ondisk_log_size":524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:01:01.063245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032205999999999999,"stat_sum":{"num_bytes":38173650,"num_objects":35,"num_object_clones":7,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":7,"num_read":3144,"num_read_kb":26032,"num_write":741,"num_write_kb":11805,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":6723939,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[{"shards":"4","objects":35}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":54,"reported_epoch":90,"state":"active+undersized","last_fresh":"2024-04-24T15:33:27.895782+0000","last_change":"2024-04-24T15:33:10.812792+0000","last_active":"2024-04-24T15:33:27.895782+0000","last_peered":"2024-04-24T15:33:27.895782+0000","last_clean":"2024-04-24T15:32:06.972928+0000","last_became_active":"2024-04-24T15:33:10.812792+0000","last_became_peered":"2024-04-24T15:33:10.812792+0000","last_unstale":"2024-04-24T15:33:27.895782+0000","last_undegraded":"2024-04-24T15:33:27.895782+0000","last_fullsized":"2024-04-24T15:33:10.812281+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_clean_scrub_stamp":"2024-04-24T15:32:06.972928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:24:28.419008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T15:33:27.909922+0000","last_change":"2024-04-24T15:32:42.207668+0000","last_active":"2024-04-24T15:33:27.909922+0000","last_peered":"2024-04-24T15:33:27.909922+0000","last_clean":"2024-04-24T15:33:27.909922+0000","last_became_active":"2024-04-24T15:32:42.207408+0000","last_became_peered":"2024-04-24T15:32:42.207408+0000","last_unstale":"2024-04-24T15:33:27.909922+0000","last_undegraded":"2024-04-24T15:33:27.909922+0000","last_fullsized":"2024-04-24T15:33:27.909922+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:03:09.182470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"90'356","reported_seq":868,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T15:33:28.920363+0000","last_change":"2024-04-24T15:33:28.920536+0000","last_active":"2024-04-24T15:33:28.920363+0000","last_peered":"2024-04-24T15:33:28.920363+0000","last_clean":"2024-04-24T15:33:28.920363+0000","last_became_active":"2024-04-24T15:32:47.635898+0000","last_became_peered":"2024-04-24T15:32:47.635898+0000","last_unstale":"2024-04-24T15:33:28.920363+0000","last_undegraded":"2024-04-24T15:33:28.920363+0000","last_fullsized":"2024-04-24T15:33:28.920363+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":356,"log_dups_size":0,"ondisk_log_size":356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:23:43.925824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011981073,"stat_sum":{"num_bytes":38370015,"num_objects":34,"num_object_clones":9,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":12,"num_read":1717,"num_read_kb":12495,"num_write":409,"num_write_kb":8054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":67213,"num_keys_recovered":18,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"24","length":"1"}]},{"pgid":"3.3","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T15:33:27.896808+0000","last_change":"2024-04-24T15:32:59.750425+0000","last_active":"2024-04-24T15:33:27.896808+0000","last_peered":"2024-04-24T15:33:27.896808+0000","last_clean":"2024-04-24T15:33:27.896808+0000","last_became_active":"2024-04-24T15:32:59.750038+0000","last_became_peered":"2024-04-24T15:32:59.750038+0000","last_unstale":"2024-04-24T15:33:27.896808+0000","last_undegraded":"2024-04-24T15:33:27.896808+0000","last_fullsized":"2024-04-24T15:33:27.896808+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_clean_scrub_stamp":"2024-04-24T15:32:06.972928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:48:12.195815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":45,"reported_epoch":90,"state":"active+undersized","last_fresh":"2024-04-24T15:33:27.899351+0000","last_change":"2024-04-24T15:33:10.809707+0000","last_active":"2024-04-24T15:33:27.899351+0000","last_peered":"2024-04-24T15:33:27.899351+0000","last_clean":"2024-04-24T15:32:16.037929+0000","last_became_active":"2024-04-24T15:33:10.809707+0000","last_became_peered":"2024-04-24T15:33:10.809707+0000","last_unstale":"2024-04-24T15:33:27.899351+0000","last_undegraded":"2024-04-24T15:33:27.899351+0000","last_fullsized":"2024-04-24T15:33:10.808259+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:32:38.810895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0],"avail_no_missing":["0"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"89'369","reported_seq":1550,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T15:33:28.905209+0000","last_change":"2024-04-24T15:33:28.905479+0000","last_active":"2024-04-24T15:33:28.905209+0000","last_peered":"2024-04-24T15:33:28.905209+0000","last_clean":"2024-04-24T15:33:28.905209+0000","last_became_active":"2024-04-24T15:32:18.044036+0000","last_became_peered":"2024-04-24T15:32:18.044036+0000","last_unstale":"2024-04-24T15:33:28.905209+0000","last_undegraded":"2024-04-24T15:33:28.905209+0000","last_fullsized":"2024-04-24T15:33:28.905209+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:48:33.464105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040112200000000001,"stat_sum":{"num_bytes":56338715,"num_objects":43,"num_object_clones":14,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":12,"num_read":1757,"num_read_kb":23706,"num_write":374,"num_write_kb":14888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"24","length":"1"}]},{"pgid":"4.2","version":"91'321","reported_seq":1547,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T15:33:28.925244+0000","last_change":"2024-04-24T15:33:28.904334+0000","last_active":"2024-04-24T15:33:28.925244+0000","last_peered":"2024-04-24T15:33:28.925244+0000","last_clean":"2024-04-24T15:33:28.925244+0000","last_became_active":"2024-04-24T15:33:02.634103+0000","last_became_peered":"2024-04-24T15:33:02.634103+0000","last_unstale":"2024-04-24T15:33:28.925244+0000","last_undegraded":"2024-04-24T15:33:28.925244+0000","last_fullsized":"2024-04-24T15:33:28.925244+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":321,"log_dups_size":0,"ondisk_log_size":321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:17:55.155562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079230100000000003,"stat_sum":{"num_bytes":54848543,"num_objects":49,"num_object_clones":18,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":14,"num_read":2040,"num_read_kb":22661,"num_write":340,"num_write_kb":16054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":578400,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"24","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":63,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T15:33:27.899468+0000","last_change":"2024-04-24T15:32:47.636966+0000","last_active":"2024-04-24T15:33:27.899468+0000","last_peered":"2024-04-24T15:33:27.899468+0000","last_clean":"2024-04-24T15:33:27.899468+0000","last_became_active":"2024-04-24T15:32:47.636739+0000","last_became_peered":"2024-04-24T15:32:47.636739+0000","last_unstale":"2024-04-24T15:33:27.899468+0000","last_undegraded":"2024-04-24T15:33:27.899468+0000","last_fullsized":"2024-04-24T15:33:27.899468+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:09:56.338464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"88'404","reported_seq":1620,"reported_epoch":90,"state":"active+undersized+degraded","last_fresh":"2024-04-24T15:33:27.896052+0000","last_change":"2024-04-24T15:33:10.809108+0000","last_active":"2024-04-24T15:33:27.896052+0000","last_peered":"2024-04-24T15:33:27.896052+0000","last_clean":"2024-04-24T15:33:09.555133+0000","last_became_active":"2024-04-24T15:33:10.809108+0000","last_became_peered":"2024-04-24T15:33:10.809108+0000","last_unstale":"2024-04-24T15:33:27.896052+0000","last_undegraded":"2024-04-24T15:33:10.808507+0000","last_fullsized":"2024-04-24T15:33:10.808413+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":404,"log_dups_size":0,"ondisk_log_size":404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:57:51.584683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032331100000000001,"stat_sum":{"num_bytes":18753140,"num_objects":34,"num_object_clones":8,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":8,"num_read":2165,"num_read_kb":14736,"num_write":490,"num_write_kb":12202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":128,"num_bytes_recovered":3330138,"num_keys_recovered":32,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[{"shards":"4","objects":34}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"91'520","reported_seq":2284,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T15:33:28.908776+0000","last_change":"2024-04-24T15:33:28.909155+0000","last_active":"2024-04-24T15:33:28.908776+0000","last_peered":"2024-04-24T15:33:28.908776+0000","last_clean":"2024-04-24T15:33:28.908776+0000","last_became_active":"2024-04-24T15:33:02.638235+0000","last_became_peered":"2024-04-24T15:33:02.638235+0000","last_unstale":"2024-04-24T15:33:28.908776+0000","last_undegraded":"2024-04-24T15:33:28.908776+0000","last_fullsized":"2024-04-24T15:33:28.908776+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":520,"log_dups_size":0,"ondisk_log_size":520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:44:57.206364+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0056358520000000002,"stat_sum":{"num_bytes":33767231,"num_objects":37,"num_object_clones":6,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":11,"num_read":3887,"num_read_kb":20759,"num_write":605,"num_write_kb":10917,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":5713612,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"24","length":"1"}]},{"pgid":"4.a","version":"90'379","reported_seq":2346,"reported_epoch":90,"state":"active+undersized+degraded","last_fresh":"2024-04-24T15:33:28.641205+0000","last_change":"2024-04-24T15:33:10.808761+0000","last_active":"2024-04-24T15:33:28.641205+0000","last_peered":"2024-04-24T15:33:28.641205+0000","last_clean":"2024-04-24T15:33:09.473621+0000","last_became_active":"2024-04-24T15:33:10.808761+0000","last_became_peered":"2024-04-24T15:33:10.808761+0000","last_unstale":"2024-04-24T15:33:28.641205+0000","last_undegraded":"2024-04-24T15:33:10.808052+0000","last_fullsized":"2024-04-24T15:33:10.807971+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:42:23.285910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032027999999999999,"stat_sum":{"num_bytes":69353257,"num_objects":40,"num_object_clones":12,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":10,"num_read":2688,"num_read_kb":27083,"num_write":358,"num_write_kb":12255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":1560706,"num_keys_recovered":19,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":40}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"91'486","reported_seq":1479,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T15:33:28.912122+0000","last_change":"2024-04-24T15:33:28.912353+0000","last_active":"2024-04-24T15:33:28.912122+0000","last_peered":"2024-04-24T15:33:28.912122+0000","last_clean":"2024-04-24T15:33:28.912122+0000","last_became_active":"2024-04-24T15:33:03.635751+0000","last_became_peered":"2024-04-24T15:33:03.635751+0000","last_unstale":"2024-04-24T15:33:28.912122+0000","last_undegraded":"2024-04-24T15:33:28.912122+0000","last_fullsized":"2024-04-24T15:33:28.912122+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:30:51.876165+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0086632930000000007,"stat_sum":{"num_bytes":38476302,"num_objects":23,"num_object_clones":5,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":4,"num_read":2613,"num_read_kb":21061,"num_write":670,"num_write_kb":8462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":1490474,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"24","length":"1"}]},{"pgid":"4.c","version":"90'980","reported_seq":2177,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T15:33:28.904377+0000","last_change":"2024-04-24T15:33:28.904674+0000","last_active":"2024-04-24T15:33:28.904377+0000","last_peered":"2024-04-24T15:33:28.904377+0000","last_clean":"2024-04-24T15:33:28.904377+0000","last_became_active":"2024-04-24T15:33:03.638316+0000","last_became_peered":"2024-04-24T15:33:03.638316+0000","last_unstale":"2024-04-24T15:33:28.904377+0000","last_undegraded":"2024-04-24T15:33:28.904377+0000","last_fullsized":"2024-04-24T15:33:28.904377+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":980,"log_dups_size":0,"ondisk_log_size":980,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:00:53.637603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010704519999999999,"stat_sum":{"num_bytes":34669962,"num_objects":24,"num_object_clones":7,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":8,"num_read":6002,"num_read_kb":16297,"num_write":1626,"num_write_kb":12113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":54,"num_bytes_recovered":1219753,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"24","length":"1"}]},{"pgid":"4.d","version":"91'256","reported_seq":2365,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T15:33:28.907778+0000","last_change":"2024-04-24T15:33:28.908002+0000","last_active":"2024-04-24T15:33:28.907778+0000","last_peered":"2024-04-24T15:33:28.907778+0000","last_clean":"2024-04-24T15:33:28.907778+0000","last_became_active":"2024-04-24T15:33:02.636600+0000","last_became_peered":"2024-04-24T15:33:02.636600+0000","last_unstale":"2024-04-24T15:33:28.907778+0000","last_undegraded":"2024-04-24T15:33:28.907778+0000","last_fullsized":"2024-04-24T15:33:28.907778+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":256,"log_dups_size":0,"ondisk_log_size":256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:59:07.207018+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0041908249999999996,"stat_sum":{"num_bytes":53158644,"num_objects":37,"num_object_clones":14,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":9,"num_read":2113,"num_read_kb":46479,"num_write":289,"num_write_kb":20603,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":4246835,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"24","length":"1"}]},{"pgid":"4.e","version":"90'482","reported_seq":1974,"reported_epoch":90,"state":"active+undersized+degraded","last_fresh":"2024-04-24T15:33:28.585961+0000","last_change":"2024-04-24T15:33:10.810920+0000","last_active":"2024-04-24T15:33:28.585961+0000","last_peered":"2024-04-24T15:33:28.585961+0000","last_clean":"2024-04-24T15:33:09.555459+0000","last_became_active":"2024-04-24T15:33:10.810920+0000","last_became_peered":"2024-04-24T15:33:10.810920+0000","last_unstale":"2024-04-24T15:33:28.585961+0000","last_undegraded":"2024-04-24T15:33:10.809034+0000","last_fullsized":"2024-04-24T15:33:10.808950+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":482,"log_dups_size":0,"ondisk_log_size":482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:51:02.143103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076249599999999999,"stat_sum":{"num_bytes":35274020,"num_objects":35,"num_object_clones":9,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":6,"num_read":2747,"num_read_kb":13238,"num_write":635,"num_write_kb":6170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":93,"num_bytes_recovered":117055,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[{"shards":"4","objects":35}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"91'798","reported_seq":2844,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T15:33:28.920067+0000","last_change":"2024-04-24T15:33:28.920279+0000","last_active":"2024-04-24T15:33:28.920067+0000","last_peered":"2024-04-24T15:33:28.920067+0000","last_clean":"2024-04-24T15:33:28.920067+0000","last_became_active":"2024-04-24T15:33:03.638156+0000","last_became_peered":"2024-04-24T15:33:03.638156+0000","last_unstale":"2024-04-24T15:33:28.920067+0000","last_undegraded":"2024-04-24T15:33:28.920067+0000","last_fullsized":"2024-04-24T15:33:28.920067+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":798,"log_dups_size":0,"ondisk_log_size":798,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:07:21.801310+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.012350641000000001,"stat_sum":{"num_bytes":79513055,"num_objects":43,"num_object_clones":11,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":8,"num_read":5335,"num_read_kb":14152,"num_write":934,"num_write_kb":8978,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":126,"num_bytes_recovered":630687,"num_keys_recovered":35,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"24","length":"1"}]},{"pgid":"4.10","version":"90'212","reported_seq":599,"reported_epoch":90,"state":"active+undersized+degraded","last_fresh":"2024-04-24T15:33:28.468577+0000","last_change":"2024-04-24T15:33:10.810108+0000","last_active":"2024-04-24T15:33:28.468577+0000","last_peered":"2024-04-24T15:33:28.468577+0000","last_clean":"2024-04-24T15:33:09.485422+0000","last_became_active":"2024-04-24T15:33:10.810108+0000","last_became_peered":"2024-04-24T15:33:10.810108+0000","last_unstale":"2024-04-24T15:33:28.468577+0000","last_undegraded":"2024-04-24T15:33:10.809123+0000","last_fullsized":"2024-04-24T15:33:10.809048+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:10.774915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000458918,"stat_sum":{"num_bytes":44263989,"num_objects":30,"num_object_clones":9,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":7,"num_read":378,"num_read_kb":4090,"num_write":221,"num_write_kb":5715,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[{"shards":"3","objects":30}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.11","version":"90'628","reported_seq":1583,"reported_epoch":90,"state":"active+undersized+degraded","last_fresh":"2024-04-24T15:33:28.585505+0000","last_change":"2024-04-24T15:33:10.811332+0000","last_active":"2024-04-24T15:33:28.585505+0000","last_peered":"2024-04-24T15:33:28.585505+0000","last_clean":"2024-04-24T15:33:08.982748+0000","last_became_active":"2024-04-24T15:33:10.811332+0000","last_became_peered":"2024-04-24T15:33:10.811332+0000","last_unstale":"2024-04-24T15:33:28.585505+0000","last_undegraded":"2024-04-24T15:33:10.809649+0000","last_fullsized":"2024-04-24T15:33:10.809577+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":628,"log_dups_size":0,"ondisk_log_size":628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:37:35.537012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057133599999999998,"stat_sum":{"num_bytes":52539726,"num_objects":38,"num_object_clones":11,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":10,"num_read":2527,"num_read_kb":21422,"num_write":837,"num_write_kb":14009,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":2255518,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[{"shards":"3","objects":38}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"88'301","reported_seq":5303,"reported_epoch":90,"state":"active+undersized+degraded","last_fresh":"2024-04-24T15:33:27.895832+0000","last_change":"2024-04-24T15:33:10.813029+0000","last_active":"2024-04-24T15:33:27.895832+0000","last_peered":"2024-04-24T15:33:27.895832+0000","last_clean":"2024-04-24T15:33:09.218784+0000","last_became_active":"2024-04-24T15:33:10.813029+0000","last_became_peered":"2024-04-24T15:33:10.813029+0000","last_unstale":"2024-04-24T15:33:27.895832+0000","last_undegraded":"2024-04-24T15:33:10.812436+0000","last_fullsized":"2024-04-24T15:33:10.812369+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":301,"log_dups_size":0,"ondisk_log_size":301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:10:27.490608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041921100000000001,"stat_sum":{"num_bytes":65264629,"num_objects":30,"num_object_clones":11,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":11,"num_read":4982,"num_read_kb":42072,"num_write":325,"num_write_kb":14625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":30}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"90'218","reported_seq":1750,"reported_epoch":90,"state":"active+undersized+degraded","last_fresh":"2024-04-24T15:33:28.639902+0000","last_change":"2024-04-24T15:33:10.810915+0000","last_active":"2024-04-24T15:33:28.639902+0000","last_peered":"2024-04-24T15:33:28.639902+0000","last_clean":"2024-04-24T15:33:09.521404+0000","last_became_active":"2024-04-24T15:33:10.810915+0000","last_became_peered":"2024-04-24T15:33:10.810915+0000","last_unstale":"2024-04-24T15:33:28.639902+0000","last_undegraded":"2024-04-24T15:33:10.808931+0000","last_fullsized":"2024-04-24T15:33:10.808855+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":218,"log_dups_size":0,"ondisk_log_size":218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:55:36.208982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091721200000000004,"stat_sum":{"num_bytes":28592545,"num_objects":23,"num_object_clones":3,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":3,"num_read":1305,"num_read_kb":18875,"num_write":236,"num_write_kb":9844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":1268965,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[{"shards":"4","objects":23}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"91'357","reported_seq":5709,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T15:33:28.914704+0000","last_change":"2024-04-24T15:33:28.915075+0000","last_active":"2024-04-24T15:33:28.914704+0000","last_peered":"2024-04-24T15:33:28.914704+0000","last_clean":"2024-04-24T15:33:28.914704+0000","last_became_active":"2024-04-24T15:32:50.657302+0000","last_became_peered":"2024-04-24T15:32:50.657302+0000","last_unstale":"2024-04-24T15:33:28.914704+0000","last_undegraded":"2024-04-24T15:33:28.914704+0000","last_fullsized":"2024-04-24T15:33:28.914704+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:30:15.118724+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0073562560000000003,"stat_sum":{"num_bytes":74655048,"num_objects":42,"num_object_clones":15,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":10,"num_read":6886,"num_read_kb":43731,"num_write":358,"num_write_kb":9838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":2229438,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"24","length":"1"}]},{"pgid":"4.15","version":"90'274","reported_seq":1282,"reported_epoch":90,"state":"active+undersized+degraded","last_fresh":"2024-04-24T15:33:28.585496+0000","last_change":"2024-04-24T15:33:10.811366+0000","last_active":"2024-04-24T15:33:28.585496+0000","last_peered":"2024-04-24T15:33:28.585496+0000","last_clean":"2024-04-24T15:33:09.514177+0000","last_became_active":"2024-04-24T15:33:10.811366+0000","last_became_peered":"2024-04-24T15:33:10.811366+0000","last_unstale":"2024-04-24T15:33:28.585496+0000","last_undegraded":"2024-04-24T15:33:10.809916+0000","last_fullsized":"2024-04-24T15:33:10.809845+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":274,"log_dups_size":0,"ondisk_log_size":274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:52:30.625222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001011678,"stat_sum":{"num_bytes":51789069,"num_objects":34,"num_object_clones":12,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":8,"num_read":837,"num_read_kb":16692,"num_write":305,"num_write_kb":15488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":4590279,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[{"shards":"3","objects":34}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"91'319","reported_seq":2127,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T15:33:28.922607+0000","last_change":"2024-04-24T15:33:28.922741+0000","last_active":"2024-04-24T15:33:28.922607+0000","last_peered":"2024-04-24T15:33:28.922607+0000","last_clean":"2024-04-24T15:33:28.922607+0000","last_became_active":"2024-04-24T15:32:24.075461+0000","last_became_peered":"2024-04-24T15:32:24.075461+0000","last_unstale":"2024-04-24T15:33:28.922607+0000","last_undegraded":"2024-04-24T15:33:28.922607+0000","last_fullsized":"2024-04-24T15:33:28.922607+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:07:28.709798+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.014303827,"stat_sum":{"num_bytes":60309940,"num_objects":36,"num_object_clones":13,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":14,"num_read":2768,"num_read_kb":17573,"num_write":313,"num_write_kb":8030,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"24","length":"1"}]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":1490589835,"num_objects":1102,"num_object_clones":308,"num_object_copies":2204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":397,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1102,"num_whiteouts":282,"num_read":106551,"num_read_kb":720483,"num_write":18667,"num_write_kb":355923,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1800,"num_bytes_recovered":55960625,"num_keys_recovered":162,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":52068352,"data_stored":556230593,"data_compressed":20839151,"data_compressed_allocated":43372544,"data_compressed_original":551311985,"omap_allocated":0,"internal_metadata":0},"log_size":14866,"ondisk_log_size":14866,"up":52,"acting":52,"num_store_stats":7},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":6,"acting":6,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":1771104,"data_compressed":15361,"data_compressed_allocated":110592,"data_compressed_original":1736704,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":1,"acting":1,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":13,"acting":13,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":0,"seq":55834574869,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303312,"kb_used_data":5944,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94068528,"statfs":{"total":96636764160,"available":96326172672,"internally_reserved":0,"allocated":6086656,"data_stored":49896719,"data_compressed":1994958,"data_compressed_allocated":4091904,"data_compressed_original":48810435,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 15:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.5},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.44500000000000001}]},{"osd":1,"last update":"Wed Apr 24 15:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.46700000000000003}]},{"osd":2,"last update":"Wed Apr 24 15:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.436},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.51000000000000001}]},{"osd":3,"last update":"Wed Apr 24 15:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.45800000000000002}]},{"osd":4,"last update":"Wed Apr 24 15:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.377},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.47699999999999998}]},{"osd":5,"last update":"Wed Apr 24 15:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.52900000000000003}]},{"osd":7,"last update":"Wed Apr 24 15:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.51800000000000002}]}]},{"osd":1,"up_from":0,"seq":55834574868,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306420,"kb_used_data":9064,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94065420,"statfs":{"total":96636764160,"available":96322990080,"internally_reserved":0,"allocated":9281536,"data_stored":88124840,"data_compressed":3473230,"data_compressed_allocated":6905856,"data_compressed_original":86896308,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Apr 24 15:33:00 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":3.7890000000000001,"5min":3.7890000000000001,"15min":3.7890000000000001},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":2.7890000000000001,"5min":2.7890000000000001,"15min":2.7890000000000001},"last":0.40899999999999997}]},{"osd":2,"last update":"Wed Apr 24 15:33:00 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":3.9950000000000001,"5min":3.9950000000000001,"15min":3.9950000000000001},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.443,"5min":0.443,"15min":0.443},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":1.8500000000000001,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":0.28199999999999997}]},{"osd":3,"last update":"Wed Apr 24 15:33:00 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":2.097,"5min":2.097,"15min":2.097},"last":0.497},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":3.2109999999999999,"5min":3.2109999999999999,"15min":3.2109999999999999},"last":0.50600000000000001}]},{"osd":4,"last update":"Wed Apr 24 15:33:00 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":3.282,"5min":3.282,"15min":3.282},"last":0.443}]},{"osd":5,"last update":"Wed Apr 24 15:33:00 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":2.2229999999999999,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":2.1699999999999999,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":0.46600000000000003}]},{"osd":6,"last update":"Wed Apr 24 15:33:00 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":3.2509999999999999,"5min":3.2509999999999999,"15min":3.2509999999999999},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":2.1909999999999998,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":0.67200000000000004}]},{"osd":7,"last update":"Wed Apr 24 15:33:00 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":2.1349999999999998,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":2.206,"5min":2.206,"15min":2.206},"last":0.53000000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":322220,"kb_used_data":8444,"kb_used_omap":1,"kb_used_meta":313726,"kb_avail":94049620,"statfs":{"total":96636764160,"available":96306810880,"internally_reserved":0,"allocated":8646656,"data_stored":82605985,"data_compressed":2877751,"data_compressed_allocated":6295552,"data_compressed_original":81318049,"omap_allocated":1704,"internal_metadata":321255768},"hb_peers":[0,1,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 15:33:03 2024","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.317,"15min":0.317},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"last":0.316},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"last":0.32900000000000001}]},{"osd":1,"last update":"Wed Apr 24 15:33:03 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":3.1110000000000002,"5min":3.1110000000000002,"15min":3.1110000000000002},"last":0.255}]},{"osd":3,"last update":"Wed Apr 24 15:33:03 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":2.056,"5min":2.056,"15min":2.056},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.38500000000000001}]},{"osd":4,"last update":"Wed Apr 24 15:33:03 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.432},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.36599999999999999}]},{"osd":5,"last update":"Wed Apr 24 15:33:03 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":2.1190000000000002,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":0.46999999999999997}]},{"osd":6,"last update":"Wed Apr 24 15:33:03 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.1000000000000001,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.376}]},{"osd":7,"last update":"Wed Apr 24 15:33:03 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":2.1360000000000001,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":0.45600000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":313100,"kb_used_data":15744,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94058740,"statfs":{"total":96636764160,"available":96316149760,"internally_reserved":0,"allocated":16121856,"data_stored":160767047,"data_compressed":6417605,"data_compressed_allocated":12484608,"data_compressed_original":158808958,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 15:33:00 2024","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.41599999999999998}]},{"osd":1,"last update":"Wed Apr 24 15:33:00 2024","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"min":{"1min":0.187,"5min":0.187,"15min":0.187},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.437},{"interface":"front","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.40400000000000003}]},{"osd":2,"last update":"Wed Apr 24 15:33:00 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.43099999999999999}]},{"osd":4,"last update":"Wed Apr 24 15:33:00 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.54200000000000004}]},{"osd":5,"last update":"Wed Apr 24 15:33:00 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.6020000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.59399999999999997}]},{"osd":6,"last update":"Wed Apr 24 15:33:00 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.52200000000000002}]},{"osd":7,"last update":"Wed Apr 24 15:33:00 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.635,"5min":1.635,"15min":1.635},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.57099999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":317416,"kb_used_data":15920,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94054424,"statfs":{"total":96636764160,"available":96311730176,"internally_reserved":0,"allocated":16302080,"data_stored":168046060,"data_compressed":6134656,"data_compressed_allocated":13238272,"data_compressed_original":166391585,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[0,1,2,3,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 15:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.436},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.42899999999999999}]},{"osd":1,"last update":"Wed Apr 24 15:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.40899999999999997}]},{"osd":2,"last update":"Wed Apr 24 15:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.35299999999999998}]},{"osd":3,"last update":"Wed Apr 24 15:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.374}]},{"osd":5,"last update":"Wed Apr 24 15:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.309}]},{"osd":6,"last update":"Wed Apr 24 15:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":5.2329999999999997},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":6.1870000000000003}]},{"osd":7,"last update":"Wed Apr 24 15:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.317},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.29399999999999998}]}]},{"osd":5,"up_from":0,"seq":55834574869,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4842,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":5677,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4842,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8011776,"data_stored":84945448,"data_compressed":3418261,"data_compressed_allocated":6778880,"data_compressed_original":84287156,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":7397376,"data_stored":79667276,"data_compressed":2763827,"data_compressed_allocated":6148096,"data_compressed_original":78930081,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":14663680,"data_stored":155168912,"data_compressed":6136785,"data_compressed_allocated":12206080,"data_compressed_original":153816243,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":14254080,"data_stored":156693808,"data_compressed":5588908,"data_compressed_allocated":12390400,"data_compressed_original":155656278,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":204800,"data_stored":181730,"data_compressed":8376,"data_compressed_allocated":32768,"data_compressed_original":77824,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":5193728,"data_stored":54700048,"data_compressed":2185115,"data_compressed_allocated":4059136,"data_compressed_original":54016451,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2342912,"data_stored":24873371,"data_compressed":737879,"data_compressed_allocated":1757184,"data_compressed_original":24527952,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T15:33:30.122 INFO:tasks.thrashosds.thrasher:Setting 4.9 to [7, 2, 6, 1] 2024-04-24T15:33:30.122 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '4.9', '7', '2', '6', '1'] 2024-04-24T15:33:30.122 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 4.9 7 2 6 1 2024-04-24T15:33:30.123 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:30.124+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:30.127 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:30.127 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:33:30.127 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-04-24T15:33:30.127 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-04-24T15:33:30.127 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:30.138 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-24T15:33:30.225 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:30.226+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:30.308 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:30.308 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-04-24T15:33:30.308 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-04-24T15:33:30.309 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-04-24T15:33:30.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.309 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:2522 4.6 4:60ceedf0:::rbd_data.114a1f60cb66.000000000000011d:head [set-alloc-hint object_size 262144 write_size 262144,write 110592~2048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-24T15:33:30.309 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:23.088240+0000", 2024-04-24T15:33:30.309 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 67.219398104000007, 2024-04-24T15:33:30.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.988232797, 2024-04-24T15:33:30.309 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:30.309 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:30.309 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:30.309 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:30.309 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:30.309 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2522 2024-04-24T15:33:30.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.310 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:30.310 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.310 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:30.310 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088240+0000", 2024-04-24T15:33:30.310 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:30.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.310 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.310 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:30.310 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088240+0000", 2024-04-24T15:33:30.310 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:30.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.311 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:30.311 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088238+0000", 2024-04-24T15:33:30.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999981 2024-04-24T15:33:30.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.311 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:30.311 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088254+0000", 2024-04-24T15:33:30.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5562000000000001e-05 2024-04-24T15:33:30.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.311 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:30.311 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088275+0000", 2024-04-24T15:33:30.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1256000000000001e-05 2024-04-24T15:33:30.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.312 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:30.312 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088283+0000", 2024-04-24T15:33:30.312 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3219999999999993e-06 2024-04-24T15:33:30.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.312 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:30.312 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088351+0000", 2024-04-24T15:33:30.312 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7595000000000006e-05 2024-04-24T15:33:30.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.312 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-04-24T15:33:30.312 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088362+0000", 2024-04-24T15:33:30.312 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1764e-05 2024-04-24T15:33:30.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.313 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.313 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:30.313 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.073748+0000", 2024-04-24T15:33:30.313 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98538541899999998 2024-04-24T15:33:30.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.313 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.313 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:30.313 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.073925+0000", 2024-04-24T15:33:30.313 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017747600000000001 2024-04-24T15:33:30.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.313 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.313 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-24T15:33:30.313 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.074092+0000", 2024-04-24T15:33:30.313 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000166622 2024-04-24T15:33:30.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.314 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.314 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:33:30.314 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.074774+0000", 2024-04-24T15:33:30.314 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00068225000000000002 2024-04-24T15:33:30.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.314 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.314 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:33:30.314 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.076412+0000", 2024-04-24T15:33:30.314 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001638189 2024-04-24T15:33:30.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.314 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.314 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:30.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.076446+0000", 2024-04-24T15:33:30.315 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3587000000000002e-05 2024-04-24T15:33:30.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.315 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:30.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.076473+0000", 2024-04-24T15:33:30.315 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6886999999999999e-05 2024-04-24T15:33:30.315 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:30.315 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:30.315 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:30.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.316 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.316 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43121 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 5865472~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:30.316 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094178+0000", 2024-04-24T15:33:30.316 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.213460421000001, 2024-04-24T15:33:30.316 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032860598999999997, 2024-04-24T15:33:30.316 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:30.316 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:30.316 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:30.316 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:30.316 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:30.316 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43121 2024-04-24T15:33:30.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.316 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:30.316 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.316 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:30.316 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094178+0000", 2024-04-24T15:33:30.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:30.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.317 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:30.317 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094178+0000", 2024-04-24T15:33:30.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:30.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.317 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:30.317 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094178+0000", 2024-04-24T15:33:30.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:30.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.318 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:30.318 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094180+0000", 2024-04-24T15:33:30.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9989999999999998e-06 2024-04-24T15:33:30.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.318 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.318 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:30.318 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094185+0000", 2024-04-24T15:33:30.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.541e-06 2024-04-24T15:33:30.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.318 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.318 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:30.319 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094189+0000", 2024-04-24T15:33:30.319 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0509999999999998e-06 2024-04-24T15:33:30.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.319 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:30.319 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.126877+0000", 2024-04-24T15:33:30.319 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032687473000000002 2024-04-24T15:33:30.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.319 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:30.319 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.126952+0000", 2024-04-24T15:33:30.319 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5924e-05 2024-04-24T15:33:30.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.319 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:30.320 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127038+0000", 2024-04-24T15:33:30.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5908999999999994e-05 2024-04-24T15:33:30.320 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:30.320 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:30.320 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:30.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.320 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43122 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 6291456~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:30.320 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094193+0000", 2024-04-24T15:33:30.320 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.213445177000001, 2024-04-24T15:33:30.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032978383999999999, 2024-04-24T15:33:30.320 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:30.320 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:30.321 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:30.321 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:30.321 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:30.321 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43122 2024-04-24T15:33:30.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.321 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:30.321 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.321 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:30.321 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094193+0000", 2024-04-24T15:33:30.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:30.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.321 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.321 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:30.321 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094193+0000", 2024-04-24T15:33:30.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:30.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.322 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.322 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:30.322 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094193+0000", 2024-04-24T15:33:30.322 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:30.322 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.322 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.322 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:30.322 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094195+0000", 2024-04-24T15:33:30.322 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3269999999999999e-06 2024-04-24T15:33:30.322 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.322 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.322 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:30.322 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094201+0000", 2024-04-24T15:33:30.323 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9850000000000003e-06 2024-04-24T15:33:30.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.323 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.323 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:30.323 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094205+0000", 2024-04-24T15:33:30.323 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8430000000000003e-06 2024-04-24T15:33:30.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.323 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.323 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:30.323 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127068+0000", 2024-04-24T15:33:30.323 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032863305000000002 2024-04-24T15:33:30.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.323 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.323 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:30.323 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127113+0000", 2024-04-24T15:33:30.323 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4412000000000001e-05 2024-04-24T15:33:30.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.324 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.324 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:30.324 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127171+0000", 2024-04-24T15:33:30.324 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8807999999999997e-05 2024-04-24T15:33:30.324 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:30.324 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:30.324 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:30.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.324 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.324 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43123 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 6717440~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:30.324 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094209+0000", 2024-04-24T15:33:30.324 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.213429010999999, 2024-04-24T15:33:30.324 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033144583999999998, 2024-04-24T15:33:30.324 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:30.325 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:30.325 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:30.325 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:30.325 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:30.325 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43123 2024-04-24T15:33:30.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.325 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:30.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.325 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:30.325 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094209+0000", 2024-04-24T15:33:30.325 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:30.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.326 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.326 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:30.326 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094209+0000", 2024-04-24T15:33:30.326 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:30.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.326 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.326 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:30.326 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094209+0000", 2024-04-24T15:33:30.326 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:30.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.327 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.327 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:30.327 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094211+0000", 2024-04-24T15:33:30.327 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2089999999999999e-06 2024-04-24T15:33:30.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.327 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.327 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:30.327 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094217+0000", 2024-04-24T15:33:30.327 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6799999999999998e-06 2024-04-24T15:33:30.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.327 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.327 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:30.327 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094220+0000", 2024-04-24T15:33:30.327 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.444e-06 2024-04-24T15:33:30.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.328 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.328 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:30.328 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127190+0000", 2024-04-24T15:33:30.328 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032969471 2024-04-24T15:33:30.328 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.328 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.328 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:30.328 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127269+0000", 2024-04-24T15:33:30.328 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9246999999999995e-05 2024-04-24T15:33:30.328 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.329 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.329 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:30.329 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127354+0000", 2024-04-24T15:33:30.329 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4853999999999998e-05 2024-04-24T15:33:30.329 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:30.329 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:30.329 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:30.329 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.329 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.329 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43124 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 7143424~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:30.329 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094224+0000", 2024-04-24T15:33:30.329 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.213414038, 2024-04-24T15:33:30.329 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033271955999999998, 2024-04-24T15:33:30.329 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:30.330 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:30.330 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:30.330 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:30.330 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:30.330 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43124 2024-04-24T15:33:30.330 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.330 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:30.330 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.330 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:30.330 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094224+0000", 2024-04-24T15:33:30.330 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:30.330 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.330 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.330 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:30.331 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094224+0000", 2024-04-24T15:33:30.331 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:30.331 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.331 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.331 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:30.331 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094224+0000", 2024-04-24T15:33:30.331 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:30.331 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.331 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.331 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:30.331 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094226+0000", 2024-04-24T15:33:30.331 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9039999999999999e-06 2024-04-24T15:33:30.331 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.331 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.331 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:30.332 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094233+0000", 2024-04-24T15:33:30.332 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7970000000000003e-06 2024-04-24T15:33:30.332 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.332 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.332 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:30.332 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094236+0000", 2024-04-24T15:33:30.332 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6339999999999999e-06 2024-04-24T15:33:30.332 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.332 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.332 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:30.332 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127390+0000", 2024-04-24T15:33:30.332 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.03315353 2024-04-24T15:33:30.332 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.332 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.332 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:30.333 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127439+0000", 2024-04-24T15:33:30.333 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8890000000000001e-05 2024-04-24T15:33:30.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.333 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.333 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:30.333 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127496+0000", 2024-04-24T15:33:30.333 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7476999999999999e-05 2024-04-24T15:33:30.333 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:30.333 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:30.333 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:30.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.333 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.333 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43125 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 7569408~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:30.333 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094240+0000", 2024-04-24T15:33:30.333 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.213397951000001, 2024-04-24T15:33:30.334 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033446278000000003, 2024-04-24T15:33:30.334 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:30.334 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:30.334 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:30.334 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:30.334 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:30.334 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43125 2024-04-24T15:33:30.334 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.334 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:30.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.334 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:30.334 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094240+0000", 2024-04-24T15:33:30.334 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:30.334 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.335 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.335 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:30.335 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094240+0000", 2024-04-24T15:33:30.335 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:30.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.335 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.336 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:30.336 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094240+0000", 2024-04-24T15:33:30.336 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:30.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.336 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.336 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:30.336 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094247+0000", 2024-04-24T15:33:30.337 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6209999999999998e-06 2024-04-24T15:33:30.337 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.337 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.337 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:30.337 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094253+0000", 2024-04-24T15:33:30.337 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0739999999999998e-06 2024-04-24T15:33:30.337 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.337 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.337 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:30.337 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094256+0000", 2024-04-24T15:33:30.337 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7699999999999999e-06 2024-04-24T15:33:30.338 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.338 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.338 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:30.338 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127526+0000", 2024-04-24T15:33:30.338 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033269703999999997 2024-04-24T15:33:30.338 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.338 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.338 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:30.338 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127603+0000", 2024-04-24T15:33:30.339 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6964000000000004e-05 2024-04-24T15:33:30.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.339 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.339 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:30.339 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127687+0000", 2024-04-24T15:33:30.339 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3431999999999997e-05 2024-04-24T15:33:30.339 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:30.339 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:30.339 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:30.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.339 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.339 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43126 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 7995392~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:30.339 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094261+0000", 2024-04-24T15:33:30.339 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.213377447999999, 2024-04-24T15:33:30.340 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033555201999999999, 2024-04-24T15:33:30.340 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:30.340 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:30.340 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:30.340 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:30.340 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:30.340 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43126 2024-04-24T15:33:30.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.340 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:30.340 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.340 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:30.340 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094261+0000", 2024-04-24T15:33:30.340 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:30.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.340 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.341 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:30.341 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094261+0000", 2024-04-24T15:33:30.341 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:30.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.341 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.341 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:30.341 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094260+0000", 2024-04-24T15:33:30.341 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:30.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.341 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.341 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:30.342 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094263+0000", 2024-04-24T15:33:30.342 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2579999999999998e-06 2024-04-24T15:33:30.342 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.342 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.342 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:30.342 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094269+0000", 2024-04-24T15:33:30.342 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.823e-06 2024-04-24T15:33:30.342 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.342 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.342 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:30.342 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094272+0000", 2024-04-24T15:33:30.342 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8999999999999999e-06 2024-04-24T15:33:30.342 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.342 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.343 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:30.343 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127720+0000", 2024-04-24T15:33:30.343 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033447521000000001 2024-04-24T15:33:30.343 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.343 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.343 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:30.343 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127765+0000", 2024-04-24T15:33:30.343 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5352000000000002e-05 2024-04-24T15:33:30.343 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.343 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.343 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:30.343 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127816+0000", 2024-04-24T15:33:30.343 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0689999999999997e-05 2024-04-24T15:33:30.343 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:30.343 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:30.344 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:30.344 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.344 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.344 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63800 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1032192~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:30.344 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555407+0000", 2024-04-24T15:33:30.344 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 10.752231461999999, 2024-04-24T15:33:30.344 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032809030000000003, 2024-04-24T15:33:30.344 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:30.344 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:30.344 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:30.344 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:30.344 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:30.344 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63800 2024-04-24T15:33:30.344 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.344 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:30.345 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.345 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:30.345 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555407+0000", 2024-04-24T15:33:30.345 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:30.345 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.345 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.345 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:30.345 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555407+0000", 2024-04-24T15:33:30.345 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:30.345 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.345 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.345 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:30.345 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555406+0000", 2024-04-24T15:33:30.345 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:30.345 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.346 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.346 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:30.346 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555409+0000", 2024-04-24T15:33:30.346 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.706e-06 2024-04-24T15:33:30.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.346 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.346 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:30.346 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555416+0000", 2024-04-24T15:33:30.346 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1609999999999997e-06 2024-04-24T15:33:30.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.347 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.347 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:30.347 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555420+0000", 2024-04-24T15:33:30.347 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0369999999999996e-06 2024-04-24T15:33:30.347 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.347 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.347 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:30.347 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588088+0000", 2024-04-24T15:33:30.347 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032668228000000001 2024-04-24T15:33:30.347 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.347 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.347 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:30.347 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588143+0000", 2024-04-24T15:33:30.347 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4852999999999998e-05 2024-04-24T15:33:30.347 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.348 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.348 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:30.348 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588216+0000", 2024-04-24T15:33:30.348 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2467999999999998e-05 2024-04-24T15:33:30.348 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:30.348 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:30.348 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:30.348 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.348 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.348 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63801 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1114112~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:30.348 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555425+0000", 2024-04-24T15:33:30.348 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 10.752213555000001, 2024-04-24T15:33:30.348 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032924691999999998, 2024-04-24T15:33:30.349 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:30.349 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:30.349 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:30.349 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:30.349 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:30.349 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63801 2024-04-24T15:33:30.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.349 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:30.349 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.349 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:30.349 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555425+0000", 2024-04-24T15:33:30.350 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:30.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.350 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.350 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:30.350 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555425+0000", 2024-04-24T15:33:30.350 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:30.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.350 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.350 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:30.350 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555424+0000", 2024-04-24T15:33:30.350 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:30.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.350 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.350 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:30.351 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555427+0000", 2024-04-24T15:33:30.351 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2350000000000002e-06 2024-04-24T15:33:30.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.351 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.351 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:30.351 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555432+0000", 2024-04-24T15:33:30.351 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.434e-06 2024-04-24T15:33:30.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.351 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.351 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:30.351 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555435+0000", 2024-04-24T15:33:30.351 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2009999999999999e-06 2024-04-24T15:33:30.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.351 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.351 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:30.351 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588237+0000", 2024-04-24T15:33:30.352 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032801508 2024-04-24T15:33:30.352 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.352 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.352 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:30.352 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588281+0000", 2024-04-24T15:33:30.352 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4100999999999997e-05 2024-04-24T15:33:30.352 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.352 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.352 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:30.352 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588349+0000", 2024-04-24T15:33:30.352 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8551000000000001e-05 2024-04-24T15:33:30.352 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:30.352 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:30.352 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:30.352 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.353 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.353 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63802 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1196032~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:30.353 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555439+0000", 2024-04-24T15:33:30.353 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 10.752199034, 2024-04-24T15:33:30.353 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033057744999999999, 2024-04-24T15:33:30.353 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:30.353 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:30.353 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:30.353 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:30.353 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:30.353 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63802 2024-04-24T15:33:30.353 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.353 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:30.353 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.354 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:30.354 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555439+0000", 2024-04-24T15:33:30.354 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:30.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.354 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.354 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:30.354 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555439+0000", 2024-04-24T15:33:30.354 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:30.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.354 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.354 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:30.354 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555439+0000", 2024-04-24T15:33:30.354 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:30.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.355 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.355 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:30.355 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555445+0000", 2024-04-24T15:33:30.355 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4019999999999999e-06 2024-04-24T15:33:30.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.355 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.355 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:30.355 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555452+0000", 2024-04-24T15:33:30.355 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8600000000000004e-06 2024-04-24T15:33:30.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.355 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.355 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:30.355 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555461+0000", 2024-04-24T15:33:30.355 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0499999999999997e-06 2024-04-24T15:33:30.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.356 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.356 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:30.356 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588378+0000", 2024-04-24T15:33:30.356 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032916527000000001 2024-04-24T15:33:30.356 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.356 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.356 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:30.356 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588423+0000", 2024-04-24T15:33:30.356 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5092999999999997e-05 2024-04-24T15:33:30.356 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.356 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.356 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:30.356 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588497+0000", 2024-04-24T15:33:30.357 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4140999999999994e-05 2024-04-24T15:33:30.357 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:30.357 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:30.357 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:30.357 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.357 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.357 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63803 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1277952~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:30.357 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555466+0000", 2024-04-24T15:33:30.357 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 10.752172089, 2024-04-24T15:33:30.357 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033162297, 2024-04-24T15:33:30.357 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:30.357 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:30.357 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:30.357 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:30.358 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:30.358 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63803 2024-04-24T15:33:30.358 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.358 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:30.358 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.358 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:30.358 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555466+0000", 2024-04-24T15:33:30.358 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:30.358 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.358 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.358 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:30.358 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555466+0000", 2024-04-24T15:33:30.358 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:30.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:30.359 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555466+0000", 2024-04-24T15:33:30.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:30.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:30.359 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555468+0000", 2024-04-24T15:33:30.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4040000000000002e-06 2024-04-24T15:33:30.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:30.359 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555475+0000", 2024-04-24T15:33:30.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9480000000000002e-06 2024-04-24T15:33:30.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:30.360 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555479+0000", 2024-04-24T15:33:30.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0289999999999999e-06 2024-04-24T15:33:30.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:30.360 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588516+0000", 2024-04-24T15:33:30.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033037094000000003 2024-04-24T15:33:30.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.361 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:30.361 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588562+0000", 2024-04-24T15:33:30.361 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6215999999999999e-05 2024-04-24T15:33:30.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.361 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.361 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:30.361 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588628+0000", 2024-04-24T15:33:30.362 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5971999999999998e-05 2024-04-24T15:33:30.362 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:30.362 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:30.362 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:30.362 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.362 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.362 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63804 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1359872~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:30.363 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555483+0000", 2024-04-24T15:33:30.363 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 10.752154922000001, 2024-04-24T15:33:30.363 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033287068000000003, 2024-04-24T15:33:30.363 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:30.363 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:30.363 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:30.364 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:30.364 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:30.364 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63804 2024-04-24T15:33:30.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.364 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:30.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.364 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:30.365 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555483+0000", 2024-04-24T15:33:30.365 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:30.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.365 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.365 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:30.365 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555483+0000", 2024-04-24T15:33:30.365 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:30.366 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.366 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.366 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:30.366 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555483+0000", 2024-04-24T15:33:30.366 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:30.366 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.367 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.367 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:30.367 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555485+0000", 2024-04-24T15:33:30.367 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0710000000000002e-06 2024-04-24T15:33:30.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.367 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.367 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:30.368 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555498+0000", 2024-04-24T15:33:30.368 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2748999999999999e-05 2024-04-24T15:33:30.368 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.368 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.368 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:30.368 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555502+0000", 2024-04-24T15:33:30.369 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0219999999999998e-06 2024-04-24T15:33:30.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.369 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.369 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:30.369 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588657+0000", 2024-04-24T15:33:30.369 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033154683999999997 2024-04-24T15:33:30.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.369 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.369 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:30.369 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588702+0000", 2024-04-24T15:33:30.370 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5809000000000001e-05 2024-04-24T15:33:30.370 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.370 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.370 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:30.370 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588770+0000", 2024-04-24T15:33:30.370 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8056e-05 2024-04-24T15:33:30.371 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:30.371 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:30.371 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:30.371 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.371 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.371 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63805 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1441792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:30.372 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555508+0000", 2024-04-24T15:33:30.372 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 10.752130073, 2024-04-24T15:33:30.372 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033375952, 2024-04-24T15:33:30.372 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:30.372 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:30.372 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:30.373 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:30.373 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:30.373 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63805 2024-04-24T15:33:30.373 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.373 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:30.374 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.374 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:30.374 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555508+0000", 2024-04-24T15:33:30.374 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:30.374 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.374 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.375 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:30.375 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555508+0000", 2024-04-24T15:33:30.375 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:30.375 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.375 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.376 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:30.376 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555508+0000", 2024-04-24T15:33:30.376 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:30.376 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.376 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.376 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:30.377 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555510+0000", 2024-04-24T15:33:30.377 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.424e-06 2024-04-24T15:33:30.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.377 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.377 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:30.377 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555516+0000", 2024-04-24T15:33:30.378 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3730000000000001e-06 2024-04-24T15:33:30.378 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.378 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.378 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:30.378 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555519+0000", 2024-04-24T15:33:30.378 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.039e-06 2024-04-24T15:33:30.379 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.379 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.379 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:30.379 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588789+0000", 2024-04-24T15:33:30.379 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0332702 2024-04-24T15:33:30.379 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.380 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.380 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:30.380 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588833+0000", 2024-04-24T15:33:30.380 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4660999999999998e-05 2024-04-24T15:33:30.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.381 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:30.381 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588884+0000", 2024-04-24T15:33:30.381 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0661999999999997e-05 2024-04-24T15:33:30.381 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:30.381 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:30.382 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:30.382 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.382 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.382 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63806 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1523712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:30.382 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555523+0000", 2024-04-24T15:33:30.383 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 10.752115492, 2024-04-24T15:33:30.383 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033479653999999998, 2024-04-24T15:33:30.383 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:30.383 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:30.383 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:30.383 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:30.384 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:30.384 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63806 2024-04-24T15:33:30.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.384 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:30.384 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.384 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:30.385 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555523+0000", 2024-04-24T15:33:30.385 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:30.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.385 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.385 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:30.385 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555523+0000", 2024-04-24T15:33:30.385 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:30.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.386 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.386 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:30.386 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555522+0000", 2024-04-24T15:33:30.386 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:30.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.387 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.387 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:30.387 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555524+0000", 2024-04-24T15:33:30.387 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0700000000000001e-06 2024-04-24T15:33:30.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.388 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.388 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:30.388 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555530+0000", 2024-04-24T15:33:30.388 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5450000000000003e-06 2024-04-24T15:33:30.388 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.388 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.389 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:30.389 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555533+0000", 2024-04-24T15:33:30.389 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2200000000000001e-06 2024-04-24T15:33:30.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.389 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.389 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:30.390 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588908+0000", 2024-04-24T15:33:30.390 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033374556 2024-04-24T15:33:30.390 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.390 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.390 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:30.390 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588950+0000", 2024-04-24T15:33:30.390 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2361999999999999e-05 2024-04-24T15:33:30.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.391 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.391 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:30.391 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589002+0000", 2024-04-24T15:33:30.391 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2224e-05 2024-04-24T15:33:30.391 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:30.391 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:30.391 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:30.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.392 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.392 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63807 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1605632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:30.392 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555537+0000", 2024-04-24T15:33:30.392 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 10.752100977, 2024-04-24T15:33:30.392 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033577227000000001, 2024-04-24T15:33:30.392 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:30.392 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:30.392 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:30.392 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:30.393 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:30.393 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63807 2024-04-24T15:33:30.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.393 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:30.393 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.393 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:30.393 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555537+0000", 2024-04-24T15:33:30.393 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:30.394 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.394 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.394 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:30.394 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555537+0000", 2024-04-24T15:33:30.394 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:30.394 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.394 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.394 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:30.394 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555537+0000", 2024-04-24T15:33:30.395 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:30.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.395 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.395 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:30.395 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555539+0000", 2024-04-24T15:33:30.395 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9769999999999999e-06 2024-04-24T15:33:30.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.395 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.395 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:30.395 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555544+0000", 2024-04-24T15:33:30.395 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.468e-06 2024-04-24T15:33:30.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.395 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.395 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:30.395 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555548+0000", 2024-04-24T15:33:30.395 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4319999999999999e-06 2024-04-24T15:33:30.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.396 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.396 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:30.396 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589021+0000", 2024-04-24T15:33:30.396 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033473655999999997 2024-04-24T15:33:30.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.396 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.396 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:30.396 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589064+0000", 2024-04-24T15:33:30.396 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2397000000000001e-05 2024-04-24T15:33:30.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.396 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.396 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:30.397 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589114+0000", 2024-04-24T15:33:30.397 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0611e-05 2024-04-24T15:33:30.397 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:30.397 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:30.397 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:30.397 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.397 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.397 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63808 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1687552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:30.397 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555552+0000", 2024-04-24T15:33:30.397 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 10.752086285000001, 2024-04-24T15:33:30.397 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033686019999999997, 2024-04-24T15:33:30.397 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:30.397 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:30.397 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:30.397 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:30.397 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:30.398 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63808 2024-04-24T15:33:30.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.398 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:30.398 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.398 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:30.398 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555552+0000", 2024-04-24T15:33:30.398 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:30.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.398 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.398 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:30.398 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555552+0000", 2024-04-24T15:33:30.399 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:30.399 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.399 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.399 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:30.399 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555552+0000", 2024-04-24T15:33:30.399 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:30.399 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.399 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.399 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:30.399 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555554+0000", 2024-04-24T15:33:30.399 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2170000000000001e-06 2024-04-24T15:33:30.399 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.399 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.399 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:30.400 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555562+0000", 2024-04-24T15:33:30.400 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8779999999999998e-06 2024-04-24T15:33:30.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.400 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.400 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:30.400 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555565+0000", 2024-04-24T15:33:30.400 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4230000000000001e-06 2024-04-24T15:33:30.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.400 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.400 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:30.400 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589143+0000", 2024-04-24T15:33:30.400 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033577382000000003 2024-04-24T15:33:30.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.400 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.401 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:30.401 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589185+0000", 2024-04-24T15:33:30.401 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2206999999999998e-05 2024-04-24T15:33:30.401 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.401 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.401 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:30.401 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589238+0000", 2024-04-24T15:33:30.401 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3211000000000003e-05 2024-04-24T15:33:30.401 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:30.401 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:30.401 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:30.401 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.401 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.401 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63809 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1769472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:30.401 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555569+0000", 2024-04-24T15:33:30.402 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 10.752068744000001, 2024-04-24T15:33:30.402 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.03378068, 2024-04-24T15:33:30.402 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:30.402 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:30.402 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:30.402 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:30.402 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:30.402 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63809 2024-04-24T15:33:30.402 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.402 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:30.402 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.402 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:30.402 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555569+0000", 2024-04-24T15:33:30.402 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:30.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.403 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.403 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:30.403 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555569+0000", 2024-04-24T15:33:30.403 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:30.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.403 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.403 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:30.403 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555569+0000", 2024-04-24T15:33:30.403 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:30.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.403 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.403 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:30.403 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555571+0000", 2024-04-24T15:33:30.403 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0889999999999998e-06 2024-04-24T15:33:30.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.404 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.404 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:30.404 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555577+0000", 2024-04-24T15:33:30.404 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4149999999999998e-06 2024-04-24T15:33:30.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.404 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.404 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:30.404 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555580+0000", 2024-04-24T15:33:30.404 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2399999999999999e-06 2024-04-24T15:33:30.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.404 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.404 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:30.404 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589258+0000", 2024-04-24T15:33:30.404 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033677817999999998 2024-04-24T15:33:30.405 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.405 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.405 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:30.405 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589300+0000", 2024-04-24T15:33:30.405 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2317999999999997e-05 2024-04-24T15:33:30.405 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.405 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.405 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:30.405 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589350+0000", 2024-04-24T15:33:30.405 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0111000000000002e-05 2024-04-24T15:33:30.405 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:30.405 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:30.405 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:30.405 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.405 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.406 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63810 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1851392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:30.406 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555584+0000", 2024-04-24T15:33:30.406 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 10.752054274000001, 2024-04-24T15:33:30.406 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033886251999999999, 2024-04-24T15:33:30.406 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:30.406 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:30.406 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:30.406 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:30.406 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:30.406 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63810 2024-04-24T15:33:30.406 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.406 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:30.407 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.407 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:30.407 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555584+0000", 2024-04-24T15:33:30.407 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:30.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.407 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.407 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:30.407 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555584+0000", 2024-04-24T15:33:30.407 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:30.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.407 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.407 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:30.407 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555584+0000", 2024-04-24T15:33:30.407 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:30.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.408 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.408 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:30.408 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555586+0000", 2024-04-24T15:33:30.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0899999999999999e-06 2024-04-24T15:33:30.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.408 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.408 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:30.408 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555591+0000", 2024-04-24T15:33:30.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5260000000000001e-06 2024-04-24T15:33:30.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.408 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.408 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:30.408 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555595+0000", 2024-04-24T15:33:30.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.41e-06 2024-04-24T15:33:30.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.409 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.409 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:30.409 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589377+0000", 2024-04-24T15:33:30.409 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033781868 2024-04-24T15:33:30.409 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.409 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.409 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:30.409 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589418+0000", 2024-04-24T15:33:30.409 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1461000000000002e-05 2024-04-24T15:33:30.409 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.409 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.409 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:30.409 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589470+0000", 2024-04-24T15:33:30.409 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2194999999999997e-05 2024-04-24T15:33:30.409 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:30.410 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:30.410 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:30.410 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.410 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.410 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63811 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1933312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:30.410 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555599+0000", 2024-04-24T15:33:30.410 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 10.752039464999999, 2024-04-24T15:33:30.410 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033987493000000001, 2024-04-24T15:33:30.410 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:30.410 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:30.410 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:30.410 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:30.410 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:30.410 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63811 2024-04-24T15:33:30.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.411 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:30.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:30.411 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555599+0000", 2024-04-24T15:33:30.411 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:30.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:30.412 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555599+0000", 2024-04-24T15:33:30.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:30.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:30.412 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555598+0000", 2024-04-24T15:33:30.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:30.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:30.413 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555600+0000", 2024-04-24T15:33:30.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.04e-06 2024-04-24T15:33:30.413 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.413 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.413 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:30.413 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555606+0000", 2024-04-24T15:33:30.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4199999999999998e-06 2024-04-24T15:33:30.413 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.413 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.413 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:30.413 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555616+0000", 2024-04-24T15:33:30.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0054000000000001e-05 2024-04-24T15:33:30.413 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.413 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.413 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:30.414 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589493+0000", 2024-04-24T15:33:30.414 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033877009999999999 2024-04-24T15:33:30.414 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.414 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.414 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:30.414 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589536+0000", 2024-04-24T15:33:30.414 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2855999999999998e-05 2024-04-24T15:33:30.414 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.414 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.414 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:30.414 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589586+0000", 2024-04-24T15:33:30.414 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0420999999999997e-05 2024-04-24T15:33:30.414 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:30.414 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:30.415 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:30.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.415 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.415 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63812 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 2015232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:30.415 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555620+0000", 2024-04-24T15:33:30.415 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 10.752017827, 2024-04-24T15:33:30.415 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034081696000000002, 2024-04-24T15:33:30.415 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:30.415 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:30.415 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:30.415 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:30.415 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:30.415 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63812 2024-04-24T15:33:30.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.416 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:30.416 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.416 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:30.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555620+0000", 2024-04-24T15:33:30.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:30.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.416 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.416 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:30.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555620+0000", 2024-04-24T15:33:30.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:30.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.416 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.416 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:30.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555620+0000", 2024-04-24T15:33:30.417 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:30.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.417 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.417 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:30.417 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555625+0000", 2024-04-24T15:33:30.417 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2859999999999999e-06 2024-04-24T15:33:30.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.417 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.417 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:30.417 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555633+0000", 2024-04-24T15:33:30.417 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8879999999999997e-06 2024-04-24T15:33:30.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.417 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.417 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:30.417 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555637+0000", 2024-04-24T15:33:30.417 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9500000000000003e-06 2024-04-24T15:33:30.418 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.418 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.418 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:30.418 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589604+0000", 2024-04-24T15:33:30.418 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033966478000000001 2024-04-24T15:33:30.418 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.418 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.418 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:30.418 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589650+0000", 2024-04-24T15:33:30.418 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6721000000000002e-05 2024-04-24T15:33:30.418 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:30.418 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:30.419 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:30.419 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589702+0000", 2024-04-24T15:33:30.419 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1709000000000002e-05 2024-04-24T15:33:30.419 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:30.419 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:30.419 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:30.419 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:30.419 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:30.419 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:30.421 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-24T15:33:30.421 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:33:30.420+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:33:30.502 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-24T15:33:30.521 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:30.521+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:30.587 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:30.587 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:33:30.587 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-04-24T15:33:30.587 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:30.598 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-24T15:33:30.622 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:30.622+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:30.645 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-04-24T15:33:30.664 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-04-24T15:33:30.682 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-04-24T15:33:30.700 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-04-24T15:33:30.719 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-04-24T15:33:30.722 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:33:30.723+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:33:30.725 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:30.726 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:30.736 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-24T15:33:30.739 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-04-24T15:33:30.823 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:33:30.824+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:33:30.858 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:30.858 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:30.868 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-24T15:33:30.915 INFO:teuthology.orchestra.run.smithi096.stderr:set 4.9 pg_upmap_items mapping to [7->2,6->1] 2024-04-24T15:33:30.924 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:33:30.924+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:33:30.959 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-04-24T15:33:30.960 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T15:33:30.960 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T15:33:30.960 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T15:33:30.960 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T15:33:30.960 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T15:33:30.960 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T15:33:30.960 INFO:teuthology.orchestra.run.smithi096.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-04-24T15:33:30.960 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T15:33:30.962 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-24T15:33:30.963 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-24T15:33:31.012 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:31.012 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:31.023 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-24T15:33:31.024 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:31.024+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:31.125 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:31.125+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:31.146 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:31.146 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:31.156 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-24T15:33:31.227 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:33:31.227+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:33:31.279 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:31.279 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:31.289 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-24T15:33:31.328 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:31.328+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:31.411 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:31.412 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:31.421 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-24T15:33:31.429 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:31.429+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:31.529 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:33:31.530+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:33:31.544 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:31.544 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:31.554 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-24T15:33:31.630 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:33:31.630+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:33:31.677 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:31.677 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:31.687 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-24T15:33:31.730 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:33:31.730+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:33:31.831 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:33:31.832+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:33:31.862 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:31.862 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:31.862 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-24T15:33:31.932 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:31.932+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:31.937 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-24T15:33:31.990 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:31.990 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:32.000 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-24T15:33:32.043 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:32.044+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:32.125 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:32.125 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:32.133 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:33:32.134+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:33:32.135 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-24T15:33:32.234 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:32.235+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:32.261 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:32.261 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:32.271 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-24T15:33:32.356 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:33:32.356+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:33:32.434 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:32.434 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:32.436 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:33:32.436+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:33:32.444 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-24T15:33:32.537 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:33:32.538+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:33:32.606 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:32.606 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:32.616 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-24T15:33:32.638 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:32.639+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:32.738 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:32.738 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:32.739 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:33:32.740+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:33:32.747 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-24T15:33:32.840 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:32.841+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:32.870 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:32.870 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:32.879 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-24T15:33:32.921 INFO:teuthology.orchestra.run.smithi096.stderr:nodeep-scrub is set 2024-04-24T15:33:32.941 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:32.942+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:32.963 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-24T15:33:33.019 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:33.019 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:33.031 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-24T15:33:33.072 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:33.072+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:33.128 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-04-24T15:33:33.142 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:33:33.143+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:33:33.146 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-04-24T15:33:33.164 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-04-24T15:33:33.182 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-04-24T15:33:33.200 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:33.200 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:33:33.200 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-04-24T15:33:33.200 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:33.201 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-04-24T15:33:33.210 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-24T15:33:33.220 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-04-24T15:33:33.241 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-04-24T15:33:33.241 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T15:33:33.241 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T15:33:33.241 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T15:33:33.241 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T15:33:33.241 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T15:33:33.241 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T15:33:33.242 INFO:teuthology.orchestra.run.smithi096.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-04-24T15:33:33.242 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T15:33:33.243 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:33.243+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:33.244 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-24T15:33:33.244 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-24T15:33:33.334 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:33.335 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:33:33.335 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-04-24T15:33:33.335 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-04-24T15:33:33.335 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:33.344 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-24T15:33:33.344 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:33.344+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:33.444 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:33:33.446+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:33:33.508 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:33.509 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-04-24T15:33:33.509 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-04-24T15:33:33.509 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-04-24T15:33:33.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.509 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:2522 4.6 4:60ceedf0:::rbd_data.114a1f60cb66.000000000000011d:head [set-alloc-hint object_size 262144 write_size 262144,write 110592~2048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-24T15:33:33.509 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:23.088240+0000", 2024-04-24T15:33:33.509 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 70.420648348, 2024-04-24T15:33:33.509 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.988232797, 2024-04-24T15:33:33.509 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:33.509 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:33.509 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:33.509 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:33.509 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:33.509 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2522 2024-04-24T15:33:33.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.510 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:33.510 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.510 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:33.510 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088240+0000", 2024-04-24T15:33:33.510 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:33.510 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.510 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.510 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:33.510 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088240+0000", 2024-04-24T15:33:33.510 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:33.510 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.510 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.510 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:33.510 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088238+0000", 2024-04-24T15:33:33.510 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999981 2024-04-24T15:33:33.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.511 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:33.511 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088254+0000", 2024-04-24T15:33:33.511 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5562000000000001e-05 2024-04-24T15:33:33.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.511 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:33.511 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088275+0000", 2024-04-24T15:33:33.511 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1256000000000001e-05 2024-04-24T15:33:33.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.511 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:33.511 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088283+0000", 2024-04-24T15:33:33.511 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3219999999999993e-06 2024-04-24T15:33:33.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.512 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.512 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:33.512 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088351+0000", 2024-04-24T15:33:33.512 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7595000000000006e-05 2024-04-24T15:33:33.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.512 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.512 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-04-24T15:33:33.512 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088362+0000", 2024-04-24T15:33:33.512 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1764e-05 2024-04-24T15:33:33.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.512 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.512 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:33.512 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.073748+0000", 2024-04-24T15:33:33.512 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98538541899999998 2024-04-24T15:33:33.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.513 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:33.513 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.073925+0000", 2024-04-24T15:33:33.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017747600000000001 2024-04-24T15:33:33.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.513 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-24T15:33:33.513 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.074092+0000", 2024-04-24T15:33:33.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000166622 2024-04-24T15:33:33.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.513 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:33:33.513 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.074774+0000", 2024-04-24T15:33:33.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00068225000000000002 2024-04-24T15:33:33.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.514 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.514 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:33:33.514 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.076412+0000", 2024-04-24T15:33:33.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001638189 2024-04-24T15:33:33.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.514 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.514 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:33.514 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.076446+0000", 2024-04-24T15:33:33.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3587000000000002e-05 2024-04-24T15:33:33.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.514 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.514 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:33.514 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.076473+0000", 2024-04-24T15:33:33.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6886999999999999e-05 2024-04-24T15:33:33.514 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:33.514 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:33.515 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:33.515 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.515 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.515 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43121 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 5865472~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:33.515 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094178+0000", 2024-04-24T15:33:33.515 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.414710665000001, 2024-04-24T15:33:33.515 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032860598999999997, 2024-04-24T15:33:33.515 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:33.515 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:33.515 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:33.515 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:33.515 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:33.515 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43121 2024-04-24T15:33:33.515 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.515 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:33.515 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.516 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:33.516 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094178+0000", 2024-04-24T15:33:33.516 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:33.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.516 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.516 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:33.516 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094178+0000", 2024-04-24T15:33:33.516 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:33.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.516 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.516 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:33.516 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094178+0000", 2024-04-24T15:33:33.516 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:33.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.516 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.517 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:33.517 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094180+0000", 2024-04-24T15:33:33.517 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9989999999999998e-06 2024-04-24T15:33:33.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.517 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.517 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:33.517 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094185+0000", 2024-04-24T15:33:33.517 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.541e-06 2024-04-24T15:33:33.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.517 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.517 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:33.517 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094189+0000", 2024-04-24T15:33:33.517 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0509999999999998e-06 2024-04-24T15:33:33.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.517 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.518 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:33.518 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.126877+0000", 2024-04-24T15:33:33.518 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032687473000000002 2024-04-24T15:33:33.518 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.518 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.518 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:33.518 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.126952+0000", 2024-04-24T15:33:33.518 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5924e-05 2024-04-24T15:33:33.518 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.518 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.518 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:33.518 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127038+0000", 2024-04-24T15:33:33.518 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5908999999999994e-05 2024-04-24T15:33:33.518 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:33.518 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:33.519 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:33.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.519 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.519 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43122 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 6291456~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:33.519 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094193+0000", 2024-04-24T15:33:33.519 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.414695421000001, 2024-04-24T15:33:33.519 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032978383999999999, 2024-04-24T15:33:33.519 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:33.519 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:33.519 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:33.519 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:33.519 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:33.519 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43122 2024-04-24T15:33:33.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.519 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:33.520 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.520 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:33.520 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094193+0000", 2024-04-24T15:33:33.520 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:33.520 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.520 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.520 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:33.520 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094193+0000", 2024-04-24T15:33:33.520 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:33.520 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.520 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.520 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:33.520 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094193+0000", 2024-04-24T15:33:33.520 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:33.520 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.520 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.521 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:33.521 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094195+0000", 2024-04-24T15:33:33.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3269999999999999e-06 2024-04-24T15:33:33.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.521 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.521 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:33.521 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094201+0000", 2024-04-24T15:33:33.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9850000000000003e-06 2024-04-24T15:33:33.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.521 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.521 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:33.521 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094205+0000", 2024-04-24T15:33:33.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8430000000000003e-06 2024-04-24T15:33:33.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.521 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.521 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:33.522 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127068+0000", 2024-04-24T15:33:33.522 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032863305000000002 2024-04-24T15:33:33.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.522 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:33.522 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127113+0000", 2024-04-24T15:33:33.522 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4412000000000001e-05 2024-04-24T15:33:33.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.522 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:33.522 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127171+0000", 2024-04-24T15:33:33.522 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8807999999999997e-05 2024-04-24T15:33:33.522 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:33.522 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:33.522 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:33.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.523 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43123 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 6717440~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:33.523 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094209+0000", 2024-04-24T15:33:33.523 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.414679254999999, 2024-04-24T15:33:33.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033144583999999998, 2024-04-24T15:33:33.523 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:33.523 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:33.523 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:33.523 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:33.523 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:33.523 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43123 2024-04-24T15:33:33.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.523 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:33.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.523 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:33.524 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094209+0000", 2024-04-24T15:33:33.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:33.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.524 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:33.524 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094209+0000", 2024-04-24T15:33:33.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:33.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.524 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:33.524 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094209+0000", 2024-04-24T15:33:33.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:33.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.524 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:33.524 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094211+0000", 2024-04-24T15:33:33.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2089999999999999e-06 2024-04-24T15:33:33.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.525 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:33.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094217+0000", 2024-04-24T15:33:33.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6799999999999998e-06 2024-04-24T15:33:33.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.525 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:33.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094220+0000", 2024-04-24T15:33:33.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.444e-06 2024-04-24T15:33:33.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.525 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:33.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127190+0000", 2024-04-24T15:33:33.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032969471 2024-04-24T15:33:33.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:33.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127269+0000", 2024-04-24T15:33:33.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9246999999999995e-05 2024-04-24T15:33:33.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:33.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127354+0000", 2024-04-24T15:33:33.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4853999999999998e-05 2024-04-24T15:33:33.526 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:33.526 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:33.526 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:33.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.527 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43124 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 7143424~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:33.527 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094224+0000", 2024-04-24T15:33:33.527 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.414664282, 2024-04-24T15:33:33.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033271955999999998, 2024-04-24T15:33:33.527 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:33.527 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:33.527 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:33.527 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:33.527 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:33.527 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43124 2024-04-24T15:33:33.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.527 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:33.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.527 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:33.527 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094224+0000", 2024-04-24T15:33:33.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:33.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:33.528 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094224+0000", 2024-04-24T15:33:33.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:33.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:33.528 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094224+0000", 2024-04-24T15:33:33.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:33.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:33.528 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094226+0000", 2024-04-24T15:33:33.529 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9039999999999999e-06 2024-04-24T15:33:33.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.529 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.529 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:33.529 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094233+0000", 2024-04-24T15:33:33.529 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7970000000000003e-06 2024-04-24T15:33:33.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.529 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.529 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:33.529 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094236+0000", 2024-04-24T15:33:33.529 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6339999999999999e-06 2024-04-24T15:33:33.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.529 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.529 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:33.529 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127390+0000", 2024-04-24T15:33:33.530 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.03315353 2024-04-24T15:33:33.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.530 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.530 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:33.530 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127439+0000", 2024-04-24T15:33:33.530 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8890000000000001e-05 2024-04-24T15:33:33.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.530 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.530 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:33.530 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127496+0000", 2024-04-24T15:33:33.530 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7476999999999999e-05 2024-04-24T15:33:33.530 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:33.530 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:33.530 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:33.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.530 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.531 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43125 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 7569408~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:33.531 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094240+0000", 2024-04-24T15:33:33.531 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.414648195000002, 2024-04-24T15:33:33.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033446278000000003, 2024-04-24T15:33:33.531 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:33.531 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:33.531 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:33.531 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:33.531 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:33.531 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43125 2024-04-24T15:33:33.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.531 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:33.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:33.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094240+0000", 2024-04-24T15:33:33.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:33.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:33.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094240+0000", 2024-04-24T15:33:33.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:33.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:33.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094240+0000", 2024-04-24T15:33:33.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:33.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:33.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094247+0000", 2024-04-24T15:33:33.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6209999999999998e-06 2024-04-24T15:33:33.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:33.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094253+0000", 2024-04-24T15:33:33.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0739999999999998e-06 2024-04-24T15:33:33.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:33.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094256+0000", 2024-04-24T15:33:33.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7699999999999999e-06 2024-04-24T15:33:33.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:33.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127526+0000", 2024-04-24T15:33:33.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033269703999999997 2024-04-24T15:33:33.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:33.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127603+0000", 2024-04-24T15:33:33.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6964000000000004e-05 2024-04-24T15:33:33.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:33.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127687+0000", 2024-04-24T15:33:33.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3431999999999997e-05 2024-04-24T15:33:33.534 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:33.534 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:33.534 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:33.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.534 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43126 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 7995392~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:33.534 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094261+0000", 2024-04-24T15:33:33.535 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.414627692, 2024-04-24T15:33:33.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033555201999999999, 2024-04-24T15:33:33.535 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:33.535 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:33.535 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:33.535 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:33.535 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:33.535 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43126 2024-04-24T15:33:33.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.535 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:33.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:33.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094261+0000", 2024-04-24T15:33:33.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:33.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:33.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094261+0000", 2024-04-24T15:33:33.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:33.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:33.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094260+0000", 2024-04-24T15:33:33.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:33.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:33.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094263+0000", 2024-04-24T15:33:33.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2579999999999998e-06 2024-04-24T15:33:33.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:33.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094269+0000", 2024-04-24T15:33:33.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.823e-06 2024-04-24T15:33:33.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:33.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094272+0000", 2024-04-24T15:33:33.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8999999999999999e-06 2024-04-24T15:33:33.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:33.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127720+0000", 2024-04-24T15:33:33.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033447521000000001 2024-04-24T15:33:33.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.538 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:33.538 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127765+0000", 2024-04-24T15:33:33.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5352000000000002e-05 2024-04-24T15:33:33.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.538 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:33.538 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127816+0000", 2024-04-24T15:33:33.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0689999999999997e-05 2024-04-24T15:33:33.538 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:33.538 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:33.538 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:33.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.538 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63800 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1032192~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:33.538 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555407+0000", 2024-04-24T15:33:33.539 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.953481706, 2024-04-24T15:33:33.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032809030000000003, 2024-04-24T15:33:33.539 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:33.539 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:33.539 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:33.539 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:33.539 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:33.539 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63800 2024-04-24T15:33:33.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.539 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:33.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.539 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:33.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555407+0000", 2024-04-24T15:33:33.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:33.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.540 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:33.540 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555407+0000", 2024-04-24T15:33:33.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:33.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.540 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:33.540 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555406+0000", 2024-04-24T15:33:33.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:33.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.540 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:33.540 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555409+0000", 2024-04-24T15:33:33.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.706e-06 2024-04-24T15:33:33.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.540 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:33.541 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555416+0000", 2024-04-24T15:33:33.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1609999999999997e-06 2024-04-24T15:33:33.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.541 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:33.541 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555420+0000", 2024-04-24T15:33:33.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0369999999999996e-06 2024-04-24T15:33:33.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.541 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:33.541 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588088+0000", 2024-04-24T15:33:33.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032668228000000001 2024-04-24T15:33:33.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.541 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:33.541 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588143+0000", 2024-04-24T15:33:33.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4852999999999998e-05 2024-04-24T15:33:33.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.542 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.542 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:33.542 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588216+0000", 2024-04-24T15:33:33.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2467999999999998e-05 2024-04-24T15:33:33.542 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:33.542 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:33.542 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:33.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.542 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.542 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63801 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1114112~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:33.542 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555425+0000", 2024-04-24T15:33:33.542 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.953463799, 2024-04-24T15:33:33.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032924691999999998, 2024-04-24T15:33:33.543 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:33.543 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:33.543 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:33.543 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:33.543 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:33.543 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63801 2024-04-24T15:33:33.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.543 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:33.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.543 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:33.543 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555425+0000", 2024-04-24T15:33:33.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:33.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.543 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:33.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555425+0000", 2024-04-24T15:33:33.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:33.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.544 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:33.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555424+0000", 2024-04-24T15:33:33.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:33.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.544 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:33.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555427+0000", 2024-04-24T15:33:33.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2350000000000002e-06 2024-04-24T15:33:33.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.544 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:33.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555432+0000", 2024-04-24T15:33:33.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.434e-06 2024-04-24T15:33:33.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:33.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555435+0000", 2024-04-24T15:33:33.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2009999999999999e-06 2024-04-24T15:33:33.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:33.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588237+0000", 2024-04-24T15:33:33.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032801508 2024-04-24T15:33:33.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:33.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588281+0000", 2024-04-24T15:33:33.546 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4100999999999997e-05 2024-04-24T15:33:33.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.546 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:33.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588349+0000", 2024-04-24T15:33:33.546 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8551000000000001e-05 2024-04-24T15:33:33.546 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:33.546 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:33.546 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:33.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.546 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63802 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1196032~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:33.547 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555439+0000", 2024-04-24T15:33:33.547 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.953449278000001, 2024-04-24T15:33:33.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033057744999999999, 2024-04-24T15:33:33.547 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:33.547 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:33.547 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:33.547 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:33.547 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:33.547 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63802 2024-04-24T15:33:33.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.547 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:33.547 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.547 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:33.547 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555439+0000", 2024-04-24T15:33:33.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:33.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.548 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:33.548 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555439+0000", 2024-04-24T15:33:33.548 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:33.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.548 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:33.548 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555439+0000", 2024-04-24T15:33:33.548 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:33.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.548 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:33.548 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555445+0000", 2024-04-24T15:33:33.548 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4019999999999999e-06 2024-04-24T15:33:33.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.549 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:33.549 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555452+0000", 2024-04-24T15:33:33.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8600000000000004e-06 2024-04-24T15:33:33.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.549 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:33.549 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555461+0000", 2024-04-24T15:33:33.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0499999999999997e-06 2024-04-24T15:33:33.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.549 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:33.549 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588378+0000", 2024-04-24T15:33:33.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032916527000000001 2024-04-24T15:33:33.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.550 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:33.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588423+0000", 2024-04-24T15:33:33.550 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5092999999999997e-05 2024-04-24T15:33:33.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.550 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:33.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588497+0000", 2024-04-24T15:33:33.550 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4140999999999994e-05 2024-04-24T15:33:33.550 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:33.550 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:33.550 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:33.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.551 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63803 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1277952~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:33.551 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555466+0000", 2024-04-24T15:33:33.551 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.953422333000001, 2024-04-24T15:33:33.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033162297, 2024-04-24T15:33:33.551 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:33.551 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:33.551 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:33.551 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:33.551 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:33.551 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63803 2024-04-24T15:33:33.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.551 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:33.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.551 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:33.551 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555466+0000", 2024-04-24T15:33:33.552 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:33.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.552 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:33.552 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555466+0000", 2024-04-24T15:33:33.552 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:33.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.552 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:33.552 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555466+0000", 2024-04-24T15:33:33.552 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:33.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.552 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:33.552 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555468+0000", 2024-04-24T15:33:33.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4040000000000002e-06 2024-04-24T15:33:33.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:33.553 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555475+0000", 2024-04-24T15:33:33.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9480000000000002e-06 2024-04-24T15:33:33.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:33.553 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555479+0000", 2024-04-24T15:33:33.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0289999999999999e-06 2024-04-24T15:33:33.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:33.553 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588516+0000", 2024-04-24T15:33:33.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033037094000000003 2024-04-24T15:33:33.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.554 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:33.554 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588562+0000", 2024-04-24T15:33:33.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6215999999999999e-05 2024-04-24T15:33:33.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.554 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:33.554 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588628+0000", 2024-04-24T15:33:33.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5971999999999998e-05 2024-04-24T15:33:33.554 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:33.554 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:33.554 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:33.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.555 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63804 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1359872~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:33.555 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555483+0000", 2024-04-24T15:33:33.555 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.953405166, 2024-04-24T15:33:33.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033287068000000003, 2024-04-24T15:33:33.555 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:33.555 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:33.555 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:33.555 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:33.555 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:33.555 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63804 2024-04-24T15:33:33.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.555 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:33.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.555 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:33.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555483+0000", 2024-04-24T15:33:33.556 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:33.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.556 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:33.556 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555483+0000", 2024-04-24T15:33:33.556 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:33.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.556 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:33.556 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555483+0000", 2024-04-24T15:33:33.556 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:33.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.556 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:33.557 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555485+0000", 2024-04-24T15:33:33.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0710000000000002e-06 2024-04-24T15:33:33.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:33.557 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555498+0000", 2024-04-24T15:33:33.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2748999999999999e-05 2024-04-24T15:33:33.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:33.557 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555502+0000", 2024-04-24T15:33:33.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0219999999999998e-06 2024-04-24T15:33:33.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:33.558 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588657+0000", 2024-04-24T15:33:33.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033154683999999997 2024-04-24T15:33:33.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.558 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.558 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:33.558 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588702+0000", 2024-04-24T15:33:33.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5809000000000001e-05 2024-04-24T15:33:33.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.558 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.558 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:33.558 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588770+0000", 2024-04-24T15:33:33.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8056e-05 2024-04-24T15:33:33.558 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:33.558 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:33.558 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:33.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.559 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63805 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1441792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:33.559 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555508+0000", 2024-04-24T15:33:33.559 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.953380317000001, 2024-04-24T15:33:33.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033375952, 2024-04-24T15:33:33.559 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:33.559 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:33.559 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:33.559 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:33.559 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:33.559 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63805 2024-04-24T15:33:33.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.559 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:33.560 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.560 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:33.560 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555508+0000", 2024-04-24T15:33:33.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:33.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.560 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.560 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:33.560 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555508+0000", 2024-04-24T15:33:33.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:33.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.560 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.560 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:33.560 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555508+0000", 2024-04-24T15:33:33.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:33.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.561 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:33.561 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555510+0000", 2024-04-24T15:33:33.561 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.424e-06 2024-04-24T15:33:33.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.561 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:33.561 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555516+0000", 2024-04-24T15:33:33.561 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3730000000000001e-06 2024-04-24T15:33:33.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.561 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:33.561 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555519+0000", 2024-04-24T15:33:33.561 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.039e-06 2024-04-24T15:33:33.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.562 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.562 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:33.562 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588789+0000", 2024-04-24T15:33:33.562 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0332702 2024-04-24T15:33:33.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.562 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.562 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:33.562 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588833+0000", 2024-04-24T15:33:33.562 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4660999999999998e-05 2024-04-24T15:33:33.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.562 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.562 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:33.562 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588884+0000", 2024-04-24T15:33:33.562 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0661999999999997e-05 2024-04-24T15:33:33.562 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:33.563 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:33.563 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:33.563 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.563 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.563 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63806 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1523712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:33.563 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555523+0000", 2024-04-24T15:33:33.563 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.953365736, 2024-04-24T15:33:33.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033479653999999998, 2024-04-24T15:33:33.563 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:33.563 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:33.563 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:33.563 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:33.563 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:33.563 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63806 2024-04-24T15:33:33.563 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.563 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:33.564 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.564 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:33.564 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555523+0000", 2024-04-24T15:33:33.564 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:33.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.564 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.564 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:33.564 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555523+0000", 2024-04-24T15:33:33.564 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:33.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.564 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.564 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:33.564 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555522+0000", 2024-04-24T15:33:33.564 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:33.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.565 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.565 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:33.565 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555524+0000", 2024-04-24T15:33:33.565 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0700000000000001e-06 2024-04-24T15:33:33.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.565 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.565 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:33.565 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555530+0000", 2024-04-24T15:33:33.565 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5450000000000003e-06 2024-04-24T15:33:33.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.565 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.565 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:33.565 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555533+0000", 2024-04-24T15:33:33.565 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2200000000000001e-06 2024-04-24T15:33:33.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.566 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.566 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:33.566 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588908+0000", 2024-04-24T15:33:33.566 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033374556 2024-04-24T15:33:33.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.566 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.566 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:33.566 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588950+0000", 2024-04-24T15:33:33.566 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2361999999999999e-05 2024-04-24T15:33:33.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.566 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.566 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:33.566 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589002+0000", 2024-04-24T15:33:33.566 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2224e-05 2024-04-24T15:33:33.566 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:33.567 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:33.567 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:33.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.567 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.567 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63807 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1605632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:33.567 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555537+0000", 2024-04-24T15:33:33.567 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.953351221, 2024-04-24T15:33:33.567 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033577227000000001, 2024-04-24T15:33:33.567 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:33.567 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:33.567 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:33.567 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:33.567 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:33.567 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63807 2024-04-24T15:33:33.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.567 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:33.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.568 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:33.568 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555537+0000", 2024-04-24T15:33:33.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:33.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.568 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:33.568 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555537+0000", 2024-04-24T15:33:33.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:33.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.568 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:33.568 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555537+0000", 2024-04-24T15:33:33.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:33.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.569 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.569 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:33.569 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555539+0000", 2024-04-24T15:33:33.569 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9769999999999999e-06 2024-04-24T15:33:33.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.569 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.569 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:33.569 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555544+0000", 2024-04-24T15:33:33.569 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.468e-06 2024-04-24T15:33:33.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.569 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.569 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:33.569 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555548+0000", 2024-04-24T15:33:33.569 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4319999999999999e-06 2024-04-24T15:33:33.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.569 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.570 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:33.570 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589021+0000", 2024-04-24T15:33:33.570 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033473655999999997 2024-04-24T15:33:33.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.570 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.570 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:33.570 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589064+0000", 2024-04-24T15:33:33.570 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2397000000000001e-05 2024-04-24T15:33:33.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.570 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.570 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:33.570 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589114+0000", 2024-04-24T15:33:33.570 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0611e-05 2024-04-24T15:33:33.570 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:33.570 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:33.571 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:33.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.571 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.571 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63808 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1687552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:33.571 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555552+0000", 2024-04-24T15:33:33.571 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.953336529, 2024-04-24T15:33:33.571 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033686019999999997, 2024-04-24T15:33:33.571 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:33.571 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:33.571 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:33.571 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:33.571 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:33.571 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63808 2024-04-24T15:33:33.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.572 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:33.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.572 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:33.572 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555552+0000", 2024-04-24T15:33:33.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:33.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.572 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:33.572 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555552+0000", 2024-04-24T15:33:33.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:33.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.572 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:33.572 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555552+0000", 2024-04-24T15:33:33.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:33.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.573 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:33.573 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555554+0000", 2024-04-24T15:33:33.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2170000000000001e-06 2024-04-24T15:33:33.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.573 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:33.573 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555562+0000", 2024-04-24T15:33:33.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8779999999999998e-06 2024-04-24T15:33:33.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.574 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:33.574 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555565+0000", 2024-04-24T15:33:33.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4230000000000001e-06 2024-04-24T15:33:33.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.574 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:33.574 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589143+0000", 2024-04-24T15:33:33.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033577382000000003 2024-04-24T15:33:33.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.574 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:33.574 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589185+0000", 2024-04-24T15:33:33.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2206999999999998e-05 2024-04-24T15:33:33.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:33.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589238+0000", 2024-04-24T15:33:33.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3211000000000003e-05 2024-04-24T15:33:33.575 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:33.575 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:33.575 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:33.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.575 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63809 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1769472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:33.575 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555569+0000", 2024-04-24T15:33:33.575 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.953318987999999, 2024-04-24T15:33:33.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.03378068, 2024-04-24T15:33:33.575 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:33.576 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:33.576 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:33.576 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:33.576 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:33.576 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63809 2024-04-24T15:33:33.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.576 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:33.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:33.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555569+0000", 2024-04-24T15:33:33.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:33.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:33.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555569+0000", 2024-04-24T15:33:33.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:33.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:33.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555569+0000", 2024-04-24T15:33:33.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:33.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:33.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555571+0000", 2024-04-24T15:33:33.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0889999999999998e-06 2024-04-24T15:33:33.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:33.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555577+0000", 2024-04-24T15:33:33.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4149999999999998e-06 2024-04-24T15:33:33.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.578 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.578 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:33.578 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555580+0000", 2024-04-24T15:33:33.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2399999999999999e-06 2024-04-24T15:33:33.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.578 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.578 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:33.578 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589258+0000", 2024-04-24T15:33:33.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033677817999999998 2024-04-24T15:33:33.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.578 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.579 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:33.579 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589300+0000", 2024-04-24T15:33:33.579 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2317999999999997e-05 2024-04-24T15:33:33.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.579 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:33.579 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589350+0000", 2024-04-24T15:33:33.579 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0111000000000002e-05 2024-04-24T15:33:33.579 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:33.579 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:33.579 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:33.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.579 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63810 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1851392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:33.580 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555584+0000", 2024-04-24T15:33:33.580 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.953304517999999, 2024-04-24T15:33:33.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033886251999999999, 2024-04-24T15:33:33.580 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:33.580 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:33.580 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:33.580 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:33.580 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:33.580 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63810 2024-04-24T15:33:33.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.580 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:33.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:33.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555584+0000", 2024-04-24T15:33:33.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:33.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:33.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555584+0000", 2024-04-24T15:33:33.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:33.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:33.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555584+0000", 2024-04-24T15:33:33.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:33.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:33.582 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555586+0000", 2024-04-24T15:33:33.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0899999999999999e-06 2024-04-24T15:33:33.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.582 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:33.582 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555591+0000", 2024-04-24T15:33:33.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5260000000000001e-06 2024-04-24T15:33:33.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.582 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:33.582 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555595+0000", 2024-04-24T15:33:33.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.41e-06 2024-04-24T15:33:33.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:33.583 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589377+0000", 2024-04-24T15:33:33.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033781868 2024-04-24T15:33:33.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:33.583 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589418+0000", 2024-04-24T15:33:33.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1461000000000002e-05 2024-04-24T15:33:33.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:33.583 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589470+0000", 2024-04-24T15:33:33.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2194999999999997e-05 2024-04-24T15:33:33.585 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:33.585 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:33.585 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:33.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.585 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63811 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1933312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:33.585 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555599+0000", 2024-04-24T15:33:33.585 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.953289709, 2024-04-24T15:33:33.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033987493000000001, 2024-04-24T15:33:33.586 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:33.586 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:33.586 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:33.586 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:33.586 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:33.586 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63811 2024-04-24T15:33:33.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.586 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:33.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:33.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555599+0000", 2024-04-24T15:33:33.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:33.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.587 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:33.587 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555599+0000", 2024-04-24T15:33:33.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:33.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.587 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:33.587 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555598+0000", 2024-04-24T15:33:33.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:33.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.587 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:33.587 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555600+0000", 2024-04-24T15:33:33.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.04e-06 2024-04-24T15:33:33.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:33.588 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555606+0000", 2024-04-24T15:33:33.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4199999999999998e-06 2024-04-24T15:33:33.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:33.588 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555616+0000", 2024-04-24T15:33:33.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0054000000000001e-05 2024-04-24T15:33:33.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:33.588 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589493+0000", 2024-04-24T15:33:33.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033877009999999999 2024-04-24T15:33:33.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.589 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:33.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589536+0000", 2024-04-24T15:33:33.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2855999999999998e-05 2024-04-24T15:33:33.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.589 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:33.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589586+0000", 2024-04-24T15:33:33.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0420999999999997e-05 2024-04-24T15:33:33.589 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:33.589 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:33.589 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:33.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.590 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63812 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 2015232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:33.590 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555620+0000", 2024-04-24T15:33:33.590 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.953268071, 2024-04-24T15:33:33.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034081696000000002, 2024-04-24T15:33:33.590 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:33.590 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:33.590 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:33.590 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:33.590 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:33.590 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63812 2024-04-24T15:33:33.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.590 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:33.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.591 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:33.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555620+0000", 2024-04-24T15:33:33.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:33.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.591 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:33.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555620+0000", 2024-04-24T15:33:33.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:33.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.591 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:33.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555620+0000", 2024-04-24T15:33:33.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:33.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:33.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555625+0000", 2024-04-24T15:33:33.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2859999999999999e-06 2024-04-24T15:33:33.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:33.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555633+0000", 2024-04-24T15:33:33.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8879999999999997e-06 2024-04-24T15:33:33.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:33.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555637+0000", 2024-04-24T15:33:33.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9500000000000003e-06 2024-04-24T15:33:33.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:33.593 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589604+0000", 2024-04-24T15:33:33.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033966478000000001 2024-04-24T15:33:33.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:33.593 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589650+0000", 2024-04-24T15:33:33.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6721000000000002e-05 2024-04-24T15:33:33.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:33.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:33.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:33.593 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589702+0000", 2024-04-24T15:33:33.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1709000000000002e-05 2024-04-24T15:33:33.594 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:33.594 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:33.594 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:33.594 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:33.594 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:33.594 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:33.595 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-24T15:33:33.596 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:33.595+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:33.695 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:33:33.696+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:33:33.759 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:33.759 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:33:33.759 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-04-24T15:33:33.759 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:33.769 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-24T15:33:33.797 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:33.797+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:33.897 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:33.897+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:33.936 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:33.936 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:33:33.936 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-04-24T15:33:33.936 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-04-24T15:33:33.936 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:33.948 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-24T15:33:33.998 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:33.999+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:34.099 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:34.099+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:34.123 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:34.123 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-04-24T15:33:34.123 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-04-24T15:33:34.123 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-04-24T15:33:34.123 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.123 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14043 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2383872~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:34.123 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892347+0000", 2024-04-24T15:33:34.123 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 60.230872251999998, 2024-04-24T15:33:34.123 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038537279000000001, 2024-04-24T15:33:34.123 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:34.124 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:34.124 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:34.124 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:34.124 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:34.124 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14043 2024-04-24T15:33:34.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.124 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:34.124 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.124 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:34.124 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892347+0000", 2024-04-24T15:33:34.124 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:34.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.124 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.124 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:34.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892347+0000", 2024-04-24T15:33:34.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:34.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.125 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:34.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892347+0000", 2024-04-24T15:33:34.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:34.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.125 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:34.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892349+0000", 2024-04-24T15:33:34.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3400000000000001e-06 2024-04-24T15:33:34.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.125 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:34.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892357+0000", 2024-04-24T15:33:34.126 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1540000000000002e-06 2024-04-24T15:33:34.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.126 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.126 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:34.126 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892360+0000", 2024-04-24T15:33:34.126 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5829999999999998e-06 2024-04-24T15:33:34.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.126 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.126 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:34.126 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930795+0000", 2024-04-24T15:33:34.126 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038434590999999997 2024-04-24T15:33:34.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.126 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.126 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:34.127 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930836+0000", 2024-04-24T15:33:34.127 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0827000000000001e-05 2024-04-24T15:33:34.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.127 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.127 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:34.127 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930885+0000", 2024-04-24T15:33:34.127 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9029000000000001e-05 2024-04-24T15:33:34.127 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:34.127 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:34.127 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:34.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.127 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.127 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14044 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2424832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:34.127 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892372+0000", 2024-04-24T15:33:34.127 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 60.230847607999998, 2024-04-24T15:33:34.128 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038624890000000002, 2024-04-24T15:33:34.128 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:34.128 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:34.128 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:34.128 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:34.128 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:34.128 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14044 2024-04-24T15:33:34.128 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.128 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:34.128 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.128 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:34.128 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892372+0000", 2024-04-24T15:33:34.128 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:34.128 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.129 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.129 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:34.129 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892372+0000", 2024-04-24T15:33:34.129 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:34.129 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.129 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.129 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:34.129 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892372+0000", 2024-04-24T15:33:34.129 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:34.129 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.129 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.129 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:34.129 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892374+0000", 2024-04-24T15:33:34.129 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4899999999999999e-06 2024-04-24T15:33:34.129 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.130 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.130 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:34.130 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892380+0000", 2024-04-24T15:33:34.130 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.631e-06 2024-04-24T15:33:34.130 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.130 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.130 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:34.130 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892383+0000", 2024-04-24T15:33:34.130 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7400000000000002e-06 2024-04-24T15:33:34.130 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.130 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.130 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:34.130 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930905+0000", 2024-04-24T15:33:34.130 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038521396999999999 2024-04-24T15:33:34.131 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.131 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:34.131 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930949+0000", 2024-04-24T15:33:34.131 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4153999999999998e-05 2024-04-24T15:33:34.131 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.131 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:34.131 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930997+0000", 2024-04-24T15:33:34.131 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7998999999999999e-05 2024-04-24T15:33:34.131 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:34.131 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:34.131 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:34.131 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.131 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14045 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2465792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:34.132 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892387+0000", 2024-04-24T15:33:34.132 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 60.230832341000003, 2024-04-24T15:33:34.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038720933999999999, 2024-04-24T15:33:34.132 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:34.132 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:34.132 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:34.132 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:34.132 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:34.132 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14045 2024-04-24T15:33:34.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.132 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:34.132 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.132 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:34.132 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892387+0000", 2024-04-24T15:33:34.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:34.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:34.133 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892387+0000", 2024-04-24T15:33:34.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:34.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:34.133 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892387+0000", 2024-04-24T15:33:34.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:34.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:34.133 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892392+0000", 2024-04-24T15:33:34.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0039999999999999e-06 2024-04-24T15:33:34.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.134 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:34.134 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892398+0000", 2024-04-24T15:33:34.134 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0900000000000001e-06 2024-04-24T15:33:34.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.134 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.134 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:34.134 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892402+0000", 2024-04-24T15:33:34.134 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.399e-06 2024-04-24T15:33:34.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.134 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.134 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:34.134 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931018+0000", 2024-04-24T15:33:34.134 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038616551999999998 2024-04-24T15:33:34.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:34.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931059+0000", 2024-04-24T15:33:34.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1205000000000003e-05 2024-04-24T15:33:34.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:34.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931108+0000", 2024-04-24T15:33:34.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8996000000000003e-05 2024-04-24T15:33:34.135 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:34.135 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:34.135 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:34.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.135 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14046 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2506752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:34.136 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892406+0000", 2024-04-24T15:33:34.136 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 60.230813750999999, 2024-04-24T15:33:34.136 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038814507999999998, 2024-04-24T15:33:34.136 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:34.136 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:34.136 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:34.136 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:34.136 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:34.136 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14046 2024-04-24T15:33:34.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.136 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:34.136 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.136 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:34.136 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892406+0000", 2024-04-24T15:33:34.137 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:34.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.137 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.137 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:34.137 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892406+0000", 2024-04-24T15:33:34.137 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:34.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.137 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.137 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:34.137 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892406+0000", 2024-04-24T15:33:34.137 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:34.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.137 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.137 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:34.138 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892413+0000", 2024-04-24T15:33:34.138 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7449999999999995e-06 2024-04-24T15:33:34.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.138 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.138 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:34.138 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892419+0000", 2024-04-24T15:33:34.138 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5790000000000003e-06 2024-04-24T15:33:34.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.138 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.138 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:34.138 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892423+0000", 2024-04-24T15:33:34.138 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8199999999999998e-06 2024-04-24T15:33:34.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.138 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.139 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:34.139 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931128+0000", 2024-04-24T15:33:34.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038705502000000003 2024-04-24T15:33:34.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.139 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.139 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:34.139 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931172+0000", 2024-04-24T15:33:34.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4066999999999997e-05 2024-04-24T15:33:34.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.139 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.139 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:34.139 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931220+0000", 2024-04-24T15:33:34.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8167999999999997e-05 2024-04-24T15:33:34.140 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:34.140 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:34.140 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:34.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.140 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.140 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14047 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2547712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:34.140 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892427+0000", 2024-04-24T15:33:34.140 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 60.230792522000002, 2024-04-24T15:33:34.140 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038904506999999998, 2024-04-24T15:33:34.140 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:34.140 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:34.140 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:34.140 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:34.140 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:34.140 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14047 2024-04-24T15:33:34.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.141 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:34.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.141 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:34.141 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892427+0000", 2024-04-24T15:33:34.141 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:34.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.141 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:34.141 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892427+0000", 2024-04-24T15:33:34.142 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:34.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.142 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.142 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:34.142 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892427+0000", 2024-04-24T15:33:34.142 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:34.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.142 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.142 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:34.142 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892429+0000", 2024-04-24T15:33:34.142 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3099999999999999e-06 2024-04-24T15:33:34.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.142 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.143 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:34.143 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892435+0000", 2024-04-24T15:33:34.143 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5520000000000004e-06 2024-04-24T15:33:34.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.143 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.143 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:34.143 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892438+0000", 2024-04-24T15:33:34.143 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4240000000000002e-06 2024-04-24T15:33:34.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.143 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.143 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:34.143 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931242+0000", 2024-04-24T15:33:34.143 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038804162000000003 2024-04-24T15:33:34.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.144 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.144 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:34.144 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931284+0000", 2024-04-24T15:33:34.144 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1414e-05 2024-04-24T15:33:34.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.144 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.144 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:34.144 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931332+0000", 2024-04-24T15:33:34.144 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8024e-05 2024-04-24T15:33:34.144 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:34.144 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:34.144 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:34.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.144 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.144 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14048 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2588672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:34.145 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892442+0000", 2024-04-24T15:33:34.145 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 60.230777559000003, 2024-04-24T15:33:34.145 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039003132000000003, 2024-04-24T15:33:34.145 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:34.145 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:34.145 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:34.145 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:34.145 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:34.145 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14048 2024-04-24T15:33:34.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.145 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:34.145 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.145 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:34.145 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892442+0000", 2024-04-24T15:33:34.145 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:34.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.146 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.146 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:34.146 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892442+0000", 2024-04-24T15:33:34.146 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:34.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.146 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.146 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:34.146 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892442+0000", 2024-04-24T15:33:34.146 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:34.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.146 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.146 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:34.147 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892444+0000", 2024-04-24T15:33:34.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1169999999999998e-06 2024-04-24T15:33:34.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.147 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:34.147 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892449+0000", 2024-04-24T15:33:34.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1599999999999997e-06 2024-04-24T15:33:34.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.147 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:34.147 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892453+0000", 2024-04-24T15:33:34.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5930000000000002e-06 2024-04-24T15:33:34.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.147 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:34.147 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931352+0000", 2024-04-24T15:33:34.148 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038899361 2024-04-24T15:33:34.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.148 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.148 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:34.148 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931398+0000", 2024-04-24T15:33:34.148 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6315e-05 2024-04-24T15:33:34.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.148 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.148 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:34.148 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931445+0000", 2024-04-24T15:33:34.148 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6950000000000003e-05 2024-04-24T15:33:34.148 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:34.148 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:34.148 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:34.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.148 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.149 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14049 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2629632~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:34.149 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892458+0000", 2024-04-24T15:33:34.149 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 60.230762138999999, 2024-04-24T15:33:34.149 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039103322000000003, 2024-04-24T15:33:34.149 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:34.149 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:34.149 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:34.149 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:34.149 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:34.149 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14049 2024-04-24T15:33:34.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.149 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:34.149 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.149 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:34.150 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892458+0000", 2024-04-24T15:33:34.150 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:34.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.150 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.150 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:34.150 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892458+0000", 2024-04-24T15:33:34.150 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:34.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.150 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.150 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:34.150 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892457+0000", 2024-04-24T15:33:34.150 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:34.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.151 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:34.151 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892459+0000", 2024-04-24T15:33:34.151 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.305e-06 2024-04-24T15:33:34.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.151 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:34.151 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892465+0000", 2024-04-24T15:33:34.151 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4360000000000001e-06 2024-04-24T15:33:34.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.151 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:34.151 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892474+0000", 2024-04-24T15:33:34.151 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7639999999999994e-06 2024-04-24T15:33:34.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.152 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:34.152 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931466+0000", 2024-04-24T15:33:34.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038992644 2024-04-24T15:33:34.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.152 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:34.152 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931512+0000", 2024-04-24T15:33:34.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5200000000000001e-05 2024-04-24T15:33:34.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.152 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:34.152 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931561+0000", 2024-04-24T15:33:34.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9357999999999997e-05 2024-04-24T15:33:34.153 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:34.153 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:34.153 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:34.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.153 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14050 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2670592~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:34.153 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892478+0000", 2024-04-24T15:33:34.153 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 60.230741551999998, 2024-04-24T15:33:34.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039195102000000002, 2024-04-24T15:33:34.153 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:34.153 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:34.153 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:34.153 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:34.153 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:34.153 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14050 2024-04-24T15:33:34.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.154 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:34.154 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.154 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:34.154 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892478+0000", 2024-04-24T15:33:34.154 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:34.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.154 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.154 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:34.154 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892478+0000", 2024-04-24T15:33:34.154 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:34.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.154 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:34.155 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892478+0000", 2024-04-24T15:33:34.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:34.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.155 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:34.155 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892480+0000", 2024-04-24T15:33:34.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3300000000000001e-06 2024-04-24T15:33:34.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.155 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:34.155 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892486+0000", 2024-04-24T15:33:34.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9200000000000001e-06 2024-04-24T15:33:34.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.155 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:34.156 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892494+0000", 2024-04-24T15:33:34.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5159999999999997e-06 2024-04-24T15:33:34.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.156 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:34.156 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931581+0000", 2024-04-24T15:33:34.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039087894999999998 2024-04-24T15:33:34.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.156 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:34.156 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931626+0000", 2024-04-24T15:33:34.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4943e-05 2024-04-24T15:33:34.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.156 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:34.157 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931673+0000", 2024-04-24T15:33:34.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6906000000000001e-05 2024-04-24T15:33:34.157 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:34.157 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:34.157 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:34.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.157 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.157 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14051 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2711552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:34.157 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892507+0000", 2024-04-24T15:33:34.157 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 60.230712853, 2024-04-24T15:33:34.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039274393999999997, 2024-04-24T15:33:34.157 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:34.157 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:34.157 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:34.157 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:34.157 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:34.158 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14051 2024-04-24T15:33:34.158 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.158 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:34.158 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.158 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:34.158 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892507+0000", 2024-04-24T15:33:34.158 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:34.158 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.158 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.158 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:34.158 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892507+0000", 2024-04-24T15:33:34.158 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:34.158 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.158 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.159 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:34.159 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892506+0000", 2024-04-24T15:33:34.159 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:34.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.159 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.159 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:34.159 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892509+0000", 2024-04-24T15:33:34.159 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8760000000000001e-06 2024-04-24T15:33:34.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.159 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.159 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:34.159 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892515+0000", 2024-04-24T15:33:34.159 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1840000000000004e-06 2024-04-24T15:33:34.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:34.160 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892519+0000", 2024-04-24T15:33:34.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7570000000000002e-06 2024-04-24T15:33:34.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:34.160 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931694+0000", 2024-04-24T15:33:34.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039174960000000002 2024-04-24T15:33:34.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:34.160 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931734+0000", 2024-04-24T15:33:34.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0284000000000003e-05 2024-04-24T15:33:34.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.161 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:34.161 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931781+0000", 2024-04-24T15:33:34.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6813000000000001e-05 2024-04-24T15:33:34.161 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:34.161 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:34.161 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:34.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.161 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14052 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2752512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:34.161 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892524+0000", 2024-04-24T15:33:34.161 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 60.230695705999999, 2024-04-24T15:33:34.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039371122000000001, 2024-04-24T15:33:34.161 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:34.162 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:34.162 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:34.162 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:34.162 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:34.162 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14052 2024-04-24T15:33:34.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.162 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:34.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:34.162 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892524+0000", 2024-04-24T15:33:34.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:34.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.163 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:34.163 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892524+0000", 2024-04-24T15:33:34.163 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:34.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.163 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.163 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:34.163 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892524+0000", 2024-04-24T15:33:34.163 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:34.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.163 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.163 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:34.163 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892526+0000", 2024-04-24T15:33:34.163 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3630000000000001e-06 2024-04-24T15:33:34.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.163 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.164 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:34.164 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892538+0000", 2024-04-24T15:33:34.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1677e-05 2024-04-24T15:33:34.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.164 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:34.164 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892541+0000", 2024-04-24T15:33:34.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.822e-06 2024-04-24T15:33:34.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.164 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:34.164 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931802+0000", 2024-04-24T15:33:34.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039260911000000003 2024-04-24T15:33:34.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.165 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:34.165 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931847+0000", 2024-04-24T15:33:34.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4450999999999998e-05 2024-04-24T15:33:34.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.165 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:34.165 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931895+0000", 2024-04-24T15:33:34.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8257999999999998e-05 2024-04-24T15:33:34.165 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:34.165 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:34.165 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:34.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.165 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14053 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2793472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:34.166 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892546+0000", 2024-04-24T15:33:34.166 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 60.230673727999999, 2024-04-24T15:33:34.166 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039457712999999998, 2024-04-24T15:33:34.166 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:34.166 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:34.166 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:34.166 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:34.166 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:34.166 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14053 2024-04-24T15:33:34.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.166 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:34.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.166 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:34.167 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892546+0000", 2024-04-24T15:33:34.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:34.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.167 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:34.167 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892546+0000", 2024-04-24T15:33:34.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:34.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.167 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:34.167 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892546+0000", 2024-04-24T15:33:34.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:34.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.168 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:34.168 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892548+0000", 2024-04-24T15:33:34.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3120000000000001e-06 2024-04-24T15:33:34.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.168 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:34.168 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892553+0000", 2024-04-24T15:33:34.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2320000000000001e-06 2024-04-24T15:33:34.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.168 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:34.168 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892559+0000", 2024-04-24T15:33:34.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5480000000000001e-06 2024-04-24T15:33:34.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.169 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:34.169 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931916+0000", 2024-04-24T15:33:34.169 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039357180999999998 2024-04-24T15:33:34.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.169 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.169 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:34.169 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931956+0000", 2024-04-24T15:33:34.169 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0565000000000002e-05 2024-04-24T15:33:34.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.169 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.169 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:34.169 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932004+0000", 2024-04-24T15:33:34.169 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7224000000000001e-05 2024-04-24T15:33:34.169 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:34.169 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:34.170 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:34.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.170 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14054 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2834432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:34.170 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892563+0000", 2024-04-24T15:33:34.170 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 60.230657076, 2024-04-24T15:33:34.170 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039552406999999998, 2024-04-24T15:33:34.170 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:34.170 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:34.170 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:34.170 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:34.170 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:34.170 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14054 2024-04-24T15:33:34.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.171 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:34.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:34.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892563+0000", 2024-04-24T15:33:34.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:34.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:34.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892563+0000", 2024-04-24T15:33:34.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:34.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:34.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892562+0000", 2024-04-24T15:33:34.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:34.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:34.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892564+0000", 2024-04-24T15:33:34.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2029999999999999e-06 2024-04-24T15:33:34.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:34.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892570+0000", 2024-04-24T15:33:34.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1460000000000004e-06 2024-04-24T15:33:34.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:34.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892573+0000", 2024-04-24T15:33:34.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2030000000000001e-06 2024-04-24T15:33:34.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.173 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:34.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932024+0000", 2024-04-24T15:33:34.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039451696000000001 2024-04-24T15:33:34.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.173 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:34.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932068+0000", 2024-04-24T15:33:34.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3970000000000001e-05 2024-04-24T15:33:34.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.173 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:34.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932115+0000", 2024-04-24T15:33:34.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6569000000000001e-05 2024-04-24T15:33:34.173 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:34.173 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:34.173 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:34.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.174 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14055 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2875392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:34.174 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892591+0000", 2024-04-24T15:33:34.174 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 60.230629102999998, 2024-04-24T15:33:34.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039634309, 2024-04-24T15:33:34.174 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:34.174 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:34.174 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:34.174 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:34.174 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:34.174 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14055 2024-04-24T15:33:34.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.174 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:34.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:34.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892591+0000", 2024-04-24T15:33:34.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:34.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:34.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892591+0000", 2024-04-24T15:33:34.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:34.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:34.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892590+0000", 2024-04-24T15:33:34.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:34.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:34.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892592+0000", 2024-04-24T15:33:34.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7519999999999999e-06 2024-04-24T15:33:34.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:34.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892596+0000", 2024-04-24T15:33:34.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.608e-06 2024-04-24T15:33:34.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:34.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892598+0000", 2024-04-24T15:33:34.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.531e-06 2024-04-24T15:33:34.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:34.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932136+0000", 2024-04-24T15:33:34.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039537668999999998 2024-04-24T15:33:34.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:34.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932177+0000", 2024-04-24T15:33:34.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0880000000000002e-05 2024-04-24T15:33:34.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:34.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932225+0000", 2024-04-24T15:33:34.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8223999999999998e-05 2024-04-24T15:33:34.177 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:34.177 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:34.178 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:34.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.178 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14056 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2916352~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:34.178 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892600+0000", 2024-04-24T15:33:34.178 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 60.230619269000002, 2024-04-24T15:33:34.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039736447000000001, 2024-04-24T15:33:34.178 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:34.178 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:34.178 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:34.178 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:34.178 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:34.178 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14056 2024-04-24T15:33:34.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.178 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:34.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.179 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:34.179 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892600+0000", 2024-04-24T15:33:34.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:34.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.179 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:34.179 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892600+0000", 2024-04-24T15:33:34.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:34.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.179 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:34.179 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892600+0000", 2024-04-24T15:33:34.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967296 2024-04-24T15:33:34.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:34.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892602+0000", 2024-04-24T15:33:34.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.576e-06 2024-04-24T15:33:34.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:34.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892605+0000", 2024-04-24T15:33:34.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.574e-06 2024-04-24T15:33:34.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:34.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892608+0000", 2024-04-24T15:33:34.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3800000000000001e-06 2024-04-24T15:33:34.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:34.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932245+0000", 2024-04-24T15:33:34.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039637378000000001 2024-04-24T15:33:34.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:34.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932290+0000", 2024-04-24T15:33:34.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4734999999999998e-05 2024-04-24T15:33:34.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:34.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932337+0000", 2024-04-24T15:33:34.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7033000000000002e-05 2024-04-24T15:33:34.182 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:34.182 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:34.182 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:34.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.182 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14063 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 3203072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:34.182 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.894564+0000", 2024-04-24T15:33:34.182 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 60.228655678000003, 2024-04-24T15:33:34.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038554995000000002, 2024-04-24T15:33:34.182 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:34.182 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:34.182 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:34.182 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:34.182 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:34.183 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14063 2024-04-24T15:33:34.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.183 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:34.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:34.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894564+0000", 2024-04-24T15:33:34.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:34.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:34.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894564+0000", 2024-04-24T15:33:34.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:34.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:34.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894564+0000", 2024-04-24T15:33:34.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967296 2024-04-24T15:33:34.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.184 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:34.184 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894565+0000", 2024-04-24T15:33:34.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2640000000000001e-06 2024-04-24T15:33:34.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.184 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:34.184 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894569+0000", 2024-04-24T15:33:34.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1710000000000004e-06 2024-04-24T15:33:34.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.184 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:34.185 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894573+0000", 2024-04-24T15:33:34.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4410000000000002e-06 2024-04-24T15:33:34.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.185 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:34.185 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933030+0000", 2024-04-24T15:33:34.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038457709999999999 2024-04-24T15:33:34.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.185 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:34.185 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933072+0000", 2024-04-24T15:33:34.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.193e-05 2024-04-24T15:33:34.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.185 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:34.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933119+0000", 2024-04-24T15:33:34.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6715000000000003e-05 2024-04-24T15:33:34.186 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:34.186 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:34.186 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:34.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.186 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14064 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 3244032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:34.186 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.894591+0000", 2024-04-24T15:33:34.186 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 60.228629036000001, 2024-04-24T15:33:34.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038639806999999998, 2024-04-24T15:33:34.186 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:34.186 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:34.186 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:34.186 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:34.187 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:34.187 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14064 2024-04-24T15:33:34.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.187 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:34.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:34.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894591+0000", 2024-04-24T15:33:34.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:34.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:34.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894591+0000", 2024-04-24T15:33:34.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:34.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:34.188 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894590+0000", 2024-04-24T15:33:34.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:34.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:34.188 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894593+0000", 2024-04-24T15:33:34.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4719999999999998e-06 2024-04-24T15:33:34.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:34.188 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894598+0000", 2024-04-24T15:33:34.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.57e-06 2024-04-24T15:33:34.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:34.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894602+0000", 2024-04-24T15:33:34.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.614e-06 2024-04-24T15:33:34.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:34.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933139+0000", 2024-04-24T15:33:34.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038536929999999997 2024-04-24T15:33:34.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:34.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933183+0000", 2024-04-24T15:33:34.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4217e-05 2024-04-24T15:33:34.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:34.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933230+0000", 2024-04-24T15:33:34.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7434e-05 2024-04-24T15:33:34.190 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:34.190 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:34.190 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:34.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.190 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14065 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 3284992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:34.190 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.894606+0000", 2024-04-24T15:33:34.190 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 60.228613654, 2024-04-24T15:33:34.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038734126000000001, 2024-04-24T15:33:34.190 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:34.191 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:34.191 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:34.191 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:34.191 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:34.191 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14065 2024-04-24T15:33:34.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.191 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:34.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:34.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894606+0000", 2024-04-24T15:33:34.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:34.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:34.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894606+0000", 2024-04-24T15:33:34.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:34.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:34.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894606+0000", 2024-04-24T15:33:34.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:34.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:34.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894608+0000", 2024-04-24T15:33:34.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0880000000000002e-06 2024-04-24T15:33:34.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:34.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894614+0000", 2024-04-24T15:33:34.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7830000000000004e-06 2024-04-24T15:33:34.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:34.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894621+0000", 2024-04-24T15:33:34.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5809999999999999e-06 2024-04-24T15:33:34.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:34.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933251+0000", 2024-04-24T15:33:34.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038629995 2024-04-24T15:33:34.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:34.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933292+0000", 2024-04-24T15:33:34.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0380000000000003e-05 2024-04-24T15:33:34.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:34.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933340+0000", 2024-04-24T15:33:34.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8662999999999998e-05 2024-04-24T15:33:34.194 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:34.194 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:34.194 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:34.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.194 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14066 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 3325952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:34.194 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.894625+0000", 2024-04-24T15:33:34.195 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 60.228594215000001, 2024-04-24T15:33:34.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038825539999999999, 2024-04-24T15:33:34.195 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:34.195 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:34.195 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:34.195 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:34.195 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:34.195 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14066 2024-04-24T15:33:34.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.195 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:34.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:34.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894625+0000", 2024-04-24T15:33:34.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:34.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:34.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894625+0000", 2024-04-24T15:33:34.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:34.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:34.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894625+0000", 2024-04-24T15:33:34.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:34.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:34.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894627+0000", 2024-04-24T15:33:34.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.092e-06 2024-04-24T15:33:34.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:34.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894635+0000", 2024-04-24T15:33:34.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6660000000000008e-06 2024-04-24T15:33:34.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:34.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894639+0000", 2024-04-24T15:33:34.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.799e-06 2024-04-24T15:33:34.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:34.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933360+0000", 2024-04-24T15:33:34.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038721569999999997 2024-04-24T15:33:34.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.198 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:34.198 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933403+0000", 2024-04-24T15:33:34.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2605999999999998e-05 2024-04-24T15:33:34.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.198 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:34.198 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933451+0000", 2024-04-24T15:33:34.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8149999999999998e-05 2024-04-24T15:33:34.198 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:34.198 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:34.198 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:34.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.198 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14067 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 3366912~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:34.198 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.894644+0000", 2024-04-24T15:33:34.199 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 60.228576156999999, 2024-04-24T15:33:34.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038926068000000001, 2024-04-24T15:33:34.199 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:34.199 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:34.199 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:34.199 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:34.199 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:34.199 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14067 2024-04-24T15:33:34.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.199 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:34.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:34.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894644+0000", 2024-04-24T15:33:34.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:34.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:34.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894644+0000", 2024-04-24T15:33:34.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:34.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:34.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894643+0000", 2024-04-24T15:33:34.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:34.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:34.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894646+0000", 2024-04-24T15:33:34.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3379999999999999e-06 2024-04-24T15:33:34.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:34.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894651+0000", 2024-04-24T15:33:34.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.485e-06 2024-04-24T15:33:34.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:34.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894655+0000", 2024-04-24T15:33:34.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.709e-06 2024-04-24T15:33:34.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:34.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933472+0000", 2024-04-24T15:33:34.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038816992000000002 2024-04-24T15:33:34.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:34.202 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933521+0000", 2024-04-24T15:33:34.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8841999999999998e-05 2024-04-24T15:33:34.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:34.202 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933570+0000", 2024-04-24T15:33:34.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9063000000000001e-05 2024-04-24T15:33:34.202 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:34.202 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:34.202 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:34.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.203 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:30240 4.0 4:059b194c:::rbd_object_map.114a2e5eebc:head [read 0~43] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-04-24T15:33:34.203 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:47.244473+0000", 2024-04-24T15:33:34.203 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 46.878746970999998, 2024-04-24T15:33:34.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4146627299999999, 2024-04-24T15:33:34.203 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:34.203 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:34.203 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:34.203 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:34.203 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:34.203 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30240 2024-04-24T15:33:34.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.203 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:34.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:34.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.244473+0000", 2024-04-24T15:33:34.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:34.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.205 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:34.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.244473+0000", 2024-04-24T15:33:34.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:34.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.206 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:34.206 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.244471+0000", 2024-04-24T15:33:34.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999986 2024-04-24T15:33:34.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.206 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.206 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:34.206 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.244477+0000", 2024-04-24T15:33:34.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7819999999999999e-06 2024-04-24T15:33:34.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.206 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.206 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:34.206 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.244487+0000", 2024-04-24T15:33:34.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0247e-05 2024-04-24T15:33:34.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.206 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.207 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:34.207 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.244499+0000", 2024-04-24T15:33:34.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1324e-05 2024-04-24T15:33:34.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.207 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:34.207 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.244545+0000", 2024-04-24T15:33:34.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6186000000000001e-05 2024-04-24T15:33:34.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.207 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-04-24T15:33:34.207 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.244553+0000", 2024-04-24T15:33:34.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5380000000000002e-06 2024-04-24T15:33:34.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:34.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.631596+0000", 2024-04-24T15:33:34.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.38704289600000003 2024-04-24T15:33:34.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-04-24T15:33:34.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.631606+0000", 2024-04-24T15:33:34.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6390000000000004e-06 2024-04-24T15:33:34.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:34.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:48.658831+0000", 2024-04-24T15:33:34.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0272250540000001 2024-04-24T15:33:34.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:34.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:48.658991+0000", 2024-04-24T15:33:34.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015994800000000001 2024-04-24T15:33:34.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:34.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:34.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:34.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:48.659135+0000", 2024-04-24T15:33:34.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000144456 2024-04-24T15:33:34.209 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:34.209 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:34.209 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:34.209 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:34.209 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:34.209 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:34.211 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-24T15:33:34.211 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:33:34.212+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:33:34.311 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:33:34.312+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:33:34.341 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:34.341 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:33:34.341 INFO:teuthology.orchestra.run.smithi151.stdout: "num_ops": 0 2024-04-24T15:33:34.341 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:34.351 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-24T15:33:34.412 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:34.413+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:34.472 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:34.473 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:33:34.473 INFO:teuthology.orchestra.run.smithi151.stdout: "complaint_time": 30, 2024-04-24T15:33:34.473 INFO:teuthology.orchestra.run.smithi151.stdout: "num_blocked_ops": 0 2024-04-24T15:33:34.473 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:34.482 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-24T15:33:34.526 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:34.527+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:34.607 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:34.607 INFO:teuthology.orchestra.run.smithi151.stdout: "size": 20, 2024-04-24T15:33:34.608 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 600, 2024-04-24T15:33:34.608 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [ 2024-04-24T15:33:34.608 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.608 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:39041 4.1 4:82c1a902:::rbd_data.114abe589119.000000000000032c:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e68)", 2024-04-24T15:33:34.608 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:01.760552+0000", 2024-04-24T15:33:34.608 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 32.846568775999998, 2024-04-24T15:33:34.608 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.87570748300000001, 2024-04-24T15:33:34.608 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:34.608 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "reached pg", 2024-04-24T15:33:34.608 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:34.608 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:34.608 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:34.608 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 39041 2024-04-24T15:33:34.608 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.609 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:34.609 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.609 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:34.609 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760552+0000", 2024-04-24T15:33:34.609 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:34.609 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.609 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.609 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:34.609 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760552+0000", 2024-04-24T15:33:34.609 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:34.609 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.609 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.609 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:34.609 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760550+0000", 2024-04-24T15:33:34.610 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999986 2024-04-24T15:33:34.610 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.610 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.610 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:34.610 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760557+0000", 2024-04-24T15:33:34.610 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1420000000000004e-06 2024-04-24T15:33:34.610 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.610 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.610 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:34.610 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760572+0000", 2024-04-24T15:33:34.610 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.483e-05 2024-04-24T15:33:34.610 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.611 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.611 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:34.611 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760578+0000", 2024-04-24T15:33:34.611 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.2820000000000002e-06 2024-04-24T15:33:34.611 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.611 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.611 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:34.611 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760628+0000", 2024-04-24T15:33:34.613 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.0040000000000002e-05 2024-04-24T15:33:34.613 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.614 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.614 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op must wait for map", 2024-04-24T15:33:34.614 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760637+0000", 2024-04-24T15:33:34.614 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.6899999999999998e-06 2024-04-24T15:33:34.614 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.614 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.614 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:34.614 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.770562+0000", 2024-04-24T15:33:34.614 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0099252739999999996 2024-04-24T15:33:34.614 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.614 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.614 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for peered", 2024-04-24T15:33:34.614 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.770572+0000", 2024-04-24T15:33:34.614 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0105999999999999e-05 2024-04-24T15:33:34.615 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.615 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.615 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:34.615 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:02.636170+0000", 2024-04-24T15:33:34.615 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.86559722299999997 2024-04-24T15:33:34.615 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.615 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.615 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:34.615 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:02.636259+0000", 2024-04-24T15:33:34.615 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.9446999999999999e-05 2024-04-24T15:33:34.615 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:34.615 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:34.615 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:34.615 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.616 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.616 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48675 4.d 4:b465d49b:::rbd_data.114a858e6641.0000000000000002:17 [read 2768896~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:34.616 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.926403+0000", 2024-04-24T15:33:34.616 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 23.680717206000001, 2024-04-24T15:33:34.616 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035742807000000001, 2024-04-24T15:33:34.616 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:34.616 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:34.616 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:34.616 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:34.616 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:34.616 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48675 2024-04-24T15:33:34.617 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.617 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:34.617 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.617 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:34.617 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926403+0000", 2024-04-24T15:33:34.617 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:34.617 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.617 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.617 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:34.617 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926403+0000", 2024-04-24T15:33:34.617 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:34.617 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.617 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.617 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:34.618 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926403+0000", 2024-04-24T15:33:34.618 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:34.618 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.618 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.618 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:34.618 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926404+0000", 2024-04-24T15:33:34.618 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.424e-06 2024-04-24T15:33:34.618 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.618 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.618 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:34.618 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926413+0000", 2024-04-24T15:33:34.618 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.6850000000000007e-06 2024-04-24T15:33:34.618 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.618 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.618 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:34.619 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926417+0000", 2024-04-24T15:33:34.619 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.7859999999999998e-06 2024-04-24T15:33:34.619 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.619 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.619 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:34.619 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.961986+0000", 2024-04-24T15:33:34.619 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035568862 2024-04-24T15:33:34.619 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.619 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.619 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:34.619 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.962063+0000", 2024-04-24T15:33:34.619 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7257999999999998e-05 2024-04-24T15:33:34.619 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.619 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.619 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:34.620 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.962146+0000", 2024-04-24T15:33:34.620 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3094000000000001e-05 2024-04-24T15:33:34.620 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:34.620 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:34.620 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:34.620 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.620 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.620 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48676 4.d 4:b465d49b:::rbd_data.114a858e6641.0000000000000002:17 [read 2834432~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:34.620 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.926421+0000", 2024-04-24T15:33:34.620 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 23.680699662999999, 2024-04-24T15:33:34.620 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035829002999999998, 2024-04-24T15:33:34.620 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:34.620 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:34.620 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:34.620 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:34.621 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:34.621 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48676 2024-04-24T15:33:34.621 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.621 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:34.621 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.621 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:34.621 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926421+0000", 2024-04-24T15:33:34.621 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:34.621 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.621 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.621 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:34.621 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926421+0000", 2024-04-24T15:33:34.621 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:34.621 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.621 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.622 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:34.622 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926420+0000", 2024-04-24T15:33:34.622 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:34.622 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.622 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.622 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:34.622 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926423+0000", 2024-04-24T15:33:34.622 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.356e-06 2024-04-24T15:33:34.622 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.622 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.622 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:34.622 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926428+0000", 2024-04-24T15:33:34.622 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.3340000000000001e-06 2024-04-24T15:33:34.622 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.622 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.622 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:34.623 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926431+0000", 2024-04-24T15:33:34.623 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.2899999999999998e-06 2024-04-24T15:33:34.623 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.623 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.623 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:34.623 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.962156+0000", 2024-04-24T15:33:34.623 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035725159999999999 2024-04-24T15:33:34.623 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.623 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.623 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:34.623 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.962197+0000", 2024-04-24T15:33:34.623 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0457999999999998e-05 2024-04-24T15:33:34.623 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.624 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.624 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:34.624 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.962250+0000", 2024-04-24T15:33:34.624 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.2812000000000002e-05 2024-04-24T15:33:34.624 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:34.624 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:34.624 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:34.624 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.624 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.624 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48703 4.d 4:b59d5735:::rbd_data.114a858e6641.0000000000000004:17 [read 2637824~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:34.624 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928531+0000", 2024-04-24T15:33:34.625 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 23.678589036999998, 2024-04-24T15:33:34.625 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035675198999999998, 2024-04-24T15:33:34.625 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:34.625 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:34.625 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:34.625 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:34.625 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:34.625 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48703 2024-04-24T15:33:34.625 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.625 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:34.625 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.625 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:34.625 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928531+0000", 2024-04-24T15:33:34.625 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:34.625 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.626 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.626 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:34.626 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928531+0000", 2024-04-24T15:33:34.626 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:34.626 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.626 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.626 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:34.626 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928531+0000", 2024-04-24T15:33:34.626 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:34.626 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.626 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.626 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:34.627 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928532+0000", 2024-04-24T15:33:34.627 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.33e-06 2024-04-24T15:33:34.627 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.627 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.627 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:34.627 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928536+0000", 2024-04-24T15:33:34.627 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.4419999999999998e-06 2024-04-24T15:33:34.627 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.627 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.627 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:34.627 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928538+0000", 2024-04-24T15:33:34.627 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.0669999999999999e-06 2024-04-24T15:33:34.627 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.627 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.627 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:34.628 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964040+0000", 2024-04-24T15:33:34.628 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035501900000000003 2024-04-24T15:33:34.628 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.628 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.628 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:34.628 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964118+0000", 2024-04-24T15:33:34.628 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.8490000000000002e-05 2024-04-24T15:33:34.628 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.628 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.628 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:34.628 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964207+0000", 2024-04-24T15:33:34.628 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.8202999999999996e-05 2024-04-24T15:33:34.628 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:34.628 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:34.629 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:34.629 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.629 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.629 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48704 4.d 4:b59d5735:::rbd_data.114a858e6641.0000000000000004:17 [read 2703360~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:34.629 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928541+0000", 2024-04-24T15:33:34.629 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 23.678579693, 2024-04-24T15:33:34.629 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035763016000000002, 2024-04-24T15:33:34.629 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:34.629 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:34.629 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:34.629 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:34.629 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:34.629 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48704 2024-04-24T15:33:34.629 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.629 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:34.629 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.630 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:34.630 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928541+0000", 2024-04-24T15:33:34.630 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:34.630 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.630 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.630 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:34.630 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928541+0000", 2024-04-24T15:33:34.630 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:34.630 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.630 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.630 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:34.630 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928540+0000", 2024-04-24T15:33:34.630 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:34.630 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.631 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.631 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:34.631 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928542+0000", 2024-04-24T15:33:34.631 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.37e-06 2024-04-24T15:33:34.631 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.631 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.631 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:34.631 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928545+0000", 2024-04-24T15:33:34.631 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8229999999999999e-06 2024-04-24T15:33:34.631 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.631 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.631 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:34.631 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928547+0000", 2024-04-24T15:33:34.631 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.9360000000000002e-06 2024-04-24T15:33:34.631 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.632 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.632 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:34.632 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964219+0000", 2024-04-24T15:33:34.632 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035672813999999997 2024-04-24T15:33:34.632 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.632 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.632 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:34.632 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964259+0000", 2024-04-24T15:33:34.632 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0070000000000001e-05 2024-04-24T15:33:34.632 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.632 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.632 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:34.632 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964304+0000", 2024-04-24T15:33:34.632 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4229e-05 2024-04-24T15:33:34.632 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:34.633 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:34.633 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:34.633 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.633 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.633 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48719 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 1851392~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:34.633 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928549+0000", 2024-04-24T15:33:34.633 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 23.678571546000001, 2024-04-24T15:33:34.633 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035890645999999998, 2024-04-24T15:33:34.633 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:34.633 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:34.633 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:34.633 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:34.633 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:34.634 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48719 2024-04-24T15:33:34.634 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.634 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:34.634 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.634 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:34.634 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928549+0000", 2024-04-24T15:33:34.634 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:34.634 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.634 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.634 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:34.634 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928549+0000", 2024-04-24T15:33:34.634 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:34.634 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.634 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.635 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:34.635 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928549+0000", 2024-04-24T15:33:34.635 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:34.635 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.635 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.635 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:34.635 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928550+0000", 2024-04-24T15:33:34.635 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3829999999999999e-06 2024-04-24T15:33:34.635 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.635 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.635 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:34.635 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928553+0000", 2024-04-24T15:33:34.635 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7369999999999998e-06 2024-04-24T15:33:34.635 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.636 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.636 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:34.636 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928555+0000", 2024-04-24T15:33:34.636 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.8920000000000001e-06 2024-04-24T15:33:34.636 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.636 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.636 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:34.636 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964240+0000", 2024-04-24T15:33:34.636 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035685537000000003 2024-04-24T15:33:34.636 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.636 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.636 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:34.636 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964318+0000", 2024-04-24T15:33:34.636 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7838999999999998e-05 2024-04-24T15:33:34.636 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.636 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.637 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:34.637 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964439+0000", 2024-04-24T15:33:34.637 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000121564 2024-04-24T15:33:34.637 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:34.637 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:34.637 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:34.637 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.637 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.637 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48720 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 1916928~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:34.637 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928557+0000", 2024-04-24T15:33:34.637 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 23.678563701000002, 2024-04-24T15:33:34.637 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036005219999999998, 2024-04-24T15:33:34.637 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:34.637 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:34.637 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:34.638 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:34.638 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:34.638 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48720 2024-04-24T15:33:34.638 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.638 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:34.638 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.638 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:34.638 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928557+0000", 2024-04-24T15:33:34.638 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:34.638 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.638 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.638 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:34.638 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928557+0000", 2024-04-24T15:33:34.638 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:34.639 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.639 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.639 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:34.639 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928556+0000", 2024-04-24T15:33:34.639 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:34.639 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.639 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.639 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:34.639 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928558+0000", 2024-04-24T15:33:34.639 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.13e-06 2024-04-24T15:33:34.639 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.639 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.639 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:34.639 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928561+0000", 2024-04-24T15:33:34.639 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5530000000000001e-06 2024-04-24T15:33:34.640 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.640 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.640 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:34.640 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928563+0000", 2024-04-24T15:33:34.640 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.8810000000000001e-06 2024-04-24T15:33:34.640 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.640 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.640 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:34.640 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964450+0000", 2024-04-24T15:33:34.640 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035887040000000002 2024-04-24T15:33:34.640 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.640 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.640 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:34.640 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964494+0000", 2024-04-24T15:33:34.640 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3699000000000003e-05 2024-04-24T15:33:34.640 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.641 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.641 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:34.641 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964562+0000", 2024-04-24T15:33:34.641 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.8149000000000001e-05 2024-04-24T15:33:34.641 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:34.641 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:34.641 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:34.641 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.641 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.641 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48721 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 1982464~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:34.641 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928565+0000", 2024-04-24T15:33:34.641 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 23.678555177, 2024-04-24T15:33:34.641 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036233466999999998, 2024-04-24T15:33:34.641 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:34.641 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:34.641 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:34.642 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:34.642 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:34.642 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48721 2024-04-24T15:33:34.642 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.642 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:34.642 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.642 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:34.642 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928565+0000", 2024-04-24T15:33:34.642 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:34.642 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.642 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.642 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:34.642 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928565+0000", 2024-04-24T15:33:34.642 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:34.642 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.643 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.643 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:34.643 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928565+0000", 2024-04-24T15:33:34.643 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:34.643 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.643 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.643 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:34.643 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928566+0000", 2024-04-24T15:33:34.643 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.336e-06 2024-04-24T15:33:34.643 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.643 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.643 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:34.643 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928569+0000", 2024-04-24T15:33:34.643 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.9189999999999999e-06 2024-04-24T15:33:34.643 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.643 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.644 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:34.644 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928571+0000", 2024-04-24T15:33:34.644 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.1909999999999999e-06 2024-04-24T15:33:34.644 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.644 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.644 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:34.644 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964592+0000", 2024-04-24T15:33:34.644 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036020120000000003 2024-04-24T15:33:34.644 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.644 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.644 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:34.644 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964673+0000", 2024-04-24T15:33:34.644 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.1254000000000005e-05 2024-04-24T15:33:34.644 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.644 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.644 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:34.645 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964799+0000", 2024-04-24T15:33:34.645 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00012588700000000001 2024-04-24T15:33:34.645 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:34.645 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:34.645 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:34.645 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.645 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.645 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48722 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2048000~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:34.645 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928586+0000", 2024-04-24T15:33:34.645 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 23.678534052, 2024-04-24T15:33:34.645 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036331347999999999, 2024-04-24T15:33:34.645 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:34.645 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:34.645 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:34.646 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:34.646 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:34.646 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48722 2024-04-24T15:33:34.646 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.646 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:34.646 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.646 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:34.646 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928586+0000", 2024-04-24T15:33:34.646 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:34.646 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.646 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.646 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:34.646 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928586+0000", 2024-04-24T15:33:34.646 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:34.646 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.646 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.647 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:34.647 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928586+0000", 2024-04-24T15:33:34.647 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:34.647 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.647 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.647 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:34.647 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928587+0000", 2024-04-24T15:33:34.647 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3519999999999999e-06 2024-04-24T15:33:34.647 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.647 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.647 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:34.647 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928590+0000", 2024-04-24T15:33:34.647 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.9699999999999999e-06 2024-04-24T15:33:34.647 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.647 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.648 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:34.648 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928592+0000", 2024-04-24T15:33:34.648 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.097e-06 2024-04-24T15:33:34.648 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.648 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.648 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:34.648 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964810+0000", 2024-04-24T15:33:34.648 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036217249 2024-04-24T15:33:34.648 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.648 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.648 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:34.648 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964851+0000", 2024-04-24T15:33:34.648 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.1151e-05 2024-04-24T15:33:34.648 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.648 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.648 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:34.649 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964918+0000", 2024-04-24T15:33:34.649 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.6898999999999998e-05 2024-04-24T15:33:34.649 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:34.649 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:34.649 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:34.649 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.649 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.649 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48723 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2113536~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:34.649 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928598+0000", 2024-04-24T15:33:34.649 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 23.678522662999999, 2024-04-24T15:33:34.649 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036552125999999997, 2024-04-24T15:33:34.649 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:34.649 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:34.649 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:34.649 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:34.650 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:34.650 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48723 2024-04-24T15:33:34.650 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.650 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:34.650 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.650 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:34.650 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928598+0000", 2024-04-24T15:33:34.650 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:34.650 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.650 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.650 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:34.650 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928598+0000", 2024-04-24T15:33:34.650 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:34.650 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.651 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.651 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:34.651 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928597+0000", 2024-04-24T15:33:34.651 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:34.651 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.651 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.651 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:34.651 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928599+0000", 2024-04-24T15:33:34.651 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.621e-06 2024-04-24T15:33:34.651 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.651 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.651 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:34.651 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928602+0000", 2024-04-24T15:33:34.651 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.9629999999999998e-06 2024-04-24T15:33:34.651 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.651 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.652 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:34.652 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928607+0000", 2024-04-24T15:33:34.652 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8940000000000001e-06 2024-04-24T15:33:34.652 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.652 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.652 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:34.652 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964947+0000", 2024-04-24T15:33:34.652 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036340439000000002 2024-04-24T15:33:34.652 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.652 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.652 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:34.652 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965028+0000", 2024-04-24T15:33:34.652 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.1054e-05 2024-04-24T15:33:34.652 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.652 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.652 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:34.653 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965150+0000", 2024-04-24T15:33:34.653 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00012154000000000001 2024-04-24T15:33:34.653 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:34.653 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:34.653 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:34.653 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.653 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.653 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48724 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2179072~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:34.653 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928609+0000", 2024-04-24T15:33:34.653 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 23.678511287999999, 2024-04-24T15:33:34.653 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036679614999999999, 2024-04-24T15:33:34.653 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:34.653 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:34.653 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:34.653 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:34.653 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:34.654 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48724 2024-04-24T15:33:34.654 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.654 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:34.654 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.654 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:34.654 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928609+0000", 2024-04-24T15:33:34.654 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:34.654 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.654 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.654 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:34.654 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928609+0000", 2024-04-24T15:33:34.654 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:34.654 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.654 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.654 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:34.654 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928609+0000", 2024-04-24T15:33:34.655 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:34.655 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.655 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.655 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:34.655 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928610+0000", 2024-04-24T15:33:34.655 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3030000000000001e-06 2024-04-24T15:33:34.655 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.655 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.655 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:34.655 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928613+0000", 2024-04-24T15:33:34.655 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.762e-06 2024-04-24T15:33:34.655 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.655 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.655 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:34.655 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928615+0000", 2024-04-24T15:33:34.655 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.832e-06 2024-04-24T15:33:34.655 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.656 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.656 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:34.656 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965174+0000", 2024-04-24T15:33:34.656 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036559633000000001 2024-04-24T15:33:34.656 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.656 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.656 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:34.656 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965221+0000", 2024-04-24T15:33:34.656 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6239999999999998e-05 2024-04-24T15:33:34.656 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.656 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.656 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:34.656 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965289+0000", 2024-04-24T15:33:34.656 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.8053999999999996e-05 2024-04-24T15:33:34.656 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:34.656 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:34.657 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:34.657 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.657 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.657 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48725 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2244608~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:34.657 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928617+0000", 2024-04-24T15:33:34.657 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 23.678503517999999, 2024-04-24T15:33:34.657 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036902659999999997, 2024-04-24T15:33:34.657 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:34.657 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:34.657 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:34.657 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:34.657 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:34.657 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48725 2024-04-24T15:33:34.657 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.657 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:34.657 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.658 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:34.658 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928617+0000", 2024-04-24T15:33:34.658 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:34.658 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.658 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.658 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:34.658 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928617+0000", 2024-04-24T15:33:34.658 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:34.658 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.658 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.658 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:34.658 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928617+0000", 2024-04-24T15:33:34.658 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:34.658 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.658 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.659 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:34.659 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928618+0000", 2024-04-24T15:33:34.659 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.206e-06 2024-04-24T15:33:34.659 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.659 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.659 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:34.659 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928621+0000", 2024-04-24T15:33:34.659 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5130000000000001e-06 2024-04-24T15:33:34.659 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.659 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.659 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:34.659 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928623+0000", 2024-04-24T15:33:34.659 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.9640000000000002e-06 2024-04-24T15:33:34.659 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.659 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.660 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:34.660 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965318+0000", 2024-04-24T15:33:34.660 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036694953000000002 2024-04-24T15:33:34.660 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.660 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.660 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:34.660 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965398+0000", 2024-04-24T15:33:34.660 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.9986000000000003e-05 2024-04-24T15:33:34.660 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.660 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.660 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:34.660 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965520+0000", 2024-04-24T15:33:34.660 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000121247 2024-04-24T15:33:34.660 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:34.660 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:34.661 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:34.661 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.661 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.661 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48726 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2310144~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:34.661 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928625+0000", 2024-04-24T15:33:34.661 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 23.678494970999999, 2024-04-24T15:33:34.661 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037015521000000003, 2024-04-24T15:33:34.661 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:34.661 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:34.661 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:34.661 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:34.661 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:34.661 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48726 2024-04-24T15:33:34.661 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.661 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:34.661 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.661 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:34.662 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928625+0000", 2024-04-24T15:33:34.662 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:34.662 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.662 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.662 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:34.662 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928625+0000", 2024-04-24T15:33:34.662 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:34.662 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.662 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.662 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:34.662 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928625+0000", 2024-04-24T15:33:34.662 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:34.662 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.662 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.662 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:34.663 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928627+0000", 2024-04-24T15:33:34.663 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3179999999999999e-06 2024-04-24T15:33:34.663 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.663 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.663 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:34.663 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928629+0000", 2024-04-24T15:33:34.663 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.819e-06 2024-04-24T15:33:34.663 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.663 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.663 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:34.663 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928634+0000", 2024-04-24T15:33:34.663 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6999999999999999e-06 2024-04-24T15:33:34.663 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.663 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.663 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:34.663 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965531+0000", 2024-04-24T15:33:34.664 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036897040999999998 2024-04-24T15:33:34.664 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.664 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.664 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:34.664 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965573+0000", 2024-04-24T15:33:34.664 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.189e-05 2024-04-24T15:33:34.664 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.664 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.664 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:34.664 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965641+0000", 2024-04-24T15:33:34.664 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.7959000000000004e-05 2024-04-24T15:33:34.664 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:34.664 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:34.664 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:34.664 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.664 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.665 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48727 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2375680~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:34.665 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928637+0000", 2024-04-24T15:33:34.665 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 23.678483610000001, 2024-04-24T15:33:34.665 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037235129999999998, 2024-04-24T15:33:34.665 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:34.665 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:34.665 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:34.665 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:34.665 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:34.665 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48727 2024-04-24T15:33:34.665 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.665 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:34.665 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.665 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:34.665 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928637+0000", 2024-04-24T15:33:34.665 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:34.666 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.666 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.666 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:34.666 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928637+0000", 2024-04-24T15:33:34.666 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:34.666 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.666 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.666 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:34.666 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928637+0000", 2024-04-24T15:33:34.666 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:34.666 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.666 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.666 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:34.666 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928638+0000", 2024-04-24T15:33:34.666 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.4279999999999999e-06 2024-04-24T15:33:34.666 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.667 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.667 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:34.667 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928641+0000", 2024-04-24T15:33:34.667 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.909e-06 2024-04-24T15:33:34.667 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.667 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.667 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:34.667 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928643+0000", 2024-04-24T15:33:34.667 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.9269999999999999e-06 2024-04-24T15:33:34.667 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.667 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.667 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:34.667 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965671+0000", 2024-04-24T15:33:34.667 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037028393 2024-04-24T15:33:34.667 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.667 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.667 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:34.668 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965752+0000", 2024-04-24T15:33:34.668 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0921999999999995e-05 2024-04-24T15:33:34.668 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.668 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.668 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:34.668 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965872+0000", 2024-04-24T15:33:34.668 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000119759 2024-04-24T15:33:34.668 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:34.668 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:34.668 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:34.668 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.668 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.668 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48728 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2441216~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:34.668 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928645+0000", 2024-04-24T15:33:34.668 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 23.678475426999999, 2024-04-24T15:33:34.668 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037346852999999999, 2024-04-24T15:33:34.669 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:34.669 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:34.669 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:34.669 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:34.669 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:34.669 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48728 2024-04-24T15:33:34.669 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.669 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:34.669 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.669 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:34.669 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928645+0000", 2024-04-24T15:33:34.669 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:34.669 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.669 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.669 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:34.669 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928645+0000", 2024-04-24T15:33:34.670 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:34.670 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.670 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.670 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:34.670 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928645+0000", 2024-04-24T15:33:34.670 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:34.670 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.670 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.670 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:34.670 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928646+0000", 2024-04-24T15:33:34.670 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.291e-06 2024-04-24T15:33:34.670 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.670 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.670 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:34.670 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928649+0000", 2024-04-24T15:33:34.671 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8509999999999999e-06 2024-04-24T15:33:34.671 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.671 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.671 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:34.671 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928651+0000", 2024-04-24T15:33:34.671 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.809e-06 2024-04-24T15:33:34.671 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.671 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.671 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:34.671 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965882+0000", 2024-04-24T15:33:34.671 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037231715999999998 2024-04-24T15:33:34.671 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.671 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.671 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:34.671 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965925+0000", 2024-04-24T15:33:34.671 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2493999999999997e-05 2024-04-24T15:33:34.672 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.672 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.672 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:34.672 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965992+0000", 2024-04-24T15:33:34.672 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.6907000000000002e-05 2024-04-24T15:33:34.672 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:34.672 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:34.672 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:34.672 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.672 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.672 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48729 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2506752~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:34.672 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928653+0000", 2024-04-24T15:33:34.672 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 23.678467552000001, 2024-04-24T15:33:34.672 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037576904000000001, 2024-04-24T15:33:34.672 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:34.673 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:34.673 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:34.673 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:34.673 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:34.673 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48729 2024-04-24T15:33:34.673 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.673 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:34.673 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.673 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:34.673 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928653+0000", 2024-04-24T15:33:34.673 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:34.673 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.673 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.673 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:34.673 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928653+0000", 2024-04-24T15:33:34.673 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:34.674 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.674 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.674 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:34.674 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928653+0000", 2024-04-24T15:33:34.674 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:34.674 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.674 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.674 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:34.674 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928654+0000", 2024-04-24T15:33:34.674 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3209999999999999e-06 2024-04-24T15:33:34.674 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.674 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.674 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:34.674 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928657+0000", 2024-04-24T15:33:34.674 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8559999999999998e-06 2024-04-24T15:33:34.674 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.675 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.675 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:34.675 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928659+0000", 2024-04-24T15:33:34.675 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7080000000000002e-06 2024-04-24T15:33:34.675 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.675 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.675 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:34.675 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966021+0000", 2024-04-24T15:33:34.675 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037361296000000002 2024-04-24T15:33:34.675 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.675 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.675 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:34.675 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966102+0000", 2024-04-24T15:33:34.675 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0939999999999994e-05 2024-04-24T15:33:34.675 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.675 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.676 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:34.676 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966230+0000", 2024-04-24T15:33:34.676 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00012799999999999999 2024-04-24T15:33:34.676 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:34.676 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:34.676 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:34.676 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.676 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.676 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48730 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2572288~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:34.676 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928662+0000", 2024-04-24T15:33:34.676 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 23.678458463999998, 2024-04-24T15:33:34.676 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037687246000000001, 2024-04-24T15:33:34.676 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:34.676 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:34.676 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:34.676 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:34.677 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:34.677 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48730 2024-04-24T15:33:34.677 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.677 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:34.677 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.677 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:34.677 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928662+0000", 2024-04-24T15:33:34.677 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:34.677 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.677 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.677 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:34.677 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928662+0000", 2024-04-24T15:33:34.678 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:34.678 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.678 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.678 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:34.678 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928662+0000", 2024-04-24T15:33:34.678 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:34.678 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.678 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.678 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:34.678 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928663+0000", 2024-04-24T15:33:34.678 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.381e-06 2024-04-24T15:33:34.678 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.678 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.678 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:34.678 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928666+0000", 2024-04-24T15:33:34.679 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.796e-06 2024-04-24T15:33:34.679 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.679 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.679 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:34.679 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928668+0000", 2024-04-24T15:33:34.679 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.8920000000000001e-06 2024-04-24T15:33:34.679 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.679 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.679 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:34.679 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966240+0000", 2024-04-24T15:33:34.679 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037572886999999999 2024-04-24T15:33:34.679 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.679 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.679 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:34.679 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966281+0000", 2024-04-24T15:33:34.680 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0450000000000001e-05 2024-04-24T15:33:34.680 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.680 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.680 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:34.680 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966349+0000", 2024-04-24T15:33:34.680 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.8215999999999999e-05 2024-04-24T15:33:34.680 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:34.680 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:34.680 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:34.680 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.680 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.680 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48731 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2637824~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:34.680 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928747+0000", 2024-04-24T15:33:34.680 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 23.678373458999999, 2024-04-24T15:33:34.680 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037833602000000001, 2024-04-24T15:33:34.680 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:34.680 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:34.681 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:34.681 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:34.681 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:34.681 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48731 2024-04-24T15:33:34.681 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.681 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:34.681 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.681 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:34.681 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928747+0000", 2024-04-24T15:33:34.681 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:34.681 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.681 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.681 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:34.681 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928747+0000", 2024-04-24T15:33:34.681 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:34.681 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.682 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.682 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:34.682 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928670+0000", 2024-04-24T15:33:34.682 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999228 2024-04-24T15:33:34.682 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.682 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.682 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:34.682 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928750+0000", 2024-04-24T15:33:34.682 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0527999999999999e-05 2024-04-24T15:33:34.682 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.682 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.682 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:34.682 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928757+0000", 2024-04-24T15:33:34.682 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.9419999999999998e-06 2024-04-24T15:33:34.682 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.683 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.683 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:34.683 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928762+0000", 2024-04-24T15:33:34.683 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4240000000000003e-06 2024-04-24T15:33:34.683 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.683 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.683 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:34.683 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966379+0000", 2024-04-24T15:33:34.683 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037617267000000003 2024-04-24T15:33:34.683 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.683 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.683 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:34.683 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966461+0000", 2024-04-24T15:33:34.683 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.2720999999999996e-05 2024-04-24T15:33:34.683 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.683 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.683 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:34.684 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966581+0000", 2024-04-24T15:33:34.684 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000119014 2024-04-24T15:33:34.684 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:34.684 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:34.684 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:34.684 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.684 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.684 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48732 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2703360~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:34.684 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928766+0000", 2024-04-24T15:33:34.684 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 23.678353975, 2024-04-24T15:33:34.684 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037930924999999997, 2024-04-24T15:33:34.684 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:34.684 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:34.684 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:34.684 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:34.684 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:34.685 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48732 2024-04-24T15:33:34.685 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.685 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:34.685 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.685 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:34.685 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928766+0000", 2024-04-24T15:33:34.685 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:34.685 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.685 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.685 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:34.685 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928766+0000", 2024-04-24T15:33:34.685 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:34.685 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.685 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.686 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:34.686 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928766+0000", 2024-04-24T15:33:34.686 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:34.686 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.686 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.686 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:34.686 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928768+0000", 2024-04-24T15:33:34.686 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.3420000000000002e-06 2024-04-24T15:33:34.686 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.686 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.686 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:34.686 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928783+0000", 2024-04-24T15:33:34.686 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.4813e-05 2024-04-24T15:33:34.686 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.686 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.686 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:34.687 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928787+0000", 2024-04-24T15:33:34.687 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.1999999999999996e-06 2024-04-24T15:33:34.687 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.687 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.687 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:34.687 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966592+0000", 2024-04-24T15:33:34.687 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037804484999999999 2024-04-24T15:33:34.687 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.687 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.687 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:34.687 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966632+0000", 2024-04-24T15:33:34.687 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0014e-05 2024-04-24T15:33:34.687 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.687 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.687 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:34.687 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966697+0000", 2024-04-24T15:33:34.688 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.5489999999999998e-05 2024-04-24T15:33:34.688 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:34.688 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:34.688 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:34.688 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.688 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.688 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48813 4.d 4:b39400aa:::rbd_data.114a8b543e91.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:34.688 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.936015+0000", 2024-04-24T15:33:34.688 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 23.671105641, 2024-04-24T15:33:34.688 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.039312294999999997, 2024-04-24T15:33:34.688 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:34.688 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:34.688 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:34.688 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:34.688 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:34.688 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48813 2024-04-24T15:33:34.689 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.689 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:34.689 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.689 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:34.689 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.936015+0000", 2024-04-24T15:33:34.689 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:34.689 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.689 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.689 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:34.689 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.936015+0000", 2024-04-24T15:33:34.689 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:34.689 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.689 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.689 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:34.689 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.936014+0000", 2024-04-24T15:33:34.690 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:34.690 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.690 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.690 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:34.690 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.942618+0000", 2024-04-24T15:33:34.690 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0066038499999999997 2024-04-24T15:33:34.690 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.690 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.690 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:34.690 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.942629+0000", 2024-04-24T15:33:34.690 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0606e-05 2024-04-24T15:33:34.690 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.690 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.690 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:34.690 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.942634+0000", 2024-04-24T15:33:34.691 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.2000000000000002e-06 2024-04-24T15:33:34.691 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.691 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.691 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:34.691 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.969006+0000", 2024-04-24T15:33:34.691 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.026371564 2024-04-24T15:33:34.691 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.691 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.691 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:34.691 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.969126+0000", 2024-04-24T15:33:34.691 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000120085 2024-04-24T15:33:34.691 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.691 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.691 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 2", 2024-04-24T15:33:34.691 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.969469+0000", 2024-04-24T15:33:34.691 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00034295400000000002 2024-04-24T15:33:34.692 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.692 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.692 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:33:34.692 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.973118+0000", 2024-04-24T15:33:34.692 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0036489700000000001 2024-04-24T15:33:34.692 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.692 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.692 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:33:34.692 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.975250+0000", 2024-04-24T15:33:34.692 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0021327030000000001 2024-04-24T15:33:34.692 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.692 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.692 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:33:34.692 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.975293+0000", 2024-04-24T15:33:34.692 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2951999999999998e-05 2024-04-24T15:33:34.693 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:34.693 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:34.693 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:34.693 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.975327+0000", 2024-04-24T15:33:34.693 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.3840000000000001e-05 2024-04-24T15:33:34.693 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:34.693 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:34.693 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:34.693 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:34.693 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:34.693 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:34.695 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-24T15:33:34.695 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:34.695+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:34.795 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:33:34.796+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:33:34.858 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:34.858 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:33:34.858 INFO:teuthology.orchestra.run.smithi151.stdout: "num_ops": 0 2024-04-24T15:33:34.859 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:34.868 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-24T15:33:34.910 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:34.910+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:34.936 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-24T15:33:34.997 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:33:34.998+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:33:35.037 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:35.037 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:33:35.037 INFO:teuthology.orchestra.run.smithi151.stdout: "complaint_time": 30, 2024-04-24T15:33:35.038 INFO:teuthology.orchestra.run.smithi151.stdout: "num_blocked_ops": 0 2024-04-24T15:33:35.038 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:35.048 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-24T15:33:35.132 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:35.133+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:35.199 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:35.199+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:35.214 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:35.214 INFO:teuthology.orchestra.run.smithi151.stdout: "size": 20, 2024-04-24T15:33:35.214 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 600, 2024-04-24T15:33:35.214 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [ 2024-04-24T15:33:35.214 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.214 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23613 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 14688256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:35.214 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.792522+0000", 2024-04-24T15:33:35.214 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 60.421792216, 2024-04-24T15:33:35.215 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040507356000000001, 2024-04-24T15:33:35.215 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:35.215 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:35.215 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:35.215 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:35.215 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:35.215 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23613 2024-04-24T15:33:35.215 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.215 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:35.215 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.215 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:35.215 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792522+0000", 2024-04-24T15:33:35.215 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:35.215 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.215 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.216 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:35.216 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792522+0000", 2024-04-24T15:33:35.216 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:35.216 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.216 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.216 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:35.216 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792522+0000", 2024-04-24T15:33:35.216 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:35.216 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.216 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.216 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:35.216 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792523+0000", 2024-04-24T15:33:35.216 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.576e-06 2024-04-24T15:33:35.216 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.216 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.217 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:35.217 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792526+0000", 2024-04-24T15:33:35.217 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7439999999999999e-06 2024-04-24T15:33:35.217 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.217 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.217 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:35.217 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792528+0000", 2024-04-24T15:33:35.217 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.0600000000000002e-06 2024-04-24T15:33:35.217 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.217 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.217 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:35.217 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.832864+0000", 2024-04-24T15:33:35.217 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040335742000000001 2024-04-24T15:33:35.217 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.217 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.218 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:35.218 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.832953+0000", 2024-04-24T15:33:35.218 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.9281000000000001e-05 2024-04-24T15:33:35.218 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.218 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.218 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:35.218 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.833029+0000", 2024-04-24T15:33:35.218 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.6259000000000003e-05 2024-04-24T15:33:35.218 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:35.218 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:35.218 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:35.218 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.218 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.218 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23614 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 14729216~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:35.219 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.792530+0000", 2024-04-24T15:33:35.219 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 60.421783951999998, 2024-04-24T15:33:35.219 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040633957999999998, 2024-04-24T15:33:35.219 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:35.219 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:35.219 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:35.219 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:35.219 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:35.219 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23614 2024-04-24T15:33:35.219 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.219 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:35.219 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.219 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:35.220 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792530+0000", 2024-04-24T15:33:35.220 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:35.220 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.220 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.220 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:35.220 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792530+0000", 2024-04-24T15:33:35.220 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:35.220 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.220 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.220 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:35.220 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792530+0000", 2024-04-24T15:33:35.220 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:35.220 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.220 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.221 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:35.221 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792531+0000", 2024-04-24T15:33:35.221 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.46e-06 2024-04-24T15:33:35.221 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.221 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.221 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:35.221 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792534+0000", 2024-04-24T15:33:35.221 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8959999999999999e-06 2024-04-24T15:33:35.221 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.221 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.222 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:35.222 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792536+0000", 2024-04-24T15:33:35.222 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.841e-06 2024-04-24T15:33:35.222 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.222 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.222 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:35.222 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.833060+0000", 2024-04-24T15:33:35.222 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040524125000000001 2024-04-24T15:33:35.222 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.222 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.222 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:35.222 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.833108+0000", 2024-04-24T15:33:35.222 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.7547999999999999e-05 2024-04-24T15:33:35.222 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.223 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.223 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:35.223 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.833164+0000", 2024-04-24T15:33:35.223 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6297e-05 2024-04-24T15:33:35.223 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:35.223 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:35.223 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:35.223 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.223 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.223 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23623 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15097856~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:35.223 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.793887+0000", 2024-04-24T15:33:35.223 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 60.420426990999999, 2024-04-24T15:33:35.223 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040432784999999999, 2024-04-24T15:33:35.223 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:35.223 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:35.223 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:35.224 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:35.224 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:35.224 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23623 2024-04-24T15:33:35.224 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.224 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:35.224 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.224 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:35.224 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793887+0000", 2024-04-24T15:33:35.224 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:35.224 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.224 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.224 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:35.224 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793887+0000", 2024-04-24T15:33:35.224 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:35.224 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.225 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.225 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:35.225 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793887+0000", 2024-04-24T15:33:35.225 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:35.225 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.225 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.225 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:35.225 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793888+0000", 2024-04-24T15:33:35.225 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3659999999999999e-06 2024-04-24T15:33:35.225 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.225 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.225 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:35.225 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793891+0000", 2024-04-24T15:33:35.225 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8389999999999998e-06 2024-04-24T15:33:35.226 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.226 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.226 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:35.226 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793893+0000", 2024-04-24T15:33:35.226 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.9549999999999999e-06 2024-04-24T15:33:35.226 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.226 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.226 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:35.226 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834239+0000", 2024-04-24T15:33:35.226 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040345487999999999 2024-04-24T15:33:35.226 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.226 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.226 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:35.226 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834280+0000", 2024-04-24T15:33:35.227 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.1878000000000001e-05 2024-04-24T15:33:35.227 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.227 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.227 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:35.227 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834320+0000", 2024-04-24T15:33:35.227 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9480000000000001e-05 2024-04-24T15:33:35.227 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:35.227 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:35.227 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:35.227 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.227 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.227 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23624 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15138816~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:35.227 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.794053+0000", 2024-04-24T15:33:35.227 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 60.420261042999996, 2024-04-24T15:33:35.228 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040370801999999997, 2024-04-24T15:33:35.228 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:35.228 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:35.228 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:35.228 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:35.228 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:35.228 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23624 2024-04-24T15:33:35.228 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.228 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:35.228 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.228 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:35.228 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794053+0000", 2024-04-24T15:33:35.228 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:35.228 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.228 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.229 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:35.229 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794053+0000", 2024-04-24T15:33:35.229 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:35.229 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.229 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.229 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:35.229 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793895+0000", 2024-04-24T15:33:35.229 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998422 2024-04-24T15:33:35.229 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.229 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.229 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:35.229 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794057+0000", 2024-04-24T15:33:35.229 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00016149400000000001 2024-04-24T15:33:35.229 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.229 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.230 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:35.230 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794065+0000", 2024-04-24T15:33:35.230 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.5259999999999993e-06 2024-04-24T15:33:35.230 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.230 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.230 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:35.230 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794071+0000", 2024-04-24T15:33:35.230 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6149999999999996e-06 2024-04-24T15:33:35.230 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.230 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.230 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:35.230 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834341+0000", 2024-04-24T15:33:35.231 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040270366000000002 2024-04-24T15:33:35.231 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.231 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.231 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:35.231 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834381+0000", 2024-04-24T15:33:35.231 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0102999999999999e-05 2024-04-24T15:33:35.231 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.231 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.231 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:35.232 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834424+0000", 2024-04-24T15:33:35.232 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2713000000000003e-05 2024-04-24T15:33:35.232 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:35.232 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:35.232 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:35.232 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.232 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.232 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23625 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15179776~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:35.232 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.794160+0000", 2024-04-24T15:33:35.232 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 60.420154160999999, 2024-04-24T15:33:35.232 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040413004000000002, 2024-04-24T15:33:35.232 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:35.232 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:35.232 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:35.233 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:35.233 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:35.233 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23625 2024-04-24T15:33:35.233 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.233 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:35.233 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.233 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:35.233 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794160+0000", 2024-04-24T15:33:35.233 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:35.233 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.233 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.233 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:35.233 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794160+0000", 2024-04-24T15:33:35.233 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:35.233 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.233 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.234 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:35.234 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794086+0000", 2024-04-24T15:33:35.234 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999256 2024-04-24T15:33:35.234 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.234 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.234 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:35.234 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794163+0000", 2024-04-24T15:33:35.234 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7896000000000001e-05 2024-04-24T15:33:35.234 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.234 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.234 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:35.234 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794171+0000", 2024-04-24T15:33:35.234 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.9500000000000001e-06 2024-04-24T15:33:35.235 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.235 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.235 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:35.235 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794179+0000", 2024-04-24T15:33:35.235 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7970000000000001e-06 2024-04-24T15:33:35.235 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.235 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.235 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:35.235 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834451+0000", 2024-04-24T15:33:35.235 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040271492999999998 2024-04-24T15:33:35.235 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.235 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.235 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:35.235 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834519+0000", 2024-04-24T15:33:35.235 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.7984000000000005e-05 2024-04-24T15:33:35.236 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.236 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.236 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:35.236 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834573+0000", 2024-04-24T15:33:35.236 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.4339999999999998e-05 2024-04-24T15:33:35.236 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:35.236 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:35.236 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:35.236 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.236 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.236 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23629 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15343616~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:35.236 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.794748+0000", 2024-04-24T15:33:35.236 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 60.419566359000001, 2024-04-24T15:33:35.236 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040419718, 2024-04-24T15:33:35.236 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:35.237 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:35.237 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:35.237 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:35.237 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:35.237 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23629 2024-04-24T15:33:35.237 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.237 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:35.237 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.237 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:35.237 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794748+0000", 2024-04-24T15:33:35.237 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:35.237 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.237 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.237 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:35.238 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794748+0000", 2024-04-24T15:33:35.238 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:35.238 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.238 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.238 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:35.238 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794673+0000", 2024-04-24T15:33:35.238 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999251 2024-04-24T15:33:35.238 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.238 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.238 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:35.238 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794751+0000", 2024-04-24T15:33:35.239 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.8480999999999996e-05 2024-04-24T15:33:35.239 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.239 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.239 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:35.239 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794760+0000", 2024-04-24T15:33:35.239 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3429999999999996e-06 2024-04-24T15:33:35.239 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.239 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.239 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:35.239 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794764+0000", 2024-04-24T15:33:35.239 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5600000000000004e-06 2024-04-24T15:33:35.239 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.239 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.239 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:35.240 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835040+0000", 2024-04-24T15:33:35.240 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040275188000000003 2024-04-24T15:33:35.240 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.240 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.240 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:35.240 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835105+0000", 2024-04-24T15:33:35.240 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.5992000000000001e-05 2024-04-24T15:33:35.240 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.240 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.240 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:35.240 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835167+0000", 2024-04-24T15:33:35.240 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.1981000000000002e-05 2024-04-24T15:33:35.240 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:35.240 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:35.240 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:35.241 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.241 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.241 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23630 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15384576~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:35.241 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.794839+0000", 2024-04-24T15:33:35.241 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 60.419475290000001, 2024-04-24T15:33:35.241 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040523894999999997, 2024-04-24T15:33:35.241 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:35.241 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:35.241 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:35.242 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:35.242 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:35.242 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23630 2024-04-24T15:33:35.242 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.242 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:35.242 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.242 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:35.242 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794839+0000", 2024-04-24T15:33:35.242 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:35.242 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.242 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.242 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:35.243 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794839+0000", 2024-04-24T15:33:35.243 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:35.243 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.243 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.243 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:35.243 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794768+0000", 2024-04-24T15:33:35.243 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999294 2024-04-24T15:33:35.243 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.243 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.243 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:35.243 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794841+0000", 2024-04-24T15:33:35.243 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2793000000000006e-05 2024-04-24T15:33:35.243 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.243 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.243 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:35.244 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794846+0000", 2024-04-24T15:33:35.244 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.7230000000000004e-06 2024-04-24T15:33:35.244 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.244 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.244 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:35.244 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794848+0000", 2024-04-24T15:33:35.244 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7659999999999999e-06 2024-04-24T15:33:35.244 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.244 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.244 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:35.244 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835223+0000", 2024-04-24T15:33:35.244 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040374275000000001 2024-04-24T15:33:35.244 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.244 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.244 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:35.245 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835295+0000", 2024-04-24T15:33:35.245 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2633999999999996e-05 2024-04-24T15:33:35.245 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.245 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.245 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:35.245 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835363+0000", 2024-04-24T15:33:35.245 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.7355999999999997e-05 2024-04-24T15:33:35.245 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:35.245 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:35.245 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:35.245 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.245 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.245 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23631 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15425536~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:35.245 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.794956+0000", 2024-04-24T15:33:35.246 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 60.419358449000001, 2024-04-24T15:33:35.246 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040562383, 2024-04-24T15:33:35.246 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:35.246 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:35.246 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:35.246 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:35.246 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:35.246 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23631 2024-04-24T15:33:35.246 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.246 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:35.246 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.246 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:35.246 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794956+0000", 2024-04-24T15:33:35.247 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:35.247 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.247 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.247 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:35.247 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794956+0000", 2024-04-24T15:33:35.247 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:35.247 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.247 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.247 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:35.247 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794859+0000", 2024-04-24T15:33:35.247 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999032 2024-04-24T15:33:35.247 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.247 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.247 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:35.247 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794959+0000", 2024-04-24T15:33:35.248 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000100207 2024-04-24T15:33:35.248 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.248 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.248 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:35.248 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794967+0000", 2024-04-24T15:33:35.248 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7640000000000005e-06 2024-04-24T15:33:35.248 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.248 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.248 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:35.248 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794971+0000", 2024-04-24T15:33:35.248 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.335e-06 2024-04-24T15:33:35.248 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.248 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.248 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:35.248 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835388+0000", 2024-04-24T15:33:35.249 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040417340000000003 2024-04-24T15:33:35.249 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.249 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.249 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:35.249 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835459+0000", 2024-04-24T15:33:35.249 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.0118999999999997e-05 2024-04-24T15:33:35.249 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.249 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.249 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:35.249 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835518+0000", 2024-04-24T15:33:35.249 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.9438999999999998e-05 2024-04-24T15:33:35.249 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:35.249 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:35.249 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:35.249 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.250 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.250 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23632 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15466496~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:35.250 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795060+0000", 2024-04-24T15:33:35.250 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 60.419253738999998, 2024-04-24T15:33:35.250 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040667993, 2024-04-24T15:33:35.250 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:35.250 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:35.250 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:35.250 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:35.250 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:35.250 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23632 2024-04-24T15:33:35.250 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.250 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:35.250 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.250 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:35.251 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795060+0000", 2024-04-24T15:33:35.251 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:35.251 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.251 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.251 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:35.251 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795060+0000", 2024-04-24T15:33:35.251 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:35.251 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.251 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.251 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:35.251 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794976+0000", 2024-04-24T15:33:35.251 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999156 2024-04-24T15:33:35.251 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.251 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.251 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:35.252 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795064+0000", 2024-04-24T15:33:35.252 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.7709999999999999e-05 2024-04-24T15:33:35.252 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.252 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.252 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:35.252 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795072+0000", 2024-04-24T15:33:35.252 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3769999999999996e-06 2024-04-24T15:33:35.252 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.252 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.253 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:35.253 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795078+0000", 2024-04-24T15:33:35.253 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.5079999999999996e-06 2024-04-24T15:33:35.253 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.253 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.253 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:35.253 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835541+0000", 2024-04-24T15:33:35.253 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040463440000000003 2024-04-24T15:33:35.253 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.253 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.253 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:35.253 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835657+0000", 2024-04-24T15:33:35.253 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00011572499999999999 2024-04-24T15:33:35.253 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.254 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.254 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:35.254 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835728+0000", 2024-04-24T15:33:35.254 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1402000000000005e-05 2024-04-24T15:33:35.254 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:35.254 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:35.254 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:35.254 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.254 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.254 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23633 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15507456~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:35.254 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795134+0000", 2024-04-24T15:33:35.254 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 60.419180083999997, 2024-04-24T15:33:35.254 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040748591000000001, 2024-04-24T15:33:35.255 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:35.255 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:35.255 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:35.255 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:35.255 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:35.255 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23633 2024-04-24T15:33:35.255 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.255 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:35.255 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.255 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:35.255 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795134+0000", 2024-04-24T15:33:35.255 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:35.255 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.255 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.256 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:35.256 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795134+0000", 2024-04-24T15:33:35.256 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:35.256 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.256 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.256 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:35.256 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795083+0000", 2024-04-24T15:33:35.256 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999949 2024-04-24T15:33:35.256 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.256 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.256 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:35.256 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795138+0000", 2024-04-24T15:33:35.256 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.4472000000000003e-05 2024-04-24T15:33:35.256 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.257 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.257 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:35.257 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795145+0000", 2024-04-24T15:33:35.257 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2779999999999996e-06 2024-04-24T15:33:35.257 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.257 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.257 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:35.257 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795149+0000", 2024-04-24T15:33:35.257 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.568e-06 2024-04-24T15:33:35.257 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.257 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.257 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:35.257 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835757+0000", 2024-04-24T15:33:35.258 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040608894999999999 2024-04-24T15:33:35.258 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.258 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.258 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:35.258 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835829+0000", 2024-04-24T15:33:35.258 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1879999999999996e-05 2024-04-24T15:33:35.258 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.258 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.258 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:35.258 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835883+0000", 2024-04-24T15:33:35.258 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.3346000000000001e-05 2024-04-24T15:33:35.258 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:35.258 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:35.259 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:35.259 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.259 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.259 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23634 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15548416~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:35.259 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795271+0000", 2024-04-24T15:33:35.259 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 60.419043049999999, 2024-04-24T15:33:35.259 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040755845999999998, 2024-04-24T15:33:35.259 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:35.259 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:35.259 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:35.259 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:35.259 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:35.259 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23634 2024-04-24T15:33:35.259 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.260 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:35.260 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.260 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:35.260 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795271+0000", 2024-04-24T15:33:35.260 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:35.260 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.260 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.260 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:35.260 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795271+0000", 2024-04-24T15:33:35.260 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:35.260 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.260 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.260 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:35.260 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795152+0000", 2024-04-24T15:33:35.261 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998813 2024-04-24T15:33:35.261 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.261 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.261 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:35.261 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795276+0000", 2024-04-24T15:33:35.261 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00012401799999999999 2024-04-24T15:33:35.261 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.261 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.261 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:35.261 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795281+0000", 2024-04-24T15:33:35.261 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8860000000000003e-06 2024-04-24T15:33:35.261 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.261 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.262 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:35.262 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795284+0000", 2024-04-24T15:33:35.262 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.0450000000000001e-06 2024-04-24T15:33:35.262 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.262 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.262 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:35.262 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835905+0000", 2024-04-24T15:33:35.262 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040621035999999999 2024-04-24T15:33:35.263 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.263 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.263 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:35.263 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835971+0000", 2024-04-24T15:33:35.263 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.5314000000000005e-05 2024-04-24T15:33:35.263 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.263 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.263 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:35.263 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836027+0000", 2024-04-24T15:33:35.263 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6305999999999999e-05 2024-04-24T15:33:35.263 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:35.263 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:35.263 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:35.263 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.263 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.264 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23635 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15589376~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:35.264 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795382+0000", 2024-04-24T15:33:35.264 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 60.418932277000003, 2024-04-24T15:33:35.264 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040805363999999997, 2024-04-24T15:33:35.264 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:35.264 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:35.264 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:35.264 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:35.264 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:35.264 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23635 2024-04-24T15:33:35.264 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.264 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:35.264 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.264 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:35.265 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795382+0000", 2024-04-24T15:33:35.265 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:35.265 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.265 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.265 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:35.265 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795382+0000", 2024-04-24T15:33:35.265 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:35.265 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.265 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.265 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:35.265 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795288+0000", 2024-04-24T15:33:35.265 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999056 2024-04-24T15:33:35.265 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.265 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.266 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:35.266 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795385+0000", 2024-04-24T15:33:35.266 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.7288999999999996e-05 2024-04-24T15:33:35.266 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.266 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.266 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:35.266 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795392+0000", 2024-04-24T15:33:35.266 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.452e-06 2024-04-24T15:33:35.266 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.266 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.266 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:35.266 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795397+0000", 2024-04-24T15:33:35.266 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2860000000000001e-06 2024-04-24T15:33:35.267 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.267 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.267 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:35.267 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836054+0000", 2024-04-24T15:33:35.267 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040657581999999998 2024-04-24T15:33:35.267 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.267 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.267 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:35.267 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836124+0000", 2024-04-24T15:33:35.267 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.0020999999999999e-05 2024-04-24T15:33:35.267 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.267 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.267 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:35.267 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836187+0000", 2024-04-24T15:33:35.268 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.3009999999999995e-05 2024-04-24T15:33:35.268 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:35.268 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:35.268 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:35.268 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.268 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.268 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23636 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15630336~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:35.268 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795610+0000", 2024-04-24T15:33:35.268 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 60.418703919000002, 2024-04-24T15:33:35.268 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040730942999999999, 2024-04-24T15:33:35.268 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:35.268 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:35.268 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:35.268 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:35.268 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:35.269 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23636 2024-04-24T15:33:35.269 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.269 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:35.269 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.269 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:35.269 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795610+0000", 2024-04-24T15:33:35.269 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:35.269 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.269 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.269 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:35.269 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795610+0000", 2024-04-24T15:33:35.269 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:35.269 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.269 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.269 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:35.270 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795401+0000", 2024-04-24T15:33:35.270 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9997911 2024-04-24T15:33:35.270 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.270 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.270 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:35.270 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795614+0000", 2024-04-24T15:33:35.270 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00021228600000000001 2024-04-24T15:33:35.270 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.270 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.270 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:35.270 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795622+0000", 2024-04-24T15:33:35.270 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3920000000000003e-06 2024-04-24T15:33:35.270 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.271 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.271 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:35.271 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795627+0000", 2024-04-24T15:33:35.271 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8199999999999996e-06 2024-04-24T15:33:35.271 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.271 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.271 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:35.271 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836211+0000", 2024-04-24T15:33:35.271 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040583803000000002 2024-04-24T15:33:35.271 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.271 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.271 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:35.271 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836284+0000", 2024-04-24T15:33:35.271 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.3617999999999999e-05 2024-04-24T15:33:35.271 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.272 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.272 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:35.272 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836341+0000", 2024-04-24T15:33:35.272 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6959999999999997e-05 2024-04-24T15:33:35.272 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:35.272 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:35.272 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:35.272 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.272 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.272 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23637 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15671296~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:35.273 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795699+0000", 2024-04-24T15:33:35.273 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 60.418614752000003, 2024-04-24T15:33:35.273 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040775556999999997, 2024-04-24T15:33:35.273 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:35.273 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:35.273 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:35.273 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:35.273 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:35.274 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23637 2024-04-24T15:33:35.274 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.274 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:35.274 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.274 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:35.274 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795699+0000", 2024-04-24T15:33:35.274 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:35.274 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.274 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.274 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:35.274 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795699+0000", 2024-04-24T15:33:35.274 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:35.274 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.275 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.275 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:35.275 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795632+0000", 2024-04-24T15:33:35.275 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999323 2024-04-24T15:33:35.275 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.275 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.275 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:35.275 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795704+0000", 2024-04-24T15:33:35.275 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2428000000000005e-05 2024-04-24T15:33:35.275 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.275 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.275 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:35.275 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795715+0000", 2024-04-24T15:33:35.275 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.1532000000000001e-05 2024-04-24T15:33:35.276 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.276 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.276 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:35.276 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795721+0000", 2024-04-24T15:33:35.276 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.8200000000000002e-06 2024-04-24T15:33:35.276 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.276 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.276 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:35.276 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836364+0000", 2024-04-24T15:33:35.276 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040642997 2024-04-24T15:33:35.276 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.276 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.276 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:35.276 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836435+0000", 2024-04-24T15:33:35.277 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.0821000000000005e-05 2024-04-24T15:33:35.277 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.277 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.277 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:35.277 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836475+0000", 2024-04-24T15:33:35.277 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9796999999999999e-05 2024-04-24T15:33:35.277 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:35.277 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:35.277 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:35.277 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.277 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.277 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23638 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15712256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:35.277 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795932+0000", 2024-04-24T15:33:35.277 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 60.418381721999999, 2024-04-24T15:33:35.278 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040668682999999997, 2024-04-24T15:33:35.278 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:35.278 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:35.278 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:35.278 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:35.278 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:35.278 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23638 2024-04-24T15:33:35.278 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.278 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:35.278 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.278 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:35.278 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795932+0000", 2024-04-24T15:33:35.278 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:35.279 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.279 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.279 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:35.279 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795932+0000", 2024-04-24T15:33:35.279 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:35.279 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.279 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.279 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:35.279 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795728+0000", 2024-04-24T15:33:35.279 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9997959 2024-04-24T15:33:35.279 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.279 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.279 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:35.280 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795936+0000", 2024-04-24T15:33:35.280 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000208128 2024-04-24T15:33:35.280 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.280 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.280 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:35.280 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795945+0000", 2024-04-24T15:33:35.280 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.7110000000000001e-06 2024-04-24T15:33:35.280 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.280 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.280 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:35.280 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795957+0000", 2024-04-24T15:33:35.280 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.1800999999999999e-05 2024-04-24T15:33:35.280 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.280 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.281 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:35.281 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836505+0000", 2024-04-24T15:33:35.281 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040547622999999998 2024-04-24T15:33:35.281 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.281 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.281 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:35.281 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836561+0000", 2024-04-24T15:33:35.281 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6598999999999998e-05 2024-04-24T15:33:35.281 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.281 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.281 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:35.281 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836601+0000", 2024-04-24T15:33:35.281 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9907000000000003e-05 2024-04-24T15:33:35.281 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:35.282 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:35.282 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:35.282 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.282 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.282 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23639 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15753216~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:35.282 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.796071+0000", 2024-04-24T15:33:35.282 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 60.418243461000003, 2024-04-24T15:33:35.282 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040648199000000003, 2024-04-24T15:33:35.282 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:35.282 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:35.282 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:35.282 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:35.282 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:35.283 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23639 2024-04-24T15:33:35.283 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.283 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:35.283 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.283 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:35.283 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796071+0000", 2024-04-24T15:33:35.283 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:35.283 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.283 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.284 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:35.284 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796071+0000", 2024-04-24T15:33:35.284 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:35.284 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.284 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.284 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:35.284 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795967+0000", 2024-04-24T15:33:35.284 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998965 2024-04-24T15:33:35.284 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.284 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.284 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:35.284 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796074+0000", 2024-04-24T15:33:35.284 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000107013 2024-04-24T15:33:35.284 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.285 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.285 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:35.285 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796081+0000", 2024-04-24T15:33:35.285 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2590000000000003e-06 2024-04-24T15:33:35.285 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.285 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.285 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:35.285 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796085+0000", 2024-04-24T15:33:35.285 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5269999999999999e-06 2024-04-24T15:33:35.285 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.285 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.285 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:35.285 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836624+0000", 2024-04-24T15:33:35.286 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040539324000000002 2024-04-24T15:33:35.286 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.286 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.286 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:35.286 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836667+0000", 2024-04-24T15:33:35.286 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2562999999999999e-05 2024-04-24T15:33:35.286 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.286 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.286 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:35.286 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836719+0000", 2024-04-24T15:33:35.286 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.2015999999999998e-05 2024-04-24T15:33:35.286 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:35.286 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:35.286 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:35.287 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.287 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.287 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23640 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15794176~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:35.287 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.796492+0000", 2024-04-24T15:33:35.287 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 60.417821703000001, 2024-04-24T15:33:35.287 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040420000999999997, 2024-04-24T15:33:35.287 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:35.287 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:35.287 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:35.287 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:35.287 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:35.287 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23640 2024-04-24T15:33:35.288 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.288 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:35.288 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.288 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:35.288 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796492+0000", 2024-04-24T15:33:35.288 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:35.288 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.288 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.288 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:35.288 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796492+0000", 2024-04-24T15:33:35.288 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:35.288 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.288 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.288 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:35.288 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796088+0000", 2024-04-24T15:33:35.289 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9995961 2024-04-24T15:33:35.289 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.289 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.289 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:35.289 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796496+0000", 2024-04-24T15:33:35.289 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00040785700000000002 2024-04-24T15:33:35.289 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.289 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.289 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:35.289 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796503+0000", 2024-04-24T15:33:35.289 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1049999999999997e-06 2024-04-24T15:33:35.289 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.289 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.289 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:35.289 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796508+0000", 2024-04-24T15:33:35.289 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.155e-06 2024-04-24T15:33:35.290 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.290 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.290 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:35.290 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836756+0000", 2024-04-24T15:33:35.290 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040248850000000003 2024-04-24T15:33:35.290 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.290 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.290 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:35.290 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836841+0000", 2024-04-24T15:33:35.290 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.4735999999999996e-05 2024-04-24T15:33:35.290 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.290 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.290 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:35.290 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836912+0000", 2024-04-24T15:33:35.291 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1229999999999994e-05 2024-04-24T15:33:35.291 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:35.291 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:35.291 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:35.291 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.291 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.291 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23641 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15835136~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:35.291 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.796658+0000", 2024-04-24T15:33:35.291 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 60.417655627000002, 2024-04-24T15:33:35.291 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040420706000000001, 2024-04-24T15:33:35.291 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:35.291 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:35.291 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:35.291 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:35.291 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:35.291 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23641 2024-04-24T15:33:35.292 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.292 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:35.292 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.292 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:35.292 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796658+0000", 2024-04-24T15:33:35.292 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:35.292 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.292 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.292 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:35.292 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796658+0000", 2024-04-24T15:33:35.292 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:35.292 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.292 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.292 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:35.293 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796512+0000", 2024-04-24T15:33:35.293 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998531 2024-04-24T15:33:35.293 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.293 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.293 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:35.293 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796669+0000", 2024-04-24T15:33:35.293 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000157577 2024-04-24T15:33:35.293 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.293 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.293 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:35.293 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796677+0000", 2024-04-24T15:33:35.293 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7800000000000001e-06 2024-04-24T15:33:35.293 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.294 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.294 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:35.294 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796681+0000", 2024-04-24T15:33:35.294 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.721e-06 2024-04-24T15:33:35.294 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.294 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.294 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:35.294 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836941+0000", 2024-04-24T15:33:35.294 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040259907999999997 2024-04-24T15:33:35.294 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.294 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.294 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:35.295 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837017+0000", 2024-04-24T15:33:35.295 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.6071999999999999e-05 2024-04-24T15:33:35.295 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.295 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.295 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:35.295 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837079+0000", 2024-04-24T15:33:35.295 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.2358000000000002e-05 2024-04-24T15:33:35.295 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:35.295 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:35.295 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:35.295 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.295 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.295 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23645 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15998976~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:35.295 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.797360+0000", 2024-04-24T15:33:35.296 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 60.416954128999997, 2024-04-24T15:33:35.296 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040387645999999999, 2024-04-24T15:33:35.296 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:35.296 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:35.296 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:35.296 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:35.296 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:35.296 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23645 2024-04-24T15:33:35.296 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.296 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:35.296 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.296 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:35.296 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797360+0000", 2024-04-24T15:33:35.296 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:35.297 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.297 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.297 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:35.297 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797360+0000", 2024-04-24T15:33:35.297 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:35.297 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.297 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.297 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:35.297 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797288+0000", 2024-04-24T15:33:35.297 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999928 2024-04-24T15:33:35.297 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.297 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.297 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:35.297 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797364+0000", 2024-04-24T15:33:35.298 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.5347999999999998e-05 2024-04-24T15:33:35.298 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.298 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.298 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:35.298 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797372+0000", 2024-04-24T15:33:35.298 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.2409999999999995e-06 2024-04-24T15:33:35.298 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.298 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.298 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:35.298 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797376+0000", 2024-04-24T15:33:35.298 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.515e-06 2024-04-24T15:33:35.298 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.298 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.299 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:35.299 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837586+0000", 2024-04-24T15:33:35.299 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040210047999999998 2024-04-24T15:33:35.299 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.299 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.299 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:35.299 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837672+0000", 2024-04-24T15:33:35.299 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.5997999999999999e-05 2024-04-24T15:33:35.299 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.299 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.299 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:35.299 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837748+0000", 2024-04-24T15:33:35.299 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.5328999999999997e-05 2024-04-24T15:33:35.299 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:35.300 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:35.300 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:35.300 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.300 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.300 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23646 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 16039936~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:35.300 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.797498+0000", 2024-04-24T15:33:35.300 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 60.416816028, 2024-04-24T15:33:35.300 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040433139, 2024-04-24T15:33:35.300 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:35.300 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:35.300 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:35.300 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:35.300 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:35.300 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23646 2024-04-24T15:33:35.300 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.301 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:35.301 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.301 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:35.301 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797498+0000", 2024-04-24T15:33:35.301 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:35.301 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.301 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.301 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:35.301 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797498+0000", 2024-04-24T15:33:35.301 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:35.301 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.301 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.301 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:35.301 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797381+0000", 2024-04-24T15:33:35.301 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998832 2024-04-24T15:33:35.302 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.302 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.302 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:35.302 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797501+0000", 2024-04-24T15:33:35.302 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000119759 2024-04-24T15:33:35.302 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.302 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.302 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:35.302 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797510+0000", 2024-04-24T15:33:35.302 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.4770000000000003e-06 2024-04-24T15:33:35.302 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.302 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.302 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:35.303 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797515+0000", 2024-04-24T15:33:35.303 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8330000000000001e-06 2024-04-24T15:33:35.303 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.303 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.303 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:35.303 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837777+0000", 2024-04-24T15:33:35.303 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040262502999999998 2024-04-24T15:33:35.303 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.303 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.303 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:35.303 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837860+0000", 2024-04-24T15:33:35.303 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3002000000000003e-05 2024-04-24T15:33:35.303 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:35.303 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:35.303 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:35.303 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837931+0000", 2024-04-24T15:33:35.304 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1222000000000004e-05 2024-04-24T15:33:35.304 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:35.304 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:35.304 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:35.304 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:35.304 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:35.304 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:35.304 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-24T15:33:35.306 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-24T15:33:35.306 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:35.306+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:35.407 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:35.407+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:35.441 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-04-24T15:33:35.459 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-04-24T15:33:35.472 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:35.472 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:33:35.472 INFO:teuthology.orchestra.run.smithi151.stdout: "num_ops": 0 2024-04-24T15:33:35.472 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:35.476 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-04-24T15:33:35.482 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-24T15:33:35.494 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-04-24T15:33:35.507 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:35.507+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:35.512 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-04-24T15:33:35.530 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-04-24T15:33:35.549 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-04-24T15:33:35.549 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T15:33:35.549 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T15:33:35.549 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T15:33:35.549 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T15:33:35.549 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T15:33:35.550 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T15:33:35.550 INFO:teuthology.orchestra.run.smithi096.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-04-24T15:33:35.550 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T15:33:35.552 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-24T15:33:35.552 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-24T15:33:35.608 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:33:35.609+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:33:35.646 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:35.646 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:35.656 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-24T15:33:35.709 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:35.710+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:35.780 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:35.780 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:35.790 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-24T15:33:35.830 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:35.831+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:35.911 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:33:35.911+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:33:35.913 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:35.913 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:35.923 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-24T15:33:35.928 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 6, 4] out_osds: [1, 5, 7] dead_osds: [6] live_osds: [0, 1, 2, 3, 4, 5, 7] 2024-04-24T15:33:35.928 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-24T15:33:35.928 INFO:tasks.thrashosds.thrasher:Reviving osd 6 2024-04-24T15:33:35.928 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-24T15:33:35.929 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-24T15:33:35.931 INFO:tasks.ceph.osd.6:Started 2024-04-24T15:33:35.931 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-24T15:33:35.939 INFO:teuthology.orchestra.run.smithi096.stderr:noscrub is unset 2024-04-24T15:33:36.012 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:36.012+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:36.066 INFO:teuthology.orchestra.run.smithi151.stderr:no valid command found; 10 closest matches: 2024-04-24T15:33:36.066 INFO:teuthology.orchestra.run.smithi151.stderr:0 2024-04-24T15:33:36.066 INFO:teuthology.orchestra.run.smithi151.stderr:1 2024-04-24T15:33:36.066 INFO:teuthology.orchestra.run.smithi151.stderr:2 2024-04-24T15:33:36.066 INFO:teuthology.orchestra.run.smithi151.stderr:abort 2024-04-24T15:33:36.066 INFO:teuthology.orchestra.run.smithi151.stderr:assert 2024-04-24T15:33:36.066 INFO:teuthology.orchestra.run.smithi151.stderr:config diff 2024-04-24T15:33:36.066 INFO:teuthology.orchestra.run.smithi151.stderr:config diff get 2024-04-24T15:33:36.066 INFO:teuthology.orchestra.run.smithi151.stderr:config get 2024-04-24T15:33:36.067 INFO:teuthology.orchestra.run.smithi151.stderr:config help [] 2024-04-24T15:33:36.067 INFO:teuthology.orchestra.run.smithi151.stderr:config set ... 2024-04-24T15:33:36.067 INFO:teuthology.orchestra.run.smithi151.stderr:admin_socket: invalid command 2024-04-24T15:33:36.067 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:36.068 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:36.068 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-24T15:33:36.069 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-6, ['dump_ops_in_flight'] 2024-04-24T15:33:36.078 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-24T15:33:36.112 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:36.113+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:36.201 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:36.201 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:36.211 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-24T15:33:36.214 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:33:36.215+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:33:36.314 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:36.314+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:36.334 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:36.334 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:36.344 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-24T15:33:36.428 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:33:36.429+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:33:36.508 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:36.508 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:36.516 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:36.516+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:36.518 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-24T15:33:36.531 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:33:36.531+0000 7f0f4e2f4540 -1 Falling back to public interface 2024-04-24T15:33:36.617 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:36.618+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:36.680 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:36.681 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:36.690 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-24T15:33:36.718 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:36.719+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:36.813 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:36.813 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:36.818 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:33:36.819+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:33:36.823 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-24T15:33:36.919 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:36.921+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:36.951 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:36.951 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:36.964 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-24T15:33:37.020 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:33:37.020+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:33:37.100 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:37.100 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:37.110 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-24T15:33:37.152 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:37.152+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:37.222 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:33:37.222+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:33:37.273 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:37.274 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:37.283 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-24T15:33:37.323 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:37.324+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:37.424 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:37.425+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:37.447 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:37.447 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:37.457 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-24T15:33:37.524 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:33:37.525+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:33:37.553 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-24T15:33:37.582 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:37.582 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:37.593 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-24T15:33:37.637 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:37.638+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:37.693 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-04-24T15:33:37.711 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-04-24T15:33:37.726 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:33:37.726+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:33:37.728 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-04-24T15:33:37.729 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:37.729 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:37.740 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-24T15:33:37.746 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-04-24T15:33:37.765 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-04-24T15:33:37.782 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-04-24T15:33:37.801 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-04-24T15:33:37.801 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T15:33:37.801 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T15:33:37.801 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T15:33:37.801 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T15:33:37.801 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T15:33:37.801 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T15:33:37.801 INFO:teuthology.orchestra.run.smithi096.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-04-24T15:33:37.801 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T15:33:37.803 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-24T15:33:37.804 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-24T15:33:37.827 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:33:37.828+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:33:37.903 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:37.903 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:33:37.903 INFO:teuthology.orchestra.run.smithi151.stdout: "complaint_time": 30, 2024-04-24T15:33:37.904 INFO:teuthology.orchestra.run.smithi151.stdout: "num_blocked_ops": 0 2024-04-24T15:33:37.904 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:37.913 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-24T15:33:37.954 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:37.955+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:37.955 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-24T15:33:38.029 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:38.029+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:38.081 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:38.081 INFO:teuthology.orchestra.run.smithi151.stdout: "size": 20, 2024-04-24T15:33:38.081 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 600, 2024-04-24T15:33:38.081 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [ 2024-04-24T15:33:38.082 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.082 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:39041 4.1 4:82c1a902:::rbd_data.114abe589119.000000000000032c:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e68)", 2024-04-24T15:33:38.082 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:01.760552+0000", 2024-04-24T15:33:38.082 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 36.320756768999999, 2024-04-24T15:33:38.082 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.87570748300000001, 2024-04-24T15:33:38.082 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:38.082 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "reached pg", 2024-04-24T15:33:38.082 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:38.082 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:38.082 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:38.082 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 39041 2024-04-24T15:33:38.082 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.083 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:38.083 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.083 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:38.083 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760552+0000", 2024-04-24T15:33:38.083 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.083 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.083 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.083 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:38.083 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760552+0000", 2024-04-24T15:33:38.083 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.083 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.083 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.083 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:38.083 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760550+0000", 2024-04-24T15:33:38.084 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999986 2024-04-24T15:33:38.084 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.084 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.084 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:38.084 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760557+0000", 2024-04-24T15:33:38.084 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1420000000000004e-06 2024-04-24T15:33:38.084 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.084 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.084 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:38.084 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760572+0000", 2024-04-24T15:33:38.084 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.483e-05 2024-04-24T15:33:38.084 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.084 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.084 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:38.085 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760578+0000", 2024-04-24T15:33:38.085 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.2820000000000002e-06 2024-04-24T15:33:38.085 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.085 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.085 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:38.085 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760628+0000", 2024-04-24T15:33:38.085 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.0040000000000002e-05 2024-04-24T15:33:38.085 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.085 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.085 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op must wait for map", 2024-04-24T15:33:38.085 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760637+0000", 2024-04-24T15:33:38.085 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.6899999999999998e-06 2024-04-24T15:33:38.085 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.085 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.086 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:38.086 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.770562+0000", 2024-04-24T15:33:38.086 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0099252739999999996 2024-04-24T15:33:38.086 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.086 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.086 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for peered", 2024-04-24T15:33:38.086 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.770572+0000", 2024-04-24T15:33:38.086 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0105999999999999e-05 2024-04-24T15:33:38.086 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.086 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.086 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:38.086 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:02.636170+0000", 2024-04-24T15:33:38.086 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.86559722299999997 2024-04-24T15:33:38.086 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.087 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.087 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:38.087 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:02.636259+0000", 2024-04-24T15:33:38.087 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.9446999999999999e-05 2024-04-24T15:33:38.087 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.087 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:38.087 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.087 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.087 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.087 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48675 4.d 4:b465d49b:::rbd_data.114a858e6641.0000000000000002:17 [read 2768896~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:38.087 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.926403+0000", 2024-04-24T15:33:38.087 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.154905199000002, 2024-04-24T15:33:38.088 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035742807000000001, 2024-04-24T15:33:38.088 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:38.088 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:38.088 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:38.088 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:38.088 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:38.088 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48675 2024-04-24T15:33:38.088 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.088 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:38.088 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.088 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:38.088 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926403+0000", 2024-04-24T15:33:38.088 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.088 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.088 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.089 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:38.089 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926403+0000", 2024-04-24T15:33:38.089 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.089 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.089 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.089 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:38.089 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926403+0000", 2024-04-24T15:33:38.089 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:38.089 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.089 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.089 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:38.089 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926404+0000", 2024-04-24T15:33:38.089 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.424e-06 2024-04-24T15:33:38.089 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.090 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.090 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:38.090 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926413+0000", 2024-04-24T15:33:38.090 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.6850000000000007e-06 2024-04-24T15:33:38.090 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.090 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.090 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:38.090 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926417+0000", 2024-04-24T15:33:38.090 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.7859999999999998e-06 2024-04-24T15:33:38.090 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.090 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.090 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:38.090 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.961986+0000", 2024-04-24T15:33:38.090 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035568862 2024-04-24T15:33:38.091 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.091 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.091 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:38.091 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.962063+0000", 2024-04-24T15:33:38.091 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7257999999999998e-05 2024-04-24T15:33:38.091 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.091 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.091 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:38.091 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.962146+0000", 2024-04-24T15:33:38.091 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3094000000000001e-05 2024-04-24T15:33:38.091 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.091 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:38.091 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.091 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.092 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.092 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48676 4.d 4:b465d49b:::rbd_data.114a858e6641.0000000000000002:17 [read 2834432~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:38.092 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.926421+0000", 2024-04-24T15:33:38.092 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.154887656, 2024-04-24T15:33:38.092 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035829002999999998, 2024-04-24T15:33:38.092 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:38.092 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:38.092 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:38.092 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:38.092 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:38.092 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48676 2024-04-24T15:33:38.092 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.092 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:38.092 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.093 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:38.093 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926421+0000", 2024-04-24T15:33:38.093 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.093 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.093 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.093 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:38.093 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926421+0000", 2024-04-24T15:33:38.093 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.093 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.093 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.093 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:38.093 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926420+0000", 2024-04-24T15:33:38.093 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:38.093 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.094 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.094 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:38.094 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926423+0000", 2024-04-24T15:33:38.094 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.356e-06 2024-04-24T15:33:38.094 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.094 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.094 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:38.094 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926428+0000", 2024-04-24T15:33:38.094 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.3340000000000001e-06 2024-04-24T15:33:38.094 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.094 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.094 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:38.094 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926431+0000", 2024-04-24T15:33:38.094 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.2899999999999998e-06 2024-04-24T15:33:38.095 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.095 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.095 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:38.095 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.962156+0000", 2024-04-24T15:33:38.095 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035725159999999999 2024-04-24T15:33:38.095 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.095 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.095 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:38.095 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.962197+0000", 2024-04-24T15:33:38.095 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0457999999999998e-05 2024-04-24T15:33:38.095 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.095 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.095 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:38.095 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.962250+0000", 2024-04-24T15:33:38.096 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.2812000000000002e-05 2024-04-24T15:33:38.096 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.096 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:38.096 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.096 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.096 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.096 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48703 4.d 4:b59d5735:::rbd_data.114a858e6641.0000000000000004:17 [read 2637824~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:38.096 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928531+0000", 2024-04-24T15:33:38.096 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.152777029999999, 2024-04-24T15:33:38.096 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035675198999999998, 2024-04-24T15:33:38.096 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:38.096 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:38.096 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:38.096 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:38.097 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:38.097 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48703 2024-04-24T15:33:38.097 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.097 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:38.097 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.097 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:38.097 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928531+0000", 2024-04-24T15:33:38.097 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.097 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.097 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.097 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:38.097 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928531+0000", 2024-04-24T15:33:38.097 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.097 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.097 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.098 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:38.098 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928531+0000", 2024-04-24T15:33:38.098 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:38.098 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.098 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.098 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:38.098 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928532+0000", 2024-04-24T15:33:38.098 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.33e-06 2024-04-24T15:33:38.098 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.098 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.098 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:38.098 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928536+0000", 2024-04-24T15:33:38.098 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.4419999999999998e-06 2024-04-24T15:33:38.099 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.099 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.099 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:38.099 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928538+0000", 2024-04-24T15:33:38.099 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.0669999999999999e-06 2024-04-24T15:33:38.099 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.099 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.099 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:38.099 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964040+0000", 2024-04-24T15:33:38.099 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035501900000000003 2024-04-24T15:33:38.099 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.099 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.099 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:38.099 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964118+0000", 2024-04-24T15:33:38.099 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.8490000000000002e-05 2024-04-24T15:33:38.100 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.100 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.100 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:38.100 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964207+0000", 2024-04-24T15:33:38.100 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.8202999999999996e-05 2024-04-24T15:33:38.100 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.100 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:38.100 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.100 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.100 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.100 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48704 4.d 4:b59d5735:::rbd_data.114a858e6641.0000000000000004:17 [read 2703360~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:38.100 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928541+0000", 2024-04-24T15:33:38.100 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.152767686000001, 2024-04-24T15:33:38.100 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035763016000000002, 2024-04-24T15:33:38.101 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:38.101 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:38.101 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:38.101 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:38.101 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:38.101 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48704 2024-04-24T15:33:38.101 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.101 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:38.101 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.101 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:38.101 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928541+0000", 2024-04-24T15:33:38.101 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.101 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.101 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.101 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:38.102 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928541+0000", 2024-04-24T15:33:38.102 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.102 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.102 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.102 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:38.102 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928540+0000", 2024-04-24T15:33:38.102 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:38.102 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.102 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.102 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:38.102 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928542+0000", 2024-04-24T15:33:38.102 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.37e-06 2024-04-24T15:33:38.102 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.102 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.103 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:38.103 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928545+0000", 2024-04-24T15:33:38.103 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8229999999999999e-06 2024-04-24T15:33:38.103 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.103 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.103 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:38.103 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928547+0000", 2024-04-24T15:33:38.103 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.9360000000000002e-06 2024-04-24T15:33:38.103 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.103 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.103 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:38.103 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964219+0000", 2024-04-24T15:33:38.103 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035672813999999997 2024-04-24T15:33:38.103 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.104 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.104 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:38.104 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964259+0000", 2024-04-24T15:33:38.104 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0070000000000001e-05 2024-04-24T15:33:38.104 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.104 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.104 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:38.104 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964304+0000", 2024-04-24T15:33:38.104 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4229e-05 2024-04-24T15:33:38.104 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.104 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:38.104 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.104 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.104 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.104 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48719 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 1851392~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:38.105 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928549+0000", 2024-04-24T15:33:38.105 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.152759539000002, 2024-04-24T15:33:38.105 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035890645999999998, 2024-04-24T15:33:38.105 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:38.105 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:38.105 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:38.105 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:38.105 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:38.105 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48719 2024-04-24T15:33:38.105 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.105 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:38.105 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.105 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:38.105 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928549+0000", 2024-04-24T15:33:38.105 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.105 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.106 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.106 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:38.106 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928549+0000", 2024-04-24T15:33:38.106 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.106 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.106 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.106 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:38.106 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928549+0000", 2024-04-24T15:33:38.106 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:38.106 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.106 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.106 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:38.106 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928550+0000", 2024-04-24T15:33:38.106 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3829999999999999e-06 2024-04-24T15:33:38.107 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.107 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.107 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:38.107 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928553+0000", 2024-04-24T15:33:38.107 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7369999999999998e-06 2024-04-24T15:33:38.107 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.107 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.107 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:38.107 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928555+0000", 2024-04-24T15:33:38.107 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.8920000000000001e-06 2024-04-24T15:33:38.107 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.107 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.107 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:38.107 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964240+0000", 2024-04-24T15:33:38.107 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035685537000000003 2024-04-24T15:33:38.108 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.108 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.108 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:38.108 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964318+0000", 2024-04-24T15:33:38.108 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7838999999999998e-05 2024-04-24T15:33:38.108 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.108 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.108 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:38.108 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964439+0000", 2024-04-24T15:33:38.108 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000121564 2024-04-24T15:33:38.108 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.108 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:38.108 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.108 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.108 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.109 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48720 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 1916928~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:38.109 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928557+0000", 2024-04-24T15:33:38.109 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.152751693999999, 2024-04-24T15:33:38.109 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036005219999999998, 2024-04-24T15:33:38.109 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:38.109 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:38.109 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:38.109 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:38.109 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:38.109 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48720 2024-04-24T15:33:38.109 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.109 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:38.109 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.109 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:38.110 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928557+0000", 2024-04-24T15:33:38.110 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.110 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.110 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.110 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:38.110 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928557+0000", 2024-04-24T15:33:38.110 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.110 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.110 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.110 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:38.110 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928556+0000", 2024-04-24T15:33:38.110 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:38.110 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.110 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.111 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:38.111 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928558+0000", 2024-04-24T15:33:38.111 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.13e-06 2024-04-24T15:33:38.111 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.111 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.111 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:38.111 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928561+0000", 2024-04-24T15:33:38.111 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5530000000000001e-06 2024-04-24T15:33:38.111 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.111 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.111 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:38.111 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928563+0000", 2024-04-24T15:33:38.111 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.8810000000000001e-06 2024-04-24T15:33:38.111 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.112 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.112 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:38.112 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964450+0000", 2024-04-24T15:33:38.112 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035887040000000002 2024-04-24T15:33:38.112 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.112 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.112 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:38.112 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964494+0000", 2024-04-24T15:33:38.112 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3699000000000003e-05 2024-04-24T15:33:38.112 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.112 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.112 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:38.112 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964562+0000", 2024-04-24T15:33:38.112 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.8149000000000001e-05 2024-04-24T15:33:38.112 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.113 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:38.113 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.113 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.113 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.113 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48721 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 1982464~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:38.113 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928565+0000", 2024-04-24T15:33:38.113 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.152743170000001, 2024-04-24T15:33:38.113 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036233466999999998, 2024-04-24T15:33:38.113 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:38.113 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:38.113 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:38.113 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:38.113 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:38.113 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48721 2024-04-24T15:33:38.113 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.114 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:38.114 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.114 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:38.114 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928565+0000", 2024-04-24T15:33:38.114 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.114 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.114 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.114 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:38.114 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928565+0000", 2024-04-24T15:33:38.114 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.114 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.114 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.114 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:38.114 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928565+0000", 2024-04-24T15:33:38.115 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:38.115 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.115 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.115 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:38.115 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928566+0000", 2024-04-24T15:33:38.115 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.336e-06 2024-04-24T15:33:38.115 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.115 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.115 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:38.115 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928569+0000", 2024-04-24T15:33:38.115 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.9189999999999999e-06 2024-04-24T15:33:38.115 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.115 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.115 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:38.115 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928571+0000", 2024-04-24T15:33:38.116 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.1909999999999999e-06 2024-04-24T15:33:38.116 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.116 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.116 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:38.116 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964592+0000", 2024-04-24T15:33:38.116 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036020120000000003 2024-04-24T15:33:38.116 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.116 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.116 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:38.116 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964673+0000", 2024-04-24T15:33:38.116 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.1254000000000005e-05 2024-04-24T15:33:38.116 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.116 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.116 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:38.116 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964799+0000", 2024-04-24T15:33:38.117 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00012588700000000001 2024-04-24T15:33:38.117 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.117 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:38.117 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.117 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.117 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.117 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48722 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2048000~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:38.117 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928586+0000", 2024-04-24T15:33:38.117 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.152722045000001, 2024-04-24T15:33:38.117 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036331347999999999, 2024-04-24T15:33:38.117 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:38.117 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:38.117 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:38.117 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:38.118 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:38.118 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48722 2024-04-24T15:33:38.118 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.118 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:38.118 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.118 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:38.118 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928586+0000", 2024-04-24T15:33:38.118 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.118 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.118 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.118 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:38.118 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928586+0000", 2024-04-24T15:33:38.118 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.119 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.119 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.119 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:38.119 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928586+0000", 2024-04-24T15:33:38.119 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:38.119 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.119 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.119 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:38.119 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928587+0000", 2024-04-24T15:33:38.119 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3519999999999999e-06 2024-04-24T15:33:38.119 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.119 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.119 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:38.120 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928590+0000", 2024-04-24T15:33:38.120 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.9699999999999999e-06 2024-04-24T15:33:38.120 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.120 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.120 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:38.120 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928592+0000", 2024-04-24T15:33:38.120 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.097e-06 2024-04-24T15:33:38.120 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.120 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.120 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:38.120 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964810+0000", 2024-04-24T15:33:38.120 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036217249 2024-04-24T15:33:38.120 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.120 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.121 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:38.121 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964851+0000", 2024-04-24T15:33:38.121 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.1151e-05 2024-04-24T15:33:38.121 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.121 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.121 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:38.121 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964918+0000", 2024-04-24T15:33:38.121 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.6898999999999998e-05 2024-04-24T15:33:38.121 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.121 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:38.121 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.121 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.121 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.121 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48723 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2113536~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:38.121 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928598+0000", 2024-04-24T15:33:38.121 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.152710656, 2024-04-24T15:33:38.122 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036552125999999997, 2024-04-24T15:33:38.122 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:38.122 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:38.122 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:38.122 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:38.122 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:38.122 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48723 2024-04-24T15:33:38.122 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.122 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:38.122 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.122 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:38.122 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928598+0000", 2024-04-24T15:33:38.122 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.122 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.123 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.123 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:38.123 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928598+0000", 2024-04-24T15:33:38.123 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.123 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.123 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.123 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:38.123 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928597+0000", 2024-04-24T15:33:38.123 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:38.123 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.123 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.123 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:38.123 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928599+0000", 2024-04-24T15:33:38.123 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.621e-06 2024-04-24T15:33:38.124 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.124 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.124 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:38.124 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928602+0000", 2024-04-24T15:33:38.124 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.9629999999999998e-06 2024-04-24T15:33:38.124 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.124 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.124 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:38.124 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928607+0000", 2024-04-24T15:33:38.124 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8940000000000001e-06 2024-04-24T15:33:38.124 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.124 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.124 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:38.124 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964947+0000", 2024-04-24T15:33:38.124 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036340439000000002 2024-04-24T15:33:38.125 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.125 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.125 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:38.125 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965028+0000", 2024-04-24T15:33:38.125 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.1054e-05 2024-04-24T15:33:38.125 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.125 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.125 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:38.125 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965150+0000", 2024-04-24T15:33:38.125 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00012154000000000001 2024-04-24T15:33:38.125 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.125 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:38.125 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.125 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.125 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.126 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48724 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2179072~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:38.126 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928609+0000", 2024-04-24T15:33:38.126 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.152699281, 2024-04-24T15:33:38.126 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036679614999999999, 2024-04-24T15:33:38.126 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:38.126 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:38.126 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:38.126 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:38.126 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:38.126 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48724 2024-04-24T15:33:38.126 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.126 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:38.127 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.127 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:38.127 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928609+0000", 2024-04-24T15:33:38.127 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.127 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.127 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.127 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:38.127 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928609+0000", 2024-04-24T15:33:38.127 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.127 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.127 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.127 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:38.127 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928609+0000", 2024-04-24T15:33:38.127 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:38.128 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.128 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.128 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:38.128 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928610+0000", 2024-04-24T15:33:38.128 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3030000000000001e-06 2024-04-24T15:33:38.128 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.128 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.128 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:38.128 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928613+0000", 2024-04-24T15:33:38.128 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.762e-06 2024-04-24T15:33:38.128 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.128 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.128 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:38.128 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928615+0000", 2024-04-24T15:33:38.128 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.832e-06 2024-04-24T15:33:38.129 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.129 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.129 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:38.129 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965174+0000", 2024-04-24T15:33:38.129 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036559633000000001 2024-04-24T15:33:38.129 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.129 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.129 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:38.129 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965221+0000", 2024-04-24T15:33:38.129 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6239999999999998e-05 2024-04-24T15:33:38.129 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.129 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.129 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:38.129 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965289+0000", 2024-04-24T15:33:38.129 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.8053999999999996e-05 2024-04-24T15:33:38.130 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.130 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:38.130 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.130 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.130 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.130 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48725 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2244608~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:38.130 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928617+0000", 2024-04-24T15:33:38.130 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.152691511, 2024-04-24T15:33:38.130 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036902659999999997, 2024-04-24T15:33:38.130 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:38.130 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:38.130 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:38.130 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:38.130 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:38.131 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48725 2024-04-24T15:33:38.131 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.131 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:38.131 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.131 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:38.131 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928617+0000", 2024-04-24T15:33:38.131 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.131 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.131 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.131 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:38.131 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928617+0000", 2024-04-24T15:33:38.131 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.131 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.131 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.131 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:38.132 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928617+0000", 2024-04-24T15:33:38.132 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:38.132 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.132 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.132 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:38.132 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928618+0000", 2024-04-24T15:33:38.132 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.206e-06 2024-04-24T15:33:38.132 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.132 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.132 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:38.132 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928621+0000", 2024-04-24T15:33:38.132 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5130000000000001e-06 2024-04-24T15:33:38.132 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.132 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.132 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:38.133 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928623+0000", 2024-04-24T15:33:38.133 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.9640000000000002e-06 2024-04-24T15:33:38.133 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.133 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.133 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:38.133 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965318+0000", 2024-04-24T15:33:38.133 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036694953000000002 2024-04-24T15:33:38.133 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.133 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.133 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:38.133 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965398+0000", 2024-04-24T15:33:38.133 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.9986000000000003e-05 2024-04-24T15:33:38.133 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.133 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.133 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:38.134 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965520+0000", 2024-04-24T15:33:38.134 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000121247 2024-04-24T15:33:38.134 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.134 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:38.134 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.134 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.134 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.134 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48726 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2310144~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:38.134 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928625+0000", 2024-04-24T15:33:38.134 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.152682964, 2024-04-24T15:33:38.134 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037015521000000003, 2024-04-24T15:33:38.134 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:38.134 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:38.134 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:38.134 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:38.135 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:38.135 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48726 2024-04-24T15:33:38.135 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.135 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:38.135 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.135 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:38.135 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928625+0000", 2024-04-24T15:33:38.135 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.135 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.135 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.135 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:38.136 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928625+0000", 2024-04-24T15:33:38.136 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.136 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.136 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.136 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:38.136 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928625+0000", 2024-04-24T15:33:38.136 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:38.136 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.136 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.136 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:38.136 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928627+0000", 2024-04-24T15:33:38.136 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3179999999999999e-06 2024-04-24T15:33:38.136 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.136 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.137 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:38.137 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928629+0000", 2024-04-24T15:33:38.137 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.819e-06 2024-04-24T15:33:38.137 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.137 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.137 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:38.137 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928634+0000", 2024-04-24T15:33:38.137 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6999999999999999e-06 2024-04-24T15:33:38.137 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.137 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.137 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:38.137 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965531+0000", 2024-04-24T15:33:38.137 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036897040999999998 2024-04-24T15:33:38.137 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.138 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.138 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:38.138 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965573+0000", 2024-04-24T15:33:38.138 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.189e-05 2024-04-24T15:33:38.138 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.138 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.138 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:38.138 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965641+0000", 2024-04-24T15:33:38.138 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.7959000000000004e-05 2024-04-24T15:33:38.138 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.138 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:38.138 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.138 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.138 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.139 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48727 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2375680~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:38.139 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928637+0000", 2024-04-24T15:33:38.139 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.152671603000002, 2024-04-24T15:33:38.139 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037235129999999998, 2024-04-24T15:33:38.139 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:38.139 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:38.139 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:38.139 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:38.139 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:38.139 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48727 2024-04-24T15:33:38.139 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.139 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:38.139 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.139 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:38.139 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928637+0000", 2024-04-24T15:33:38.139 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.140 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.140 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.140 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:38.140 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928637+0000", 2024-04-24T15:33:38.140 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.140 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.140 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.140 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:38.140 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928637+0000", 2024-04-24T15:33:38.140 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:38.140 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.140 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.140 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:38.140 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928638+0000", 2024-04-24T15:33:38.140 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.4279999999999999e-06 2024-04-24T15:33:38.140 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.141 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.141 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:38.141 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928641+0000", 2024-04-24T15:33:38.141 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.909e-06 2024-04-24T15:33:38.141 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.141 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.141 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:38.141 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928643+0000", 2024-04-24T15:33:38.141 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.9269999999999999e-06 2024-04-24T15:33:38.141 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.141 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.141 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:38.141 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965671+0000", 2024-04-24T15:33:38.141 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037028393 2024-04-24T15:33:38.141 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.142 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.142 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:38.142 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965752+0000", 2024-04-24T15:33:38.142 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0921999999999995e-05 2024-04-24T15:33:38.142 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.142 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.142 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:38.142 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965872+0000", 2024-04-24T15:33:38.142 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000119759 2024-04-24T15:33:38.142 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.142 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:38.142 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.142 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.142 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.143 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48728 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2441216~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:38.143 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928645+0000", 2024-04-24T15:33:38.143 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.15266342, 2024-04-24T15:33:38.143 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037346852999999999, 2024-04-24T15:33:38.143 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:38.143 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:38.143 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:38.143 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:38.143 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:38.143 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48728 2024-04-24T15:33:38.143 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.143 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:38.143 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.143 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:38.143 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928645+0000", 2024-04-24T15:33:38.143 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.144 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.144 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.144 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:38.144 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928645+0000", 2024-04-24T15:33:38.144 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.144 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.144 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.144 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:38.144 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928645+0000", 2024-04-24T15:33:38.144 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:38.144 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.144 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.144 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:38.144 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928646+0000", 2024-04-24T15:33:38.145 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.291e-06 2024-04-24T15:33:38.145 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.145 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.145 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:38.145 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928649+0000", 2024-04-24T15:33:38.145 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8509999999999999e-06 2024-04-24T15:33:38.145 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.145 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.145 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:38.145 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928651+0000", 2024-04-24T15:33:38.145 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.809e-06 2024-04-24T15:33:38.145 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.145 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.145 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:38.145 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965882+0000", 2024-04-24T15:33:38.146 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037231715999999998 2024-04-24T15:33:38.146 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.146 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.146 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:38.146 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965925+0000", 2024-04-24T15:33:38.146 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2493999999999997e-05 2024-04-24T15:33:38.146 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.146 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.146 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:38.146 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965992+0000", 2024-04-24T15:33:38.146 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.6907000000000002e-05 2024-04-24T15:33:38.146 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.146 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:38.146 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.146 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.147 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.147 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48729 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2506752~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:38.147 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928653+0000", 2024-04-24T15:33:38.147 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.152655545000002, 2024-04-24T15:33:38.147 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037576904000000001, 2024-04-24T15:33:38.147 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:38.147 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:38.147 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:38.147 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:38.147 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:38.147 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48729 2024-04-24T15:33:38.147 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.147 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:38.147 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.147 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:38.148 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928653+0000", 2024-04-24T15:33:38.148 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.148 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.148 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.148 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:38.148 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928653+0000", 2024-04-24T15:33:38.148 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.148 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.148 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.148 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:38.148 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928653+0000", 2024-04-24T15:33:38.148 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:38.148 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.148 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.148 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:38.148 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928654+0000", 2024-04-24T15:33:38.149 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3209999999999999e-06 2024-04-24T15:33:38.149 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.149 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.149 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:38.149 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928657+0000", 2024-04-24T15:33:38.149 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8559999999999998e-06 2024-04-24T15:33:38.149 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.149 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.149 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:38.149 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928659+0000", 2024-04-24T15:33:38.149 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7080000000000002e-06 2024-04-24T15:33:38.149 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.149 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.149 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:38.150 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966021+0000", 2024-04-24T15:33:38.150 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037361296000000002 2024-04-24T15:33:38.150 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.150 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.150 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:38.150 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966102+0000", 2024-04-24T15:33:38.150 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0939999999999994e-05 2024-04-24T15:33:38.150 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.150 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.150 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:38.150 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966230+0000", 2024-04-24T15:33:38.150 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00012799999999999999 2024-04-24T15:33:38.150 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.150 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:38.151 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.151 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.151 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.151 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48730 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2572288~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:38.151 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928662+0000", 2024-04-24T15:33:38.151 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.152646456999999, 2024-04-24T15:33:38.151 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037687246000000001, 2024-04-24T15:33:38.151 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:38.151 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:38.151 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:38.151 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:38.151 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:38.151 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48730 2024-04-24T15:33:38.151 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.151 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:38.152 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.152 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:38.152 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928662+0000", 2024-04-24T15:33:38.152 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.152 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.152 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.152 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:38.152 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928662+0000", 2024-04-24T15:33:38.152 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.152 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.152 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.152 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:38.152 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928662+0000", 2024-04-24T15:33:38.152 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:38.153 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.153 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.153 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:38.153 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928663+0000", 2024-04-24T15:33:38.153 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.381e-06 2024-04-24T15:33:38.153 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.153 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.153 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:38.153 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928666+0000", 2024-04-24T15:33:38.153 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.796e-06 2024-04-24T15:33:38.153 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.153 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.153 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:38.153 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928668+0000", 2024-04-24T15:33:38.153 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.8920000000000001e-06 2024-04-24T15:33:38.154 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.154 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.154 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:38.154 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966240+0000", 2024-04-24T15:33:38.154 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037572886999999999 2024-04-24T15:33:38.154 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.154 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.154 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:38.154 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966281+0000", 2024-04-24T15:33:38.154 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0450000000000001e-05 2024-04-24T15:33:38.154 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.154 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.154 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:38.155 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966349+0000", 2024-04-24T15:33:38.155 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.8215999999999999e-05 2024-04-24T15:33:38.155 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.155 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:38.155 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.155 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.155 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.155 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48731 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2637824~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:38.155 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928747+0000", 2024-04-24T15:33:38.155 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.152561452, 2024-04-24T15:33:38.155 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037833602000000001, 2024-04-24T15:33:38.155 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:38.155 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:38.155 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:38.156 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:38.156 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:38.156 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48731 2024-04-24T15:33:38.156 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.156 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:38.156 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.156 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:38.156 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928747+0000", 2024-04-24T15:33:38.156 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.156 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.156 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.156 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:38.156 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928747+0000", 2024-04-24T15:33:38.156 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.157 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.157 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.157 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:38.157 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928670+0000", 2024-04-24T15:33:38.157 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999228 2024-04-24T15:33:38.157 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.157 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.157 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:38.157 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928750+0000", 2024-04-24T15:33:38.157 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0527999999999999e-05 2024-04-24T15:33:38.157 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.157 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.157 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:38.157 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928757+0000", 2024-04-24T15:33:38.157 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.9419999999999998e-06 2024-04-24T15:33:38.158 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.158 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.158 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:38.158 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928762+0000", 2024-04-24T15:33:38.158 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4240000000000003e-06 2024-04-24T15:33:38.158 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.158 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.158 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:38.158 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966379+0000", 2024-04-24T15:33:38.158 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037617267000000003 2024-04-24T15:33:38.158 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.158 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.158 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:38.159 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966461+0000", 2024-04-24T15:33:38.159 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.2720999999999996e-05 2024-04-24T15:33:38.159 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.159 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.159 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:38.159 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966581+0000", 2024-04-24T15:33:38.159 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000119014 2024-04-24T15:33:38.159 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.159 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:38.159 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.159 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.159 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.159 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48732 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2703360~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:38.159 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928766+0000", 2024-04-24T15:33:38.160 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.152541968000001, 2024-04-24T15:33:38.160 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037930924999999997, 2024-04-24T15:33:38.160 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:38.160 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:38.160 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:38.160 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:38.160 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:38.160 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48732 2024-04-24T15:33:38.160 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.160 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:38.160 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.160 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:38.160 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928766+0000", 2024-04-24T15:33:38.160 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.160 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.160 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.161 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:38.161 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928766+0000", 2024-04-24T15:33:38.161 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.161 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.161 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.161 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:38.161 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928766+0000", 2024-04-24T15:33:38.161 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:38.161 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.161 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.161 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:38.161 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928768+0000", 2024-04-24T15:33:38.161 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.3420000000000002e-06 2024-04-24T15:33:38.161 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.161 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.162 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:38.162 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928783+0000", 2024-04-24T15:33:38.162 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.4813e-05 2024-04-24T15:33:38.162 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.162 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.162 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:38.162 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928787+0000", 2024-04-24T15:33:38.162 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.1999999999999996e-06 2024-04-24T15:33:38.162 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.162 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.162 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:38.162 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966592+0000", 2024-04-24T15:33:38.162 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037804484999999999 2024-04-24T15:33:38.163 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.163 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.163 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:38.163 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966632+0000", 2024-04-24T15:33:38.163 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0014e-05 2024-04-24T15:33:38.163 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.163 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.163 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:38.163 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966697+0000", 2024-04-24T15:33:38.163 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.5489999999999998e-05 2024-04-24T15:33:38.163 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.163 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:38.163 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.163 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.164 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.164 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48813 4.d 4:b39400aa:::rbd_data.114a8b543e91.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:38.164 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.936015+0000", 2024-04-24T15:33:38.164 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 27.145293634000002, 2024-04-24T15:33:38.164 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.039312294999999997, 2024-04-24T15:33:38.164 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:38.164 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:38.164 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:38.164 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:38.164 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:38.164 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48813 2024-04-24T15:33:38.164 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.164 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:38.164 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.165 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:38.165 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.936015+0000", 2024-04-24T15:33:38.165 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.165 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.165 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.165 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:38.165 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.936015+0000", 2024-04-24T15:33:38.165 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.165 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.165 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.165 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:38.165 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.936014+0000", 2024-04-24T15:33:38.165 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:38.165 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.165 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.166 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:38.166 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.942618+0000", 2024-04-24T15:33:38.166 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0066038499999999997 2024-04-24T15:33:38.166 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.166 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.166 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:38.166 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.942629+0000", 2024-04-24T15:33:38.166 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0606e-05 2024-04-24T15:33:38.166 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.166 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.166 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:38.166 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.942634+0000", 2024-04-24T15:33:38.166 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.2000000000000002e-06 2024-04-24T15:33:38.166 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.167 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.167 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:38.167 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.969006+0000", 2024-04-24T15:33:38.167 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.026371564 2024-04-24T15:33:38.167 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.167 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.167 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:38.167 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.969126+0000", 2024-04-24T15:33:38.167 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000120085 2024-04-24T15:33:38.167 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.167 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.167 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 2", 2024-04-24T15:33:38.167 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.969469+0000", 2024-04-24T15:33:38.167 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00034295400000000002 2024-04-24T15:33:38.168 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.168 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.168 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:33:38.168 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.973118+0000", 2024-04-24T15:33:38.168 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0036489700000000001 2024-04-24T15:33:38.168 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.168 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.168 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:33:38.168 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.975250+0000", 2024-04-24T15:33:38.168 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0021327030000000001 2024-04-24T15:33:38.168 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.168 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.168 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:33:38.168 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.975293+0000", 2024-04-24T15:33:38.168 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2951999999999998e-05 2024-04-24T15:33:38.169 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.169 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.169 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:38.169 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.975327+0000", 2024-04-24T15:33:38.169 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.3840000000000001e-05 2024-04-24T15:33:38.169 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.169 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:38.169 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.169 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.169 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:38.169 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:38.171 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-24T15:33:38.171 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:33:38.170+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:33:38.271 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:38.272+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:38.294 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:38.294 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:33:38.294 INFO:teuthology.orchestra.run.smithi151.stdout: "num_ops": 0 2024-04-24T15:33:38.294 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:38.304 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-24T15:33:38.388 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:38.389+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:38.468 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:38.468 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:33:38.468 INFO:teuthology.orchestra.run.smithi151.stdout: "complaint_time": 30, 2024-04-24T15:33:38.468 INFO:teuthology.orchestra.run.smithi151.stdout: "num_blocked_ops": 0 2024-04-24T15:33:38.468 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:38.473 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:38.474+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:38.477 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-24T15:33:38.574 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:38.575+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:38.642 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:38.642 INFO:teuthology.orchestra.run.smithi151.stdout: "size": 20, 2024-04-24T15:33:38.642 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 600, 2024-04-24T15:33:38.642 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [ 2024-04-24T15:33:38.642 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.642 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23613 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 14688256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:38.642 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.792522+0000", 2024-04-24T15:33:38.643 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 63.849800657999999, 2024-04-24T15:33:38.643 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040507356000000001, 2024-04-24T15:33:38.643 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:38.643 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:38.643 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:38.643 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:38.643 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:38.643 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23613 2024-04-24T15:33:38.643 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.643 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:38.643 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.643 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:38.643 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792522+0000", 2024-04-24T15:33:38.643 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.644 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.644 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.644 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:38.644 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792522+0000", 2024-04-24T15:33:38.644 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.644 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.644 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.644 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:38.644 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792522+0000", 2024-04-24T15:33:38.644 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:38.644 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.644 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.644 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:38.644 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792523+0000", 2024-04-24T15:33:38.645 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.576e-06 2024-04-24T15:33:38.645 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.645 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.645 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:38.645 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792526+0000", 2024-04-24T15:33:38.645 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7439999999999999e-06 2024-04-24T15:33:38.645 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.645 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.645 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:38.645 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792528+0000", 2024-04-24T15:33:38.645 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.0600000000000002e-06 2024-04-24T15:33:38.645 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.645 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.645 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:38.645 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.832864+0000", 2024-04-24T15:33:38.646 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040335742000000001 2024-04-24T15:33:38.646 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.646 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.646 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:38.646 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.832953+0000", 2024-04-24T15:33:38.646 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.9281000000000001e-05 2024-04-24T15:33:38.646 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.646 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.646 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:38.646 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.833029+0000", 2024-04-24T15:33:38.646 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.6259000000000003e-05 2024-04-24T15:33:38.646 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.646 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:38.646 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.647 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.647 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.647 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23614 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 14729216~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:38.647 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.792530+0000", 2024-04-24T15:33:38.647 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 63.849792393999998, 2024-04-24T15:33:38.647 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040633957999999998, 2024-04-24T15:33:38.647 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:38.647 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:38.647 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:38.647 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:38.647 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:38.647 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23614 2024-04-24T15:33:38.647 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.647 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:38.648 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.648 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:38.648 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792530+0000", 2024-04-24T15:33:38.648 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.648 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.648 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.648 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:38.648 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792530+0000", 2024-04-24T15:33:38.648 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.648 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.649 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.649 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:38.649 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792530+0000", 2024-04-24T15:33:38.649 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:38.649 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.649 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.649 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:38.649 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792531+0000", 2024-04-24T15:33:38.649 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.46e-06 2024-04-24T15:33:38.649 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.649 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.649 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:38.649 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792534+0000", 2024-04-24T15:33:38.649 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8959999999999999e-06 2024-04-24T15:33:38.650 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.650 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.650 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:38.650 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792536+0000", 2024-04-24T15:33:38.650 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.841e-06 2024-04-24T15:33:38.650 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.650 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.650 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:38.650 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.833060+0000", 2024-04-24T15:33:38.650 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040524125000000001 2024-04-24T15:33:38.650 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.650 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.650 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:38.650 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.833108+0000", 2024-04-24T15:33:38.651 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.7547999999999999e-05 2024-04-24T15:33:38.651 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.651 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.651 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:38.651 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.833164+0000", 2024-04-24T15:33:38.651 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6297e-05 2024-04-24T15:33:38.651 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.651 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:38.651 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.651 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.651 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.651 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23623 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15097856~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:38.651 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.793887+0000", 2024-04-24T15:33:38.651 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 63.848435432999999, 2024-04-24T15:33:38.651 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040432784999999999, 2024-04-24T15:33:38.652 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:38.652 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:38.652 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:38.652 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:38.652 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:38.652 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23623 2024-04-24T15:33:38.652 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.652 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:38.652 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.652 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:38.652 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793887+0000", 2024-04-24T15:33:38.652 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.652 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.652 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.652 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:38.653 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793887+0000", 2024-04-24T15:33:38.653 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.653 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.653 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.653 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:38.653 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793887+0000", 2024-04-24T15:33:38.653 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:38.653 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.653 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.653 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:38.653 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793888+0000", 2024-04-24T15:33:38.653 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3659999999999999e-06 2024-04-24T15:33:38.653 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.653 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.653 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:38.654 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793891+0000", 2024-04-24T15:33:38.654 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8389999999999998e-06 2024-04-24T15:33:38.654 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.654 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.654 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:38.654 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793893+0000", 2024-04-24T15:33:38.654 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.9549999999999999e-06 2024-04-24T15:33:38.654 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.654 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.654 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:38.654 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834239+0000", 2024-04-24T15:33:38.654 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040345487999999999 2024-04-24T15:33:38.654 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.655 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.655 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:38.655 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834280+0000", 2024-04-24T15:33:38.655 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.1878000000000001e-05 2024-04-24T15:33:38.655 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.655 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.655 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:38.655 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834320+0000", 2024-04-24T15:33:38.655 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9480000000000001e-05 2024-04-24T15:33:38.655 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.655 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:38.655 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.655 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.655 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.655 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23624 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15138816~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:38.655 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.794053+0000", 2024-04-24T15:33:38.656 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 63.848269485000003, 2024-04-24T15:33:38.656 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040370801999999997, 2024-04-24T15:33:38.656 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:38.656 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:38.656 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:38.656 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:38.656 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:38.656 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23624 2024-04-24T15:33:38.656 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.656 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:38.656 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.656 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:38.656 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794053+0000", 2024-04-24T15:33:38.656 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.657 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.657 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.657 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:38.657 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794053+0000", 2024-04-24T15:33:38.657 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.657 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.657 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.657 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:38.657 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793895+0000", 2024-04-24T15:33:38.657 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998422 2024-04-24T15:33:38.657 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.657 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.657 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:38.657 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794057+0000", 2024-04-24T15:33:38.658 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00016149400000000001 2024-04-24T15:33:38.658 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.658 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.658 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:38.658 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794065+0000", 2024-04-24T15:33:38.658 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.5259999999999993e-06 2024-04-24T15:33:38.658 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.658 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.658 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:38.658 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794071+0000", 2024-04-24T15:33:38.658 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6149999999999996e-06 2024-04-24T15:33:38.658 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.658 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.658 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:38.659 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834341+0000", 2024-04-24T15:33:38.659 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040270366000000002 2024-04-24T15:33:38.659 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.659 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.659 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:38.659 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834381+0000", 2024-04-24T15:33:38.659 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0102999999999999e-05 2024-04-24T15:33:38.659 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.659 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.659 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:38.659 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834424+0000", 2024-04-24T15:33:38.659 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2713000000000003e-05 2024-04-24T15:33:38.659 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.659 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:38.660 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.660 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.660 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.660 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23625 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15179776~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:38.660 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.794160+0000", 2024-04-24T15:33:38.660 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 63.848162602999999, 2024-04-24T15:33:38.660 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040413004000000002, 2024-04-24T15:33:38.660 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:38.660 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:38.660 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:38.660 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:38.660 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:38.660 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23625 2024-04-24T15:33:38.660 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.660 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:38.661 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.661 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:38.661 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794160+0000", 2024-04-24T15:33:38.661 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.661 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.661 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.661 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:38.661 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794160+0000", 2024-04-24T15:33:38.661 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.661 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.661 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.661 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:38.661 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794086+0000", 2024-04-24T15:33:38.661 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999256 2024-04-24T15:33:38.662 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.662 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.662 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:38.662 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794163+0000", 2024-04-24T15:33:38.662 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7896000000000001e-05 2024-04-24T15:33:38.662 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.662 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.662 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:38.662 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794171+0000", 2024-04-24T15:33:38.662 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.9500000000000001e-06 2024-04-24T15:33:38.662 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.662 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.662 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:38.662 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794179+0000", 2024-04-24T15:33:38.663 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7970000000000001e-06 2024-04-24T15:33:38.663 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.663 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.663 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:38.663 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834451+0000", 2024-04-24T15:33:38.663 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040271492999999998 2024-04-24T15:33:38.663 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.663 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.663 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:38.663 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834519+0000", 2024-04-24T15:33:38.663 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.7984000000000005e-05 2024-04-24T15:33:38.663 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.663 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.663 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:38.663 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834573+0000", 2024-04-24T15:33:38.664 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.4339999999999998e-05 2024-04-24T15:33:38.664 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.664 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:38.664 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.664 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.664 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.664 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23629 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15343616~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:38.664 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.794748+0000", 2024-04-24T15:33:38.664 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 63.847574801, 2024-04-24T15:33:38.664 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040419718, 2024-04-24T15:33:38.664 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:38.664 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:38.664 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:38.664 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:38.664 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:38.665 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23629 2024-04-24T15:33:38.665 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.665 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:38.665 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.665 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:38.665 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794748+0000", 2024-04-24T15:33:38.665 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.665 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.665 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.665 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:38.665 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794748+0000", 2024-04-24T15:33:38.665 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.665 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.665 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.666 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:38.666 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794673+0000", 2024-04-24T15:33:38.666 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999251 2024-04-24T15:33:38.666 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.666 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.666 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:38.666 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794751+0000", 2024-04-24T15:33:38.666 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.8480999999999996e-05 2024-04-24T15:33:38.666 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.666 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.666 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:38.666 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794760+0000", 2024-04-24T15:33:38.666 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3429999999999996e-06 2024-04-24T15:33:38.666 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.667 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.667 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:38.667 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794764+0000", 2024-04-24T15:33:38.667 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5600000000000004e-06 2024-04-24T15:33:38.667 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.667 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.667 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:38.667 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835040+0000", 2024-04-24T15:33:38.667 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040275188000000003 2024-04-24T15:33:38.667 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.667 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.667 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:38.667 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835105+0000", 2024-04-24T15:33:38.667 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.5992000000000001e-05 2024-04-24T15:33:38.668 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.668 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.668 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:38.668 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835167+0000", 2024-04-24T15:33:38.668 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.1981000000000002e-05 2024-04-24T15:33:38.668 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.668 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:38.668 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.668 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.668 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.668 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23630 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15384576~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:38.668 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.794839+0000", 2024-04-24T15:33:38.668 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 63.847483732000001, 2024-04-24T15:33:38.668 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040523894999999997, 2024-04-24T15:33:38.668 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:38.668 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:38.669 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:38.669 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:38.669 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:38.669 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23630 2024-04-24T15:33:38.669 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.669 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:38.669 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.669 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:38.669 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794839+0000", 2024-04-24T15:33:38.669 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.669 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.669 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.669 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:38.669 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794839+0000", 2024-04-24T15:33:38.670 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.670 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.670 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.670 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:38.670 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794768+0000", 2024-04-24T15:33:38.670 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999294 2024-04-24T15:33:38.670 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.670 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.670 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:38.670 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794841+0000", 2024-04-24T15:33:38.670 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2793000000000006e-05 2024-04-24T15:33:38.670 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.670 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.671 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:38.671 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794846+0000", 2024-04-24T15:33:38.671 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.7230000000000004e-06 2024-04-24T15:33:38.671 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.671 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.671 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:38.671 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794848+0000", 2024-04-24T15:33:38.671 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7659999999999999e-06 2024-04-24T15:33:38.671 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.671 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.671 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:38.671 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835223+0000", 2024-04-24T15:33:38.671 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040374275000000001 2024-04-24T15:33:38.671 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.671 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.672 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:38.672 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835295+0000", 2024-04-24T15:33:38.672 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2633999999999996e-05 2024-04-24T15:33:38.672 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.672 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.672 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:38.672 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835363+0000", 2024-04-24T15:33:38.672 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.7355999999999997e-05 2024-04-24T15:33:38.672 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.672 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:38.672 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.672 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.672 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.672 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23631 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15425536~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:38.672 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.794956+0000", 2024-04-24T15:33:38.673 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 63.847366891, 2024-04-24T15:33:38.673 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040562383, 2024-04-24T15:33:38.673 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:38.673 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:38.673 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:38.673 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:38.673 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:38.673 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23631 2024-04-24T15:33:38.673 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.673 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:38.673 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.673 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:38.673 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794956+0000", 2024-04-24T15:33:38.673 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.673 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.674 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.674 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:38.674 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794956+0000", 2024-04-24T15:33:38.674 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.674 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.674 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.674 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:38.674 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794859+0000", 2024-04-24T15:33:38.674 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999032 2024-04-24T15:33:38.674 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.674 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.674 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:38.674 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794959+0000", 2024-04-24T15:33:38.674 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000100207 2024-04-24T15:33:38.675 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.675 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.675 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:38.675 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794967+0000", 2024-04-24T15:33:38.675 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7640000000000005e-06 2024-04-24T15:33:38.675 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.675 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.675 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:38.675 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794971+0000", 2024-04-24T15:33:38.675 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.335e-06 2024-04-24T15:33:38.675 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.675 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.675 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:38.675 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835388+0000", 2024-04-24T15:33:38.675 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040417340000000003 2024-04-24T15:33:38.675 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.676 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.676 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:38.676 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835459+0000", 2024-04-24T15:33:38.676 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.0118999999999997e-05 2024-04-24T15:33:38.676 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.676 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.676 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:38.676 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835518+0000", 2024-04-24T15:33:38.676 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.9438999999999998e-05 2024-04-24T15:33:38.676 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.676 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:38.676 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.676 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.676 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.676 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23632 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15466496~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:38.677 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795060+0000", 2024-04-24T15:33:38.677 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 63.847262180999998, 2024-04-24T15:33:38.677 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040667993, 2024-04-24T15:33:38.677 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:38.677 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:38.677 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:38.677 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:38.677 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:38.677 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23632 2024-04-24T15:33:38.677 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.677 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:38.677 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.677 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:38.677 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795060+0000", 2024-04-24T15:33:38.678 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.678 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.678 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.678 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:38.678 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795060+0000", 2024-04-24T15:33:38.678 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.678 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.678 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.678 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:38.678 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794976+0000", 2024-04-24T15:33:38.678 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999156 2024-04-24T15:33:38.678 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.678 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.678 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:38.679 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795064+0000", 2024-04-24T15:33:38.679 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.7709999999999999e-05 2024-04-24T15:33:38.679 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.679 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.679 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:38.679 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795072+0000", 2024-04-24T15:33:38.679 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3769999999999996e-06 2024-04-24T15:33:38.679 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.679 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.679 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:38.679 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795078+0000", 2024-04-24T15:33:38.679 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.5079999999999996e-06 2024-04-24T15:33:38.679 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.679 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.679 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:38.680 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835541+0000", 2024-04-24T15:33:38.680 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040463440000000003 2024-04-24T15:33:38.680 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.680 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.680 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:38.680 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835657+0000", 2024-04-24T15:33:38.680 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00011572499999999999 2024-04-24T15:33:38.680 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.680 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.680 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:38.680 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835728+0000", 2024-04-24T15:33:38.680 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1402000000000005e-05 2024-04-24T15:33:38.680 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.680 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:38.680 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.680 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.681 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.681 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23633 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15507456~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:38.681 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795134+0000", 2024-04-24T15:33:38.681 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 63.847188525999997, 2024-04-24T15:33:38.681 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040748591000000001, 2024-04-24T15:33:38.681 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:38.681 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:38.681 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:38.681 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:38.681 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:38.681 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23633 2024-04-24T15:33:38.681 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.681 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:38.681 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.681 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:38.682 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795134+0000", 2024-04-24T15:33:38.682 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.682 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.682 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.682 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:38.682 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795134+0000", 2024-04-24T15:33:38.682 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.682 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.682 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.682 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:38.682 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795083+0000", 2024-04-24T15:33:38.682 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999949 2024-04-24T15:33:38.682 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.682 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.683 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:38.683 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795138+0000", 2024-04-24T15:33:38.683 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.4472000000000003e-05 2024-04-24T15:33:38.683 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.683 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.683 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:38.683 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795145+0000", 2024-04-24T15:33:38.683 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2779999999999996e-06 2024-04-24T15:33:38.683 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.683 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.683 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:38.683 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795149+0000", 2024-04-24T15:33:38.683 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.568e-06 2024-04-24T15:33:38.683 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.683 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.684 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:38.684 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835757+0000", 2024-04-24T15:33:38.684 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040608894999999999 2024-04-24T15:33:38.684 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.684 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.684 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:38.684 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835829+0000", 2024-04-24T15:33:38.684 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1879999999999996e-05 2024-04-24T15:33:38.684 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.684 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.684 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:38.684 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835883+0000", 2024-04-24T15:33:38.684 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.3346000000000001e-05 2024-04-24T15:33:38.684 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.684 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:38.685 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.685 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.685 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.685 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23634 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15548416~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:38.685 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795271+0000", 2024-04-24T15:33:38.685 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 63.847051491999999, 2024-04-24T15:33:38.685 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040755845999999998, 2024-04-24T15:33:38.685 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:38.685 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:38.685 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:38.685 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:38.685 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:38.685 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23634 2024-04-24T15:33:38.685 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.685 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:38.686 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.686 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:38.686 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795271+0000", 2024-04-24T15:33:38.686 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.686 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.686 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.686 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:38.686 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795271+0000", 2024-04-24T15:33:38.686 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.686 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.686 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.686 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:38.686 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795152+0000", 2024-04-24T15:33:38.686 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998813 2024-04-24T15:33:38.687 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.687 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.687 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:38.687 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795276+0000", 2024-04-24T15:33:38.687 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00012401799999999999 2024-04-24T15:33:38.687 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.687 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.687 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:38.687 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795281+0000", 2024-04-24T15:33:38.687 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8860000000000003e-06 2024-04-24T15:33:38.687 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.687 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.687 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:38.687 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795284+0000", 2024-04-24T15:33:38.687 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.0450000000000001e-06 2024-04-24T15:33:38.687 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.688 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.688 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:38.688 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835905+0000", 2024-04-24T15:33:38.688 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040621035999999999 2024-04-24T15:33:38.688 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.688 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.688 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:38.688 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835971+0000", 2024-04-24T15:33:38.688 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.5314000000000005e-05 2024-04-24T15:33:38.688 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.688 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.688 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:38.688 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836027+0000", 2024-04-24T15:33:38.688 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6305999999999999e-05 2024-04-24T15:33:38.688 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.689 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:38.689 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.689 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.689 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.689 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23635 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15589376~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:38.689 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795382+0000", 2024-04-24T15:33:38.689 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 63.846940719000003, 2024-04-24T15:33:38.689 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040805363999999997, 2024-04-24T15:33:38.689 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:38.689 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:38.689 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:38.689 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:38.689 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:38.689 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23635 2024-04-24T15:33:38.689 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.690 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:38.690 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.690 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:38.690 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795382+0000", 2024-04-24T15:33:38.690 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.690 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.690 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.690 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:38.690 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795382+0000", 2024-04-24T15:33:38.690 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.690 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.690 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.690 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:38.691 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795288+0000", 2024-04-24T15:33:38.691 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999056 2024-04-24T15:33:38.691 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.691 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.691 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:38.691 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795385+0000", 2024-04-24T15:33:38.691 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.7288999999999996e-05 2024-04-24T15:33:38.691 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.691 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.691 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:38.691 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795392+0000", 2024-04-24T15:33:38.691 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.452e-06 2024-04-24T15:33:38.691 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.691 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.691 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:38.692 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795397+0000", 2024-04-24T15:33:38.692 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2860000000000001e-06 2024-04-24T15:33:38.692 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.692 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.692 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:38.692 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836054+0000", 2024-04-24T15:33:38.692 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040657581999999998 2024-04-24T15:33:38.692 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.692 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.692 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:38.692 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836124+0000", 2024-04-24T15:33:38.692 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.0020999999999999e-05 2024-04-24T15:33:38.692 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.692 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.692 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:38.693 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836187+0000", 2024-04-24T15:33:38.693 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.3009999999999995e-05 2024-04-24T15:33:38.693 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.693 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:38.693 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.693 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.693 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.693 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23636 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15630336~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:38.693 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795610+0000", 2024-04-24T15:33:38.693 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 63.846712361000002, 2024-04-24T15:33:38.693 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040730942999999999, 2024-04-24T15:33:38.693 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:38.693 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:38.693 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:38.693 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:38.693 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:38.694 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23636 2024-04-24T15:33:38.694 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.694 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:38.694 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.694 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:38.694 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795610+0000", 2024-04-24T15:33:38.694 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.694 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.694 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.694 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:38.694 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795610+0000", 2024-04-24T15:33:38.694 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.695 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.695 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.695 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:38.695 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795401+0000", 2024-04-24T15:33:38.695 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9997911 2024-04-24T15:33:38.695 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.695 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.695 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:38.695 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795614+0000", 2024-04-24T15:33:38.695 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00021228600000000001 2024-04-24T15:33:38.695 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.695 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.695 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:38.695 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795622+0000", 2024-04-24T15:33:38.696 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3920000000000003e-06 2024-04-24T15:33:38.696 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.696 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.696 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:38.696 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795627+0000", 2024-04-24T15:33:38.696 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8199999999999996e-06 2024-04-24T15:33:38.696 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.696 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.696 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:38.696 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836211+0000", 2024-04-24T15:33:38.696 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040583803000000002 2024-04-24T15:33:38.696 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.696 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.696 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:38.696 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836284+0000", 2024-04-24T15:33:38.697 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.3617999999999999e-05 2024-04-24T15:33:38.697 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.697 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.697 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:38.697 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836341+0000", 2024-04-24T15:33:38.697 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6959999999999997e-05 2024-04-24T15:33:38.697 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.697 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:38.697 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.697 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.697 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.697 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23637 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15671296~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:38.697 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795699+0000", 2024-04-24T15:33:38.697 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 63.846623194000003, 2024-04-24T15:33:38.697 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040775556999999997, 2024-04-24T15:33:38.698 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:38.698 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:38.698 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:38.698 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:38.698 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:38.698 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23637 2024-04-24T15:33:38.698 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.698 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:38.698 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.698 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:38.698 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795699+0000", 2024-04-24T15:33:38.698 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.698 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.698 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.699 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:38.699 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795699+0000", 2024-04-24T15:33:38.699 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.699 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.699 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.699 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:38.699 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795632+0000", 2024-04-24T15:33:38.699 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999323 2024-04-24T15:33:38.699 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.699 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.699 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:38.699 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795704+0000", 2024-04-24T15:33:38.699 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2428000000000005e-05 2024-04-24T15:33:38.699 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.699 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.699 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:38.700 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795715+0000", 2024-04-24T15:33:38.700 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.1532000000000001e-05 2024-04-24T15:33:38.700 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.700 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.700 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:38.700 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795721+0000", 2024-04-24T15:33:38.700 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.8200000000000002e-06 2024-04-24T15:33:38.700 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.700 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.700 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:38.700 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836364+0000", 2024-04-24T15:33:38.700 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040642997 2024-04-24T15:33:38.700 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.700 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.700 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:38.701 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836435+0000", 2024-04-24T15:33:38.701 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.0821000000000005e-05 2024-04-24T15:33:38.701 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.701 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.701 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:38.701 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836475+0000", 2024-04-24T15:33:38.701 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9796999999999999e-05 2024-04-24T15:33:38.701 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.701 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:38.701 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.701 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.701 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.701 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23638 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15712256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:38.701 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795932+0000", 2024-04-24T15:33:38.702 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 63.846390163999999, 2024-04-24T15:33:38.702 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040668682999999997, 2024-04-24T15:33:38.702 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:38.702 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:38.702 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:38.702 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:38.702 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:38.702 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23638 2024-04-24T15:33:38.702 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.702 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:38.702 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.702 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:38.702 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795932+0000", 2024-04-24T15:33:38.702 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.703 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.703 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.703 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:38.703 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795932+0000", 2024-04-24T15:33:38.703 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.703 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.703 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.703 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:38.703 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795728+0000", 2024-04-24T15:33:38.703 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9997959 2024-04-24T15:33:38.703 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.703 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.703 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:38.703 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795936+0000", 2024-04-24T15:33:38.704 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000208128 2024-04-24T15:33:38.704 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.704 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.704 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:38.704 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795945+0000", 2024-04-24T15:33:38.704 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.7110000000000001e-06 2024-04-24T15:33:38.704 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.704 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.704 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:38.704 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795957+0000", 2024-04-24T15:33:38.704 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.1800999999999999e-05 2024-04-24T15:33:38.704 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.704 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.704 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:38.705 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836505+0000", 2024-04-24T15:33:38.705 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040547622999999998 2024-04-24T15:33:38.705 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.705 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.705 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:38.705 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836561+0000", 2024-04-24T15:33:38.705 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6598999999999998e-05 2024-04-24T15:33:38.705 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.705 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.705 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:38.705 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836601+0000", 2024-04-24T15:33:38.705 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9907000000000003e-05 2024-04-24T15:33:38.705 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.705 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:38.705 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.706 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.706 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.706 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23639 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15753216~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:38.706 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.796071+0000", 2024-04-24T15:33:38.706 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 63.846251903000002, 2024-04-24T15:33:38.706 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040648199000000003, 2024-04-24T15:33:38.706 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:38.706 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:38.706 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:38.706 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:38.706 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:38.706 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23639 2024-04-24T15:33:38.706 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.706 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:38.707 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.707 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:38.707 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796071+0000", 2024-04-24T15:33:38.707 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.707 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.707 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.707 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:38.707 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796071+0000", 2024-04-24T15:33:38.707 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.707 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.707 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.707 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:38.707 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795967+0000", 2024-04-24T15:33:38.707 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998965 2024-04-24T15:33:38.707 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.708 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.708 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:38.708 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796074+0000", 2024-04-24T15:33:38.708 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000107013 2024-04-24T15:33:38.708 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.708 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.708 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:38.708 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796081+0000", 2024-04-24T15:33:38.708 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2590000000000003e-06 2024-04-24T15:33:38.708 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.708 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.708 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:38.708 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796085+0000", 2024-04-24T15:33:38.708 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5269999999999999e-06 2024-04-24T15:33:38.709 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.709 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.709 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:38.709 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836624+0000", 2024-04-24T15:33:38.709 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040539324000000002 2024-04-24T15:33:38.709 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.709 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.709 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:38.709 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836667+0000", 2024-04-24T15:33:38.709 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2562999999999999e-05 2024-04-24T15:33:38.709 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.709 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.709 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:38.710 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836719+0000", 2024-04-24T15:33:38.710 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.2015999999999998e-05 2024-04-24T15:33:38.710 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.710 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:38.710 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.710 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.710 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.710 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23640 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15794176~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:38.710 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.796492+0000", 2024-04-24T15:33:38.710 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 63.845830145000001, 2024-04-24T15:33:38.710 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040420000999999997, 2024-04-24T15:33:38.710 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:38.710 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:38.710 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:38.711 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:38.711 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:38.711 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23640 2024-04-24T15:33:38.711 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.711 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:38.711 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.711 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:38.711 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796492+0000", 2024-04-24T15:33:38.711 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.711 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.711 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.711 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:38.711 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796492+0000", 2024-04-24T15:33:38.711 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.711 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.712 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.712 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:38.712 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796088+0000", 2024-04-24T15:33:38.712 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9995961 2024-04-24T15:33:38.712 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.712 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.712 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:38.712 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796496+0000", 2024-04-24T15:33:38.712 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00040785700000000002 2024-04-24T15:33:38.712 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.712 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.712 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:38.712 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796503+0000", 2024-04-24T15:33:38.712 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1049999999999997e-06 2024-04-24T15:33:38.712 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.713 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.713 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:38.713 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796508+0000", 2024-04-24T15:33:38.713 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.155e-06 2024-04-24T15:33:38.713 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.713 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.713 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:38.713 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836756+0000", 2024-04-24T15:33:38.713 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040248850000000003 2024-04-24T15:33:38.713 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.713 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.713 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:38.713 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836841+0000", 2024-04-24T15:33:38.713 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.4735999999999996e-05 2024-04-24T15:33:38.714 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.714 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.714 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:38.714 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836912+0000", 2024-04-24T15:33:38.714 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1229999999999994e-05 2024-04-24T15:33:38.714 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.714 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:38.714 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.714 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.714 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.714 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23641 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15835136~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:38.714 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.796658+0000", 2024-04-24T15:33:38.714 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 63.845664069000001, 2024-04-24T15:33:38.715 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040420706000000001, 2024-04-24T15:33:38.715 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:38.715 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:38.715 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:38.715 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:38.715 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:38.715 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23641 2024-04-24T15:33:38.715 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.715 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:38.715 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.715 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:38.715 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796658+0000", 2024-04-24T15:33:38.715 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.715 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.715 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.716 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:38.716 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796658+0000", 2024-04-24T15:33:38.716 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.716 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.716 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.716 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:38.716 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796512+0000", 2024-04-24T15:33:38.716 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998531 2024-04-24T15:33:38.716 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.716 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.716 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:38.716 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796669+0000", 2024-04-24T15:33:38.716 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000157577 2024-04-24T15:33:38.716 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.717 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.717 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:38.717 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796677+0000", 2024-04-24T15:33:38.717 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7800000000000001e-06 2024-04-24T15:33:38.717 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.717 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.717 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:38.717 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796681+0000", 2024-04-24T15:33:38.717 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.721e-06 2024-04-24T15:33:38.717 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.717 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.717 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:38.717 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836941+0000", 2024-04-24T15:33:38.717 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040259907999999997 2024-04-24T15:33:38.717 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.718 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.718 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:38.718 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837017+0000", 2024-04-24T15:33:38.718 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.6071999999999999e-05 2024-04-24T15:33:38.718 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.718 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.718 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:38.718 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837079+0000", 2024-04-24T15:33:38.718 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.2358000000000002e-05 2024-04-24T15:33:38.718 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.718 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:38.718 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.718 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.718 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.719 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23645 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15998976~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:38.719 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.797360+0000", 2024-04-24T15:33:38.719 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 63.844962571000003, 2024-04-24T15:33:38.719 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040387645999999999, 2024-04-24T15:33:38.719 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:38.719 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:38.719 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:38.719 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:38.719 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:38.719 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23645 2024-04-24T15:33:38.719 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.719 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:38.719 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.719 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:38.719 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797360+0000", 2024-04-24T15:33:38.720 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.720 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.720 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.720 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:38.720 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797360+0000", 2024-04-24T15:33:38.720 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.720 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.720 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.720 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:38.720 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797288+0000", 2024-04-24T15:33:38.720 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999928 2024-04-24T15:33:38.720 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.720 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.721 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:38.721 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797364+0000", 2024-04-24T15:33:38.721 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.5347999999999998e-05 2024-04-24T15:33:38.721 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.721 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.721 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:38.721 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797372+0000", 2024-04-24T15:33:38.721 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.2409999999999995e-06 2024-04-24T15:33:38.721 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.721 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.721 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:38.721 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797376+0000", 2024-04-24T15:33:38.721 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.515e-06 2024-04-24T15:33:38.721 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.721 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.722 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:38.722 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837586+0000", 2024-04-24T15:33:38.722 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040210047999999998 2024-04-24T15:33:38.722 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.722 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.722 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:38.722 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837672+0000", 2024-04-24T15:33:38.722 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.5997999999999999e-05 2024-04-24T15:33:38.722 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.722 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.722 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:38.722 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837748+0000", 2024-04-24T15:33:38.722 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.5328999999999997e-05 2024-04-24T15:33:38.723 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.723 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:38.723 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.723 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.723 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.723 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23646 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 16039936~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:38.723 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.797498+0000", 2024-04-24T15:33:38.723 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 63.844824469999999, 2024-04-24T15:33:38.723 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040433139, 2024-04-24T15:33:38.723 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:38.723 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:38.723 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:38.723 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:38.723 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:38.723 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23646 2024-04-24T15:33:38.724 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.724 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:38.724 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.724 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:38.724 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797498+0000", 2024-04-24T15:33:38.724 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.724 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.724 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.724 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:38.724 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797498+0000", 2024-04-24T15:33:38.724 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:38.724 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.724 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.724 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:38.724 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797381+0000", 2024-04-24T15:33:38.725 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998832 2024-04-24T15:33:38.725 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.725 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.725 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:38.725 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797501+0000", 2024-04-24T15:33:38.725 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000119759 2024-04-24T15:33:38.725 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.725 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.725 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:38.725 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797510+0000", 2024-04-24T15:33:38.725 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.4770000000000003e-06 2024-04-24T15:33:38.725 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.725 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.725 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:38.725 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797515+0000", 2024-04-24T15:33:38.726 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8330000000000001e-06 2024-04-24T15:33:38.726 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.726 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.726 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:38.726 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837777+0000", 2024-04-24T15:33:38.726 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040262502999999998 2024-04-24T15:33:38.726 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.726 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.726 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:38.726 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837860+0000", 2024-04-24T15:33:38.726 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3002000000000003e-05 2024-04-24T15:33:38.726 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:38.726 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:38.726 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:38.726 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837931+0000", 2024-04-24T15:33:38.727 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1222000000000004e-05 2024-04-24T15:33:38.727 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.727 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:38.727 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.727 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:38.727 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:38.727 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:38.729 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-24T15:33:38.729 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:38.729+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:38.829 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:38.830+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:38.852 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:38.852 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:33:38.852 INFO:teuthology.orchestra.run.smithi151.stdout: "num_ops": 0 2024-04-24T15:33:38.852 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:38.862 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-24T15:33:38.930 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:33:38.931+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:33:38.959 INFO:teuthology.orchestra.run.smithi096.stderr:nodeep-scrub is unset 2024-04-24T15:33:38.994 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:38.994 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:33:38.994 INFO:teuthology.orchestra.run.smithi151.stdout: "complaint_time": 30, 2024-04-24T15:33:38.994 INFO:teuthology.orchestra.run.smithi151.stdout: "num_blocked_ops": 0 2024-04-24T15:33:38.995 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:39.006 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-24T15:33:39.032 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:33:39.032+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:33:39.132 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:33:39.133+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:33:39.133 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:39.133 INFO:teuthology.orchestra.run.smithi151.stdout: "size": 20, 2024-04-24T15:33:39.134 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 600, 2024-04-24T15:33:39.134 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [ 2024-04-24T15:33:39.134 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.134 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18006 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12419072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:39.134 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.207911+0000", 2024-04-24T15:33:39.134 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 64.925704081000006, 2024-04-24T15:33:39.134 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042899496000000002, 2024-04-24T15:33:39.134 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:39.134 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:39.134 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:39.134 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:39.134 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:39.135 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18006 2024-04-24T15:33:39.135 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.135 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:39.135 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.135 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:39.135 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207911+0000", 2024-04-24T15:33:39.135 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:39.135 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.135 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.135 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:39.135 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207911+0000", 2024-04-24T15:33:39.135 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:39.135 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.135 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.135 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:39.136 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207910+0000", 2024-04-24T15:33:39.136 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:39.136 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.136 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.136 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:39.136 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207913+0000", 2024-04-24T15:33:39.136 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.2309999999999999e-06 2024-04-24T15:33:39.136 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.136 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.136 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:39.136 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207918+0000", 2024-04-24T15:33:39.136 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.3399999999999997e-06 2024-04-24T15:33:39.136 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.136 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.136 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:39.136 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207922+0000", 2024-04-24T15:33:39.137 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5439999999999999e-06 2024-04-24T15:33:39.137 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.137 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.137 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:39.137 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250726+0000", 2024-04-24T15:33:39.137 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042804129000000003 2024-04-24T15:33:39.137 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.137 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.137 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:39.137 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250765+0000", 2024-04-24T15:33:39.137 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9554000000000001e-05 2024-04-24T15:33:39.137 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.137 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.137 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:39.137 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250810+0000", 2024-04-24T15:33:39.137 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5027000000000001e-05 2024-04-24T15:33:39.138 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:39.138 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:39.138 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:39.138 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.138 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.138 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18007 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12460032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:39.138 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.207961+0000", 2024-04-24T15:33:39.138 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 64.92565424, 2024-04-24T15:33:39.138 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042965470999999998, 2024-04-24T15:33:39.138 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:39.138 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:39.138 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:39.138 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:39.138 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:39.138 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18007 2024-04-24T15:33:39.139 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.139 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:39.139 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.139 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:39.139 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207961+0000", 2024-04-24T15:33:39.139 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:39.139 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.139 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.139 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:39.139 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207961+0000", 2024-04-24T15:33:39.139 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:39.139 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.139 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.139 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:39.139 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207960+0000", 2024-04-24T15:33:39.139 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:39.140 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.140 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.140 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:39.140 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207963+0000", 2024-04-24T15:33:39.140 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.3759999999999999e-06 2024-04-24T15:33:39.140 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.140 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.140 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:39.140 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207970+0000", 2024-04-24T15:33:39.140 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.0489999999999998e-06 2024-04-24T15:33:39.140 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.140 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.140 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:39.140 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207975+0000", 2024-04-24T15:33:39.140 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.1640000000000003e-06 2024-04-24T15:33:39.141 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.141 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.141 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:39.141 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250834+0000", 2024-04-24T15:33:39.141 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042859060999999997 2024-04-24T15:33:39.141 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.141 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.141 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:39.141 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250874+0000", 2024-04-24T15:33:39.141 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0674999999999999e-05 2024-04-24T15:33:39.141 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.141 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.141 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:39.141 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250926+0000", 2024-04-24T15:33:39.141 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.1693000000000001e-05 2024-04-24T15:33:39.142 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:39.142 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:39.142 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:39.142 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.142 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.142 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18008 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12500992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:39.142 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208030+0000", 2024-04-24T15:33:39.142 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 64.925585147000007, 2024-04-24T15:33:39.142 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043003337000000003, 2024-04-24T15:33:39.142 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:39.142 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:39.142 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:39.142 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:39.143 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:39.143 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18008 2024-04-24T15:33:39.143 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.143 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:39.143 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.143 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:39.143 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208030+0000", 2024-04-24T15:33:39.143 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:39.143 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.143 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.143 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:39.143 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208030+0000", 2024-04-24T15:33:39.143 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:39.143 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.143 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.143 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:39.144 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208029+0000", 2024-04-24T15:33:39.144 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:39.144 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.144 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.144 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:39.144 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208032+0000", 2024-04-24T15:33:39.144 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.117e-06 2024-04-24T15:33:39.144 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.144 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.144 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:39.144 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208039+0000", 2024-04-24T15:33:39.144 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.0129999999999996e-06 2024-04-24T15:33:39.144 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.144 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.144 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:39.144 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208043+0000", 2024-04-24T15:33:39.145 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0600000000000001e-06 2024-04-24T15:33:39.145 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.145 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.145 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:39.145 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250948+0000", 2024-04-24T15:33:39.145 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042904577999999999 2024-04-24T15:33:39.145 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.145 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.145 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:39.145 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250989+0000", 2024-04-24T15:33:39.145 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0978e-05 2024-04-24T15:33:39.145 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.145 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.145 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:39.145 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251033+0000", 2024-04-24T15:33:39.145 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4161e-05 2024-04-24T15:33:39.146 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:39.146 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:39.146 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:39.146 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.146 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.146 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18009 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12541952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:39.146 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208093+0000", 2024-04-24T15:33:39.146 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 64.925521400999997, 2024-04-24T15:33:39.146 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043043451000000003, 2024-04-24T15:33:39.146 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:39.146 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:39.146 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:39.146 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:39.146 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:39.147 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18009 2024-04-24T15:33:39.147 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.147 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:39.147 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.147 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:39.147 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208093+0000", 2024-04-24T15:33:39.147 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:39.147 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.147 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.147 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:39.147 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208093+0000", 2024-04-24T15:33:39.147 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:39.147 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.147 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.147 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:39.148 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208093+0000", 2024-04-24T15:33:39.148 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:39.148 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.148 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.148 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:39.148 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208096+0000", 2024-04-24T15:33:39.148 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.439e-06 2024-04-24T15:33:39.148 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.148 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.148 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:39.149 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208103+0000", 2024-04-24T15:33:39.149 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1019999999999999e-06 2024-04-24T15:33:39.149 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.149 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.149 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:39.149 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208107+0000", 2024-04-24T15:33:39.149 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0910000000000003e-06 2024-04-24T15:33:39.150 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.150 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.150 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:39.150 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251054+0000", 2024-04-24T15:33:39.150 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042946826 2024-04-24T15:33:39.150 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.150 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.150 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:39.150 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251092+0000", 2024-04-24T15:33:39.150 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.7941999999999997e-05 2024-04-24T15:33:39.150 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.150 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.151 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:39.151 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251137+0000", 2024-04-24T15:33:39.151 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4638000000000002e-05 2024-04-24T15:33:39.151 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:39.151 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:39.151 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:39.152 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.152 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.152 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18010 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12615680~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:39.152 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208202+0000", 2024-04-24T15:33:39.152 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 64.925412570000006, 2024-04-24T15:33:39.152 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043046549000000003, 2024-04-24T15:33:39.152 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:39.152 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:39.152 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:39.152 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:39.152 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:39.152 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18010 2024-04-24T15:33:39.152 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.153 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:39.153 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.153 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:39.153 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208202+0000", 2024-04-24T15:33:39.153 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:39.153 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.153 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.153 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:39.153 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208202+0000", 2024-04-24T15:33:39.153 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:39.153 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.153 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.153 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:39.153 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208202+0000", 2024-04-24T15:33:39.154 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:39.154 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.154 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.154 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:39.154 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208205+0000", 2024-04-24T15:33:39.154 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.2490000000000001e-06 2024-04-24T15:33:39.154 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.154 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.154 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:39.154 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208215+0000", 2024-04-24T15:33:39.154 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.9070000000000002e-06 2024-04-24T15:33:39.154 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.154 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.154 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:39.154 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208220+0000", 2024-04-24T15:33:39.155 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8849999999999998e-06 2024-04-24T15:33:39.155 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.155 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.155 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:39.155 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251158+0000", 2024-04-24T15:33:39.155 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042938416 2024-04-24T15:33:39.155 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.155 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.155 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:39.155 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251197+0000", 2024-04-24T15:33:39.155 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9341000000000002e-05 2024-04-24T15:33:39.155 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.155 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.155 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:39.155 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251249+0000", 2024-04-24T15:33:39.156 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.1391000000000003e-05 2024-04-24T15:33:39.156 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:39.156 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:39.156 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:39.156 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.156 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.156 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18011 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12656640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:39.156 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208258+0000", 2024-04-24T15:33:39.156 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 64.925356723999997, 2024-04-24T15:33:39.156 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043102708000000003, 2024-04-24T15:33:39.156 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:39.156 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:39.156 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:39.156 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:39.156 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:39.157 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18011 2024-04-24T15:33:39.157 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.157 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:39.157 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.157 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:39.157 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208258+0000", 2024-04-24T15:33:39.157 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:39.157 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.157 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.157 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:39.157 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208258+0000", 2024-04-24T15:33:39.157 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:39.157 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.157 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.157 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:39.157 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208258+0000", 2024-04-24T15:33:39.157 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:39.158 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.158 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.158 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:39.158 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208260+0000", 2024-04-24T15:33:39.158 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7810000000000001e-06 2024-04-24T15:33:39.158 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.158 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.158 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:39.158 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208267+0000", 2024-04-24T15:33:39.158 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.7610000000000002e-06 2024-04-24T15:33:39.158 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.158 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.158 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:39.158 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208271+0000", 2024-04-24T15:33:39.158 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0890000000000002e-06 2024-04-24T15:33:39.159 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.159 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.159 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:39.159 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251271+0000", 2024-04-24T15:33:39.159 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043000159000000003 2024-04-24T15:33:39.159 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.159 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.159 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:39.159 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251315+0000", 2024-04-24T15:33:39.159 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3446999999999999e-05 2024-04-24T15:33:39.159 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.159 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.159 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:39.159 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251361+0000", 2024-04-24T15:33:39.159 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5952000000000003e-05 2024-04-24T15:33:39.159 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:39.160 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:39.160 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:39.160 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.160 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.160 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18012 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12697600~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:39.160 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208289+0000", 2024-04-24T15:33:39.160 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 64.92532568, 2024-04-24T15:33:39.160 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043174774999999999, 2024-04-24T15:33:39.160 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:39.160 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:39.160 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:39.160 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:39.160 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:39.160 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18012 2024-04-24T15:33:39.160 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.160 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:39.161 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.161 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:39.161 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208289+0000", 2024-04-24T15:33:39.161 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:39.161 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.161 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.161 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:39.161 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208289+0000", 2024-04-24T15:33:39.161 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:39.161 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.161 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.161 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:39.161 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208289+0000", 2024-04-24T15:33:39.161 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:39.161 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.162 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.162 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:39.162 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208291+0000", 2024-04-24T15:33:39.162 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.542e-06 2024-04-24T15:33:39.162 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.162 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.162 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:39.162 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208298+0000", 2024-04-24T15:33:39.162 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.3899999999999998e-06 2024-04-24T15:33:39.162 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.162 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.162 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:39.162 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208302+0000", 2024-04-24T15:33:39.162 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.019e-06 2024-04-24T15:33:39.163 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.163 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.163 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:39.163 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251382+0000", 2024-04-24T15:33:39.163 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043080790000000001 2024-04-24T15:33:39.163 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.163 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.163 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:39.163 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251421+0000", 2024-04-24T15:33:39.163 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8473000000000002e-05 2024-04-24T15:33:39.163 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.163 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.163 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:39.163 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251464+0000", 2024-04-24T15:33:39.163 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3133999999999997e-05 2024-04-24T15:33:39.163 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:39.164 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:39.164 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:39.164 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.164 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.164 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18013 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12738560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:39.164 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208423+0000", 2024-04-24T15:33:39.164 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 64.925191572000003, 2024-04-24T15:33:39.164 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043145554000000003, 2024-04-24T15:33:39.164 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:39.164 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:39.164 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:39.164 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:39.164 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:39.164 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18013 2024-04-24T15:33:39.164 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.165 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:39.165 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.165 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:39.165 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208423+0000", 2024-04-24T15:33:39.165 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:39.165 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.165 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.165 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:39.165 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208423+0000", 2024-04-24T15:33:39.165 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:39.165 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.165 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.165 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:39.165 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208423+0000", 2024-04-24T15:33:39.165 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:39.166 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.166 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.166 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:39.166 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208426+0000", 2024-04-24T15:33:39.166 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.0539999999999999e-06 2024-04-24T15:33:39.166 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.166 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.166 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:39.166 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208433+0000", 2024-04-24T15:33:39.166 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.43e-06 2024-04-24T15:33:39.166 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.166 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.166 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:39.166 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208437+0000", 2024-04-24T15:33:39.166 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5590000000000001e-06 2024-04-24T15:33:39.167 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.167 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.167 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:39.167 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251485+0000", 2024-04-24T15:33:39.167 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043048259999999998 2024-04-24T15:33:39.167 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.167 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.167 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:39.167 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251525+0000", 2024-04-24T15:33:39.167 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9916999999999997e-05 2024-04-24T15:33:39.167 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.167 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.167 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:39.168 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251569+0000", 2024-04-24T15:33:39.168 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3939000000000001e-05 2024-04-24T15:33:39.168 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:39.168 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:39.168 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:39.168 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.168 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.168 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18015 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12779520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:39.168 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208630+0000", 2024-04-24T15:33:39.168 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 64.924985047000007, 2024-04-24T15:33:39.168 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043047328000000003, 2024-04-24T15:33:39.168 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:39.168 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:39.168 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:39.169 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:39.169 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:39.169 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18015 2024-04-24T15:33:39.169 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.169 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:39.169 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.169 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:39.169 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208630+0000", 2024-04-24T15:33:39.169 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:39.169 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.169 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.169 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:39.169 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208630+0000", 2024-04-24T15:33:39.169 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:39.169 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.170 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.170 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:39.170 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208629+0000", 2024-04-24T15:33:39.170 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999999 2024-04-24T15:33:39.170 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.170 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.170 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:39.170 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208634+0000", 2024-04-24T15:33:39.170 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8960000000000002e-06 2024-04-24T15:33:39.170 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.170 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.170 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:39.170 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208645+0000", 2024-04-24T15:33:39.170 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0939e-05 2024-04-24T15:33:39.170 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.170 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.171 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:39.171 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208651+0000", 2024-04-24T15:33:39.171 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.896e-06 2024-04-24T15:33:39.171 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.171 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.171 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:39.171 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251591+0000", 2024-04-24T15:33:39.171 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042939862000000002 2024-04-24T15:33:39.171 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.171 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.171 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:39.171 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251631+0000", 2024-04-24T15:33:39.171 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0216000000000003e-05 2024-04-24T15:33:39.171 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.171 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.171 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:39.172 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251677+0000", 2024-04-24T15:33:39.172 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6390999999999997e-05 2024-04-24T15:33:39.172 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:39.172 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:39.172 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:39.172 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.172 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.172 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18016 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12820480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:39.172 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208660+0000", 2024-04-24T15:33:39.172 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 64.924955257999997, 2024-04-24T15:33:39.172 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043127510000000001, 2024-04-24T15:33:39.172 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:39.172 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:39.172 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:39.172 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:39.172 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:39.173 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18016 2024-04-24T15:33:39.173 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.173 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:39.173 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.173 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:39.173 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208660+0000", 2024-04-24T15:33:39.173 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:39.173 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.173 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.173 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:39.173 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208660+0000", 2024-04-24T15:33:39.173 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:39.173 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.173 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.173 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:39.173 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208659+0000", 2024-04-24T15:33:39.173 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:39.174 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.174 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.174 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:39.174 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208662+0000", 2024-04-24T15:33:39.174 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.0979999999999998e-06 2024-04-24T15:33:39.174 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.174 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.174 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:39.174 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208669+0000", 2024-04-24T15:33:39.174 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.4470000000000003e-06 2024-04-24T15:33:39.174 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.174 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.174 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:39.174 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208673+0000", 2024-04-24T15:33:39.175 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2810000000000002e-06 2024-04-24T15:33:39.175 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.175 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.175 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:39.175 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251699+0000", 2024-04-24T15:33:39.175 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043025728999999999 2024-04-24T15:33:39.175 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.175 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.175 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:39.175 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251743+0000", 2024-04-24T15:33:39.175 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4469999999999999e-05 2024-04-24T15:33:39.175 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.175 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.175 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:39.175 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251787+0000", 2024-04-24T15:33:39.176 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3949999999999998e-05 2024-04-24T15:33:39.176 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:39.176 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:39.176 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:39.176 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.176 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.176 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18017 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12861440~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:39.176 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208721+0000", 2024-04-24T15:33:39.177 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 64.92489381, 2024-04-24T15:33:39.177 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043167749999999998, 2024-04-24T15:33:39.177 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:39.177 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:39.177 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:39.177 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:39.177 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:39.177 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18017 2024-04-24T15:33:39.177 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.177 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:39.178 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.178 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:39.178 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208721+0000", 2024-04-24T15:33:39.178 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:39.178 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.178 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.178 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:39.178 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208721+0000", 2024-04-24T15:33:39.178 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:39.178 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.178 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.178 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:39.178 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208720+0000", 2024-04-24T15:33:39.178 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:39.179 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.179 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.179 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:39.179 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208724+0000", 2024-04-24T15:33:39.179 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.3919999999999999e-06 2024-04-24T15:33:39.180 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.180 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.180 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:39.180 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208729+0000", 2024-04-24T15:33:39.180 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.3480000000000003e-06 2024-04-24T15:33:39.180 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.180 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.180 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:39.180 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208732+0000", 2024-04-24T15:33:39.180 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.9529999999999999e-06 2024-04-24T15:33:39.180 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.180 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.180 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:39.180 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251807+0000", 2024-04-24T15:33:39.181 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043074492999999998 2024-04-24T15:33:39.181 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.181 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.181 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:39.181 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251846+0000", 2024-04-24T15:33:39.181 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8958999999999997e-05 2024-04-24T15:33:39.181 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.181 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.181 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:39.181 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251889+0000", 2024-04-24T15:33:39.181 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3253000000000001e-05 2024-04-24T15:33:39.181 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:39.181 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:39.181 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:39.181 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.182 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.182 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18018 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12902400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:39.182 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208752+0000", 2024-04-24T15:33:39.182 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 64.924863196999993, 2024-04-24T15:33:39.182 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043241012000000002, 2024-04-24T15:33:39.182 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:39.182 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:39.182 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:39.182 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:39.182 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:39.182 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18018 2024-04-24T15:33:39.182 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.182 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:39.182 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.183 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:39.183 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208752+0000", 2024-04-24T15:33:39.183 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:39.183 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.183 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.183 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:39.183 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208752+0000", 2024-04-24T15:33:39.183 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:39.183 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.183 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.183 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:39.183 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208751+0000", 2024-04-24T15:33:39.183 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:39.183 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.183 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.183 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:39.184 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208753+0000", 2024-04-24T15:33:39.184 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.894e-06 2024-04-24T15:33:39.184 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.184 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.184 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:39.184 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208757+0000", 2024-04-24T15:33:39.184 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.856e-06 2024-04-24T15:33:39.184 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.184 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.184 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:39.184 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208760+0000", 2024-04-24T15:33:39.184 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.6749999999999998e-06 2024-04-24T15:33:39.184 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.184 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.184 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:39.185 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251909+0000", 2024-04-24T15:33:39.185 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043149145999999999 2024-04-24T15:33:39.185 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.185 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.185 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:39.185 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251948+0000", 2024-04-24T15:33:39.185 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9159000000000002e-05 2024-04-24T15:33:39.185 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.185 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.185 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:39.185 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251993+0000", 2024-04-24T15:33:39.185 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4638999999999997e-05 2024-04-24T15:33:39.185 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:39.185 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:39.185 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:39.186 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.186 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.186 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18019 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12943360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:39.186 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208938+0000", 2024-04-24T15:33:39.186 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 64.924676906000002, 2024-04-24T15:33:39.186 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043162272000000002, 2024-04-24T15:33:39.186 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:39.186 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:39.186 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:39.186 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:39.186 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:39.186 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18019 2024-04-24T15:33:39.186 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.186 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:39.186 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.187 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:39.187 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208938+0000", 2024-04-24T15:33:39.187 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:39.187 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.187 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.187 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:39.187 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208938+0000", 2024-04-24T15:33:39.187 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:39.187 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.187 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.187 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:39.187 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208762+0000", 2024-04-24T15:33:39.187 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999824 2024-04-24T15:33:39.187 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.187 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.187 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:39.188 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208942+0000", 2024-04-24T15:33:39.188 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000180008 2024-04-24T15:33:39.188 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.188 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.188 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:39.188 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208950+0000", 2024-04-24T15:33:39.188 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.1550000000000007e-06 2024-04-24T15:33:39.188 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.188 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.188 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:39.188 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208956+0000", 2024-04-24T15:33:39.188 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.7910000000000001e-06 2024-04-24T15:33:39.188 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.188 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.188 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:39.189 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252014+0000", 2024-04-24T15:33:39.189 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043058427000000003 2024-04-24T15:33:39.189 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.189 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.189 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:39.189 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252054+0000", 2024-04-24T15:33:39.189 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0071999999999999e-05 2024-04-24T15:33:39.189 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.189 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.189 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:39.189 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252100+0000", 2024-04-24T15:33:39.189 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5917000000000001e-05 2024-04-24T15:33:39.189 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:39.189 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:39.190 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:39.190 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.190 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.190 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18020 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12984320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:39.190 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.209092+0000", 2024-04-24T15:33:39.190 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 64.924522908, 2024-04-24T15:33:39.190 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043111295000000001, 2024-04-24T15:33:39.190 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:39.190 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:39.190 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:39.190 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:39.190 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:39.190 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18020 2024-04-24T15:33:39.190 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.190 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:39.191 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.191 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:39.191 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209092+0000", 2024-04-24T15:33:39.191 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:39.191 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.191 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.191 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:39.191 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209092+0000", 2024-04-24T15:33:39.191 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:39.191 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.191 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.191 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:39.191 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208964+0000", 2024-04-24T15:33:39.191 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998722 2024-04-24T15:33:39.191 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.192 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.192 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:39.192 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209095+0000", 2024-04-24T15:33:39.192 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000130648 2024-04-24T15:33:39.192 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.192 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.192 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:39.192 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209103+0000", 2024-04-24T15:33:39.192 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0320000000000003e-06 2024-04-24T15:33:39.192 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.192 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.192 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:39.192 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209108+0000", 2024-04-24T15:33:39.192 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8690000000000003e-06 2024-04-24T15:33:39.192 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.192 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.193 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:39.193 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252121+0000", 2024-04-24T15:33:39.193 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043013323999999999 2024-04-24T15:33:39.193 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.193 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.193 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:39.193 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252160+0000", 2024-04-24T15:33:39.193 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8376e-05 2024-04-24T15:33:39.193 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.193 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.193 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:39.193 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252203+0000", 2024-04-24T15:33:39.193 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.371e-05 2024-04-24T15:33:39.193 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:39.193 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:39.194 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:39.194 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.194 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.194 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18021 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 13025280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:39.194 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.209132+0000", 2024-04-24T15:33:39.194 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 64.924482549999993, 2024-04-24T15:33:39.194 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043183903000000003, 2024-04-24T15:33:39.194 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:39.194 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:39.194 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:39.194 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:39.194 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:39.194 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18021 2024-04-24T15:33:39.194 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.194 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:39.195 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.195 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:39.195 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209132+0000", 2024-04-24T15:33:39.195 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:39.195 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.195 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.195 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:39.195 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209132+0000", 2024-04-24T15:33:39.195 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:39.195 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.195 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.195 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:39.195 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209132+0000", 2024-04-24T15:33:39.195 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:39.196 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.196 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.196 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:39.196 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209134+0000", 2024-04-24T15:33:39.196 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.891e-06 2024-04-24T15:33:39.196 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.196 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.196 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:39.196 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209138+0000", 2024-04-24T15:33:39.196 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0659999999999997e-06 2024-04-24T15:33:39.196 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.196 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.196 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:39.197 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209140+0000", 2024-04-24T15:33:39.197 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.5339999999999998e-06 2024-04-24T15:33:39.197 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.197 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.197 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:39.197 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252232+0000", 2024-04-24T15:33:39.197 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043091142999999998 2024-04-24T15:33:39.197 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.197 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.197 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:39.198 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252270+0000", 2024-04-24T15:33:39.198 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8872000000000003e-05 2024-04-24T15:33:39.198 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.198 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.198 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:39.198 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252316+0000", 2024-04-24T15:33:39.198 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5788000000000003e-05 2024-04-24T15:33:39.198 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:39.198 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:39.198 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:39.198 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.198 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.198 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18022 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 13066240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:39.198 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.209411+0000", 2024-04-24T15:33:39.198 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 64.924204001999996, 2024-04-24T15:33:39.199 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043016616000000001, 2024-04-24T15:33:39.199 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:39.199 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:39.199 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:39.199 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:39.199 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:39.199 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18022 2024-04-24T15:33:39.199 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.199 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:39.199 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.199 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:39.199 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209411+0000", 2024-04-24T15:33:39.199 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:39.199 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.199 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.199 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:39.200 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209411+0000", 2024-04-24T15:33:39.200 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:39.200 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.200 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.200 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:39.200 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209143+0000", 2024-04-24T15:33:39.200 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999732 2024-04-24T15:33:39.200 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.200 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.200 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:39.200 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209415+0000", 2024-04-24T15:33:39.200 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000271873 2024-04-24T15:33:39.200 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.200 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.200 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:39.201 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209423+0000", 2024-04-24T15:33:39.201 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3080000000000008e-06 2024-04-24T15:33:39.201 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.201 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.201 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:39.201 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209428+0000", 2024-04-24T15:33:39.201 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.5040000000000002e-06 2024-04-24T15:33:39.201 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.201 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.201 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:39.201 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252339+0000", 2024-04-24T15:33:39.201 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042910419999999998 2024-04-24T15:33:39.201 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.201 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.201 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:39.202 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252380+0000", 2024-04-24T15:33:39.202 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0991000000000001e-05 2024-04-24T15:33:39.202 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.202 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.202 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:39.202 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252427+0000", 2024-04-24T15:33:39.202 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.7647e-05 2024-04-24T15:33:39.202 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:39.202 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:39.202 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:39.202 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.202 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.202 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18023 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 13107200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:39.202 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.209594+0000", 2024-04-24T15:33:39.203 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 64.924020489, 2024-04-24T15:33:39.203 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042938543000000003, 2024-04-24T15:33:39.203 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:39.203 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:39.203 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:39.203 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:39.203 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:39.203 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18023 2024-04-24T15:33:39.203 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.203 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:39.203 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.203 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:39.203 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209594+0000", 2024-04-24T15:33:39.203 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:39.204 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.204 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.204 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:39.204 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209594+0000", 2024-04-24T15:33:39.204 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:39.204 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.204 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.204 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:39.204 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209440+0000", 2024-04-24T15:33:39.204 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999846 2024-04-24T15:33:39.204 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.204 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.204 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:39.204 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209598+0000", 2024-04-24T15:33:39.205 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00015731100000000001 2024-04-24T15:33:39.205 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.205 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.205 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:39.205 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209604+0000", 2024-04-24T15:33:39.205 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.2910000000000004e-06 2024-04-24T15:33:39.205 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.205 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.205 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:39.205 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209607+0000", 2024-04-24T15:33:39.205 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5389999999999999e-06 2024-04-24T15:33:39.205 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.205 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.205 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:39.206 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252449+0000", 2024-04-24T15:33:39.206 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042841833000000003 2024-04-24T15:33:39.206 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.206 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.206 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:39.206 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252489+0000", 2024-04-24T15:33:39.206 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9898000000000003e-05 2024-04-24T15:33:39.207 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.207 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.207 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:39.207 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252533+0000", 2024-04-24T15:33:39.207 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3702999999999998e-05 2024-04-24T15:33:39.207 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:39.207 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:39.207 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:39.207 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.208 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.208 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18025 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 13189120~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:39.208 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.209832+0000", 2024-04-24T15:33:39.208 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 64.923782987999999, 2024-04-24T15:33:39.208 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042913556999999998, 2024-04-24T15:33:39.208 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:39.208 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:39.208 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:39.208 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:39.208 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:39.208 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18025 2024-04-24T15:33:39.208 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.208 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:39.208 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.208 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:39.209 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209832+0000", 2024-04-24T15:33:39.209 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:39.209 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.209 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.209 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:39.209 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209832+0000", 2024-04-24T15:33:39.209 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:39.209 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.209 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.209 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:39.209 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209774+0000", 2024-04-24T15:33:39.209 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999418 2024-04-24T15:33:39.209 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.209 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.210 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:39.210 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209835+0000", 2024-04-24T15:33:39.210 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.1377000000000005e-05 2024-04-24T15:33:39.210 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.210 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.210 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:39.210 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209843+0000", 2024-04-24T15:33:39.210 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.2330000000000006e-06 2024-04-24T15:33:39.210 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.210 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.210 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:39.210 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209848+0000", 2024-04-24T15:33:39.210 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6149999999999999e-06 2024-04-24T15:33:39.210 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.211 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.211 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:39.211 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252658+0000", 2024-04-24T15:33:39.211 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042810002 2024-04-24T15:33:39.211 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.211 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.211 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:39.211 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252697+0000", 2024-04-24T15:33:39.211 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8757999999999997e-05 2024-04-24T15:33:39.211 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.211 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.211 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:39.211 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252745+0000", 2024-04-24T15:33:39.211 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.863e-05 2024-04-24T15:33:39.211 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:39.211 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:39.212 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:39.212 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.212 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.212 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:24749 4.8 4:137f5701:::rbd_header.114ae15b9a20:head [watch reconnect cookie 140672595753696 gen 11] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-04-24T15:33:39.212 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:37.628703+0000", 2024-04-24T15:33:39.212 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 61.504911368999998, 2024-04-24T15:33:39.212 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0102918700000001, 2024-04-24T15:33:39.212 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:39.212 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:39.212 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:39.212 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:39.212 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:39.212 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 24749 2024-04-24T15:33:39.212 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.212 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:39.213 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.213 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:39.213 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628703+0000", 2024-04-24T15:33:39.213 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:39.213 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.213 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.213 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:39.213 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628703+0000", 2024-04-24T15:33:39.213 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:39.213 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.213 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.213 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:39.213 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628703+0000", 2024-04-24T15:33:39.213 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999999 2024-04-24T15:33:39.213 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.213 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.213 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:39.214 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628708+0000", 2024-04-24T15:33:39.214 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.4589999999999997e-06 2024-04-24T15:33:39.214 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.214 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.214 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:39.214 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628721+0000", 2024-04-24T15:33:39.214 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.2601e-05 2024-04-24T15:33:39.214 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.214 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.214 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:39.214 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628728+0000", 2024-04-24T15:33:39.214 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.9059999999999996e-06 2024-04-24T15:33:39.214 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.214 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.214 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:39.215 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628789+0000", 2024-04-24T15:33:39.215 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.1672000000000002e-05 2024-04-24T15:33:39.215 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.215 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.215 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting_for_map not empty", 2024-04-24T15:33:39.215 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628797+0000", 2024-04-24T15:33:39.215 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.8189999999999992e-06 2024-04-24T15:33:39.215 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.215 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.215 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:39.215 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.631255+0000", 2024-04-24T15:33:39.215 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0024577610000000001 2024-04-24T15:33:39.215 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.215 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.215 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for peered", 2024-04-24T15:33:39.216 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.631263+0000", 2024-04-24T15:33:39.216 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.4109999999999996e-06 2024-04-24T15:33:39.216 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.216 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.216 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:39.216 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:38.637560+0000", 2024-04-24T15:33:39.216 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0062970339999999 2024-04-24T15:33:39.216 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.216 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.216 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:39.216 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:38.637747+0000", 2024-04-24T15:33:39.216 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00018693400000000001 2024-04-24T15:33:39.216 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.216 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.217 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:39.217 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:38.638995+0000", 2024-04-24T15:33:39.217 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0012481300000000001 2024-04-24T15:33:39.217 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:39.217 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:39.217 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:39.217 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.218 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.218 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:24750 4.8 4:137f5701:::rbd_header.114ae15b9a20:head [notify cookie 140672594434416] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-04-24T15:33:39.218 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:37.628736+0000", 2024-04-24T15:33:39.218 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 61.504879109000001, 2024-04-24T15:33:39.218 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0096094309999999, 2024-04-24T15:33:39.218 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:39.218 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:39.219 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:39.219 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:39.219 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:39.219 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 24750 2024-04-24T15:33:39.219 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.219 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:39.220 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.220 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:39.220 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628736+0000", 2024-04-24T15:33:39.220 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:39.220 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.220 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.221 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:39.221 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628736+0000", 2024-04-24T15:33:39.221 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:39.221 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.221 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.221 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:39.222 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628735+0000", 2024-04-24T15:33:39.222 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:39.222 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.222 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.222 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:39.222 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628739+0000", 2024-04-24T15:33:39.222 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5109999999999999e-06 2024-04-24T15:33:39.223 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.223 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.223 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:39.223 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628745+0000", 2024-04-24T15:33:39.223 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.8300000000000001e-06 2024-04-24T15:33:39.223 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.224 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.224 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:39.224 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628749+0000", 2024-04-24T15:33:39.224 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.7110000000000003e-06 2024-04-24T15:33:39.224 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.224 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.225 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:39.225 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628826+0000", 2024-04-24T15:33:39.225 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.6678e-05 2024-04-24T15:33:39.225 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.225 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.225 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting_for_map not empty", 2024-04-24T15:33:39.226 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628834+0000", 2024-04-24T15:33:39.226 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.6190000000000002e-06 2024-04-24T15:33:39.226 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.226 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.226 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:39.226 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.631305+0000", 2024-04-24T15:33:39.226 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0024714400000000001 2024-04-24T15:33:39.226 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.227 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.227 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for peered", 2024-04-24T15:33:39.227 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.631313+0000", 2024-04-24T15:33:39.227 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.6529999999999994e-06 2024-04-24T15:33:39.227 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.227 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.228 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:39.228 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:38.638068+0000", 2024-04-24T15:33:39.228 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0067548369999999 2024-04-24T15:33:39.228 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.228 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.228 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:39.228 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:38.638158+0000", 2024-04-24T15:33:39.229 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.0079000000000003e-05 2024-04-24T15:33:39.229 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:39.229 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:39.229 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:39.229 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:38.638345+0000", 2024-04-24T15:33:39.229 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00018757100000000001 2024-04-24T15:33:39.230 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:39.230 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:39.230 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:39.230 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:39.230 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:39.230 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:39.232 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-24T15:33:39.234 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:39.234+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:39.334 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:39.334+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:39.358 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:39.358 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:33:39.359 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-04-24T15:33:39.359 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:39.368 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-24T15:33:39.452 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:33:39.453+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:33:39.533 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:39.534 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:33:39.534 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-04-24T15:33:39.534 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-04-24T15:33:39.534 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:39.535 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:39.536+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:39.541 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-24T15:33:39.636 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:39.637+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:39.706 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:39.707 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-04-24T15:33:39.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-04-24T15:33:39.707 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-04-24T15:33:39.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.707 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:47384 4.1f 4:fdac2fcc:::rbd_header.114a8b543e91:head [watch reconnect cookie 140672527126976 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-04-24T15:33:39.707 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:09.802333+0000", 2024-04-24T15:33:39.707 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.904083486000001, 2024-04-24T15:33:39.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0093213050000001, 2024-04-24T15:33:39.707 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:39.707 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:39.707 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:39.707 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:39.707 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:39.707 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 47384 2024-04-24T15:33:39.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.707 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:39.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:39.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.802333+0000", 2024-04-24T15:33:39.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:39.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:39.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.802333+0000", 2024-04-24T15:33:39.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:39.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:39.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.802332+0000", 2024-04-24T15:33:39.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999981 2024-04-24T15:33:39.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:39.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.802340+0000", 2024-04-24T15:33:39.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7700000000000007e-06 2024-04-24T15:33:39.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:39.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.802360+0000", 2024-04-24T15:33:39.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9959999999999999e-05 2024-04-24T15:33:39.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:39.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.802371+0000", 2024-04-24T15:33:39.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0473e-05 2024-04-24T15:33:39.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:39.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.802447+0000", 2024-04-24T15:33:39.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6089999999999998e-05 2024-04-24T15:33:39.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op must wait for map", 2024-04-24T15:33:39.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.802456+0000", 2024-04-24T15:33:39.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0599999999999997e-06 2024-04-24T15:33:39.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:39.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.808378+0000", 2024-04-24T15:33:39.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0059224910000000002 2024-04-24T15:33:39.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-04-24T15:33:39.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.808384+0000", 2024-04-24T15:33:39.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7740000000000001e-06 2024-04-24T15:33:39.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:39.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.809812+0000", 2024-04-24T15:33:39.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.001428043 2024-04-24T15:33:39.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:39.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.810082+0000", 2024-04-24T15:33:39.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00026971800000000001 2024-04-24T15:33:39.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:39.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.811655+0000", 2024-04-24T15:33:39.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015726869999999999 2024-04-24T15:33:39.712 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:39.712 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:39.713 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:39.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.713 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:47385 4.1f 4:fdac2fcc:::rbd_header.114a8b543e91:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-04-24T15:33:39.713 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:09.803953+0000", 2024-04-24T15:33:39.713 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.902463571999998, 2024-04-24T15:33:39.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0080910729999999, 2024-04-24T15:33:39.713 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:39.713 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:39.713 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:39.713 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:39.713 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:39.713 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 47385 2024-04-24T15:33:39.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.713 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:39.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:39.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.803953+0000", 2024-04-24T15:33:39.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:39.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:39.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.803953+0000", 2024-04-24T15:33:39.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:39.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:39.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.803952+0000", 2024-04-24T15:33:39.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999981 2024-04-24T15:33:39.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:39.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.803961+0000", 2024-04-24T15:33:39.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9549999999999998e-06 2024-04-24T15:33:39.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:39.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.803977+0000", 2024-04-24T15:33:39.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6498999999999999e-05 2024-04-24T15:33:39.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:39.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.803985+0000", 2024-04-24T15:33:39.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5000000000000002e-06 2024-04-24T15:33:39.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.716 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:39.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.804032+0000", 2024-04-24T15:33:39.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7657999999999997e-05 2024-04-24T15:33:39.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.716 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2024-04-24T15:33:39.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.804039+0000", 2024-04-24T15:33:39.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5719999999999999e-06 2024-04-24T15:33:39.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.716 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:39.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.808397+0000", 2024-04-24T15:33:39.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0043579860000000003 2024-04-24T15:33:39.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.716 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-04-24T15:33:39.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.808401+0000", 2024-04-24T15:33:39.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3089999999999997e-06 2024-04-24T15:33:39.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:39.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.810700+0000", 2024-04-24T15:33:39.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0022984699999999 2024-04-24T15:33:39.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:39.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.810758+0000", 2024-04-24T15:33:39.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8121000000000001e-05 2024-04-24T15:33:39.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:33:39.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.811932+0000", 2024-04-24T15:33:39.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001174542 2024-04-24T15:33:39.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:39.718 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.811978+0000", 2024-04-24T15:33:39.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5645000000000001e-05 2024-04-24T15:33:39.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:39.718 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.812044+0000", 2024-04-24T15:33:39.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6575999999999994e-05 2024-04-24T15:33:39.718 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:39.718 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:39.719 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:39.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.719 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48590 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2686976~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:39.719 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889219+0000", 2024-04-24T15:33:39.719 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.81719764, 2024-04-24T15:33:39.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016760856000000001, 2024-04-24T15:33:39.719 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:39.719 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:39.719 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:39.719 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:39.719 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:39.719 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48590 2024-04-24T15:33:39.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.719 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:39.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:39.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889219+0000", 2024-04-24T15:33:39.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:39.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:39.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889219+0000", 2024-04-24T15:33:39.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:39.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:39.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889219+0000", 2024-04-24T15:33:39.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:39.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.721 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:39.721 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889221+0000", 2024-04-24T15:33:39.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4499999999999998e-06 2024-04-24T15:33:39.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.721 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:39.721 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889227+0000", 2024-04-24T15:33:39.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6869999999999999e-06 2024-04-24T15:33:39.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.721 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:39.721 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889231+0000", 2024-04-24T15:33:39.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6320000000000001e-06 2024-04-24T15:33:39.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.721 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:39.722 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.905769+0000", 2024-04-24T15:33:39.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016538850000000001 2024-04-24T15:33:39.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.722 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.722 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:39.722 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.905855+0000", 2024-04-24T15:33:39.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5296000000000005e-05 2024-04-24T15:33:39.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.722 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.722 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:39.722 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.905980+0000", 2024-04-24T15:33:39.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000125406 2024-04-24T15:33:39.722 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:39.722 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:39.722 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:39.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.723 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48705 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 1851392~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:39.723 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889236+0000", 2024-04-24T15:33:39.723 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.817181032000001, 2024-04-24T15:33:39.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016984302, 2024-04-24T15:33:39.723 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:39.723 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:39.723 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:39.723 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:39.723 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:39.723 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48705 2024-04-24T15:33:39.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.723 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:39.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:39.724 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889236+0000", 2024-04-24T15:33:39.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:39.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.724 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:39.724 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889236+0000", 2024-04-24T15:33:39.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:39.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.724 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:39.724 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889235+0000", 2024-04-24T15:33:39.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:39.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.724 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:39.725 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889238+0000", 2024-04-24T15:33:39.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.249e-06 2024-04-24T15:33:39.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.725 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:39.725 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889243+0000", 2024-04-24T15:33:39.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6810000000000003e-06 2024-04-24T15:33:39.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.725 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:39.725 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889247+0000", 2024-04-24T15:33:39.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6780000000000002e-06 2024-04-24T15:33:39.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.726 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:39.726 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906008+0000", 2024-04-24T15:33:39.726 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01676099 2024-04-24T15:33:39.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.726 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.726 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:39.726 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906090+0000", 2024-04-24T15:33:39.726 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1508999999999995e-05 2024-04-24T15:33:39.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.726 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.726 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:39.726 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906220+0000", 2024-04-24T15:33:39.726 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013056999999999999 2024-04-24T15:33:39.726 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:39.726 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:39.727 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:39.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.727 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48706 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 1916928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:39.727 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889251+0000", 2024-04-24T15:33:39.727 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.817165598999999, 2024-04-24T15:33:39.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01718693, 2024-04-24T15:33:39.727 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:39.727 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:39.727 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:39.727 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:39.727 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:39.727 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48706 2024-04-24T15:33:39.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.727 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:39.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:39.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889251+0000", 2024-04-24T15:33:39.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:39.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:39.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889251+0000", 2024-04-24T15:33:39.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:39.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:39.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889251+0000", 2024-04-24T15:33:39.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:39.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:39.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889253+0000", 2024-04-24T15:33:39.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.244e-06 2024-04-24T15:33:39.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:39.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889259+0000", 2024-04-24T15:33:39.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3640000000000004e-06 2024-04-24T15:33:39.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:39.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889263+0000", 2024-04-24T15:33:39.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3720000000000001e-06 2024-04-24T15:33:39.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:39.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906258+0000", 2024-04-24T15:33:39.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016995310999999999 2024-04-24T15:33:39.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.730 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.730 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:39.730 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906343+0000", 2024-04-24T15:33:39.730 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4536000000000005e-05 2024-04-24T15:33:39.730 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.730 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.730 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:39.730 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906438+0000", 2024-04-24T15:33:39.730 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5502999999999997e-05 2024-04-24T15:33:39.730 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:39.730 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:39.730 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:39.730 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.730 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.730 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48707 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 1982464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:39.731 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889267+0000", 2024-04-24T15:33:39.731 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.817150039000001, 2024-04-24T15:33:39.731 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017425395, 2024-04-24T15:33:39.731 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:39.731 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:39.731 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:39.731 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:39.731 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:39.731 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48707 2024-04-24T15:33:39.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.731 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:39.731 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.731 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:39.731 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889267+0000", 2024-04-24T15:33:39.731 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:39.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:39.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889267+0000", 2024-04-24T15:33:39.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:39.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:39.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889266+0000", 2024-04-24T15:33:39.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:39.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:39.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889275+0000", 2024-04-24T15:33:39.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.568e-06 2024-04-24T15:33:39.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:39.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889281+0000", 2024-04-24T15:33:39.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3779999999999998e-06 2024-04-24T15:33:39.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:39.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889291+0000", 2024-04-24T15:33:39.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2769999999999994e-06 2024-04-24T15:33:39.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:39.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906476+0000", 2024-04-24T15:33:39.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017185277999999998 2024-04-24T15:33:39.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.734 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:39.734 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906567+0000", 2024-04-24T15:33:39.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0902999999999993e-05 2024-04-24T15:33:39.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.734 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.734 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:39.734 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906692+0000", 2024-04-24T15:33:39.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012541700000000001 2024-04-24T15:33:39.734 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:39.734 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:39.734 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:39.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.737 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48708 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2048000~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:39.737 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889295+0000", 2024-04-24T15:33:39.738 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.817121547999999, 2024-04-24T15:33:39.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017534984999999999, 2024-04-24T15:33:39.738 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:39.738 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:39.738 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:39.738 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:39.738 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:39.738 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48708 2024-04-24T15:33:39.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.738 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:39.738 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.738 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:39.738 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889295+0000", 2024-04-24T15:33:39.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:39.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:39.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889295+0000", 2024-04-24T15:33:39.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:39.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:39.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889295+0000", 2024-04-24T15:33:39.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:39.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:39.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889297+0000", 2024-04-24T15:33:39.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3479999999999998e-06 2024-04-24T15:33:39.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:39.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889304+0000", 2024-04-24T15:33:39.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3720000000000001e-06 2024-04-24T15:33:39.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:39.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889307+0000", 2024-04-24T15:33:39.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7160000000000001e-06 2024-04-24T15:33:39.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:39.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906711+0000", 2024-04-24T15:33:39.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017403452999999999 2024-04-24T15:33:39.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:39.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906765+0000", 2024-04-24T15:33:39.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4219999999999999e-05 2024-04-24T15:33:39.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:39.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906830+0000", 2024-04-24T15:33:39.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5277000000000005e-05 2024-04-24T15:33:39.741 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:39.741 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:39.741 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:39.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.741 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48709 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2113536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:39.741 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889312+0000", 2024-04-24T15:33:39.741 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.817105241, 2024-04-24T15:33:39.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017772013, 2024-04-24T15:33:39.742 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:39.742 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:39.742 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:39.742 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:39.742 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:39.742 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48709 2024-04-24T15:33:39.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.742 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:39.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.742 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:39.742 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889312+0000", 2024-04-24T15:33:39.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:39.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:39.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889312+0000", 2024-04-24T15:33:39.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:39.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:39.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889311+0000", 2024-04-24T15:33:39.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:39.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:39.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889314+0000", 2024-04-24T15:33:39.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2759999999999999e-06 2024-04-24T15:33:39.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:39.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889322+0000", 2024-04-24T15:33:39.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8589999999999994e-06 2024-04-24T15:33:39.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:39.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889326+0000", 2024-04-24T15:33:39.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6380000000000002e-06 2024-04-24T15:33:39.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:39.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906862+0000", 2024-04-24T15:33:39.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017536403999999998 2024-04-24T15:33:39.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:39.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906949+0000", 2024-04-24T15:33:39.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6217000000000005e-05 2024-04-24T15:33:39.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:39.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907084+0000", 2024-04-24T15:33:39.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013497100000000001 2024-04-24T15:33:39.745 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:39.745 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:39.745 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:39.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.746 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.746 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48710 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2179072~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:39.746 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889332+0000", 2024-04-24T15:33:39.746 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.817084427000001, 2024-04-24T15:33:39.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017944720000000001, 2024-04-24T15:33:39.746 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:39.746 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:39.746 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:39.746 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:39.746 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:39.746 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48710 2024-04-24T15:33:39.746 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.746 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:39.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:39.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889332+0000", 2024-04-24T15:33:39.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:39.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:39.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889332+0000", 2024-04-24T15:33:39.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:39.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:39.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889332+0000", 2024-04-24T15:33:39.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:39.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:39.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889334+0000", 2024-04-24T15:33:39.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4210000000000002e-06 2024-04-24T15:33:39.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:39.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889340+0000", 2024-04-24T15:33:39.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7209999999999999e-06 2024-04-24T15:33:39.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:39.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889344+0000", 2024-04-24T15:33:39.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.726e-06 2024-04-24T15:33:39.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:39.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907119+0000", 2024-04-24T15:33:39.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017775551000000001 2024-04-24T15:33:39.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:39.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907190+0000", 2024-04-24T15:33:39.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0634000000000002e-05 2024-04-24T15:33:39.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:39.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907277+0000", 2024-04-24T15:33:39.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7120999999999995e-05 2024-04-24T15:33:39.750 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:39.750 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:39.750 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:39.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.750 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48711 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2244608~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:39.750 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889348+0000", 2024-04-24T15:33:39.750 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.817068809999999, 2024-04-24T15:33:39.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018188109000000001, 2024-04-24T15:33:39.750 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:39.751 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:39.751 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:39.751 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:39.751 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:39.751 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48711 2024-04-24T15:33:39.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.751 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:39.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:39.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889348+0000", 2024-04-24T15:33:39.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:39.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:39.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889348+0000", 2024-04-24T15:33:39.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:39.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:39.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889348+0000", 2024-04-24T15:33:39.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:39.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:39.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889350+0000", 2024-04-24T15:33:39.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2330000000000001e-06 2024-04-24T15:33:39.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:39.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889356+0000", 2024-04-24T15:33:39.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8769999999999998e-06 2024-04-24T15:33:39.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:39.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889359+0000", 2024-04-24T15:33:39.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3699999999999999e-06 2024-04-24T15:33:39.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:39.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907316+0000", 2024-04-24T15:33:39.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017956857999999999 2024-04-24T15:33:39.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:39.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907402+0000", 2024-04-24T15:33:39.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5927000000000006e-05 2024-04-24T15:33:39.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:39.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907536+0000", 2024-04-24T15:33:39.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000134258 2024-04-24T15:33:39.754 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:39.754 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:39.754 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:39.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.754 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48712 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2310144~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:39.755 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889363+0000", 2024-04-24T15:33:39.755 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.817053490999999, 2024-04-24T15:33:39.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018314815000000002, 2024-04-24T15:33:39.755 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:39.755 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:39.755 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:39.755 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:39.755 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:39.755 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48712 2024-04-24T15:33:39.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.755 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:39.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:39.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889363+0000", 2024-04-24T15:33:39.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:39.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:39.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889363+0000", 2024-04-24T15:33:39.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:39.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:39.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889363+0000", 2024-04-24T15:33:39.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:39.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:39.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889366+0000", 2024-04-24T15:33:39.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6709999999999999e-06 2024-04-24T15:33:39.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:39.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889373+0000", 2024-04-24T15:33:39.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1269999999999997e-06 2024-04-24T15:33:39.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:39.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889377+0000", 2024-04-24T15:33:39.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1219999999999997e-06 2024-04-24T15:33:39.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:39.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907560+0000", 2024-04-24T15:33:39.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018183357000000001 2024-04-24T15:33:39.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:39.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907612+0000", 2024-04-24T15:33:39.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2172999999999997e-05 2024-04-24T15:33:39.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:39.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907678+0000", 2024-04-24T15:33:39.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5736999999999998e-05 2024-04-24T15:33:39.758 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:39.758 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:39.759 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:39.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.759 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48713 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2375680~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:39.759 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889381+0000", 2024-04-24T15:33:39.759 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.817035598, 2024-04-24T15:33:39.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018556019, 2024-04-24T15:33:39.759 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:39.759 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:39.759 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:39.759 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:39.759 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:39.759 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48713 2024-04-24T15:33:39.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.760 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:39.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:39.760 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889381+0000", 2024-04-24T15:33:39.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:39.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:39.760 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889381+0000", 2024-04-24T15:33:39.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:39.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:39.760 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889381+0000", 2024-04-24T15:33:39.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:39.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:39.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889388+0000", 2024-04-24T15:33:39.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8809999999999998e-06 2024-04-24T15:33:39.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:39.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889394+0000", 2024-04-24T15:33:39.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0410000000000003e-06 2024-04-24T15:33:39.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:39.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889397+0000", 2024-04-24T15:33:39.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5939999999999998e-06 2024-04-24T15:33:39.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:39.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907711+0000", 2024-04-24T15:33:39.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018313949999999999 2024-04-24T15:33:39.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:39.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907802+0000", 2024-04-24T15:33:39.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0307000000000001e-05 2024-04-24T15:33:39.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:39.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907937+0000", 2024-04-24T15:33:39.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013561899999999999 2024-04-24T15:33:39.763 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:39.763 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:39.763 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:39.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.763 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48714 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2441216~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:39.763 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889435+0000", 2024-04-24T15:33:39.763 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.816982249999999, 2024-04-24T15:33:39.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018636040999999999, 2024-04-24T15:33:39.763 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:39.764 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:39.764 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:39.764 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:39.764 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:39.764 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48714 2024-04-24T15:33:39.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.764 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:39.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:39.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889435+0000", 2024-04-24T15:33:39.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:39.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:39.765 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889435+0000", 2024-04-24T15:33:39.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:39.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.765 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:39.765 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889434+0000", 2024-04-24T15:33:39.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:39.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.765 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:39.765 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889437+0000", 2024-04-24T15:33:39.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.875e-06 2024-04-24T15:33:39.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.765 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:39.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889444+0000", 2024-04-24T15:33:39.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9870000000000002e-06 2024-04-24T15:33:39.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:39.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889448+0000", 2024-04-24T15:33:39.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3030000000000001e-06 2024-04-24T15:33:39.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:39.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907952+0000", 2024-04-24T15:33:39.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018504245999999999 2024-04-24T15:33:39.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:39.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908007+0000", 2024-04-24T15:33:39.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4579e-05 2024-04-24T15:33:39.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:39.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908071+0000", 2024-04-24T15:33:39.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3615999999999995e-05 2024-04-24T15:33:39.767 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:39.767 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:39.767 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:39.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.767 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48715 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2506752~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:39.768 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889453+0000", 2024-04-24T15:33:39.768 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.816963886, 2024-04-24T15:33:39.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018873935000000001, 2024-04-24T15:33:39.768 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:39.768 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:39.768 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:39.768 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:39.768 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:39.768 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48715 2024-04-24T15:33:39.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.768 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:39.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:39.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889453+0000", 2024-04-24T15:33:39.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:39.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:39.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889453+0000", 2024-04-24T15:33:39.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:39.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:39.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889452+0000", 2024-04-24T15:33:39.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:39.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:39.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889459+0000", 2024-04-24T15:33:39.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9269999999999999e-06 2024-04-24T15:33:39.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:39.770 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889468+0000", 2024-04-24T15:33:39.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2390000000000002e-06 2024-04-24T15:33:39.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:39.770 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889475+0000", 2024-04-24T15:33:39.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3939999999999998e-06 2024-04-24T15:33:39.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:39.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908103+0000", 2024-04-24T15:33:39.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018628388999999999 2024-04-24T15:33:39.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:39.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908193+0000", 2024-04-24T15:33:39.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9343000000000002e-05 2024-04-24T15:33:39.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:39.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908327+0000", 2024-04-24T15:33:39.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013415899999999999 2024-04-24T15:33:39.771 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:39.771 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:39.771 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:39.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.772 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48716 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2572288~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:39.772 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889480+0000", 2024-04-24T15:33:39.772 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.816937035999999, 2024-04-24T15:33:39.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018992862999999999, 2024-04-24T15:33:39.772 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:39.772 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:39.772 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:39.772 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:39.772 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:39.772 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48716 2024-04-24T15:33:39.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.772 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:39.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:39.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889480+0000", 2024-04-24T15:33:39.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:39.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:39.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889480+0000", 2024-04-24T15:33:39.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:39.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:39.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889479+0000", 2024-04-24T15:33:39.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:39.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:39.774 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889482+0000", 2024-04-24T15:33:39.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4729999999999999e-06 2024-04-24T15:33:39.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:39.774 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889488+0000", 2024-04-24T15:33:39.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1750000000000002e-06 2024-04-24T15:33:39.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:39.774 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889501+0000", 2024-04-24T15:33:39.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.288e-05 2024-04-24T15:33:39.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:39.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908345+0000", 2024-04-24T15:33:39.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018843835999999999 2024-04-24T15:33:39.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:39.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908395+0000", 2024-04-24T15:33:39.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9765999999999998e-05 2024-04-24T15:33:39.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:39.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908473+0000", 2024-04-24T15:33:39.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8159999999999997e-05 2024-04-24T15:33:39.775 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:39.776 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:39.776 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:39.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.776 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48717 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2637824~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:39.776 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889506+0000", 2024-04-24T15:33:39.776 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.816910785000001, 2024-04-24T15:33:39.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.019241864000000001, 2024-04-24T15:33:39.776 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:39.776 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:39.776 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:39.776 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:39.776 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:39.776 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48717 2024-04-24T15:33:39.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.777 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:39.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:39.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889506+0000", 2024-04-24T15:33:39.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:39.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:39.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889506+0000", 2024-04-24T15:33:39.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:39.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:39.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889506+0000", 2024-04-24T15:33:39.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:39.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:39.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889508+0000", 2024-04-24T15:33:39.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6900000000000001e-06 2024-04-24T15:33:39.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:39.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889515+0000", 2024-04-24T15:33:39.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5409999999999997e-06 2024-04-24T15:33:39.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:39.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889519+0000", 2024-04-24T15:33:39.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7129999999999999e-06 2024-04-24T15:33:39.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:39.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908520+0000", 2024-04-24T15:33:39.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.019001606000000001 2024-04-24T15:33:39.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:39.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908612+0000", 2024-04-24T15:33:39.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1909000000000004e-05 2024-04-24T15:33:39.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:39.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908748+0000", 2024-04-24T15:33:39.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000135847 2024-04-24T15:33:39.780 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:39.780 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:39.780 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:39.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.780 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48718 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2703360~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:39.780 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889523+0000", 2024-04-24T15:33:39.780 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.816893792999998, 2024-04-24T15:33:39.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.019451725999999999, 2024-04-24T15:33:39.780 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:39.780 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:39.780 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:39.781 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:39.781 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:39.781 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48718 2024-04-24T15:33:39.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.781 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:39.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:39.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889523+0000", 2024-04-24T15:33:39.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:39.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:39.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889523+0000", 2024-04-24T15:33:39.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:39.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:39.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889523+0000", 2024-04-24T15:33:39.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:39.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:39.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889525+0000", 2024-04-24T15:33:39.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9970000000000001e-06 2024-04-24T15:33:39.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:39.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889530+0000", 2024-04-24T15:33:39.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7610000000000004e-06 2024-04-24T15:33:39.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:39.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889534+0000", 2024-04-24T15:33:39.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6119999999999999e-06 2024-04-24T15:33:39.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:39.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908775+0000", 2024-04-24T15:33:39.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.019240709000000002 2024-04-24T15:33:39.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:39.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908856+0000", 2024-04-24T15:33:39.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0788e-05 2024-04-24T15:33:39.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:39.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908975+0000", 2024-04-24T15:33:39.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011927800000000001 2024-04-24T15:33:39.784 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:39.784 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:39.784 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:39.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.784 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:49872 4.1f 4:fe48f4c1:::rbd_data.114ab402c616.0000000000000014:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-04-24T15:33:39.784 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:12.416730+0000", 2024-04-24T15:33:39.784 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.289686815, 2024-04-24T15:33:39.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.020522327999999999, 2024-04-24T15:33:39.785 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:39.785 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:39.785 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:39.785 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:39.785 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:39.785 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 49872 2024-04-24T15:33:39.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.785 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:39.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:39.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.416730+0000", 2024-04-24T15:33:39.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:39.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:39.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.416730+0000", 2024-04-24T15:33:39.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:39.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:39.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.416729+0000", 2024-04-24T15:33:39.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:33:39.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:39.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.416734+0000", 2024-04-24T15:33:39.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0180000000000001e-06 2024-04-24T15:33:39.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:39.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.416744+0000", 2024-04-24T15:33:39.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0069e-05 2024-04-24T15:33:39.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:39.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.416749+0000", 2024-04-24T15:33:39.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7190000000000001e-06 2024-04-24T15:33:39.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:39.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.416777+0000", 2024-04-24T15:33:39.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7532999999999999e-05 2024-04-24T15:33:39.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:39.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.416824+0000", 2024-04-24T15:33:39.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7738000000000003e-05 2024-04-24T15:33:39.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:33:39.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.437145+0000", 2024-04-24T15:33:39.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.020320416000000001 2024-04-24T15:33:39.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:39.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.437189+0000", 2024-04-24T15:33:39.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4493000000000002e-05 2024-04-24T15:33:39.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:39.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.437252+0000", 2024-04-24T15:33:39.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3110999999999999e-05 2024-04-24T15:33:39.789 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:39.789 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:39.789 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:39.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.789 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:52545 4.a 4:5331820a:::rbd_data.114ab867d54d.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-04-24T15:33:39.789 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:14.399696+0000", 2024-04-24T15:33:39.789 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 25.306720412000001, 2024-04-24T15:33:39.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016877893000000001, 2024-04-24T15:33:39.790 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:39.790 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:39.790 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:39.790 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:39.790 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:39.790 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 52545 2024-04-24T15:33:39.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.790 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:39.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:39.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.399696+0000", 2024-04-24T15:33:39.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:39.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:39.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.399696+0000", 2024-04-24T15:33:39.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:39.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:39.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.399695+0000", 2024-04-24T15:33:39.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999981 2024-04-24T15:33:39.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:39.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.399704+0000", 2024-04-24T15:33:39.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.098e-06 2024-04-24T15:33:39.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:39.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.399721+0000", 2024-04-24T15:33:39.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7088999999999999e-05 2024-04-24T15:33:39.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:39.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.399728+0000", 2024-04-24T15:33:39.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5699999999999998e-06 2024-04-24T15:33:39.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:39.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.399775+0000", 2024-04-24T15:33:39.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7445999999999999e-05 2024-04-24T15:33:39.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:39.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.399840+0000", 2024-04-24T15:33:39.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5492000000000002e-05 2024-04-24T15:33:39.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:33:39.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.416468+0000", 2024-04-24T15:33:39.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016627527 2024-04-24T15:33:39.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:39.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.416521+0000", 2024-04-24T15:33:39.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2988000000000002e-05 2024-04-24T15:33:39.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:39.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.416574+0000", 2024-04-24T15:33:39.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3363999999999999e-05 2024-04-24T15:33:39.794 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:39.794 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:39.794 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:39.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.794 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:66966 4.1f 4:fb3b69af:::rbd_data.114aed92ca6e.000000000000001a:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-04-24T15:33:39.794 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:21.265525+0000", 2024-04-24T15:33:39.795 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.440891871000002, 2024-04-24T15:33:39.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018862027999999999, 2024-04-24T15:33:39.795 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:39.795 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:39.795 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:39.795 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:39.795 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:39.795 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 66966 2024-04-24T15:33:39.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.795 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:39.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:39.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:21.265525+0000", 2024-04-24T15:33:39.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:39.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:39.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:21.265525+0000", 2024-04-24T15:33:39.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:39.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:39.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:21.265523+0000", 2024-04-24T15:33:39.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999981 2024-04-24T15:33:39.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:39.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:21.265536+0000", 2024-04-24T15:33:39.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2489e-05 2024-04-24T15:33:39.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:39.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:21.265550+0000", 2024-04-24T15:33:39.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4218e-05 2024-04-24T15:33:39.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:39.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:21.265556+0000", 2024-04-24T15:33:39.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.241e-06 2024-04-24T15:33:39.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:39.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:21.265606+0000", 2024-04-24T15:33:39.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9972000000000002e-05 2024-04-24T15:33:39.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:39.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:21.265670+0000", 2024-04-24T15:33:39.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3425999999999999e-05 2024-04-24T15:33:39.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:33:39.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:21.284297+0000", 2024-04-24T15:33:39.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018627517999999999 2024-04-24T15:33:39.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:39.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:21.284339+0000", 2024-04-24T15:33:39.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1628000000000002e-05 2024-04-24T15:33:39.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:39.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:39.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:39.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:21.284387+0000", 2024-04-24T15:33:39.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8242000000000003e-05 2024-04-24T15:33:39.799 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:39.799 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:39.799 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:39.799 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:39.799 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:39.799 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:39.801 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-24T15:33:39.801 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:39.801+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:39.805 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-24T15:33:39.902 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:39.902+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:39.982 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:39.983 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:33:39.983 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-04-24T15:33:39.983 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:39.993 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-24T15:33:39.994 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-04-24T15:33:40.020 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-04-24T15:33:40.020 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:40.020+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:40.043 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-04-24T15:33:40.048 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:33:40.049+0000 7f0f4e2f4540 -1 osd.6 75 log_to_monitors true 2024-04-24T15:33:40.063 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-04-24T15:33:40.087 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-04-24T15:33:40.103 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:33:40.103+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:33:40.107 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-04-24T15:33:40.126 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-04-24T15:33:40.126 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T15:33:40.126 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T15:33:40.127 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T15:33:40.127 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T15:33:40.127 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T15:33:40.127 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T15:33:40.127 INFO:teuthology.orchestra.run.smithi096.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-04-24T15:33:40.127 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T15:33:40.130 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-24T15:33:40.130 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-24T15:33:40.134 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:40.134 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:40.145 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-24T15:33:40.229 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:33:40.230+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:33:40.305 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:40.305+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:40.308 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:40.308 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:40.319 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-24T15:33:40.405 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:40.407+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:40.482 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:40.482 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:40.491 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-24T15:33:40.535 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:33:40.535+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:33:40.607 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:40.608+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:40.615 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:40.615 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:40.625 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-24T15:33:40.708 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:40.709+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:40.788 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:40.788 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:40.798 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-24T15:33:40.809 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:33:40.810+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:33:40.909 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:40.909+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:40.922 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:40.922 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:40.932 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-24T15:33:40.978 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-24T15:33:41.010 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:33:41.010+0000 7f0f3dcf0700 -1 osd.6 103 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T15:33:41.017 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:41.017+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:41.069 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-24T15:33:41.111 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:41.111+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:41.130 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:41.130 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:41.141 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-24T15:33:41.200 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:41.200 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:33:41.200 INFO:teuthology.orchestra.run.smithi151.stdout: "num_ops": 0 2024-04-24T15:33:41.200 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:41.225 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:33:41.225+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:33:41.306 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:41.306 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:41.312 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:33:41.314+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:33:41.316 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-24T15:33:41.414 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:33:41.414+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:33:41.440 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:41.440 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:41.450 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-24T15:33:41.515 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:41.515+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:41.572 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:41.572 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:41.582 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-24T15:33:41.616 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:33:41.616+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:33:41.705 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:41.705 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:41.715 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-24T15:33:41.717 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:41.717+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:41.818 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:41.818+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:41.839 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:41.839 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:41.849 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-24T15:33:41.918 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:41.919+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:41.979 INFO:teuthology.orchestra.run.smithi096.stderr:noscrub is set 2024-04-24T15:33:41.982 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:41.982 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:41.990 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-24T15:33:42.034 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:42.035+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:42.120 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:42.121+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:42.132 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-24T15:33:42.153 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:42.153 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:42.163 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-24T15:33:42.220 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:42.220+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:42.270 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-04-24T15:33:42.286 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:42.286 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:42.288 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-04-24T15:33:42.296 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-24T15:33:42.306 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-04-24T15:33:42.323 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-04-24T15:33:42.339 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:42.340+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:42.342 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-04-24T15:33:42.361 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-04-24T15:33:42.379 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-04-24T15:33:42.397 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-04-24T15:33:42.398 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T15:33:42.399 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T15:33:42.399 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T15:33:42.399 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T15:33:42.399 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T15:33:42.399 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T15:33:42.399 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_enable_op_tracker = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' 2024-04-24T15:33:42.399 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T15:33:42.422 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:42.423+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:42.423 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:42.423 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:33:42.423 INFO:teuthology.orchestra.run.smithi151.stdout: "complaint_time": 30, 2024-04-24T15:33:42.424 INFO:teuthology.orchestra.run.smithi151.stdout: "num_blocked_ops": 0 2024-04-24T15:33:42.424 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:42.433 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-24T15:33:42.523 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:33:42.524+0000 7f0f45efb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 59992) UID: 0 2024-04-24T15:33:42.599 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:42.599 INFO:teuthology.orchestra.run.smithi151.stdout: "size": 20, 2024-04-24T15:33:42.599 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 600, 2024-04-24T15:33:42.599 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [ 2024-04-24T15:33:42.599 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.599 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18006 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12419072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:42.599 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.207911+0000", 2024-04-24T15:33:42.599 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 68.390784674000002, 2024-04-24T15:33:42.599 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042899496000000002, 2024-04-24T15:33:42.599 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:42.599 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:42.599 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:42.600 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:42.600 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:42.600 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18006 2024-04-24T15:33:42.600 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.600 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:42.600 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.600 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:42.600 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207911+0000", 2024-04-24T15:33:42.600 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:42.600 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.600 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.600 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:42.600 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207911+0000", 2024-04-24T15:33:42.600 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:42.600 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.600 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.601 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:42.601 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207910+0000", 2024-04-24T15:33:42.601 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:42.601 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.601 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.601 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:42.601 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207913+0000", 2024-04-24T15:33:42.601 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.2309999999999999e-06 2024-04-24T15:33:42.601 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.601 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.601 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:42.601 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207918+0000", 2024-04-24T15:33:42.601 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.3399999999999997e-06 2024-04-24T15:33:42.602 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.602 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.602 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:42.602 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207922+0000", 2024-04-24T15:33:42.602 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5439999999999999e-06 2024-04-24T15:33:42.602 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.602 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.602 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:42.602 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250726+0000", 2024-04-24T15:33:42.602 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042804129000000003 2024-04-24T15:33:42.602 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.602 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.602 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:42.603 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250765+0000", 2024-04-24T15:33:42.603 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9554000000000001e-05 2024-04-24T15:33:42.603 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.603 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.603 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:42.603 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250810+0000", 2024-04-24T15:33:42.603 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5027000000000001e-05 2024-04-24T15:33:42.603 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:42.603 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:42.603 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:42.603 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.603 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.603 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18007 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12460032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:42.603 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.207961+0000", 2024-04-24T15:33:42.603 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 68.390734832999996, 2024-04-24T15:33:42.604 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042965470999999998, 2024-04-24T15:33:42.604 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:42.604 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:42.604 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:42.604 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:42.604 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:42.604 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18007 2024-04-24T15:33:42.604 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.604 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:42.604 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.604 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:42.604 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207961+0000", 2024-04-24T15:33:42.604 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:42.604 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.604 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.605 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:42.605 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207961+0000", 2024-04-24T15:33:42.605 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:42.605 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.605 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.605 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:42.605 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207960+0000", 2024-04-24T15:33:42.605 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:42.605 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.605 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.605 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:42.605 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207963+0000", 2024-04-24T15:33:42.605 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.3759999999999999e-06 2024-04-24T15:33:42.605 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.605 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.606 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:42.606 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207970+0000", 2024-04-24T15:33:42.606 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.0489999999999998e-06 2024-04-24T15:33:42.606 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.606 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.606 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:42.606 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207975+0000", 2024-04-24T15:33:42.606 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.1640000000000003e-06 2024-04-24T15:33:42.606 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.606 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.606 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:42.606 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250834+0000", 2024-04-24T15:33:42.606 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042859060999999997 2024-04-24T15:33:42.606 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.607 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.607 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:42.607 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250874+0000", 2024-04-24T15:33:42.607 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0674999999999999e-05 2024-04-24T15:33:42.607 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.607 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.607 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:42.607 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250926+0000", 2024-04-24T15:33:42.607 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.1693000000000001e-05 2024-04-24T15:33:42.607 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:42.607 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:42.607 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:42.607 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.607 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.607 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18008 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12500992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:42.607 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208030+0000", 2024-04-24T15:33:42.607 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 68.390665740000003, 2024-04-24T15:33:42.608 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043003337000000003, 2024-04-24T15:33:42.608 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:42.608 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:42.608 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:42.608 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:42.608 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:42.608 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18008 2024-04-24T15:33:42.608 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.608 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:42.608 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.608 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:42.608 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208030+0000", 2024-04-24T15:33:42.608 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:42.608 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.608 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.609 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:42.609 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208030+0000", 2024-04-24T15:33:42.609 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:42.609 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.609 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.609 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:42.609 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208029+0000", 2024-04-24T15:33:42.609 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:42.609 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.609 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.609 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:42.609 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208032+0000", 2024-04-24T15:33:42.609 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.117e-06 2024-04-24T15:33:42.609 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.609 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.609 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:42.610 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208039+0000", 2024-04-24T15:33:42.610 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.0129999999999996e-06 2024-04-24T15:33:42.610 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.610 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.610 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:42.610 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208043+0000", 2024-04-24T15:33:42.610 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0600000000000001e-06 2024-04-24T15:33:42.610 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.610 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.610 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:42.610 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250948+0000", 2024-04-24T15:33:42.610 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042904577999999999 2024-04-24T15:33:42.610 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.610 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.611 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:42.611 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250989+0000", 2024-04-24T15:33:42.611 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0978e-05 2024-04-24T15:33:42.611 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.611 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.611 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:42.611 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251033+0000", 2024-04-24T15:33:42.611 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4161e-05 2024-04-24T15:33:42.611 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:42.611 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:42.611 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:42.611 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.611 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.611 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18009 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12541952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:42.611 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208093+0000", 2024-04-24T15:33:42.612 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 68.390601993999994, 2024-04-24T15:33:42.612 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043043451000000003, 2024-04-24T15:33:42.612 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:42.612 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:42.612 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:42.612 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:42.612 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:42.612 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18009 2024-04-24T15:33:42.612 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.612 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:42.612 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.612 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:42.612 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208093+0000", 2024-04-24T15:33:42.612 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:42.612 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.613 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.613 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:42.613 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208093+0000", 2024-04-24T15:33:42.613 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:42.613 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.613 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.613 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:42.613 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208093+0000", 2024-04-24T15:33:42.613 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:42.613 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.613 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.613 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:42.613 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208096+0000", 2024-04-24T15:33:42.613 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.439e-06 2024-04-24T15:33:42.613 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.613 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.614 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:42.614 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208103+0000", 2024-04-24T15:33:42.614 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1019999999999999e-06 2024-04-24T15:33:42.614 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.614 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.614 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:42.614 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208107+0000", 2024-04-24T15:33:42.614 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0910000000000003e-06 2024-04-24T15:33:42.614 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.614 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.614 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:42.614 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251054+0000", 2024-04-24T15:33:42.614 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042946826 2024-04-24T15:33:42.614 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.615 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.615 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:42.615 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251092+0000", 2024-04-24T15:33:42.615 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.7941999999999997e-05 2024-04-24T15:33:42.615 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.615 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.615 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:42.615 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251137+0000", 2024-04-24T15:33:42.615 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4638000000000002e-05 2024-04-24T15:33:42.615 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:42.615 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:42.615 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:42.615 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.615 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.615 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18010 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12615680~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:42.616 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208202+0000", 2024-04-24T15:33:42.616 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 68.390493163000002, 2024-04-24T15:33:42.616 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043046549000000003, 2024-04-24T15:33:42.616 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:42.616 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:42.616 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:42.616 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:42.616 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:42.616 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18010 2024-04-24T15:33:42.616 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.616 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:42.616 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.616 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:42.616 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208202+0000", 2024-04-24T15:33:42.616 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:42.616 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.617 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.617 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:42.617 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208202+0000", 2024-04-24T15:33:42.617 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:42.617 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.617 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.617 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:42.617 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208202+0000", 2024-04-24T15:33:42.617 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:42.617 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.617 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.617 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:42.617 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208205+0000", 2024-04-24T15:33:42.617 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.2490000000000001e-06 2024-04-24T15:33:42.617 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.618 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.618 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:42.618 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208215+0000", 2024-04-24T15:33:42.618 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.9070000000000002e-06 2024-04-24T15:33:42.618 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.618 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.618 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:42.618 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208220+0000", 2024-04-24T15:33:42.618 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8849999999999998e-06 2024-04-24T15:33:42.618 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.618 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.618 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:42.618 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251158+0000", 2024-04-24T15:33:42.618 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042938416 2024-04-24T15:33:42.619 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.619 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.619 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:42.619 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251197+0000", 2024-04-24T15:33:42.619 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9341000000000002e-05 2024-04-24T15:33:42.619 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.619 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.619 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:42.619 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251249+0000", 2024-04-24T15:33:42.619 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.1391000000000003e-05 2024-04-24T15:33:42.619 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:42.619 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:42.619 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:42.619 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.619 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.620 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18011 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12656640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:42.620 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208258+0000", 2024-04-24T15:33:42.620 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 68.390437317000007, 2024-04-24T15:33:42.620 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043102708000000003, 2024-04-24T15:33:42.620 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:42.620 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:42.620 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:42.620 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:42.620 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:42.620 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18011 2024-04-24T15:33:42.620 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.620 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:42.620 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.620 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:42.620 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208258+0000", 2024-04-24T15:33:42.620 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:42.621 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.621 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.621 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:42.621 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208258+0000", 2024-04-24T15:33:42.621 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:42.621 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.621 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.621 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:42.621 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208258+0000", 2024-04-24T15:33:42.621 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:42.621 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.621 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.621 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:42.621 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208260+0000", 2024-04-24T15:33:42.621 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7810000000000001e-06 2024-04-24T15:33:42.622 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.622 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.622 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:42.622 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208267+0000", 2024-04-24T15:33:42.622 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.7610000000000002e-06 2024-04-24T15:33:42.622 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.622 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.622 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:42.622 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208271+0000", 2024-04-24T15:33:42.622 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0890000000000002e-06 2024-04-24T15:33:42.622 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.622 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.622 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:42.622 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251271+0000", 2024-04-24T15:33:42.623 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043000159000000003 2024-04-24T15:33:42.623 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.623 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.623 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:42.623 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251315+0000", 2024-04-24T15:33:42.623 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3446999999999999e-05 2024-04-24T15:33:42.623 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.623 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.623 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:42.623 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251361+0000", 2024-04-24T15:33:42.623 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5952000000000003e-05 2024-04-24T15:33:42.623 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:42.623 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:42.623 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:42.623 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.624 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.624 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18012 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12697600~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:42.624 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208289+0000", 2024-04-24T15:33:42.624 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 68.390406272999996, 2024-04-24T15:33:42.624 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043174774999999999, 2024-04-24T15:33:42.624 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:42.624 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:42.624 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:42.624 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:42.624 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:42.624 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18012 2024-04-24T15:33:42.624 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.624 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:42.624 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.624 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:42.624 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208289+0000", 2024-04-24T15:33:42.625 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:42.625 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.625 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.625 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:42.625 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208289+0000", 2024-04-24T15:33:42.625 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:42.625 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.625 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.625 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:42.625 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208289+0000", 2024-04-24T15:33:42.625 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:42.625 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.625 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.625 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:42.625 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208291+0000", 2024-04-24T15:33:42.626 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.542e-06 2024-04-24T15:33:42.626 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.626 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.626 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:42.626 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208298+0000", 2024-04-24T15:33:42.626 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.3899999999999998e-06 2024-04-24T15:33:42.626 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.626 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.626 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:42.626 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208302+0000", 2024-04-24T15:33:42.626 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.019e-06 2024-04-24T15:33:42.626 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.626 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.626 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:42.627 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251382+0000", 2024-04-24T15:33:42.627 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043080790000000001 2024-04-24T15:33:42.627 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.627 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.627 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:42.627 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251421+0000", 2024-04-24T15:33:42.627 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8473000000000002e-05 2024-04-24T15:33:42.627 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.627 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.627 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:42.627 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251464+0000", 2024-04-24T15:33:42.627 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3133999999999997e-05 2024-04-24T15:33:42.627 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:42.627 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:42.627 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:42.628 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.628 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.628 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18013 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12738560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:42.628 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208423+0000", 2024-04-24T15:33:42.628 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 68.390272164999999, 2024-04-24T15:33:42.628 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043145554000000003, 2024-04-24T15:33:42.628 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:42.628 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:42.628 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:42.628 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:42.628 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:42.628 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18013 2024-04-24T15:33:42.628 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.628 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:42.628 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.628 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:42.629 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208423+0000", 2024-04-24T15:33:42.629 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:42.629 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.629 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.629 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:42.629 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208423+0000", 2024-04-24T15:33:42.629 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:42.629 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.629 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.629 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:42.629 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208423+0000", 2024-04-24T15:33:42.629 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:42.629 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.629 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.629 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:42.630 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208426+0000", 2024-04-24T15:33:42.630 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.0539999999999999e-06 2024-04-24T15:33:42.630 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.630 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.630 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:42.630 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208433+0000", 2024-04-24T15:33:42.630 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.43e-06 2024-04-24T15:33:42.630 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.630 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.630 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:42.630 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208437+0000", 2024-04-24T15:33:42.630 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5590000000000001e-06 2024-04-24T15:33:42.630 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.630 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.631 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:42.631 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251485+0000", 2024-04-24T15:33:42.631 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043048259999999998 2024-04-24T15:33:42.631 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.631 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.631 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:42.631 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251525+0000", 2024-04-24T15:33:42.631 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9916999999999997e-05 2024-04-24T15:33:42.631 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.631 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.631 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:42.631 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251569+0000", 2024-04-24T15:33:42.631 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3939000000000001e-05 2024-04-24T15:33:42.631 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:42.631 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:42.631 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:42.632 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.632 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.632 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18015 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12779520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:42.632 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208630+0000", 2024-04-24T15:33:42.632 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 68.390065640000003, 2024-04-24T15:33:42.632 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043047328000000003, 2024-04-24T15:33:42.632 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:42.632 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:42.632 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:42.632 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:42.632 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:42.632 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18015 2024-04-24T15:33:42.632 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.632 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:42.632 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.633 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:42.633 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208630+0000", 2024-04-24T15:33:42.633 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:42.633 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.633 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.633 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:42.633 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208630+0000", 2024-04-24T15:33:42.633 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:42.633 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.633 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.633 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:42.633 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208629+0000", 2024-04-24T15:33:42.633 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999999 2024-04-24T15:33:42.633 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.633 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.633 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:42.633 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208634+0000", 2024-04-24T15:33:42.634 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8960000000000002e-06 2024-04-24T15:33:42.634 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.634 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.634 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:42.634 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208645+0000", 2024-04-24T15:33:42.634 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0939e-05 2024-04-24T15:33:42.634 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.634 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.634 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:42.634 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208651+0000", 2024-04-24T15:33:42.634 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.896e-06 2024-04-24T15:33:42.634 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.634 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.634 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:42.634 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251591+0000", 2024-04-24T15:33:42.634 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042939862000000002 2024-04-24T15:33:42.635 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.635 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.635 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:42.635 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251631+0000", 2024-04-24T15:33:42.635 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0216000000000003e-05 2024-04-24T15:33:42.635 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.635 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.635 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:42.635 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251677+0000", 2024-04-24T15:33:42.635 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6390999999999997e-05 2024-04-24T15:33:42.635 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:42.635 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:42.635 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:42.635 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.635 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.635 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18016 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12820480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:42.636 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208660+0000", 2024-04-24T15:33:42.636 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 68.390035850999993, 2024-04-24T15:33:42.636 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043127510000000001, 2024-04-24T15:33:42.636 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:42.636 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:42.636 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:42.636 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:42.636 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:42.636 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18016 2024-04-24T15:33:42.636 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.636 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:42.636 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.636 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:42.636 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208660+0000", 2024-04-24T15:33:42.636 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:42.636 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.636 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.637 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:42.637 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208660+0000", 2024-04-24T15:33:42.637 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:42.637 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.637 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.637 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:42.637 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208659+0000", 2024-04-24T15:33:42.637 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:42.637 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.637 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.637 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:42.637 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208662+0000", 2024-04-24T15:33:42.637 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.0979999999999998e-06 2024-04-24T15:33:42.637 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.637 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.638 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:42.638 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208669+0000", 2024-04-24T15:33:42.638 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.4470000000000003e-06 2024-04-24T15:33:42.638 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.638 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.638 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:42.638 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208673+0000", 2024-04-24T15:33:42.638 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2810000000000002e-06 2024-04-24T15:33:42.638 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.638 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.638 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:42.638 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251699+0000", 2024-04-24T15:33:42.638 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043025728999999999 2024-04-24T15:33:42.638 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.638 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.639 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:42.639 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251743+0000", 2024-04-24T15:33:42.639 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4469999999999999e-05 2024-04-24T15:33:42.639 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.639 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.639 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:42.639 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251787+0000", 2024-04-24T15:33:42.639 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3949999999999998e-05 2024-04-24T15:33:42.639 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:42.639 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:42.639 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:42.639 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.639 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.639 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18017 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12861440~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:42.639 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208721+0000", 2024-04-24T15:33:42.639 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 68.389974402999997, 2024-04-24T15:33:42.640 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043167749999999998, 2024-04-24T15:33:42.640 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:42.640 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:42.640 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:42.640 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:42.640 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:42.640 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18017 2024-04-24T15:33:42.640 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.640 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:42.640 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.640 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:42.640 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208721+0000", 2024-04-24T15:33:42.640 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:42.640 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.640 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.640 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:42.641 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208721+0000", 2024-04-24T15:33:42.641 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:42.641 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.641 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.641 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:42.641 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208720+0000", 2024-04-24T15:33:42.641 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:42.641 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.641 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.641 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:42.641 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208724+0000", 2024-04-24T15:33:42.641 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.3919999999999999e-06 2024-04-24T15:33:42.641 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.641 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.641 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:42.641 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208729+0000", 2024-04-24T15:33:42.642 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.3480000000000003e-06 2024-04-24T15:33:42.642 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.642 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.642 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:42.642 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208732+0000", 2024-04-24T15:33:42.642 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.9529999999999999e-06 2024-04-24T15:33:42.642 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.642 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.642 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:42.642 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251807+0000", 2024-04-24T15:33:42.642 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043074492999999998 2024-04-24T15:33:42.642 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.642 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.642 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:42.642 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251846+0000", 2024-04-24T15:33:42.643 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8958999999999997e-05 2024-04-24T15:33:42.643 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.643 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.643 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:42.643 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251889+0000", 2024-04-24T15:33:42.643 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3253000000000001e-05 2024-04-24T15:33:42.643 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:42.643 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:42.643 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:42.643 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.643 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.643 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18018 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12902400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:42.643 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208752+0000", 2024-04-24T15:33:42.643 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 68.389943790000004, 2024-04-24T15:33:42.643 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043241012000000002, 2024-04-24T15:33:42.643 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:42.644 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:42.644 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:42.644 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:42.644 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:42.644 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18018 2024-04-24T15:33:42.644 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.644 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:42.644 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.644 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:42.644 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208752+0000", 2024-04-24T15:33:42.644 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:42.644 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.644 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.644 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:42.644 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208752+0000", 2024-04-24T15:33:42.644 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:42.644 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.645 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.645 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:42.645 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208751+0000", 2024-04-24T15:33:42.645 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:42.645 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.645 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.645 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:42.645 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208753+0000", 2024-04-24T15:33:42.645 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.894e-06 2024-04-24T15:33:42.645 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.645 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.645 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:42.645 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208757+0000", 2024-04-24T15:33:42.645 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.856e-06 2024-04-24T15:33:42.645 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.645 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.645 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:42.646 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208760+0000", 2024-04-24T15:33:42.646 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.6749999999999998e-06 2024-04-24T15:33:42.646 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.646 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.646 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:42.646 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251909+0000", 2024-04-24T15:33:42.646 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043149145999999999 2024-04-24T15:33:42.646 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.646 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.646 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:42.646 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251948+0000", 2024-04-24T15:33:42.646 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9159000000000002e-05 2024-04-24T15:33:42.646 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.646 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.646 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:42.647 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251993+0000", 2024-04-24T15:33:42.647 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4638999999999997e-05 2024-04-24T15:33:42.647 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:42.647 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:42.647 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:42.647 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.647 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.647 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18019 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12943360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:42.647 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208938+0000", 2024-04-24T15:33:42.647 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 68.389757498999998, 2024-04-24T15:33:42.647 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043162272000000002, 2024-04-24T15:33:42.647 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:42.647 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:42.647 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:42.647 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:42.647 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:42.647 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18019 2024-04-24T15:33:42.648 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.648 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:42.648 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.648 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:42.648 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208938+0000", 2024-04-24T15:33:42.648 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:42.648 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.648 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.648 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:42.648 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208938+0000", 2024-04-24T15:33:42.648 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:42.648 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.648 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.648 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:42.648 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208762+0000", 2024-04-24T15:33:42.648 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999824 2024-04-24T15:33:42.649 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.649 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.649 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:42.649 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208942+0000", 2024-04-24T15:33:42.649 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000180008 2024-04-24T15:33:42.649 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.649 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.649 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:42.649 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208950+0000", 2024-04-24T15:33:42.649 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.1550000000000007e-06 2024-04-24T15:33:42.649 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.649 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.649 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:42.649 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208956+0000", 2024-04-24T15:33:42.649 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.7910000000000001e-06 2024-04-24T15:33:42.649 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.650 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.650 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:42.650 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252014+0000", 2024-04-24T15:33:42.650 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043058427000000003 2024-04-24T15:33:42.650 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.650 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.650 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:42.650 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252054+0000", 2024-04-24T15:33:42.650 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0071999999999999e-05 2024-04-24T15:33:42.650 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.650 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.650 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:42.650 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252100+0000", 2024-04-24T15:33:42.651 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5917000000000001e-05 2024-04-24T15:33:42.651 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:42.651 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:42.651 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:42.651 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.651 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.651 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18020 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12984320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:42.651 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.209092+0000", 2024-04-24T15:33:42.651 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 68.389603500999996, 2024-04-24T15:33:42.651 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043111295000000001, 2024-04-24T15:33:42.651 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:42.651 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:42.651 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:42.651 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:42.651 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:42.652 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18020 2024-04-24T15:33:42.652 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.652 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:42.652 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.652 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:42.652 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209092+0000", 2024-04-24T15:33:42.652 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:42.652 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.652 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.652 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:42.652 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209092+0000", 2024-04-24T15:33:42.652 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:42.652 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.652 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.652 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:42.653 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208964+0000", 2024-04-24T15:33:42.653 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998722 2024-04-24T15:33:42.653 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.653 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.653 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:42.653 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209095+0000", 2024-04-24T15:33:42.653 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000130648 2024-04-24T15:33:42.653 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.653 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.653 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:42.653 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209103+0000", 2024-04-24T15:33:42.653 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0320000000000003e-06 2024-04-24T15:33:42.653 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.653 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.653 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:42.654 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209108+0000", 2024-04-24T15:33:42.654 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8690000000000003e-06 2024-04-24T15:33:42.654 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.654 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.654 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:42.654 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252121+0000", 2024-04-24T15:33:42.654 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043013323999999999 2024-04-24T15:33:42.654 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.654 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.654 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:42.654 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252160+0000", 2024-04-24T15:33:42.654 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8376e-05 2024-04-24T15:33:42.654 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.654 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.655 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:42.655 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252203+0000", 2024-04-24T15:33:42.655 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.371e-05 2024-04-24T15:33:42.655 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:42.655 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:42.655 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:42.655 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.655 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.655 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18021 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 13025280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:42.655 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.209132+0000", 2024-04-24T15:33:42.655 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 68.389563143000004, 2024-04-24T15:33:42.655 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043183903000000003, 2024-04-24T15:33:42.655 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:42.655 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:42.655 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:42.656 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:42.656 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:42.656 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18021 2024-04-24T15:33:42.656 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.656 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:42.656 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.656 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:42.656 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209132+0000", 2024-04-24T15:33:42.656 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:42.656 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.656 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.656 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:42.656 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209132+0000", 2024-04-24T15:33:42.656 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:42.656 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.657 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.657 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:42.657 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209132+0000", 2024-04-24T15:33:42.657 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:42.657 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.657 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.657 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:42.657 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209134+0000", 2024-04-24T15:33:42.657 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.891e-06 2024-04-24T15:33:42.657 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.658 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.658 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:42.658 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209138+0000", 2024-04-24T15:33:42.658 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0659999999999997e-06 2024-04-24T15:33:42.658 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.658 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.658 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:42.658 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209140+0000", 2024-04-24T15:33:42.658 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.5339999999999998e-06 2024-04-24T15:33:42.658 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.658 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.658 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:42.658 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252232+0000", 2024-04-24T15:33:42.658 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043091142999999998 2024-04-24T15:33:42.659 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.659 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.659 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:42.659 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252270+0000", 2024-04-24T15:33:42.659 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8872000000000003e-05 2024-04-24T15:33:42.659 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.659 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.659 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:42.659 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252316+0000", 2024-04-24T15:33:42.659 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5788000000000003e-05 2024-04-24T15:33:42.659 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:42.659 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:42.659 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:42.659 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.659 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.659 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18022 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 13066240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:42.660 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.209411+0000", 2024-04-24T15:33:42.660 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 68.389284595000007, 2024-04-24T15:33:42.660 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043016616000000001, 2024-04-24T15:33:42.660 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:42.660 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:42.660 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:42.660 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:42.660 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:42.660 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18022 2024-04-24T15:33:42.660 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.660 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:42.660 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.660 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:42.660 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209411+0000", 2024-04-24T15:33:42.660 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:42.661 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.661 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.661 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:42.661 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209411+0000", 2024-04-24T15:33:42.661 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:42.661 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.661 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.661 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:42.661 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209143+0000", 2024-04-24T15:33:42.661 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999732 2024-04-24T15:33:42.661 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.661 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.661 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:42.661 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209415+0000", 2024-04-24T15:33:42.662 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000271873 2024-04-24T15:33:42.662 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.662 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.662 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:42.662 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209423+0000", 2024-04-24T15:33:42.662 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3080000000000008e-06 2024-04-24T15:33:42.662 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.662 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.662 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:42.662 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209428+0000", 2024-04-24T15:33:42.662 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.5040000000000002e-06 2024-04-24T15:33:42.662 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.662 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.662 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:42.663 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252339+0000", 2024-04-24T15:33:42.663 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042910419999999998 2024-04-24T15:33:42.663 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.663 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.663 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:42.663 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252380+0000", 2024-04-24T15:33:42.663 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0991000000000001e-05 2024-04-24T15:33:42.663 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.663 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.663 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:42.663 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252427+0000", 2024-04-24T15:33:42.663 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.7647e-05 2024-04-24T15:33:42.663 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:42.663 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:42.663 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:42.664 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.664 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.664 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18023 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 13107200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:42.664 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.209594+0000", 2024-04-24T15:33:42.664 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 68.389101081999996, 2024-04-24T15:33:42.664 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042938543000000003, 2024-04-24T15:33:42.664 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:42.664 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:42.664 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:42.664 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:42.664 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:42.664 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18023 2024-04-24T15:33:42.664 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.664 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:42.664 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.665 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:42.665 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209594+0000", 2024-04-24T15:33:42.665 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:42.665 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.665 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.665 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:42.665 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209594+0000", 2024-04-24T15:33:42.665 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:42.665 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.665 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.665 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:42.665 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209440+0000", 2024-04-24T15:33:42.665 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999846 2024-04-24T15:33:42.665 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.665 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.666 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:42.666 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209598+0000", 2024-04-24T15:33:42.666 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00015731100000000001 2024-04-24T15:33:42.666 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.666 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.666 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:42.666 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209604+0000", 2024-04-24T15:33:42.666 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.2910000000000004e-06 2024-04-24T15:33:42.666 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.666 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.666 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:42.666 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209607+0000", 2024-04-24T15:33:42.666 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5389999999999999e-06 2024-04-24T15:33:42.666 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.667 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.667 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:42.667 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252449+0000", 2024-04-24T15:33:42.667 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042841833000000003 2024-04-24T15:33:42.667 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.667 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.667 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:42.667 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252489+0000", 2024-04-24T15:33:42.667 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9898000000000003e-05 2024-04-24T15:33:42.667 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.667 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.667 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:42.667 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252533+0000", 2024-04-24T15:33:42.667 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3702999999999998e-05 2024-04-24T15:33:42.667 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:42.667 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:42.667 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:42.668 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.668 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.668 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18025 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 13189120~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:42.668 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.209832+0000", 2024-04-24T15:33:42.668 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 68.388863580999995, 2024-04-24T15:33:42.668 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042913556999999998, 2024-04-24T15:33:42.668 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:42.668 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:42.668 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:42.668 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:42.668 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:42.668 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18025 2024-04-24T15:33:42.668 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.668 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:42.668 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.669 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:42.669 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209832+0000", 2024-04-24T15:33:42.669 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:42.669 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.669 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.669 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:42.669 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209832+0000", 2024-04-24T15:33:42.669 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:42.669 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.669 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.669 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:42.669 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209774+0000", 2024-04-24T15:33:42.669 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999418 2024-04-24T15:33:42.669 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.669 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.669 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:42.670 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209835+0000", 2024-04-24T15:33:42.670 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.1377000000000005e-05 2024-04-24T15:33:42.670 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.670 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.670 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:42.670 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209843+0000", 2024-04-24T15:33:42.670 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.2330000000000006e-06 2024-04-24T15:33:42.670 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.670 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.670 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:42.670 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209848+0000", 2024-04-24T15:33:42.670 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6149999999999999e-06 2024-04-24T15:33:42.670 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.670 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.670 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:42.671 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252658+0000", 2024-04-24T15:33:42.671 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042810002 2024-04-24T15:33:42.671 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.671 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.671 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:42.671 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252697+0000", 2024-04-24T15:33:42.671 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8757999999999997e-05 2024-04-24T15:33:42.671 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.671 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.671 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:42.671 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252745+0000", 2024-04-24T15:33:42.671 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.863e-05 2024-04-24T15:33:42.671 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:42.671 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:42.671 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:42.671 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.672 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.672 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:24749 4.8 4:137f5701:::rbd_header.114ae15b9a20:head [watch reconnect cookie 140672595753696 gen 11] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-04-24T15:33:42.672 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:37.628703+0000", 2024-04-24T15:33:42.672 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 64.969991961999995, 2024-04-24T15:33:42.672 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0102918700000001, 2024-04-24T15:33:42.672 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:42.672 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:42.672 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:42.672 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:42.672 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:42.672 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 24749 2024-04-24T15:33:42.672 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.672 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:42.672 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.672 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:42.672 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628703+0000", 2024-04-24T15:33:42.673 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:42.673 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.673 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.673 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:42.673 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628703+0000", 2024-04-24T15:33:42.673 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:42.673 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.673 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.673 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:42.673 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628703+0000", 2024-04-24T15:33:42.673 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999999 2024-04-24T15:33:42.673 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.673 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.673 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:42.673 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628708+0000", 2024-04-24T15:33:42.674 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.4589999999999997e-06 2024-04-24T15:33:42.674 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.674 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.674 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:42.674 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628721+0000", 2024-04-24T15:33:42.674 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.2601e-05 2024-04-24T15:33:42.674 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.674 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.674 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:42.674 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628728+0000", 2024-04-24T15:33:42.674 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.9059999999999996e-06 2024-04-24T15:33:42.674 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.674 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.674 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:42.674 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628789+0000", 2024-04-24T15:33:42.675 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.1672000000000002e-05 2024-04-24T15:33:42.675 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.675 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.675 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting_for_map not empty", 2024-04-24T15:33:42.675 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628797+0000", 2024-04-24T15:33:42.675 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.8189999999999992e-06 2024-04-24T15:33:42.675 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.675 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.675 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:42.675 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.631255+0000", 2024-04-24T15:33:42.675 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0024577610000000001 2024-04-24T15:33:42.675 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.675 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.675 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for peered", 2024-04-24T15:33:42.675 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.631263+0000", 2024-04-24T15:33:42.675 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.4109999999999996e-06 2024-04-24T15:33:42.676 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.676 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.676 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:42.676 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:38.637560+0000", 2024-04-24T15:33:42.676 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0062970339999999 2024-04-24T15:33:42.676 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.676 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.676 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:42.676 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:38.637747+0000", 2024-04-24T15:33:42.676 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00018693400000000001 2024-04-24T15:33:42.676 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.676 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.676 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:42.676 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:38.638995+0000", 2024-04-24T15:33:42.676 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0012481300000000001 2024-04-24T15:33:42.676 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:42.677 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:42.677 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:42.677 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.677 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.677 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:24750 4.8 4:137f5701:::rbd_header.114ae15b9a20:head [notify cookie 140672594434416] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-04-24T15:33:42.677 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:37.628736+0000", 2024-04-24T15:33:42.677 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 64.969959701999997, 2024-04-24T15:33:42.677 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0096094309999999, 2024-04-24T15:33:42.677 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:42.677 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:42.677 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:42.677 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:42.677 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:42.677 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 24750 2024-04-24T15:33:42.677 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.678 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:42.678 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.678 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:42.678 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628736+0000", 2024-04-24T15:33:42.678 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:42.678 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.678 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.678 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:42.678 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628736+0000", 2024-04-24T15:33:42.678 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:42.678 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.678 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.678 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:42.678 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628735+0000", 2024-04-24T15:33:42.678 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:42.679 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.679 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.679 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:42.679 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628739+0000", 2024-04-24T15:33:42.679 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5109999999999999e-06 2024-04-24T15:33:42.679 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.679 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.679 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:42.679 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628745+0000", 2024-04-24T15:33:42.679 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.8300000000000001e-06 2024-04-24T15:33:42.679 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.679 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.679 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:42.679 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628749+0000", 2024-04-24T15:33:42.679 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.7110000000000003e-06 2024-04-24T15:33:42.680 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.680 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.680 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:42.680 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628826+0000", 2024-04-24T15:33:42.680 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.6678e-05 2024-04-24T15:33:42.680 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.680 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.680 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting_for_map not empty", 2024-04-24T15:33:42.680 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628834+0000", 2024-04-24T15:33:42.680 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.6190000000000002e-06 2024-04-24T15:33:42.680 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.680 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.680 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:42.680 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.631305+0000", 2024-04-24T15:33:42.680 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0024714400000000001 2024-04-24T15:33:42.680 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.681 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.681 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for peered", 2024-04-24T15:33:42.681 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.631313+0000", 2024-04-24T15:33:42.681 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.6529999999999994e-06 2024-04-24T15:33:42.681 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.681 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.681 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:42.681 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:38.638068+0000", 2024-04-24T15:33:42.681 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0067548369999999 2024-04-24T15:33:42.681 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.681 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.681 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:42.681 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:38.638158+0000", 2024-04-24T15:33:42.681 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.0079000000000003e-05 2024-04-24T15:33:42.681 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:42.681 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:42.682 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:42.682 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:38.638345+0000", 2024-04-24T15:33:42.682 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00018757100000000001 2024-04-24T15:33:42.682 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:42.682 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:42.682 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:42.682 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:42.682 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:42.682 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:42.683 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-24T15:33:42.684 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:42.683+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:42.784 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:33:42.784+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:33:42.861 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:42.861 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:33:42.861 INFO:teuthology.orchestra.run.smithi151.stdout: "num_ops": 0 2024-04-24T15:33:42.861 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:42.870 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-24T15:33:42.885 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:33:42.885+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:33:42.986 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:42.987+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:43.036 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:43.036 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:33:43.036 INFO:teuthology.orchestra.run.smithi151.stdout: "complaint_time": 30, 2024-04-24T15:33:43.036 INFO:teuthology.orchestra.run.smithi151.stdout: "num_blocked_ops": 0 2024-04-24T15:33:43.036 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:43.046 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-24T15:33:43.087 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:33:43.087+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:33:43.188 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:43.188+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:43.210 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:43.210 INFO:teuthology.orchestra.run.smithi151.stdout: "size": 20, 2024-04-24T15:33:43.210 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 600, 2024-04-24T15:33:43.211 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [ 2024-04-24T15:33:43.211 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.211 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:75519 4.a 4.2a5fcea (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e104)", 2024-04-24T15:33:43.211 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:42.066841+0000", 2024-04-24T15:33:43.211 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 1.14362326, 2024-04-24T15:33:43.211 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.57234695599999996, 2024-04-24T15:33:43.211 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:43.211 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "reached pg", 2024-04-24T15:33:43.211 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:43.211 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:43.211 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:43.213 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 75519 2024-04-24T15:33:43.213 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.213 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:43.213 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.213 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:43.213 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.066841+0000", 2024-04-24T15:33:43.213 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:43.213 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.213 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.213 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:43.214 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.066841+0000", 2024-04-24T15:33:43.214 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:43.214 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.214 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.214 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:43.214 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.066840+0000", 2024-04-24T15:33:43.214 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999999 2024-04-24T15:33:43.214 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.214 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.214 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:43.214 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.066846+0000", 2024-04-24T15:33:43.214 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6289999999999998e-06 2024-04-24T15:33:43.214 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.214 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.215 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:43.215 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.066854+0000", 2024-04-24T15:33:43.215 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.9219999999999997e-06 2024-04-24T15:33:43.215 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.215 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.215 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:43.215 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.066861+0000", 2024-04-24T15:33:43.215 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.6760000000000001e-06 2024-04-24T15:33:43.215 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.215 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.215 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:43.215 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.066915+0000", 2024-04-24T15:33:43.215 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.4083999999999999e-05 2024-04-24T15:33:43.215 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.215 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.216 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for peered", 2024-04-24T15:33:43.216 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.066924+0000", 2024-04-24T15:33:43.216 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.0350000000000007e-06 2024-04-24T15:33:43.216 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.216 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.216 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:43.216 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.639176+0000", 2024-04-24T15:33:43.216 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.57225240700000002 2024-04-24T15:33:43.216 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.216 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.216 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:43.216 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.639188+0000", 2024-04-24T15:33:43.216 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.2377e-05 2024-04-24T15:33:43.216 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:43.216 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:43.217 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:43.217 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.217 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.217 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:75520 4.5 4.787c8fa5 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e104)", 2024-04-24T15:33:43.217 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:42.066869+0000", 2024-04-24T15:33:43.217 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 1.1435953599999999, 2024-04-24T15:33:43.217 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.57110218499999998, 2024-04-24T15:33:43.217 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:43.217 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "reached pg", 2024-04-24T15:33:43.217 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:43.217 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:43.217 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:43.217 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 75520 2024-04-24T15:33:43.217 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.217 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:43.218 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.218 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:43.218 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.066869+0000", 2024-04-24T15:33:43.218 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:43.218 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.218 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.218 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:43.218 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.066869+0000", 2024-04-24T15:33:43.218 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:43.218 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.218 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.218 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:43.218 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.066869+0000", 2024-04-24T15:33:43.218 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:43.219 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.219 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.219 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:43.219 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.066872+0000", 2024-04-24T15:33:43.219 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.9840000000000001e-06 2024-04-24T15:33:43.219 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.219 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.219 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:43.219 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.066883+0000", 2024-04-24T15:33:43.219 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0883000000000001e-05 2024-04-24T15:33:43.219 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.219 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.219 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:43.219 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.066887+0000", 2024-04-24T15:33:43.220 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3309999999999997e-06 2024-04-24T15:33:43.220 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.220 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.220 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:43.220 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.066958+0000", 2024-04-24T15:33:43.220 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1290000000000004e-05 2024-04-24T15:33:43.220 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.220 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.220 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for peered", 2024-04-24T15:33:43.220 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.066969+0000", 2024-04-24T15:33:43.220 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0927000000000001e-05 2024-04-24T15:33:43.220 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.220 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.220 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:43.221 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.637952+0000", 2024-04-24T15:33:43.221 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.57098265400000003 2024-04-24T15:33:43.221 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.221 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.221 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:43.221 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.637972+0000", 2024-04-24T15:33:43.221 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.9519000000000001e-05 2024-04-24T15:33:43.221 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:43.221 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:43.221 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:43.221 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.221 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.221 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_repop(client.4426.0:75517 4.18 e105/104 4:1d3ff9a6:::rbd_data.114a858e6641.0000000000000003:head v 105'404, mlcod=0'0)", 2024-04-24T15:33:43.221 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:42.643751+0000", 2024-04-24T15:33:43.221 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 0.56671338699999996, 2024-04-24T15:33:43.221 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0060238999999999996, 2024-04-24T15:33:43.222 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:43.222 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:43.222 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:43.222 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.222 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:43.222 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.643751+0000", 2024-04-24T15:33:43.222 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:43.222 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.222 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.222 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:43.222 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.643751+0000", 2024-04-24T15:33:43.222 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:43.222 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.222 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.223 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:43.223 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.643751+0000", 2024-04-24T15:33:43.223 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:43.223 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.223 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.223 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:43.223 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.643762+0000", 2024-04-24T15:33:43.223 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0845e-05 2024-04-24T15:33:43.223 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.223 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.223 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:43.223 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.643770+0000", 2024-04-24T15:33:43.223 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7149999999999998e-06 2024-04-24T15:33:43.223 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.223 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.224 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:43.224 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.643777+0000", 2024-04-24T15:33:43.224 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.029e-06 2024-04-24T15:33:43.224 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.224 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.224 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:43.224 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.645926+0000", 2024-04-24T15:33:43.224 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0021492600000000001 2024-04-24T15:33:43.224 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.224 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.224 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:43.224 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.645948+0000", 2024-04-24T15:33:43.224 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.1837000000000001e-05 2024-04-24T15:33:43.224 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.224 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.224 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:33:43.225 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.649745+0000", 2024-04-24T15:33:43.225 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0037970719999999999 2024-04-24T15:33:43.225 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.225 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.225 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:43.225 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.649775+0000", 2024-04-24T15:33:43.225 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.0480999999999998e-05 2024-04-24T15:33:43.225 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:43.225 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:43.225 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:43.225 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.225 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.225 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_repop(client.4426.0:75522 4.1 e105/104 4:83ace853:::rbd_data.114a858e6641.0000000000000008:head v 105'287, mlcod=0'0)", 2024-04-24T15:33:43.226 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:42.645889+0000", 2024-04-24T15:33:43.226 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 0.56457552899999996, 2024-04-24T15:33:43.226 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0025720669999999999, 2024-04-24T15:33:43.226 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:43.226 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:43.226 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:43.226 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.226 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:43.226 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.645889+0000", 2024-04-24T15:33:43.226 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:43.226 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.226 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.226 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:43.226 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.645889+0000", 2024-04-24T15:33:43.227 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:43.227 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.227 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.227 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:43.227 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.645889+0000", 2024-04-24T15:33:43.227 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:43.227 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.227 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.227 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:43.227 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.645894+0000", 2024-04-24T15:33:43.227 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.587e-06 2024-04-24T15:33:43.227 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.227 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.227 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:43.227 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.645903+0000", 2024-04-24T15:33:43.228 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.9090000000000006e-06 2024-04-24T15:33:43.228 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.228 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.228 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:43.228 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.645910+0000", 2024-04-24T15:33:43.228 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.5490000000000003e-06 2024-04-24T15:33:43.228 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.228 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.228 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:43.228 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.646318+0000", 2024-04-24T15:33:43.228 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00040830700000000001 2024-04-24T15:33:43.228 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.228 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.228 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:43.228 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.646337+0000", 2024-04-24T15:33:43.229 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.9108999999999999e-05 2024-04-24T15:33:43.229 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.229 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.229 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:33:43.229 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.648423+0000", 2024-04-24T15:33:43.229 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0020859009999999998 2024-04-24T15:33:43.229 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.229 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.229 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:43.229 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.648461+0000", 2024-04-24T15:33:43.229 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8271e-05 2024-04-24T15:33:43.229 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:43.229 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:43.229 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:43.229 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.230 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.230 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_repop(client.4426.0:75514 4.13 e105/104 4:cfbf367e:::rbd_data.114a858e6641.0000000000000000:head v 105'224, mlcod=0'0)", 2024-04-24T15:33:43.230 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:42.646298+0000", 2024-04-24T15:33:43.230 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 0.56416709099999995, 2024-04-24T15:33:43.230 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.002490969, 2024-04-24T15:33:43.230 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:43.230 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:43.230 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:43.230 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.230 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:43.230 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.646298+0000", 2024-04-24T15:33:43.230 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:43.230 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.231 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.231 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:43.231 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.646298+0000", 2024-04-24T15:33:43.231 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:43.231 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.231 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.231 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:43.231 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.646297+0000", 2024-04-24T15:33:43.231 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:43.231 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.231 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.231 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:43.231 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.646314+0000", 2024-04-24T15:33:43.231 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.6444999999999999e-05 2024-04-24T15:33:43.231 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.232 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.232 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:43.232 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.646322+0000", 2024-04-24T15:33:43.232 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.8289999999999997e-06 2024-04-24T15:33:43.232 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.232 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.232 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:43.232 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.646327+0000", 2024-04-24T15:33:43.232 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.0359999999999997e-06 2024-04-24T15:33:43.232 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.232 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.232 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:43.232 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.646359+0000", 2024-04-24T15:33:43.232 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.2014000000000002e-05 2024-04-24T15:33:43.232 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.232 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.233 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:43.233 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.646383+0000", 2024-04-24T15:33:43.233 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.3898000000000001e-05 2024-04-24T15:33:43.233 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.233 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.233 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:33:43.233 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.648759+0000", 2024-04-24T15:33:43.233 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0023756620000000002 2024-04-24T15:33:43.233 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.233 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.233 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:43.233 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.648789+0000", 2024-04-24T15:33:43.233 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.9536999999999999e-05 2024-04-24T15:33:43.233 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:43.233 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:43.233 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:43.234 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.234 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.234 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "MOSDPGPush(4.12 105/104 [PushOp(4:4c712395:::rbd_data.114ab402c616.0000000000000006:1a, version: 79'279, data_included: [0~655360,720896~131072,917504~131072], data_size: 917504, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(4:4c712395:::rbd_data.114ab402c616.0000000000000006:1a@79'279, size: 2097152, copy_subset: [0~2097152], clone_subset: {}, snapset: 1a=[]:{1a=[1a]}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:2097152, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-04-24T15:33:43.234 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:42.651483+0000", 2024-04-24T15:33:43.234 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 0.55898128499999999, 2024-04-24T15:33:43.234 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.002709309, 2024-04-24T15:33:43.234 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:43.234 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:43.234 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:43.234 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.234 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:43.234 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.651483+0000", 2024-04-24T15:33:43.234 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:43.234 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.235 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.235 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:43.235 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.651483+0000", 2024-04-24T15:33:43.235 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:43.235 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.235 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.235 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:43.235 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.651483+0000", 2024-04-24T15:33:43.235 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:43.235 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.235 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.235 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:43.235 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.652426+0000", 2024-04-24T15:33:43.235 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00094301300000000003 2024-04-24T15:33:43.235 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.236 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.236 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:43.236 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.652442+0000", 2024-04-24T15:33:43.236 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.5563999999999999e-05 2024-04-24T15:33:43.236 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.236 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.236 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:43.236 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.652454+0000", 2024-04-24T15:33:43.236 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.2262999999999999e-05 2024-04-24T15:33:43.236 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.236 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.236 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:43.236 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.652521+0000", 2024-04-24T15:33:43.236 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.7073e-05 2024-04-24T15:33:43.236 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.236 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.237 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:43.237 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.652530+0000", 2024-04-24T15:33:43.237 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.465e-06 2024-04-24T15:33:43.237 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.237 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.237 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:43.237 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.654193+0000", 2024-04-24T15:33:43.237 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0016623740000000001 2024-04-24T15:33:43.237 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:43.237 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:43.237 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:43.237 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.237 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.237 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "MOSDPGPush(4.12 105/104 [PushOp(4:4bb634d6:::rbd_data.114a9052fca3.0000000000000015:1e, version: 83'292, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(4:4bb634d6:::rbd_data.114a9052fca3.0000000000000015:1e@83'292, size: 1425408, copy_subset: [0~1425408], clone_subset: {}, snapset: 1e=[]:{1e=[1e]}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:1048576, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-04-24T15:33:43.237 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:42.658761+0000", 2024-04-24T15:33:43.238 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 0.55170393399999995, 2024-04-24T15:33:43.238 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.002286826, 2024-04-24T15:33:43.238 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:43.238 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:43.238 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:43.238 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.238 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:43.238 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.658761+0000", 2024-04-24T15:33:43.238 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:43.238 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.238 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.238 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:43.238 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.658761+0000", 2024-04-24T15:33:43.238 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:43.239 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.239 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.239 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:43.239 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.658760+0000", 2024-04-24T15:33:43.239 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:43.239 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.239 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.239 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:43.239 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.659932+0000", 2024-04-24T15:33:43.239 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.001171645 2024-04-24T15:33:43.239 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.239 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.239 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:43.239 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.659946+0000", 2024-04-24T15:33:43.239 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3959e-05 2024-04-24T15:33:43.240 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.240 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.240 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:43.240 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.659953+0000", 2024-04-24T15:33:43.240 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.52e-06 2024-04-24T15:33:43.240 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.240 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.240 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:43.240 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.660009+0000", 2024-04-24T15:33:43.240 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6110999999999999e-05 2024-04-24T15:33:43.240 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.240 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.240 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:43.240 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.660016+0000", 2024-04-24T15:33:43.240 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.9009999999999997e-06 2024-04-24T15:33:43.241 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.241 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.241 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:43.241 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.661048+0000", 2024-04-24T15:33:43.241 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0010312710000000001 2024-04-24T15:33:43.241 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:43.241 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:43.241 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:43.241 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.241 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.241 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "MOSDPGPush(4.12 105/104 [PushOp(4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e, version: 83'294, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e@83'294, size: 1415104, copy_subset: [0~1415104], clone_subset: {}, snapset: 1e=[]:{1e=[1e]}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:1048576, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-04-24T15:33:43.241 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:42.665272+0000", 2024-04-24T15:33:43.241 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 0.54519284499999998, 2024-04-24T15:33:43.241 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0022331830000000001, 2024-04-24T15:33:43.242 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:43.242 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:43.242 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:43.242 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.242 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:43.242 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.665272+0000", 2024-04-24T15:33:43.242 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:43.242 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.242 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.242 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:43.242 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.665272+0000", 2024-04-24T15:33:43.242 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:43.242 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.242 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.243 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:43.243 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.665269+0000", 2024-04-24T15:33:43.243 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999967 2024-04-24T15:33:43.243 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.243 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.243 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:43.243 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.666490+0000", 2024-04-24T15:33:43.243 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0012209339999999999 2024-04-24T15:33:43.243 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.243 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.243 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:43.243 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.666510+0000", 2024-04-24T15:33:43.243 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.0284000000000001e-05 2024-04-24T15:33:43.243 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.243 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.243 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:43.244 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.666525+0000", 2024-04-24T15:33:43.244 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.4661000000000001e-05 2024-04-24T15:33:43.244 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.244 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.244 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:43.244 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.666571+0000", 2024-04-24T15:33:43.244 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6746999999999998e-05 2024-04-24T15:33:43.244 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.244 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.244 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:43.244 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.666576+0000", 2024-04-24T15:33:43.244 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4120000000000003e-06 2024-04-24T15:33:43.244 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.244 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.245 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:43.245 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.667505+0000", 2024-04-24T15:33:43.245 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00092927600000000004 2024-04-24T15:33:43.245 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:43.245 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:43.245 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:43.245 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.245 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.245 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_repop(osd.4.0:2759 4.18 e105/104 4:1fc87759:::rbd_data.114a8b543e91.0000000000000002:18 v 105'408, mlcod=105'404)", 2024-04-24T15:33:43.245 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:42.674602+0000", 2024-04-24T15:33:43.245 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 0.53586224699999996, 2024-04-24T15:33:43.245 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0043182719999999997, 2024-04-24T15:33:43.245 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:43.245 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:43.245 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:43.246 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.246 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:43.246 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.674602+0000", 2024-04-24T15:33:43.246 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:43.246 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.246 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.246 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:43.246 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.674602+0000", 2024-04-24T15:33:43.246 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:43.246 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.246 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.246 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:43.246 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.674602+0000", 2024-04-24T15:33:43.246 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:43.247 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.247 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.247 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:43.247 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.674609+0000", 2024-04-24T15:33:43.247 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.6889999999999998e-06 2024-04-24T15:33:43.247 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.247 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.247 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:43.247 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.674618+0000", 2024-04-24T15:33:43.247 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.1130000000000007e-06 2024-04-24T15:33:43.247 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.247 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.247 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:43.247 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.674623+0000", 2024-04-24T15:33:43.247 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.3959999999999997e-06 2024-04-24T15:33:43.248 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.248 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.248 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:43.248 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.674657+0000", 2024-04-24T15:33:43.248 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.3847999999999998e-05 2024-04-24T15:33:43.248 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.248 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.248 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:43.248 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.674673+0000", 2024-04-24T15:33:43.248 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.5619000000000001e-05 2024-04-24T15:33:43.248 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.248 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.248 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:33:43.248 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.678897+0000", 2024-04-24T15:33:43.248 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0042239280000000001 2024-04-24T15:33:43.249 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.249 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.249 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:43.249 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.678921+0000", 2024-04-24T15:33:43.249 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.4178000000000001e-05 2024-04-24T15:33:43.249 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:43.249 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:43.249 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:43.249 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.249 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.249 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_repop(osd.4.0:2760 4.18 e105/104 4:1c88e57e:::rbd_data.114ab402c616.000000000000001a:1a v 105'410, mlcod=105'406)", 2024-04-24T15:33:43.249 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:42.682611+0000", 2024-04-24T15:33:43.249 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 0.52785396200000001, 2024-04-24T15:33:43.249 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0050158809999999998, 2024-04-24T15:33:43.249 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:43.250 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:43.250 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:43.250 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.250 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:43.250 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.682611+0000", 2024-04-24T15:33:43.250 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:43.250 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.250 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.250 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:43.250 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.682611+0000", 2024-04-24T15:33:43.250 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:43.250 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.250 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.250 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:43.251 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.682610+0000", 2024-04-24T15:33:43.251 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:43.251 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.251 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.251 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:43.251 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.682616+0000", 2024-04-24T15:33:43.251 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.207e-06 2024-04-24T15:33:43.251 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.251 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.251 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:43.251 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.682624+0000", 2024-04-24T15:33:43.251 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.9410000000000007e-06 2024-04-24T15:33:43.251 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.251 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.252 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:43.252 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.682629+0000", 2024-04-24T15:33:43.252 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.9779999999999996e-06 2024-04-24T15:33:43.252 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.252 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.252 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:43.252 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.682665+0000", 2024-04-24T15:33:43.252 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5581999999999999e-05 2024-04-24T15:33:43.252 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.252 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.252 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:43.252 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.682684+0000", 2024-04-24T15:33:43.252 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.8788999999999999e-05 2024-04-24T15:33:43.252 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.252 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.253 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:33:43.253 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.687594+0000", 2024-04-24T15:33:43.253 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0049105490000000002 2024-04-24T15:33:43.253 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.253 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.253 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:43.253 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.687627+0000", 2024-04-24T15:33:43.253 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.2396e-05 2024-04-24T15:33:43.253 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:43.253 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:43.253 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:43.253 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.253 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.253 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "MOSDPGPush(4.10 105/104 [PushOp(4:0ef81912:::rbd_data.114aee5fd3a9.000000000000000e:1c, version: 81'181, data_included: [0~786432,1310720~9581], data_size: 796013, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(4:0ef81912:::rbd_data.114aee5fd3a9.000000000000000e:1c@81'181, size: 6553600, copy_subset: [0~6553600], clone_subset: {}, snapset: 1c=[]:{1c=[1c]}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:6553600, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-04-24T15:33:43.254 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:42.690941+0000", 2024-04-24T15:33:43.254 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 0.51952409200000005, 2024-04-24T15:33:43.254 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0027029699999999999, 2024-04-24T15:33:43.254 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:43.254 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:43.254 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:43.254 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.254 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:43.254 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.690941+0000", 2024-04-24T15:33:43.254 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:43.254 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.254 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.254 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:43.254 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.690941+0000", 2024-04-24T15:33:43.254 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:43.255 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.255 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.255 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:43.255 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.690940+0000", 2024-04-24T15:33:43.255 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:43.255 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.255 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.255 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:43.255 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.691765+0000", 2024-04-24T15:33:43.255 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00082501799999999995 2024-04-24T15:33:43.255 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.255 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.255 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:43.255 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.691784+0000", 2024-04-24T15:33:43.255 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.9361999999999999e-05 2024-04-24T15:33:43.256 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.256 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.256 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:43.256 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.691795+0000", 2024-04-24T15:33:43.256 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.044e-05 2024-04-24T15:33:43.256 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.256 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.256 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:43.256 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.691874+0000", 2024-04-24T15:33:43.256 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.9474000000000005e-05 2024-04-24T15:33:43.256 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.256 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.256 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:43.256 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.691878+0000", 2024-04-24T15:33:43.256 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0849999999999999e-06 2024-04-24T15:33:43.256 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.257 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.257 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:43.257 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.693644+0000", 2024-04-24T15:33:43.257 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0017652 2024-04-24T15:33:43.257 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:43.257 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:43.257 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:43.257 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.257 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.257 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "MOSDPGPush(4.15 105/104 [PushOp(4:aca5755e:::rbd_data.114a8b543e91.0000000000000009:18, version: 77'224, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(4:aca5755e:::rbd_data.114a8b543e91.0000000000000009:18@77'224, size: 2097152, copy_subset: [0~2097152], clone_subset: {}, snapset: 18=[]:{18=[18]}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:1048576, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-04-24T15:33:43.257 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:42.747643+0000", 2024-04-24T15:33:43.257 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 0.46282173500000001, 2024-04-24T15:33:43.257 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0023312210000000001, 2024-04-24T15:33:43.257 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:43.257 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:43.257 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:43.258 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.258 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:43.258 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.747643+0000", 2024-04-24T15:33:43.258 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:43.258 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.258 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.258 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:43.258 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.747643+0000", 2024-04-24T15:33:43.258 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:43.258 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.258 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.258 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:43.258 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.747642+0000", 2024-04-24T15:33:43.258 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:43.259 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.259 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.259 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:43.259 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.748954+0000", 2024-04-24T15:33:43.259 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00131128 2024-04-24T15:33:43.259 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.259 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.259 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:43.259 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.748973+0000", 2024-04-24T15:33:43.259 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.9823e-05 2024-04-24T15:33:43.259 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.259 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.259 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:43.259 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.748986+0000", 2024-04-24T15:33:43.259 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.2179e-05 2024-04-24T15:33:43.259 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.259 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.260 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:43.260 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.749044+0000", 2024-04-24T15:33:43.260 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.7902999999999998e-05 2024-04-24T15:33:43.260 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.260 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.260 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:43.260 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.749048+0000", 2024-04-24T15:33:43.260 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8049999999999997e-06 2024-04-24T15:33:43.260 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.260 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.260 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:43.260 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.749974+0000", 2024-04-24T15:33:43.260 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00092589099999999998 2024-04-24T15:33:43.260 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:43.261 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:43.261 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:43.261 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.261 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.261 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "MOSDPGPush(4.15 105/104 [PushOp(4:aca5755e:::rbd_data.114a8b543e91.0000000000000009:18, version: 77'224, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(4:aca5755e:::rbd_data.114a8b543e91.0000000000000009:18@77'224, size: 2097152, copy_subset: [0~2097152], clone_subset: {}, snapset: 18=[]:{18=[18]}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:2097152, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to:1048576, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2024-04-24T15:33:43.261 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:42.752086+0000", 2024-04-24T15:33:43.261 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 0.45837837199999998, 2024-04-24T15:33:43.261 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0025164599999999999, 2024-04-24T15:33:43.261 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:43.261 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:43.261 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:43.261 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.261 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:43.261 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.752086+0000", 2024-04-24T15:33:43.262 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:43.262 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.262 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.262 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:43.262 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.752086+0000", 2024-04-24T15:33:43.262 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:43.262 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.262 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.262 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:43.262 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.752086+0000", 2024-04-24T15:33:43.262 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:43.262 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.262 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.262 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:43.263 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.753322+0000", 2024-04-24T15:33:43.263 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0012367719999999999 2024-04-24T15:33:43.263 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.263 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.263 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:43.263 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.753343+0000", 2024-04-24T15:33:43.263 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.0194999999999999e-05 2024-04-24T15:33:43.263 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.263 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.263 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:43.263 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.753354+0000", 2024-04-24T15:33:43.263 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0889e-05 2024-04-24T15:33:43.263 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.263 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.264 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:43.264 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.753410+0000", 2024-04-24T15:33:43.264 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6752999999999997e-05 2024-04-24T15:33:43.264 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.264 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.264 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:43.264 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.753417+0000", 2024-04-24T15:33:43.264 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.5320000000000003e-06 2024-04-24T15:33:43.264 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.264 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.264 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:43.264 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.754603+0000", 2024-04-24T15:33:43.264 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.001185974 2024-04-24T15:33:43.264 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:43.265 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:43.265 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:43.265 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.265 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.265 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "MOSDPGPush(4.15 105/104 [PushOp(4:a939818e:::rbd_data.114a9052fca3.000000000000000b:1e, version: 83'240, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(4:a939818e:::rbd_data.114a9052fca3.000000000000000b:1e@83'240, size: 2097152, copy_subset: [0~2097152], clone_subset: {}, snapset: 1e=[]:{1e=[1e]}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:1048576, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-04-24T15:33:43.265 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:42.758490+0000", 2024-04-24T15:33:43.265 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 0.45197502299999998, 2024-04-24T15:33:43.265 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0024740919999999998, 2024-04-24T15:33:43.265 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:43.265 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:43.265 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:43.265 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.265 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:43.265 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.758490+0000", 2024-04-24T15:33:43.266 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:43.266 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.266 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.266 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:43.266 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.758490+0000", 2024-04-24T15:33:43.266 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:43.266 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.266 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.266 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:43.266 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.758484+0000", 2024-04-24T15:33:43.266 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999943 2024-04-24T15:33:43.266 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.266 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.266 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:43.267 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.759748+0000", 2024-04-24T15:33:43.267 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.001264083 2024-04-24T15:33:43.267 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.267 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.267 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:43.267 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.759763+0000", 2024-04-24T15:33:43.267 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.5221999999999999e-05 2024-04-24T15:33:43.267 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.267 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.267 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:43.267 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.759772+0000", 2024-04-24T15:33:43.267 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.2500000000000006e-06 2024-04-24T15:33:43.267 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.267 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.268 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:43.268 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.759828+0000", 2024-04-24T15:33:43.268 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.5942000000000001e-05 2024-04-24T15:33:43.268 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.268 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.268 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:43.268 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.759832+0000", 2024-04-24T15:33:43.268 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.7870000000000001e-06 2024-04-24T15:33:43.268 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.268 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.268 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:43.268 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.760964+0000", 2024-04-24T15:33:43.268 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.001131359 2024-04-24T15:33:43.268 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:43.268 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:43.269 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:43.269 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.269 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.269 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_repop(osd.4.0:2767 4.1 e105/104 4:827a7463:::rbd_data.114a8191d38d.0000000000000000:23 v 105'299, mlcod=105'295)", 2024-04-24T15:33:43.269 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:42.760819+0000", 2024-04-24T15:33:43.269 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 0.44964599, 2024-04-24T15:33:43.269 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0051269829999999999, 2024-04-24T15:33:43.269 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:43.269 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:43.269 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:43.269 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.269 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:43.269 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.760819+0000", 2024-04-24T15:33:43.269 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:43.270 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.270 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.270 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:43.270 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.760819+0000", 2024-04-24T15:33:43.270 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:43.270 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.270 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.270 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:43.270 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.760818+0000", 2024-04-24T15:33:43.270 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:43.270 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.270 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.270 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:43.270 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.760826+0000", 2024-04-24T15:33:43.271 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7370000000000006e-06 2024-04-24T15:33:43.271 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.271 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.271 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:43.271 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.760837+0000", 2024-04-24T15:33:43.271 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.1516e-05 2024-04-24T15:33:43.271 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.271 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.271 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:43.271 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.760844+0000", 2024-04-24T15:33:43.271 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.1890000000000004e-06 2024-04-24T15:33:43.271 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.271 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.271 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:43.271 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.760881+0000", 2024-04-24T15:33:43.271 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.7642999999999999e-05 2024-04-24T15:33:43.271 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.272 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.272 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:43.272 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.760900+0000", 2024-04-24T15:33:43.272 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.9154e-05 2024-04-24T15:33:43.272 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.272 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.272 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:33:43.272 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.765915+0000", 2024-04-24T15:33:43.272 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0050150150000000003 2024-04-24T15:33:43.272 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.272 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.272 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:43.272 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.765946+0000", 2024-04-24T15:33:43.272 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.0388000000000001e-05 2024-04-24T15:33:43.272 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:43.273 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:43.273 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:43.273 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.273 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.273 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_repop(osd.4.0:2768 4.1 e105/104 4:8506c281:::rbd_data.114a8191d38d.0000000000000012:23 v 105'301, mlcod=105'295)", 2024-04-24T15:33:43.273 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:42.761567+0000", 2024-04-24T15:33:43.273 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 0.448898046, 2024-04-24T15:33:43.273 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0069725309999999997, 2024-04-24T15:33:43.273 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:43.273 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:43.273 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:43.273 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.273 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:43.273 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.761567+0000", 2024-04-24T15:33:43.273 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:43.274 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.274 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.274 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:43.274 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.761567+0000", 2024-04-24T15:33:43.274 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:43.274 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.274 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.274 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:43.274 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.761566+0000", 2024-04-24T15:33:43.274 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:43.274 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.274 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.274 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:43.275 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.761571+0000", 2024-04-24T15:33:43.275 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.809e-06 2024-04-24T15:33:43.275 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.275 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.275 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:43.275 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.761578+0000", 2024-04-24T15:33:43.275 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.0829999999999998e-06 2024-04-24T15:33:43.275 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.275 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.275 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:43.275 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.761582+0000", 2024-04-24T15:33:43.275 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.1699999999999999e-06 2024-04-24T15:33:43.275 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.275 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.275 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:43.276 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.761607+0000", 2024-04-24T15:33:43.276 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.4946000000000001e-05 2024-04-24T15:33:43.276 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.276 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.276 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:43.276 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.761623+0000", 2024-04-24T15:33:43.276 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.5899000000000001e-05 2024-04-24T15:33:43.276 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.276 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.276 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:33:43.276 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.768508+0000", 2024-04-24T15:33:43.276 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0068846209999999996 2024-04-24T15:33:43.276 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.276 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.277 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:43.277 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.768539+0000", 2024-04-24T15:33:43.277 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.1525000000000001e-05 2024-04-24T15:33:43.277 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:43.277 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:43.277 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:43.277 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.277 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.277 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "MOSDPGPush(4.15 105/104 [PushOp(4:a939818e:::rbd_data.114a9052fca3.000000000000000b:1e, version: 83'240, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(4:a939818e:::rbd_data.114a9052fca3.000000000000000b:1e@83'240, size: 2097152, copy_subset: [0~2097152], clone_subset: {}, snapset: 1e=[]:{1e=[1e]}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:2097152, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to:1048576, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2024-04-24T15:33:43.277 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:42.766457+0000", 2024-04-24T15:33:43.277 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 0.44400803, 2024-04-24T15:33:43.277 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00257128, 2024-04-24T15:33:43.277 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:43.277 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:43.278 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:43.278 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.278 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:43.278 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.766457+0000", 2024-04-24T15:33:43.278 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:43.278 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.278 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.278 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:43.278 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.766457+0000", 2024-04-24T15:33:43.278 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:43.278 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.278 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.278 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:43.278 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.766456+0000", 2024-04-24T15:33:43.279 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:43.279 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.279 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.279 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:43.279 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.767975+0000", 2024-04-24T15:33:43.279 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0015188650000000001 2024-04-24T15:33:43.279 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.279 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.279 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:43.279 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.767989+0000", 2024-04-24T15:33:43.279 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3994e-05 2024-04-24T15:33:43.279 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.279 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.279 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:43.279 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.767996+0000", 2024-04-24T15:33:43.280 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.255e-06 2024-04-24T15:33:43.280 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.280 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.280 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:43.280 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.768044+0000", 2024-04-24T15:33:43.280 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.7729999999999999e-05 2024-04-24T15:33:43.280 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.280 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.280 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:43.280 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.768050+0000", 2024-04-24T15:33:43.280 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.9660000000000001e-06 2024-04-24T15:33:43.280 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.280 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.280 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:43.280 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.769028+0000", 2024-04-24T15:33:43.280 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00097793899999999994 2024-04-24T15:33:43.281 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:43.281 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:43.281 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:43.281 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.281 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.281 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "MOSDPGPush(4.15 105/104 [PushOp(4:ad717d5c:::rbd_data.114a9052fca3.0000000000000002:1e, version: 83'243, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(4:ad717d5c:::rbd_data.114a9052fca3.0000000000000002:1e@83'243, size: 2097152, copy_subset: [0~2097152], clone_subset: {}, snapset: 1e=[]:{1e=[1e]}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:1048576, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-04-24T15:33:43.281 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:42.777619+0000", 2024-04-24T15:33:43.281 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 0.43284571900000002, 2024-04-24T15:33:43.281 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0025080139999999998, 2024-04-24T15:33:43.281 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:43.281 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:43.281 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:43.281 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.281 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:43.281 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.777619+0000", 2024-04-24T15:33:43.282 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:43.282 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.282 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.282 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:43.282 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.777619+0000", 2024-04-24T15:33:43.282 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:43.282 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.282 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.282 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:43.282 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.777619+0000", 2024-04-24T15:33:43.282 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:43.282 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.282 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.282 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:43.282 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.779004+0000", 2024-04-24T15:33:43.283 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.001385145 2024-04-24T15:33:43.283 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.283 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.283 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:43.283 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.779017+0000", 2024-04-24T15:33:43.283 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3668e-05 2024-04-24T15:33:43.283 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.283 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.283 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:43.283 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.779025+0000", 2024-04-24T15:33:43.283 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0439999999999996e-06 2024-04-24T15:33:43.283 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.283 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.284 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:43.284 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.779085+0000", 2024-04-24T15:33:43.284 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.9435999999999998e-05 2024-04-24T15:33:43.284 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.284 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.284 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:43.284 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.779093+0000", 2024-04-24T15:33:43.284 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7190000000000001e-06 2024-04-24T15:33:43.284 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.284 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.284 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:43.284 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.780127+0000", 2024-04-24T15:33:43.284 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.001034456 2024-04-24T15:33:43.284 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:43.284 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:43.285 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:43.285 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.285 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.285 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_repop(osd.4.0:2769 4.13 e105/104 4:cfbf367e:::rbd_data.114a858e6641.0000000000000000:17 v 105'226, mlcod=103'223)", 2024-04-24T15:33:43.285 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:42.815674+0000", 2024-04-24T15:33:43.285 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 0.39479117899999999, 2024-04-24T15:33:43.285 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0031768130000000001, 2024-04-24T15:33:43.285 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:43.285 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:43.285 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:43.285 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.285 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:43.285 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.815674+0000", 2024-04-24T15:33:43.285 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:43.285 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.285 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.286 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:43.286 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.815674+0000", 2024-04-24T15:33:43.286 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:43.286 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.286 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.286 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:43.286 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.815673+0000", 2024-04-24T15:33:43.286 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:43.286 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.286 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.286 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:43.286 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.815682+0000", 2024-04-24T15:33:43.286 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.6410000000000008e-06 2024-04-24T15:33:43.286 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.286 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.287 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:43.287 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.815691+0000", 2024-04-24T15:33:43.287 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.4199999999999996e-06 2024-04-24T15:33:43.287 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.287 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.287 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:43.287 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.815697+0000", 2024-04-24T15:33:43.287 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6300000000000003e-06 2024-04-24T15:33:43.287 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.287 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.287 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:43.287 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.815728+0000", 2024-04-24T15:33:43.287 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.1433999999999998e-05 2024-04-24T15:33:43.287 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.287 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.287 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:43.288 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.815747+0000", 2024-04-24T15:33:43.288 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.929e-05 2024-04-24T15:33:43.288 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.288 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.288 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:33:43.288 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.818807+0000", 2024-04-24T15:33:43.288 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0030592309999999999 2024-04-24T15:33:43.288 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.288 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.288 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:43.288 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.818850+0000", 2024-04-24T15:33:43.288 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3819000000000002e-05 2024-04-24T15:33:43.288 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:43.288 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:43.288 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:43.288 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.289 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.289 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_repop(osd.4.0:2770 4.13 e105/104 4:ce8f4ace:::rbd_data.114a9052fca3.0000000000000018:1e v 105'228, mlcod=105'224)", 2024-04-24T15:33:43.289 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:42.821450+0000", 2024-04-24T15:33:43.289 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 0.389015112, 2024-04-24T15:33:43.289 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0025787380000000001, 2024-04-24T15:33:43.289 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:43.289 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:43.289 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:43.289 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.289 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:43.289 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.821450+0000", 2024-04-24T15:33:43.289 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:43.289 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.289 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.289 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:43.289 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.821450+0000", 2024-04-24T15:33:43.289 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:43.290 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.290 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.290 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:43.290 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.821449+0000", 2024-04-24T15:33:43.290 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:43.290 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.290 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.290 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:43.290 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.821456+0000", 2024-04-24T15:33:43.290 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2150000000000004e-06 2024-04-24T15:33:43.290 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.290 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.290 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:43.290 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.821467+0000", 2024-04-24T15:33:43.291 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.1188e-05 2024-04-24T15:33:43.291 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.291 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.291 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:43.291 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.821474+0000", 2024-04-24T15:33:43.291 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.905e-06 2024-04-24T15:33:43.291 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.291 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.291 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:43.291 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.821511+0000", 2024-04-24T15:33:43.291 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.6769e-05 2024-04-24T15:33:43.291 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.291 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.291 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:43.291 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.821537+0000", 2024-04-24T15:33:43.291 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.5782000000000002e-05 2024-04-24T15:33:43.292 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.292 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.292 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:33:43.292 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.824004+0000", 2024-04-24T15:33:43.292 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0024674929999999999 2024-04-24T15:33:43.292 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:43.292 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:43.292 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:43.292 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.824028+0000", 2024-04-24T15:33:43.292 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.3966000000000001e-05 2024-04-24T15:33:43.292 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:43.292 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:43.292 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:43.292 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:43.292 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:43.292 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:43.294 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:43.294+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:43.294 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-24T15:33:43.394 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:33:43.395+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:33:43.417 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:43.418 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-04-24T15:33:43.418 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.418 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:75519 4.a 4.2a5fcea (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e105)", 2024-04-24T15:33:43.418 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:42.638138+0000", 2024-04-24T15:33:43.418 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.78127403500000003, 2024-04-24T15:33:43.418 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78129891699999998, 2024-04-24T15:33:43.418 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:43.418 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for peered", 2024-04-24T15:33:43.418 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:43.418 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:43.418 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:43.418 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 75519 2024-04-24T15:33:43.418 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.419 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:43.419 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.419 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:43.419 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:42.638138+0000", 2024-04-24T15:33:43.419 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:43.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.419 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.419 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:43.419 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:42.638138+0000", 2024-04-24T15:33:43.419 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:43.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.419 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.419 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:43.419 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:42.638137+0000", 2024-04-24T15:33:43.419 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999986 2024-04-24T15:33:43.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.420 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:43.420 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:42.638142+0000", 2024-04-24T15:33:43.420 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8989999999999998e-06 2024-04-24T15:33:43.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.420 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:43.420 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:42.638154+0000", 2024-04-24T15:33:43.420 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1572e-05 2024-04-24T15:33:43.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.420 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:43.420 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:42.638160+0000", 2024-04-24T15:33:43.420 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3899999999999998e-06 2024-04-24T15:33:43.421 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.421 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.421 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:43.421 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:42.638203+0000", 2024-04-24T15:33:43.421 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2252000000000001e-05 2024-04-24T15:33:43.421 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.421 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.421 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op must wait for map", 2024-04-24T15:33:43.421 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:42.638209+0000", 2024-04-24T15:33:43.421 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4359999999999999e-06 2024-04-24T15:33:43.421 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.421 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.421 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:43.421 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:42.639589+0000", 2024-04-24T15:33:43.421 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001379903 2024-04-24T15:33:43.422 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.422 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.422 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-04-24T15:33:43.422 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:42.639600+0000", 2024-04-24T15:33:43.422 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0719e-05 2024-04-24T15:33:43.422 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:43.422 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:43.422 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:43.422 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:43.422 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-24T15:33:43.422 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 1 2024-04-24T15:33:43.422 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:43.427 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-24T15:33:43.495 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:33:43.496+0000 7f0f45efb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 59992) UID: 0 2024-04-24T15:33:43.551 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:43.551 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:33:43.551 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-04-24T15:33:43.551 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-04-24T15:33:43.551 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:43.561 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-24T15:33:43.596 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:33:43.597+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:33:43.696 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:43.697+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:43.697 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:43.697 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-04-24T15:33:43.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-04-24T15:33:43.698 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-04-24T15:33:43.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.698 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:47384 4.1f 4:fdac2fcc:::rbd_header.114a8b543e91:head [watch reconnect cookie 140672527126976 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-04-24T15:33:43.698 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:09.802333+0000", 2024-04-24T15:33:43.698 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.89512938, 2024-04-24T15:33:43.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0093213050000001, 2024-04-24T15:33:43.698 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:43.698 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:43.698 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:43.698 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:43.698 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:43.698 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 47384 2024-04-24T15:33:43.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.698 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:43.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:43.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.802333+0000", 2024-04-24T15:33:43.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:43.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:43.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.802333+0000", 2024-04-24T15:33:43.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:43.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:43.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.802332+0000", 2024-04-24T15:33:43.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999981 2024-04-24T15:33:43.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:43.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.802340+0000", 2024-04-24T15:33:43.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7700000000000007e-06 2024-04-24T15:33:43.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:43.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.802360+0000", 2024-04-24T15:33:43.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9959999999999999e-05 2024-04-24T15:33:43.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:43.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.802371+0000", 2024-04-24T15:33:43.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0473e-05 2024-04-24T15:33:43.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:43.701 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.802447+0000", 2024-04-24T15:33:43.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6089999999999998e-05 2024-04-24T15:33:43.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op must wait for map", 2024-04-24T15:33:43.701 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.802456+0000", 2024-04-24T15:33:43.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0599999999999997e-06 2024-04-24T15:33:43.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:43.701 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.808378+0000", 2024-04-24T15:33:43.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0059224910000000002 2024-04-24T15:33:43.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.702 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-04-24T15:33:43.702 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.808384+0000", 2024-04-24T15:33:43.702 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7740000000000001e-06 2024-04-24T15:33:43.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.702 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:43.702 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.809812+0000", 2024-04-24T15:33:43.702 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.001428043 2024-04-24T15:33:43.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.702 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:43.702 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.810082+0000", 2024-04-24T15:33:43.702 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00026971800000000001 2024-04-24T15:33:43.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.703 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:43.703 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.811655+0000", 2024-04-24T15:33:43.703 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015726869999999999 2024-04-24T15:33:43.703 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:43.703 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:43.703 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:43.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.703 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:47385 4.1f 4:fdac2fcc:::rbd_header.114a8b543e91:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-04-24T15:33:43.703 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:09.803953+0000", 2024-04-24T15:33:43.703 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.893509465999998, 2024-04-24T15:33:43.703 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0080910729999999, 2024-04-24T15:33:43.703 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:43.703 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:43.703 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:43.704 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:43.704 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:43.704 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 47385 2024-04-24T15:33:43.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.704 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:43.704 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.704 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:43.704 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.803953+0000", 2024-04-24T15:33:43.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:43.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.704 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.704 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:43.704 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.803953+0000", 2024-04-24T15:33:43.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:43.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.704 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.704 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:43.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.803952+0000", 2024-04-24T15:33:43.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999981 2024-04-24T15:33:43.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:43.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.803961+0000", 2024-04-24T15:33:43.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9549999999999998e-06 2024-04-24T15:33:43.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:43.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.803977+0000", 2024-04-24T15:33:43.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6498999999999999e-05 2024-04-24T15:33:43.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:43.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.803985+0000", 2024-04-24T15:33:43.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5000000000000002e-06 2024-04-24T15:33:43.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.706 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:43.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.804032+0000", 2024-04-24T15:33:43.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7657999999999997e-05 2024-04-24T15:33:43.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.706 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2024-04-24T15:33:43.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.804039+0000", 2024-04-24T15:33:43.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5719999999999999e-06 2024-04-24T15:33:43.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.706 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:43.707 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.808397+0000", 2024-04-24T15:33:43.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0043579860000000003 2024-04-24T15:33:43.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.707 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-04-24T15:33:43.707 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.808401+0000", 2024-04-24T15:33:43.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3089999999999997e-06 2024-04-24T15:33:43.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.707 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:43.707 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.810700+0000", 2024-04-24T15:33:43.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0022984699999999 2024-04-24T15:33:43.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:43.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.810758+0000", 2024-04-24T15:33:43.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8121000000000001e-05 2024-04-24T15:33:43.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:33:43.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.811932+0000", 2024-04-24T15:33:43.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001174542 2024-04-24T15:33:43.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:43.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.811978+0000", 2024-04-24T15:33:43.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5645000000000001e-05 2024-04-24T15:33:43.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:43.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.812044+0000", 2024-04-24T15:33:43.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6575999999999994e-05 2024-04-24T15:33:43.709 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:43.709 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:43.709 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:43.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.709 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48590 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2686976~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:43.709 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889219+0000", 2024-04-24T15:33:43.709 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.808243533999999, 2024-04-24T15:33:43.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016760856000000001, 2024-04-24T15:33:43.709 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:43.709 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:43.709 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:43.709 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:43.710 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:43.710 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48590 2024-04-24T15:33:43.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.710 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:43.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:43.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889219+0000", 2024-04-24T15:33:43.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:43.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:43.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889219+0000", 2024-04-24T15:33:43.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:43.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:43.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889219+0000", 2024-04-24T15:33:43.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:43.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:43.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889221+0000", 2024-04-24T15:33:43.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4499999999999998e-06 2024-04-24T15:33:43.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:43.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889227+0000", 2024-04-24T15:33:43.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6869999999999999e-06 2024-04-24T15:33:43.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:43.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889231+0000", 2024-04-24T15:33:43.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6320000000000001e-06 2024-04-24T15:33:43.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:43.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.905769+0000", 2024-04-24T15:33:43.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016538850000000001 2024-04-24T15:33:43.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:43.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.905855+0000", 2024-04-24T15:33:43.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5296000000000005e-05 2024-04-24T15:33:43.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.713 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:43.713 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.905980+0000", 2024-04-24T15:33:43.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000125406 2024-04-24T15:33:43.713 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:43.713 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:43.713 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:43.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.713 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48705 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 1851392~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:43.713 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889236+0000", 2024-04-24T15:33:43.713 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.808226926000003, 2024-04-24T15:33:43.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016984302, 2024-04-24T15:33:43.713 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:43.713 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:43.713 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:43.714 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:43.714 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:43.714 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48705 2024-04-24T15:33:43.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.714 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:43.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:43.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889236+0000", 2024-04-24T15:33:43.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:43.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:43.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889236+0000", 2024-04-24T15:33:43.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:43.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:43.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889235+0000", 2024-04-24T15:33:43.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:43.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:43.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889238+0000", 2024-04-24T15:33:43.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.249e-06 2024-04-24T15:33:43.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:43.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889243+0000", 2024-04-24T15:33:43.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6810000000000003e-06 2024-04-24T15:33:43.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.716 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:43.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889247+0000", 2024-04-24T15:33:43.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6780000000000002e-06 2024-04-24T15:33:43.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.716 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:43.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906008+0000", 2024-04-24T15:33:43.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01676099 2024-04-24T15:33:43.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.716 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:43.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906090+0000", 2024-04-24T15:33:43.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1508999999999995e-05 2024-04-24T15:33:43.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:43.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906220+0000", 2024-04-24T15:33:43.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013056999999999999 2024-04-24T15:33:43.717 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:43.717 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:43.717 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:43.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.717 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48706 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 1916928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:43.717 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889251+0000", 2024-04-24T15:33:43.717 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.808211493000002, 2024-04-24T15:33:43.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01718693, 2024-04-24T15:33:43.717 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:43.717 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:43.717 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:43.718 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:43.718 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:43.718 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48706 2024-04-24T15:33:43.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.718 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:43.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:43.718 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889251+0000", 2024-04-24T15:33:43.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:43.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:43.718 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889251+0000", 2024-04-24T15:33:43.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:43.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:43.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889251+0000", 2024-04-24T15:33:43.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:43.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:43.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889253+0000", 2024-04-24T15:33:43.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.244e-06 2024-04-24T15:33:43.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:43.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889259+0000", 2024-04-24T15:33:43.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3640000000000004e-06 2024-04-24T15:33:43.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:43.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889263+0000", 2024-04-24T15:33:43.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3720000000000001e-06 2024-04-24T15:33:43.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:43.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906258+0000", 2024-04-24T15:33:43.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016995310999999999 2024-04-24T15:33:43.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:43.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906343+0000", 2024-04-24T15:33:43.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4536000000000005e-05 2024-04-24T15:33:43.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.721 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:43.721 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906438+0000", 2024-04-24T15:33:43.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5502999999999997e-05 2024-04-24T15:33:43.721 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:43.721 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:43.721 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:43.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.721 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48707 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 1982464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:43.721 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889267+0000", 2024-04-24T15:33:43.721 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.808195933, 2024-04-24T15:33:43.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017425395, 2024-04-24T15:33:43.721 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:43.721 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:43.721 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:43.722 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:43.722 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:43.722 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48707 2024-04-24T15:33:43.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.722 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:43.722 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.722 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:43.722 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889267+0000", 2024-04-24T15:33:43.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:43.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.722 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.722 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:43.722 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889267+0000", 2024-04-24T15:33:43.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:43.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:43.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889266+0000", 2024-04-24T15:33:43.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:43.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:43.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889275+0000", 2024-04-24T15:33:43.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.568e-06 2024-04-24T15:33:43.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:43.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889281+0000", 2024-04-24T15:33:43.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3779999999999998e-06 2024-04-24T15:33:43.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.724 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:43.724 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889291+0000", 2024-04-24T15:33:43.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2769999999999994e-06 2024-04-24T15:33:43.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.724 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:43.724 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906476+0000", 2024-04-24T15:33:43.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017185277999999998 2024-04-24T15:33:43.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.724 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:43.724 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906567+0000", 2024-04-24T15:33:43.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0902999999999993e-05 2024-04-24T15:33:43.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.725 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:43.725 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906692+0000", 2024-04-24T15:33:43.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012541700000000001 2024-04-24T15:33:43.725 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:43.725 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:43.725 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:43.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.725 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48708 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2048000~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:43.725 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889295+0000", 2024-04-24T15:33:43.725 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.808167441999998, 2024-04-24T15:33:43.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017534984999999999, 2024-04-24T15:33:43.725 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:43.725 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:43.726 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:43.726 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:43.726 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:43.726 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48708 2024-04-24T15:33:43.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.726 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:43.726 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.726 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:43.726 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889295+0000", 2024-04-24T15:33:43.726 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:43.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.726 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.726 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:43.726 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889295+0000", 2024-04-24T15:33:43.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:43.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:43.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889295+0000", 2024-04-24T15:33:43.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:43.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:43.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889297+0000", 2024-04-24T15:33:43.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3479999999999998e-06 2024-04-24T15:33:43.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:43.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889304+0000", 2024-04-24T15:33:43.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3720000000000001e-06 2024-04-24T15:33:43.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:43.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889307+0000", 2024-04-24T15:33:43.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7160000000000001e-06 2024-04-24T15:33:43.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:43.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906711+0000", 2024-04-24T15:33:43.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017403452999999999 2024-04-24T15:33:43.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:43.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906765+0000", 2024-04-24T15:33:43.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4219999999999999e-05 2024-04-24T15:33:43.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:43.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906830+0000", 2024-04-24T15:33:43.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5277000000000005e-05 2024-04-24T15:33:43.729 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:43.729 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:43.729 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:43.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.729 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48709 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2113536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:43.729 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889312+0000", 2024-04-24T15:33:43.729 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.808151135000003, 2024-04-24T15:33:43.730 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017772013, 2024-04-24T15:33:43.730 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:43.730 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:43.730 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:43.730 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:43.730 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:43.730 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48709 2024-04-24T15:33:43.730 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.730 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:43.730 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.730 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:43.730 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889312+0000", 2024-04-24T15:33:43.730 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:43.730 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.730 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.731 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:43.731 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889312+0000", 2024-04-24T15:33:43.731 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:43.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.731 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.731 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:43.731 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889311+0000", 2024-04-24T15:33:43.731 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:43.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.731 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.731 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:43.731 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889314+0000", 2024-04-24T15:33:43.731 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2759999999999999e-06 2024-04-24T15:33:43.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.731 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:43.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889322+0000", 2024-04-24T15:33:43.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8589999999999994e-06 2024-04-24T15:33:43.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:43.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889326+0000", 2024-04-24T15:33:43.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6380000000000002e-06 2024-04-24T15:33:43.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:43.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906862+0000", 2024-04-24T15:33:43.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017536403999999998 2024-04-24T15:33:43.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:43.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906949+0000", 2024-04-24T15:33:43.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6217000000000005e-05 2024-04-24T15:33:43.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:43.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907084+0000", 2024-04-24T15:33:43.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013497100000000001 2024-04-24T15:33:43.733 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:43.733 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:43.733 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:43.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.733 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48710 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2179072~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:43.733 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889332+0000", 2024-04-24T15:33:43.734 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.808130321, 2024-04-24T15:33:43.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017944720000000001, 2024-04-24T15:33:43.734 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:43.734 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:43.734 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:43.734 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:43.734 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:43.734 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48710 2024-04-24T15:33:43.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.734 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:43.734 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.734 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:43.734 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889332+0000", 2024-04-24T15:33:43.735 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:43.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.735 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:43.735 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889332+0000", 2024-04-24T15:33:43.735 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:43.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.735 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:43.735 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889332+0000", 2024-04-24T15:33:43.735 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:43.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.735 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:43.735 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889334+0000", 2024-04-24T15:33:43.735 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4210000000000002e-06 2024-04-24T15:33:43.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.736 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.736 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:43.736 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889340+0000", 2024-04-24T15:33:43.736 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7209999999999999e-06 2024-04-24T15:33:43.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.736 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.736 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:43.736 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889344+0000", 2024-04-24T15:33:43.736 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.726e-06 2024-04-24T15:33:43.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.736 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.736 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:43.736 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907119+0000", 2024-04-24T15:33:43.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017775551000000001 2024-04-24T15:33:43.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:43.737 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907190+0000", 2024-04-24T15:33:43.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0634000000000002e-05 2024-04-24T15:33:43.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:43.737 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907277+0000", 2024-04-24T15:33:43.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7120999999999995e-05 2024-04-24T15:33:43.737 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:43.737 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:43.737 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:43.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.738 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48711 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2244608~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:43.738 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889348+0000", 2024-04-24T15:33:43.738 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.808114703999998, 2024-04-24T15:33:43.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018188109000000001, 2024-04-24T15:33:43.738 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:43.738 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:43.738 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:43.738 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:43.738 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:43.738 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48711 2024-04-24T15:33:43.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.738 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:43.738 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.738 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:43.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889348+0000", 2024-04-24T15:33:43.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:43.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:43.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889348+0000", 2024-04-24T15:33:43.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:43.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:43.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889348+0000", 2024-04-24T15:33:43.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:43.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:43.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889350+0000", 2024-04-24T15:33:43.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2330000000000001e-06 2024-04-24T15:33:43.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:43.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889356+0000", 2024-04-24T15:33:43.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8769999999999998e-06 2024-04-24T15:33:43.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:43.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889359+0000", 2024-04-24T15:33:43.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3699999999999999e-06 2024-04-24T15:33:43.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:43.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907316+0000", 2024-04-24T15:33:43.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017956857999999999 2024-04-24T15:33:43.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:43.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907402+0000", 2024-04-24T15:33:43.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5927000000000006e-05 2024-04-24T15:33:43.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:43.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907536+0000", 2024-04-24T15:33:43.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000134258 2024-04-24T15:33:43.741 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:43.741 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:43.742 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:43.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.742 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48712 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2310144~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:43.742 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889363+0000", 2024-04-24T15:33:43.742 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.808099384999998, 2024-04-24T15:33:43.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018314815000000002, 2024-04-24T15:33:43.742 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:43.742 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:43.742 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:43.742 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:43.742 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:43.742 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48712 2024-04-24T15:33:43.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.742 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:43.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:43.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889363+0000", 2024-04-24T15:33:43.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:43.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:43.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889363+0000", 2024-04-24T15:33:43.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:43.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:43.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889363+0000", 2024-04-24T15:33:43.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:43.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:43.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889366+0000", 2024-04-24T15:33:43.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6709999999999999e-06 2024-04-24T15:33:43.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:43.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889373+0000", 2024-04-24T15:33:43.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1269999999999997e-06 2024-04-24T15:33:43.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:43.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889377+0000", 2024-04-24T15:33:43.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1219999999999997e-06 2024-04-24T15:33:43.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:43.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907560+0000", 2024-04-24T15:33:43.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018183357000000001 2024-04-24T15:33:43.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:43.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907612+0000", 2024-04-24T15:33:43.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2172999999999997e-05 2024-04-24T15:33:43.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:43.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907678+0000", 2024-04-24T15:33:43.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5736999999999998e-05 2024-04-24T15:33:43.745 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:43.746 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:43.746 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:43.746 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.746 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.746 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48713 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2375680~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:43.746 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889381+0000", 2024-04-24T15:33:43.746 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.808081491999999, 2024-04-24T15:33:43.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018556019, 2024-04-24T15:33:43.746 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:43.746 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:43.746 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:43.746 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:43.746 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:43.746 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48713 2024-04-24T15:33:43.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.747 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:43.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:43.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889381+0000", 2024-04-24T15:33:43.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:43.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:43.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889381+0000", 2024-04-24T15:33:43.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:43.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:43.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889381+0000", 2024-04-24T15:33:43.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:43.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:43.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889388+0000", 2024-04-24T15:33:43.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8809999999999998e-06 2024-04-24T15:33:43.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:43.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889394+0000", 2024-04-24T15:33:43.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0410000000000003e-06 2024-04-24T15:33:43.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:43.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889397+0000", 2024-04-24T15:33:43.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5939999999999998e-06 2024-04-24T15:33:43.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:43.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907711+0000", 2024-04-24T15:33:43.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018313949999999999 2024-04-24T15:33:43.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:43.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907802+0000", 2024-04-24T15:33:43.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0307000000000001e-05 2024-04-24T15:33:43.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:43.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907937+0000", 2024-04-24T15:33:43.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013561899999999999 2024-04-24T15:33:43.750 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:43.750 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:43.750 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:43.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.750 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48714 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2441216~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:43.750 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889435+0000", 2024-04-24T15:33:43.750 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.808028143999998, 2024-04-24T15:33:43.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018636040999999999, 2024-04-24T15:33:43.750 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:43.750 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:43.750 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:43.751 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:43.751 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:43.751 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48714 2024-04-24T15:33:43.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.751 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:43.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:43.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889435+0000", 2024-04-24T15:33:43.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:43.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:43.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889435+0000", 2024-04-24T15:33:43.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:43.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:43.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889434+0000", 2024-04-24T15:33:43.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:43.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:43.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889437+0000", 2024-04-24T15:33:43.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.875e-06 2024-04-24T15:33:43.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:43.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889444+0000", 2024-04-24T15:33:43.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9870000000000002e-06 2024-04-24T15:33:43.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:43.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889448+0000", 2024-04-24T15:33:43.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3030000000000001e-06 2024-04-24T15:33:43.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:43.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907952+0000", 2024-04-24T15:33:43.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018504245999999999 2024-04-24T15:33:43.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:43.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908007+0000", 2024-04-24T15:33:43.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4579e-05 2024-04-24T15:33:43.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:43.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908071+0000", 2024-04-24T15:33:43.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3615999999999995e-05 2024-04-24T15:33:43.754 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:43.754 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:43.754 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:43.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.754 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48715 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2506752~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:43.754 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889453+0000", 2024-04-24T15:33:43.754 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.808009779999999, 2024-04-24T15:33:43.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018873935000000001, 2024-04-24T15:33:43.754 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:43.754 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:43.755 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:43.755 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:43.755 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:43.755 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48715 2024-04-24T15:33:43.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.755 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:43.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:43.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889453+0000", 2024-04-24T15:33:43.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:43.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:43.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889453+0000", 2024-04-24T15:33:43.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:43.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:43.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889452+0000", 2024-04-24T15:33:43.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:43.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:43.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889459+0000", 2024-04-24T15:33:43.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9269999999999999e-06 2024-04-24T15:33:43.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:43.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889468+0000", 2024-04-24T15:33:43.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2390000000000002e-06 2024-04-24T15:33:43.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:43.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889475+0000", 2024-04-24T15:33:43.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3939999999999998e-06 2024-04-24T15:33:43.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:43.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908103+0000", 2024-04-24T15:33:43.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018628388999999999 2024-04-24T15:33:43.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:43.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908193+0000", 2024-04-24T15:33:43.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9343000000000002e-05 2024-04-24T15:33:43.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:43.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908327+0000", 2024-04-24T15:33:43.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013415899999999999 2024-04-24T15:33:43.758 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:43.758 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:43.758 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:43.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.758 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48716 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2572288~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:43.758 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889480+0000", 2024-04-24T15:33:43.758 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.807982930000001, 2024-04-24T15:33:43.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018992862999999999, 2024-04-24T15:33:43.758 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:43.758 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:43.758 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:43.758 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:43.758 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:43.759 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48716 2024-04-24T15:33:43.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.759 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:43.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:43.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889480+0000", 2024-04-24T15:33:43.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:43.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:43.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889480+0000", 2024-04-24T15:33:43.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:43.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:43.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889479+0000", 2024-04-24T15:33:43.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:43.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:43.760 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889482+0000", 2024-04-24T15:33:43.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4729999999999999e-06 2024-04-24T15:33:43.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:43.760 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889488+0000", 2024-04-24T15:33:43.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1750000000000002e-06 2024-04-24T15:33:43.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:43.760 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889501+0000", 2024-04-24T15:33:43.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.288e-05 2024-04-24T15:33:43.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:43.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908345+0000", 2024-04-24T15:33:43.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018843835999999999 2024-04-24T15:33:43.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:43.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908395+0000", 2024-04-24T15:33:43.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9765999999999998e-05 2024-04-24T15:33:43.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:43.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908473+0000", 2024-04-24T15:33:43.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8159999999999997e-05 2024-04-24T15:33:43.762 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:43.762 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:43.762 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:43.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.762 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48717 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2637824~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:43.762 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889506+0000", 2024-04-24T15:33:43.762 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.807956679, 2024-04-24T15:33:43.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.019241864000000001, 2024-04-24T15:33:43.762 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:43.762 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:43.762 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:43.762 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:43.762 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:43.763 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48717 2024-04-24T15:33:43.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.763 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:43.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:43.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889506+0000", 2024-04-24T15:33:43.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:43.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:43.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889506+0000", 2024-04-24T15:33:43.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:43.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:43.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889506+0000", 2024-04-24T15:33:43.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:43.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:43.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889508+0000", 2024-04-24T15:33:43.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6900000000000001e-06 2024-04-24T15:33:43.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:43.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889515+0000", 2024-04-24T15:33:43.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5409999999999997e-06 2024-04-24T15:33:43.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.765 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:43.765 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889519+0000", 2024-04-24T15:33:43.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7129999999999999e-06 2024-04-24T15:33:43.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.765 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:43.765 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908520+0000", 2024-04-24T15:33:43.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.019001606000000001 2024-04-24T15:33:43.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.765 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:43.765 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908612+0000", 2024-04-24T15:33:43.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1909000000000004e-05 2024-04-24T15:33:43.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:43.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908748+0000", 2024-04-24T15:33:43.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000135847 2024-04-24T15:33:43.766 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:43.766 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:43.766 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:43.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.766 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48718 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2703360~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:43.766 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889523+0000", 2024-04-24T15:33:43.766 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.807939687000001, 2024-04-24T15:33:43.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.019451725999999999, 2024-04-24T15:33:43.766 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:43.766 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:43.767 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:43.767 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:43.767 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:43.767 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48718 2024-04-24T15:33:43.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.767 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:43.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:43.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889523+0000", 2024-04-24T15:33:43.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:43.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:43.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889523+0000", 2024-04-24T15:33:43.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:43.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:43.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889523+0000", 2024-04-24T15:33:43.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:43.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:43.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889525+0000", 2024-04-24T15:33:43.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9970000000000001e-06 2024-04-24T15:33:43.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:43.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889530+0000", 2024-04-24T15:33:43.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7610000000000004e-06 2024-04-24T15:33:43.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:43.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889534+0000", 2024-04-24T15:33:43.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6119999999999999e-06 2024-04-24T15:33:43.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:43.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908775+0000", 2024-04-24T15:33:43.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.019240709000000002 2024-04-24T15:33:43.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:43.770 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908856+0000", 2024-04-24T15:33:43.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0788e-05 2024-04-24T15:33:43.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:43.770 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908975+0000", 2024-04-24T15:33:43.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011927800000000001 2024-04-24T15:33:43.770 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:43.770 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:43.770 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:43.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.770 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:49872 4.1f 4:fe48f4c1:::rbd_data.114ab402c616.0000000000000014:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-04-24T15:33:43.770 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:12.416730+0000", 2024-04-24T15:33:43.771 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.280732708999999, 2024-04-24T15:33:43.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.020522327999999999, 2024-04-24T15:33:43.771 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:43.771 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:43.771 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:43.771 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:43.771 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:43.771 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 49872 2024-04-24T15:33:43.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.771 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:43.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:43.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.416730+0000", 2024-04-24T15:33:43.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:43.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:43.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.416730+0000", 2024-04-24T15:33:43.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:43.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:43.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.416729+0000", 2024-04-24T15:33:43.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:33:43.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:43.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.416734+0000", 2024-04-24T15:33:43.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0180000000000001e-06 2024-04-24T15:33:43.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:43.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.416744+0000", 2024-04-24T15:33:43.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0069e-05 2024-04-24T15:33:43.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:43.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.416749+0000", 2024-04-24T15:33:43.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7190000000000001e-06 2024-04-24T15:33:43.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:43.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.416777+0000", 2024-04-24T15:33:43.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7532999999999999e-05 2024-04-24T15:33:43.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:43.774 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.416824+0000", 2024-04-24T15:33:43.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7738000000000003e-05 2024-04-24T15:33:43.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:33:43.774 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.437145+0000", 2024-04-24T15:33:43.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.020320416000000001 2024-04-24T15:33:43.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:43.774 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.437189+0000", 2024-04-24T15:33:43.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4493000000000002e-05 2024-04-24T15:33:43.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:43.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.437252+0000", 2024-04-24T15:33:43.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3110999999999999e-05 2024-04-24T15:33:43.775 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:43.775 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:43.775 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:43.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.775 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:52545 4.a 4:5331820a:::rbd_data.114ab867d54d.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-04-24T15:33:43.775 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:14.399696+0000", 2024-04-24T15:33:43.775 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.297766306, 2024-04-24T15:33:43.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016877893000000001, 2024-04-24T15:33:43.776 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:43.776 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:43.776 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:43.776 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:43.776 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:43.776 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 52545 2024-04-24T15:33:43.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.776 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:43.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:43.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.399696+0000", 2024-04-24T15:33:43.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:43.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:43.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.399696+0000", 2024-04-24T15:33:43.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:43.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:43.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.399695+0000", 2024-04-24T15:33:43.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999981 2024-04-24T15:33:43.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:43.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.399704+0000", 2024-04-24T15:33:43.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.098e-06 2024-04-24T15:33:43.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:43.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.399721+0000", 2024-04-24T15:33:43.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7088999999999999e-05 2024-04-24T15:33:43.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:43.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.399728+0000", 2024-04-24T15:33:43.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5699999999999998e-06 2024-04-24T15:33:43.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:43.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.399775+0000", 2024-04-24T15:33:43.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7445999999999999e-05 2024-04-24T15:33:43.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:43.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.399840+0000", 2024-04-24T15:33:43.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5492000000000002e-05 2024-04-24T15:33:43.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:33:43.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.416468+0000", 2024-04-24T15:33:43.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016627527 2024-04-24T15:33:43.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:43.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.416521+0000", 2024-04-24T15:33:43.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2988000000000002e-05 2024-04-24T15:33:43.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:43.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.416574+0000", 2024-04-24T15:33:43.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3363999999999999e-05 2024-04-24T15:33:43.780 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:43.780 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:43.780 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:43.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.780 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:66966 4.1f 4:fb3b69af:::rbd_data.114aed92ca6e.000000000000001a:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-04-24T15:33:43.780 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:21.265525+0000", 2024-04-24T15:33:43.780 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 22.431937765000001, 2024-04-24T15:33:43.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018862027999999999, 2024-04-24T15:33:43.780 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:43.780 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:43.781 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:43.781 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:43.781 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:43.781 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 66966 2024-04-24T15:33:43.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.781 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:43.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:43.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:21.265525+0000", 2024-04-24T15:33:43.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:43.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:43.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:21.265525+0000", 2024-04-24T15:33:43.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:43.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:43.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:21.265523+0000", 2024-04-24T15:33:43.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999981 2024-04-24T15:33:43.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:43.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:21.265536+0000", 2024-04-24T15:33:43.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2489e-05 2024-04-24T15:33:43.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:43.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:21.265550+0000", 2024-04-24T15:33:43.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4218e-05 2024-04-24T15:33:43.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:43.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:21.265556+0000", 2024-04-24T15:33:43.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.241e-06 2024-04-24T15:33:43.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:43.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:21.265606+0000", 2024-04-24T15:33:43.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9972000000000002e-05 2024-04-24T15:33:43.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:43.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:21.265670+0000", 2024-04-24T15:33:43.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3425999999999999e-05 2024-04-24T15:33:43.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:33:43.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:21.284297+0000", 2024-04-24T15:33:43.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018627517999999999 2024-04-24T15:33:43.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:43.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:21.284339+0000", 2024-04-24T15:33:43.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1628000000000002e-05 2024-04-24T15:33:43.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:43.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:43.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:43.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:21.284387+0000", 2024-04-24T15:33:43.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8242000000000003e-05 2024-04-24T15:33:43.785 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:43.785 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:43.785 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:43.785 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:43.785 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:43.785 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:43.786 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-24T15:33:43.797 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:33:43.798+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:33:43.898 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:43.898+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:43.951 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:43.951 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:33:43.951 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-04-24T15:33:43.951 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:43.960 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-24T15:33:43.994 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-24T15:33:43.999 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:33:44.000+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:33:44.100 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:33:44.100+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:33:44.128 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:44.128 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:33:44.128 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-04-24T15:33:44.128 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-04-24T15:33:44.128 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:44.141 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-24T15:33:44.224 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:33:44.225+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:33:44.302 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:44.302+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:44.306 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:44.306 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-04-24T15:33:44.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-04-24T15:33:44.306 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [] 2024-04-24T15:33:44.307 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:44.316 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-24T15:33:44.402 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:33:44.404+0000 7f0f45efb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 59992) UID: 0 2024-04-24T15:33:44.410 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-24T15:33:44.484 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:44.484 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:33:44.484 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-04-24T15:33:44.484 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:44.495 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-24T15:33:44.538 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:33:44.539+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:33:44.604 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:33:44.605+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:33:44.622 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:44.622 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:33:44.623 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-04-24T15:33:44.623 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-04-24T15:33:44.623 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:44.632 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-24T15:33:44.647 INFO:teuthology.orchestra.run.smithi096.stderr:nodeep-scrub is set 2024-04-24T15:33:44.704 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:44.704+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:44.757 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-04-24T15:33:44.774 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:44.774 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-04-24T15:33:44.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-04-24T15:33:44.774 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-04-24T15:33:44.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.774 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:2522 4.6 4:60ceedf0:::rbd_data.114a1f60cb66.000000000000011d:head [set-alloc-hint object_size 262144 write_size 262144,write 110592~2048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-24T15:33:44.774 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:23.088240+0000", 2024-04-24T15:33:44.774 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 81.685841002999993, 2024-04-24T15:33:44.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.988232797, 2024-04-24T15:33:44.774 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:44.775 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:44.775 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:44.775 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:44.775 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:44.775 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2522 2024-04-24T15:33:44.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.775 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:44.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:44.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088240+0000", 2024-04-24T15:33:44.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:44.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:44.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088240+0000", 2024-04-24T15:33:44.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:44.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:44.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088238+0000", 2024-04-24T15:33:44.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999981 2024-04-24T15:33:44.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:44.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088254+0000", 2024-04-24T15:33:44.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5562000000000001e-05 2024-04-24T15:33:44.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:44.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088275+0000", 2024-04-24T15:33:44.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1256000000000001e-05 2024-04-24T15:33:44.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:44.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088283+0000", 2024-04-24T15:33:44.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3219999999999993e-06 2024-04-24T15:33:44.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:44.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088351+0000", 2024-04-24T15:33:44.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7595000000000006e-05 2024-04-24T15:33:44.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-04-24T15:33:44.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088362+0000", 2024-04-24T15:33:44.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1764e-05 2024-04-24T15:33:44.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:44.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.073748+0000", 2024-04-24T15:33:44.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98538541899999998 2024-04-24T15:33:44.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:44.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.073925+0000", 2024-04-24T15:33:44.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017747600000000001 2024-04-24T15:33:44.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-24T15:33:44.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.074092+0000", 2024-04-24T15:33:44.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000166622 2024-04-24T15:33:44.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:33:44.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.074774+0000", 2024-04-24T15:33:44.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00068225000000000002 2024-04-24T15:33:44.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:33:44.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.076412+0000", 2024-04-24T15:33:44.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001638189 2024-04-24T15:33:44.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:44.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.076446+0000", 2024-04-24T15:33:44.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3587000000000002e-05 2024-04-24T15:33:44.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:44.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.076473+0000", 2024-04-24T15:33:44.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6886999999999999e-05 2024-04-24T15:33:44.780 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:44.780 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:44.780 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:44.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.781 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43121 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 5865472~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:44.781 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094178+0000", 2024-04-24T15:33:44.781 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.679903320000001, 2024-04-24T15:33:44.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032860598999999997, 2024-04-24T15:33:44.781 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:44.781 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:44.781 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:44.781 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:44.781 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:44.781 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43121 2024-04-24T15:33:44.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.781 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:44.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:44.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094178+0000", 2024-04-24T15:33:44.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:44.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:44.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094178+0000", 2024-04-24T15:33:44.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:44.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:44.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094178+0000", 2024-04-24T15:33:44.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:44.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:44.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094180+0000", 2024-04-24T15:33:44.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9989999999999998e-06 2024-04-24T15:33:44.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:44.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094185+0000", 2024-04-24T15:33:44.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.541e-06 2024-04-24T15:33:44.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:44.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094189+0000", 2024-04-24T15:33:44.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0509999999999998e-06 2024-04-24T15:33:44.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:44.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.126877+0000", 2024-04-24T15:33:44.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032687473000000002 2024-04-24T15:33:44.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:44.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.126952+0000", 2024-04-24T15:33:44.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5924e-05 2024-04-24T15:33:44.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:44.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127038+0000", 2024-04-24T15:33:44.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5908999999999994e-05 2024-04-24T15:33:44.785 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:44.785 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:44.785 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:44.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.785 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43122 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 6291456~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:44.785 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094193+0000", 2024-04-24T15:33:44.785 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.679888075999997, 2024-04-24T15:33:44.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032978383999999999, 2024-04-24T15:33:44.785 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:44.785 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:44.785 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:44.785 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:44.785 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:44.785 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43122 2024-04-24T15:33:44.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.786 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:44.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:44.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094193+0000", 2024-04-24T15:33:44.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:44.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:44.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094193+0000", 2024-04-24T15:33:44.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:44.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:44.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094193+0000", 2024-04-24T15:33:44.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:44.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:44.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094195+0000", 2024-04-24T15:33:44.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3269999999999999e-06 2024-04-24T15:33:44.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:44.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094201+0000", 2024-04-24T15:33:44.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9850000000000003e-06 2024-04-24T15:33:44.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:44.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094205+0000", 2024-04-24T15:33:44.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8430000000000003e-06 2024-04-24T15:33:44.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:44.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127068+0000", 2024-04-24T15:33:44.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032863305000000002 2024-04-24T15:33:44.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:44.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127113+0000", 2024-04-24T15:33:44.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4412000000000001e-05 2024-04-24T15:33:44.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:44.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127171+0000", 2024-04-24T15:33:44.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8807999999999997e-05 2024-04-24T15:33:44.789 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:44.789 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:44.789 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:44.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.789 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43123 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 6717440~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:44.789 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094209+0000", 2024-04-24T15:33:44.789 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.679871910000003, 2024-04-24T15:33:44.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033144583999999998, 2024-04-24T15:33:44.789 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:44.789 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:44.789 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:44.790 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:44.790 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:44.790 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43123 2024-04-24T15:33:44.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.790 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:44.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:44.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094209+0000", 2024-04-24T15:33:44.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:44.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:44.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094209+0000", 2024-04-24T15:33:44.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:44.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:44.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094209+0000", 2024-04-24T15:33:44.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:44.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:44.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094211+0000", 2024-04-24T15:33:44.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2089999999999999e-06 2024-04-24T15:33:44.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:44.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094217+0000", 2024-04-24T15:33:44.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6799999999999998e-06 2024-04-24T15:33:44.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:44.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094220+0000", 2024-04-24T15:33:44.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.444e-06 2024-04-24T15:33:44.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:44.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127190+0000", 2024-04-24T15:33:44.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032969471 2024-04-24T15:33:44.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:44.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127269+0000", 2024-04-24T15:33:44.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9246999999999995e-05 2024-04-24T15:33:44.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:44.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127354+0000", 2024-04-24T15:33:44.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4853999999999998e-05 2024-04-24T15:33:44.793 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:44.793 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:44.793 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:44.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.793 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43124 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 7143424~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:44.793 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094224+0000", 2024-04-24T15:33:44.793 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.679856936999997, 2024-04-24T15:33:44.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033271955999999998, 2024-04-24T15:33:44.793 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:44.793 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:44.793 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:44.794 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:44.794 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:44.794 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43124 2024-04-24T15:33:44.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.794 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:44.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:44.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094224+0000", 2024-04-24T15:33:44.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:44.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:44.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094224+0000", 2024-04-24T15:33:44.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:44.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:44.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094224+0000", 2024-04-24T15:33:44.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:44.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:44.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094226+0000", 2024-04-24T15:33:44.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9039999999999999e-06 2024-04-24T15:33:44.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:44.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094233+0000", 2024-04-24T15:33:44.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7970000000000003e-06 2024-04-24T15:33:44.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:44.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094236+0000", 2024-04-24T15:33:44.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6339999999999999e-06 2024-04-24T15:33:44.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:44.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127390+0000", 2024-04-24T15:33:44.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.03315353 2024-04-24T15:33:44.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:44.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127439+0000", 2024-04-24T15:33:44.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8890000000000001e-05 2024-04-24T15:33:44.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:44.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127496+0000", 2024-04-24T15:33:44.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7476999999999999e-05 2024-04-24T15:33:44.797 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:44.797 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:44.797 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:44.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.797 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43125 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 7569408~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:44.797 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094240+0000", 2024-04-24T15:33:44.797 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.679840849999998, 2024-04-24T15:33:44.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033446278000000003, 2024-04-24T15:33:44.797 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:44.797 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:44.797 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:44.797 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:44.797 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:44.797 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43125 2024-04-24T15:33:44.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.798 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:44.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:44.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094240+0000", 2024-04-24T15:33:44.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:44.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:44.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094240+0000", 2024-04-24T15:33:44.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:44.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:44.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094240+0000", 2024-04-24T15:33:44.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:44.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:44.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094247+0000", 2024-04-24T15:33:44.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6209999999999998e-06 2024-04-24T15:33:44.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:44.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094253+0000", 2024-04-24T15:33:44.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0739999999999998e-06 2024-04-24T15:33:44.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:44.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094256+0000", 2024-04-24T15:33:44.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7699999999999999e-06 2024-04-24T15:33:44.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:44.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127526+0000", 2024-04-24T15:33:44.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033269703999999997 2024-04-24T15:33:44.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:44.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127603+0000", 2024-04-24T15:33:44.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6964000000000004e-05 2024-04-24T15:33:44.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:44.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127687+0000", 2024-04-24T15:33:44.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3431999999999997e-05 2024-04-24T15:33:44.801 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:44.801 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:44.801 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:44.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.801 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43126 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 7995392~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:44.801 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094261+0000", 2024-04-24T15:33:44.801 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.679820347000003, 2024-04-24T15:33:44.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033555201999999999, 2024-04-24T15:33:44.801 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:44.802 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:44.802 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:44.802 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:44.802 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:44.802 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43126 2024-04-24T15:33:44.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.802 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:44.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.802 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:44.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094261+0000", 2024-04-24T15:33:44.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:44.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:44.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094261+0000", 2024-04-24T15:33:44.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:44.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:44.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094260+0000", 2024-04-24T15:33:44.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:44.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:44.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094263+0000", 2024-04-24T15:33:44.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2579999999999998e-06 2024-04-24T15:33:44.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:44.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094269+0000", 2024-04-24T15:33:44.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.823e-06 2024-04-24T15:33:44.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:44.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094272+0000", 2024-04-24T15:33:44.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8999999999999999e-06 2024-04-24T15:33:44.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:44.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127720+0000", 2024-04-24T15:33:44.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033447521000000001 2024-04-24T15:33:44.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.805 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:44.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127765+0000", 2024-04-24T15:33:44.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5352000000000002e-05 2024-04-24T15:33:44.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.805 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:44.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127816+0000", 2024-04-24T15:33:44.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0689999999999997e-05 2024-04-24T15:33:44.805 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:44.805 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:44.805 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:44.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.806 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63800 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1032192~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:44.806 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555407+0000", 2024-04-24T15:33:44.806 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 25.218674361000001, 2024-04-24T15:33:44.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032809030000000003, 2024-04-24T15:33:44.806 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:44.806 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:44.806 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:44.806 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:44.806 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:44.806 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63800 2024-04-24T15:33:44.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.806 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:44.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.806 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:44.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555407+0000", 2024-04-24T15:33:44.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:44.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:44.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555407+0000", 2024-04-24T15:33:44.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:44.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:44.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555406+0000", 2024-04-24T15:33:44.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:44.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:44.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555409+0000", 2024-04-24T15:33:44.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.706e-06 2024-04-24T15:33:44.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:44.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555416+0000", 2024-04-24T15:33:44.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1609999999999997e-06 2024-04-24T15:33:44.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:44.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555420+0000", 2024-04-24T15:33:44.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0369999999999996e-06 2024-04-24T15:33:44.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:44.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588088+0000", 2024-04-24T15:33:44.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032668228000000001 2024-04-24T15:33:44.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:44.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588143+0000", 2024-04-24T15:33:44.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4852999999999998e-05 2024-04-24T15:33:44.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:44.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588216+0000", 2024-04-24T15:33:44.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2467999999999998e-05 2024-04-24T15:33:44.810 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:44.810 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:44.810 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:44.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.810 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.810 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63801 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1114112~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:44.810 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555425+0000", 2024-04-24T15:33:44.810 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 25.218656454000001, 2024-04-24T15:33:44.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032924691999999998, 2024-04-24T15:33:44.810 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:44.810 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:44.810 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:44.811 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:44.811 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:44.811 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63801 2024-04-24T15:33:44.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.811 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:44.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:44.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555425+0000", 2024-04-24T15:33:44.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:44.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:44.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555425+0000", 2024-04-24T15:33:44.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:44.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:44.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555424+0000", 2024-04-24T15:33:44.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:44.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:44.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555427+0000", 2024-04-24T15:33:44.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2350000000000002e-06 2024-04-24T15:33:44.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:44.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555432+0000", 2024-04-24T15:33:44.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.434e-06 2024-04-24T15:33:44.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:44.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555435+0000", 2024-04-24T15:33:44.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2009999999999999e-06 2024-04-24T15:33:44.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:44.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588237+0000", 2024-04-24T15:33:44.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032801508 2024-04-24T15:33:44.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:44.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588281+0000", 2024-04-24T15:33:44.814 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4100999999999997e-05 2024-04-24T15:33:44.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.814 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:44.814 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588349+0000", 2024-04-24T15:33:44.814 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8551000000000001e-05 2024-04-24T15:33:44.814 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:44.814 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:44.814 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:44.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.814 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63802 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1196032~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:44.814 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555439+0000", 2024-04-24T15:33:44.814 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 25.218641933000001, 2024-04-24T15:33:44.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033057744999999999, 2024-04-24T15:33:44.815 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:44.815 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:44.815 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:44.815 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:44.815 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:44.815 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63802 2024-04-24T15:33:44.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.815 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:44.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:44.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555439+0000", 2024-04-24T15:33:44.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:44.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:44.816 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555439+0000", 2024-04-24T15:33:44.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:44.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:44.816 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555439+0000", 2024-04-24T15:33:44.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:44.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:44.816 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555445+0000", 2024-04-24T15:33:44.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4019999999999999e-06 2024-04-24T15:33:44.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:44.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555452+0000", 2024-04-24T15:33:44.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8600000000000004e-06 2024-04-24T15:33:44.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:44.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555461+0000", 2024-04-24T15:33:44.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0499999999999997e-06 2024-04-24T15:33:44.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:44.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588378+0000", 2024-04-24T15:33:44.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032916527000000001 2024-04-24T15:33:44.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.818 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:44.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588423+0000", 2024-04-24T15:33:44.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5092999999999997e-05 2024-04-24T15:33:44.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.818 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:44.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588497+0000", 2024-04-24T15:33:44.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4140999999999994e-05 2024-04-24T15:33:44.818 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:44.818 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:44.818 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:44.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.819 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63803 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1277952~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:44.819 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555466+0000", 2024-04-24T15:33:44.819 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 25.218614987999999, 2024-04-24T15:33:44.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033162297, 2024-04-24T15:33:44.819 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:44.819 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:44.819 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:44.819 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:44.819 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:44.819 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63803 2024-04-24T15:33:44.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.819 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:44.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.820 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:44.820 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555466+0000", 2024-04-24T15:33:44.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:44.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.820 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:44.820 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555466+0000", 2024-04-24T15:33:44.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:44.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.820 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:44.820 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555466+0000", 2024-04-24T15:33:44.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:44.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:44.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555468+0000", 2024-04-24T15:33:44.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4040000000000002e-06 2024-04-24T15:33:44.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:44.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555475+0000", 2024-04-24T15:33:44.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9480000000000002e-06 2024-04-24T15:33:44.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:44.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555479+0000", 2024-04-24T15:33:44.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0289999999999999e-06 2024-04-24T15:33:44.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.822 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:44.822 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588516+0000", 2024-04-24T15:33:44.822 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033037094000000003 2024-04-24T15:33:44.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.822 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:44.822 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588562+0000", 2024-04-24T15:33:44.822 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6215999999999999e-05 2024-04-24T15:33:44.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.822 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:44.822 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588628+0000", 2024-04-24T15:33:44.822 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5971999999999998e-05 2024-04-24T15:33:44.822 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:44.823 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:44.823 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:44.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.823 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63804 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1359872~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:44.823 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555483+0000", 2024-04-24T15:33:44.823 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 25.218597820999999, 2024-04-24T15:33:44.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033287068000000003, 2024-04-24T15:33:44.823 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:44.823 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:44.823 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:44.823 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:44.823 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:44.823 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63804 2024-04-24T15:33:44.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.824 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:44.824 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.824 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:44.824 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555483+0000", 2024-04-24T15:33:44.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:44.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.824 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.824 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:44.824 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555483+0000", 2024-04-24T15:33:44.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:44.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.824 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.824 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:44.824 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555483+0000", 2024-04-24T15:33:44.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:44.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.825 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.825 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:44.825 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555485+0000", 2024-04-24T15:33:44.825 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0710000000000002e-06 2024-04-24T15:33:44.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.825 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.825 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:44.825 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555498+0000", 2024-04-24T15:33:44.825 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2748999999999999e-05 2024-04-24T15:33:44.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.825 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.825 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:44.825 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555502+0000", 2024-04-24T15:33:44.825 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0219999999999998e-06 2024-04-24T15:33:44.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.826 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:44.826 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588657+0000", 2024-04-24T15:33:44.826 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033154683999999997 2024-04-24T15:33:44.826 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.826 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:44.826 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588702+0000", 2024-04-24T15:33:44.826 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5809000000000001e-05 2024-04-24T15:33:44.826 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.826 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:44.826 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588770+0000", 2024-04-24T15:33:44.826 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8056e-05 2024-04-24T15:33:44.827 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:44.827 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:44.827 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:44.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.827 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.827 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63805 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1441792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:44.827 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555508+0000", 2024-04-24T15:33:44.827 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 25.218572972, 2024-04-24T15:33:44.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033375952, 2024-04-24T15:33:44.827 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:44.827 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:44.827 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:44.827 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:44.827 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:44.827 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63805 2024-04-24T15:33:44.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.828 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:44.828 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.828 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:44.828 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555508+0000", 2024-04-24T15:33:44.828 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:44.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.828 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.828 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:44.828 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555508+0000", 2024-04-24T15:33:44.828 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:44.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.828 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.828 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:44.828 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555508+0000", 2024-04-24T15:33:44.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:44.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.829 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:44.829 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555510+0000", 2024-04-24T15:33:44.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.424e-06 2024-04-24T15:33:44.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.829 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:44.829 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555516+0000", 2024-04-24T15:33:44.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3730000000000001e-06 2024-04-24T15:33:44.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.830 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:44.830 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555519+0000", 2024-04-24T15:33:44.830 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.039e-06 2024-04-24T15:33:44.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.830 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:44.830 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588789+0000", 2024-04-24T15:33:44.830 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0332702 2024-04-24T15:33:44.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.830 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:44.830 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588833+0000", 2024-04-24T15:33:44.830 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4660999999999998e-05 2024-04-24T15:33:44.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.831 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.831 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:44.831 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588884+0000", 2024-04-24T15:33:44.831 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0661999999999997e-05 2024-04-24T15:33:44.831 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:44.831 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:44.831 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:44.831 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.831 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.831 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63806 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1523712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:44.831 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555523+0000", 2024-04-24T15:33:44.831 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 25.218558390999998, 2024-04-24T15:33:44.831 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033479653999999998, 2024-04-24T15:33:44.831 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:44.831 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:44.831 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:44.832 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:44.832 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:44.832 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63806 2024-04-24T15:33:44.832 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.832 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:44.832 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.832 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:44.832 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555523+0000", 2024-04-24T15:33:44.832 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:44.832 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.832 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.832 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:44.832 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555523+0000", 2024-04-24T15:33:44.832 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:44.832 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.832 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.833 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:44.833 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555522+0000", 2024-04-24T15:33:44.833 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:44.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.833 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:44.833 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555524+0000", 2024-04-24T15:33:44.833 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0700000000000001e-06 2024-04-24T15:33:44.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.833 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:44.833 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555530+0000", 2024-04-24T15:33:44.833 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5450000000000003e-06 2024-04-24T15:33:44.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.833 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:44.833 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555533+0000", 2024-04-24T15:33:44.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2200000000000001e-06 2024-04-24T15:33:44.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.834 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:44.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588908+0000", 2024-04-24T15:33:44.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033374556 2024-04-24T15:33:44.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.834 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:44.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588950+0000", 2024-04-24T15:33:44.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2361999999999999e-05 2024-04-24T15:33:44.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.834 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:44.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589002+0000", 2024-04-24T15:33:44.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2224e-05 2024-04-24T15:33:44.835 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:44.835 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:44.835 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:44.835 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.835 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.835 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63807 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1605632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:44.835 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555537+0000", 2024-04-24T15:33:44.835 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 25.218543875999998, 2024-04-24T15:33:44.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033577227000000001, 2024-04-24T15:33:44.835 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:44.835 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:44.835 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:44.835 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:44.835 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:44.836 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63807 2024-04-24T15:33:44.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.836 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:44.836 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.836 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:44.836 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555537+0000", 2024-04-24T15:33:44.836 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:44.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.836 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.836 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:44.836 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555537+0000", 2024-04-24T15:33:44.836 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:44.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.836 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.836 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:44.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555537+0000", 2024-04-24T15:33:44.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:44.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.837 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:44.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555539+0000", 2024-04-24T15:33:44.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9769999999999999e-06 2024-04-24T15:33:44.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.837 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:44.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555544+0000", 2024-04-24T15:33:44.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.468e-06 2024-04-24T15:33:44.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.837 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:44.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555548+0000", 2024-04-24T15:33:44.838 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4319999999999999e-06 2024-04-24T15:33:44.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.838 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:44.838 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589021+0000", 2024-04-24T15:33:44.838 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033473655999999997 2024-04-24T15:33:44.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.838 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:44.838 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589064+0000", 2024-04-24T15:33:44.838 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2397000000000001e-05 2024-04-24T15:33:44.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.838 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:44.838 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589114+0000", 2024-04-24T15:33:44.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0611e-05 2024-04-24T15:33:44.839 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:44.839 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:44.839 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:44.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.839 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63808 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1687552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:44.839 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555552+0000", 2024-04-24T15:33:44.839 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 25.218529184000001, 2024-04-24T15:33:44.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033686019999999997, 2024-04-24T15:33:44.839 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:44.839 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:44.839 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:44.839 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:44.839 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:44.839 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63808 2024-04-24T15:33:44.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.840 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:44.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.840 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:44.840 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555552+0000", 2024-04-24T15:33:44.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:44.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.840 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:44.840 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555552+0000", 2024-04-24T15:33:44.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:44.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.840 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:44.840 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555552+0000", 2024-04-24T15:33:44.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:44.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.841 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:44.841 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555554+0000", 2024-04-24T15:33:44.841 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2170000000000001e-06 2024-04-24T15:33:44.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.841 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:44.841 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555562+0000", 2024-04-24T15:33:44.841 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8779999999999998e-06 2024-04-24T15:33:44.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.841 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:44.841 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555565+0000", 2024-04-24T15:33:44.841 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4230000000000001e-06 2024-04-24T15:33:44.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.842 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.842 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:44.842 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589143+0000", 2024-04-24T15:33:44.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033577382000000003 2024-04-24T15:33:44.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.842 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.842 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:44.842 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589185+0000", 2024-04-24T15:33:44.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2206999999999998e-05 2024-04-24T15:33:44.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.842 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.842 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:44.842 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589238+0000", 2024-04-24T15:33:44.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3211000000000003e-05 2024-04-24T15:33:44.843 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:44.843 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:44.843 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:44.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.843 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63809 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1769472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:44.843 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555569+0000", 2024-04-24T15:33:44.843 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 25.218511642999999, 2024-04-24T15:33:44.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.03378068, 2024-04-24T15:33:44.843 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:44.843 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:44.843 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:44.843 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:44.843 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:44.843 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63809 2024-04-24T15:33:44.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.844 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:44.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.844 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:44.844 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555569+0000", 2024-04-24T15:33:44.844 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:44.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.844 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:44.844 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555569+0000", 2024-04-24T15:33:44.844 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:44.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.844 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:44.844 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555569+0000", 2024-04-24T15:33:44.844 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:44.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.845 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.845 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:44.845 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555571+0000", 2024-04-24T15:33:44.845 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0889999999999998e-06 2024-04-24T15:33:44.845 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.845 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.845 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:44.845 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555577+0000", 2024-04-24T15:33:44.845 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4149999999999998e-06 2024-04-24T15:33:44.845 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.845 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.845 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:44.845 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555580+0000", 2024-04-24T15:33:44.845 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2399999999999999e-06 2024-04-24T15:33:44.845 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.846 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.846 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:44.846 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589258+0000", 2024-04-24T15:33:44.846 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033677817999999998 2024-04-24T15:33:44.846 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.846 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.846 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:44.846 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589300+0000", 2024-04-24T15:33:44.846 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2317999999999997e-05 2024-04-24T15:33:44.846 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.846 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.846 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:44.846 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589350+0000", 2024-04-24T15:33:44.846 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0111000000000002e-05 2024-04-24T15:33:44.846 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:44.847 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:44.847 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:44.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.847 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.847 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63810 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1851392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:44.847 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555584+0000", 2024-04-24T15:33:44.847 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 25.218497172999999, 2024-04-24T15:33:44.847 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033886251999999999, 2024-04-24T15:33:44.847 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:44.847 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:44.847 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:44.847 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:44.847 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:44.847 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63810 2024-04-24T15:33:44.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.848 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:44.848 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.848 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:44.848 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555584+0000", 2024-04-24T15:33:44.848 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:44.848 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.848 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.848 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:44.848 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555584+0000", 2024-04-24T15:33:44.848 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:44.848 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.848 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.848 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:44.848 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555584+0000", 2024-04-24T15:33:44.848 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:44.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.849 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.849 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:44.849 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555586+0000", 2024-04-24T15:33:44.849 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0899999999999999e-06 2024-04-24T15:33:44.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.849 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.849 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:44.849 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555591+0000", 2024-04-24T15:33:44.849 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5260000000000001e-06 2024-04-24T15:33:44.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.849 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.849 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:44.849 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555595+0000", 2024-04-24T15:33:44.849 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.41e-06 2024-04-24T15:33:44.850 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.850 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.850 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:44.850 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589377+0000", 2024-04-24T15:33:44.850 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033781868 2024-04-24T15:33:44.850 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.850 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.850 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:44.850 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589418+0000", 2024-04-24T15:33:44.850 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1461000000000002e-05 2024-04-24T15:33:44.850 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.850 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.850 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:44.850 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589470+0000", 2024-04-24T15:33:44.851 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2194999999999997e-05 2024-04-24T15:33:44.851 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:44.851 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:44.851 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:44.851 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.851 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.851 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63811 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1933312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:44.851 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555599+0000", 2024-04-24T15:33:44.851 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 25.218482364, 2024-04-24T15:33:44.851 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033987493000000001, 2024-04-24T15:33:44.851 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:44.851 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:44.851 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:44.851 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:44.851 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:44.852 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63811 2024-04-24T15:33:44.852 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.852 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:44.852 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.852 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:44.852 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555599+0000", 2024-04-24T15:33:44.852 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:44.852 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.852 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.852 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:44.852 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555599+0000", 2024-04-24T15:33:44.852 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:44.852 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.852 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.852 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:44.853 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555598+0000", 2024-04-24T15:33:44.853 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:44.853 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.853 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.853 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:44.853 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555600+0000", 2024-04-24T15:33:44.853 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.04e-06 2024-04-24T15:33:44.853 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.853 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.853 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:44.853 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555606+0000", 2024-04-24T15:33:44.853 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4199999999999998e-06 2024-04-24T15:33:44.853 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.854 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.854 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:44.854 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555616+0000", 2024-04-24T15:33:44.854 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0054000000000001e-05 2024-04-24T15:33:44.854 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.854 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.854 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:44.854 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589493+0000", 2024-04-24T15:33:44.854 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033877009999999999 2024-04-24T15:33:44.854 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.854 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.854 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:44.854 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589536+0000", 2024-04-24T15:33:44.854 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2855999999999998e-05 2024-04-24T15:33:44.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.855 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.855 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:44.855 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589586+0000", 2024-04-24T15:33:44.855 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0420999999999997e-05 2024-04-24T15:33:44.855 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:44.855 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:44.855 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:44.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.855 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.855 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63812 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 2015232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:44.855 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555620+0000", 2024-04-24T15:33:44.855 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 25.218460726, 2024-04-24T15:33:44.855 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034081696000000002, 2024-04-24T15:33:44.855 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:44.855 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:44.856 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:44.856 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:44.856 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:44.856 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63812 2024-04-24T15:33:44.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.856 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:44.856 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.856 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:44.856 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555620+0000", 2024-04-24T15:33:44.856 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:44.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.856 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.856 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:44.856 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555620+0000", 2024-04-24T15:33:44.856 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:44.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.857 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.857 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:44.857 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555620+0000", 2024-04-24T15:33:44.857 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:44.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.857 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.857 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:44.857 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555625+0000", 2024-04-24T15:33:44.857 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2859999999999999e-06 2024-04-24T15:33:44.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.857 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.857 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:44.857 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555633+0000", 2024-04-24T15:33:44.857 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8879999999999997e-06 2024-04-24T15:33:44.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.858 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.858 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:44.858 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555637+0000", 2024-04-24T15:33:44.858 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9500000000000003e-06 2024-04-24T15:33:44.858 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.858 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.858 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:44.858 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589604+0000", 2024-04-24T15:33:44.858 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033966478000000001 2024-04-24T15:33:44.858 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.858 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.858 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:44.858 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589650+0000", 2024-04-24T15:33:44.858 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6721000000000002e-05 2024-04-24T15:33:44.858 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:44.859 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:44.859 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:44.859 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589702+0000", 2024-04-24T15:33:44.859 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1709000000000002e-05 2024-04-24T15:33:44.859 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:44.859 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:44.859 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:44.859 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:44.859 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:44.859 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:44.860 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-04-24T15:33:44.861 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-24T15:33:44.861 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-04-24T15:33:44.861 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-04-24T15:33:44.862 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-04-24T15:33:44.862 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-04-24T15:33:44.862 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:33:44.860+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:33:44.868 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-04-24T15:33:44.889 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-04-24T15:33:44.891 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T15:33:44.891 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T15:33:44.891 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T15:33:44.891 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T15:33:44.891 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T15:33:44.891 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T15:33:44.891 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T15:33:44.891 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T15:33:44.961 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:33:44.961+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:33:44.992 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:44.992 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:45.001 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-24T15:33:45.086 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:45.086+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:45.163 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:45.164+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:45.164 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:45.164 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:45.174 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-24T15:33:45.264 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:45.264+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:45.337 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:45.337 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:45.347 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-24T15:33:45.388 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:33:45.388+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:33:45.465 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:33:45.466+0000 7f0f45efb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 59992) UID: 0 2024-04-24T15:33:45.483 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:45.483 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:45.493 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-24T15:33:45.577 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:33:45.578+0000 7f0f45efb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 59992) UID: 0 2024-04-24T15:33:45.661 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:45.662 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:45.667 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:45.668+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:45.676 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-24T15:33:45.769 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:45.769+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:45.847 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:45.848 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:45.857 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-24T15:33:45.870 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:45.870+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:45.970 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:33:45.971+0000 7f0f45efb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 59992) UID: 0 2024-04-24T15:33:45.980 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:45.980 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:45.990 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-24T15:33:46.071 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:46.072+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:46.153 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:46.153 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:46.162 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-24T15:33:46.206 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:46.207+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:46.210 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 6, 4] out_osds: [1, 5, 7] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 7, 6] 2024-04-24T15:33:46.210 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-24T15:33:46.210 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-04-24T15:33:46.210 INFO:tasks.thrashosds.thrasher:Setting osd 4 primary_affinity to 0.175896 2024-04-24T15:33:46.210 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 0.1758957724165392 2024-04-24T15:33:46.273 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:33:46.273+0000 7f0f45efb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 59992) UID: 0 2024-04-24T15:33:46.285 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:46.285 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:46.295 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-24T15:33:46.379 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:46.380+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:46.458 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:46.459 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:46.468 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-24T15:33:46.509 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:46.510+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:46.575 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:33:46.576+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:33:46.631 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:46.631 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:46.641 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-24T15:33:46.659 INFO:teuthology.orchestra.run.smithi096.stderr:set osd.4 primary-affinity to 0.175896 (8115272) 2024-04-24T15:33:46.662 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-24T15:33:46.684 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:33:46.685+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:33:46.768 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:46.768 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:46.776 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:33:46.776+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:33:46.778 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-24T15:33:46.877 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:33:46.877+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:33:46.902 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-24T15:33:46.941 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:46.941 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:46.951 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-24T15:33:46.994 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:46.995+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:47.042 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-04-24T15:33:47.059 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-04-24T15:33:47.074 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:47.074 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:47.077 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-04-24T15:33:47.078 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:33:47.078+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:33:47.084 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-24T15:33:47.095 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-04-24T15:33:47.113 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-04-24T15:33:47.131 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-04-24T15:33:47.148 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-04-24T15:33:47.166 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-04-24T15:33:47.168 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T15:33:47.168 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T15:33:47.168 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T15:33:47.168 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T15:33:47.168 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T15:33:47.168 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T15:33:47.168 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T15:33:47.168 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T15:33:47.178 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:47.178+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:47.250 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:47.250 INFO:teuthology.orchestra.run.smithi151.stdout: "size": 20, 2024-04-24T15:33:47.250 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 600, 2024-04-24T15:33:47.250 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [ 2024-04-24T15:33:47.250 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.251 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:75519 4.a 4.2a5fcea (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e104)", 2024-04-24T15:33:47.251 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:42.066841+0000", 2024-04-24T15:33:47.251 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 5.1832916610000002, 2024-04-24T15:33:47.251 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.57234695599999996, 2024-04-24T15:33:47.251 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:47.251 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "reached pg", 2024-04-24T15:33:47.251 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:47.251 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:47.251 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:47.251 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 75519 2024-04-24T15:33:47.251 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.251 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:47.251 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.251 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:47.251 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.066841+0000", 2024-04-24T15:33:47.252 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:47.252 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.252 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.252 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:47.252 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.066841+0000", 2024-04-24T15:33:47.252 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:47.252 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.252 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.252 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:47.252 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.066840+0000", 2024-04-24T15:33:47.252 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999999 2024-04-24T15:33:47.252 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.252 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.252 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:47.253 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.066846+0000", 2024-04-24T15:33:47.253 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6289999999999998e-06 2024-04-24T15:33:47.253 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.253 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.253 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:47.253 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.066854+0000", 2024-04-24T15:33:47.253 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.9219999999999997e-06 2024-04-24T15:33:47.253 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.253 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.253 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:47.253 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.066861+0000", 2024-04-24T15:33:47.253 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.6760000000000001e-06 2024-04-24T15:33:47.253 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.253 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.254 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:47.254 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.066915+0000", 2024-04-24T15:33:47.254 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.4083999999999999e-05 2024-04-24T15:33:47.254 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.254 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.254 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for peered", 2024-04-24T15:33:47.254 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.066924+0000", 2024-04-24T15:33:47.254 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.0350000000000007e-06 2024-04-24T15:33:47.254 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.254 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.254 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:47.254 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.639176+0000", 2024-04-24T15:33:47.254 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.57225240700000002 2024-04-24T15:33:47.254 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.254 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.255 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:47.255 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.639188+0000", 2024-04-24T15:33:47.255 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.2377e-05 2024-04-24T15:33:47.255 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:47.255 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:47.255 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:47.255 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.255 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.255 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:75520 4.5 4.787c8fa5 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e104)", 2024-04-24T15:33:47.255 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:42.066869+0000", 2024-04-24T15:33:47.255 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 5.1832637610000001, 2024-04-24T15:33:47.255 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.57110218499999998, 2024-04-24T15:33:47.255 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:47.255 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "reached pg", 2024-04-24T15:33:47.256 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:47.256 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:47.256 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:47.256 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 75520 2024-04-24T15:33:47.256 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.256 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:47.256 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.256 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:47.256 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.066869+0000", 2024-04-24T15:33:47.256 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:47.256 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.256 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.256 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:47.256 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.066869+0000", 2024-04-24T15:33:47.256 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:47.257 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.257 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.257 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:47.257 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.066869+0000", 2024-04-24T15:33:47.257 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:47.257 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.257 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.257 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:47.257 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.066872+0000", 2024-04-24T15:33:47.257 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.9840000000000001e-06 2024-04-24T15:33:47.257 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.257 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.257 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:47.257 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.066883+0000", 2024-04-24T15:33:47.257 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0883000000000001e-05 2024-04-24T15:33:47.258 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.258 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.258 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:47.258 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.066887+0000", 2024-04-24T15:33:47.258 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3309999999999997e-06 2024-04-24T15:33:47.258 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.258 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.258 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:47.258 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.066958+0000", 2024-04-24T15:33:47.258 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1290000000000004e-05 2024-04-24T15:33:47.258 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.258 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.258 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for peered", 2024-04-24T15:33:47.258 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.066969+0000", 2024-04-24T15:33:47.259 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0927000000000001e-05 2024-04-24T15:33:47.259 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.259 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.259 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:47.259 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.637952+0000", 2024-04-24T15:33:47.259 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.57098265400000003 2024-04-24T15:33:47.259 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.259 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.259 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:47.259 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.637972+0000", 2024-04-24T15:33:47.259 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.9519000000000001e-05 2024-04-24T15:33:47.259 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:47.259 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:47.259 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:47.260 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.260 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.260 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_repop(client.4426.0:75517 4.18 e105/104 4:1d3ff9a6:::rbd_data.114a858e6641.0000000000000003:head v 105'404, mlcod=0'0)", 2024-04-24T15:33:47.260 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:42.643751+0000", 2024-04-24T15:33:47.260 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 4.6063817880000002, 2024-04-24T15:33:47.260 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0060238999999999996, 2024-04-24T15:33:47.260 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:47.260 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:47.260 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:47.260 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.260 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:47.260 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.643751+0000", 2024-04-24T15:33:47.260 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:47.260 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.261 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.261 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:47.261 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.643751+0000", 2024-04-24T15:33:47.261 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:47.261 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.261 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.261 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:47.261 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.643751+0000", 2024-04-24T15:33:47.261 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:47.261 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.261 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.261 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:47.261 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.643762+0000", 2024-04-24T15:33:47.261 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0845e-05 2024-04-24T15:33:47.262 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.262 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.262 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:47.262 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.643770+0000", 2024-04-24T15:33:47.262 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7149999999999998e-06 2024-04-24T15:33:47.262 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.262 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.262 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:47.262 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.643777+0000", 2024-04-24T15:33:47.262 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.029e-06 2024-04-24T15:33:47.262 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.262 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.262 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:47.262 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.645926+0000", 2024-04-24T15:33:47.263 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0021492600000000001 2024-04-24T15:33:47.263 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.263 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.263 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:47.263 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.645948+0000", 2024-04-24T15:33:47.263 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.1837000000000001e-05 2024-04-24T15:33:47.263 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.263 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.263 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:33:47.263 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.649745+0000", 2024-04-24T15:33:47.263 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0037970719999999999 2024-04-24T15:33:47.263 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.263 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.263 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:47.263 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.649775+0000", 2024-04-24T15:33:47.264 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.0480999999999998e-05 2024-04-24T15:33:47.264 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:47.264 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:47.264 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:47.264 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.264 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.265 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_repop(client.4426.0:75522 4.1 e105/104 4:83ace853:::rbd_data.114a858e6641.0000000000000008:head v 105'287, mlcod=0'0)", 2024-04-24T15:33:47.265 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:42.645889+0000", 2024-04-24T15:33:47.265 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 4.60424393, 2024-04-24T15:33:47.265 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0025720669999999999, 2024-04-24T15:33:47.265 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:47.265 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:47.265 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:47.265 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.265 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:47.265 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.645889+0000", 2024-04-24T15:33:47.266 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:47.266 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.266 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.266 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:47.266 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.645889+0000", 2024-04-24T15:33:47.266 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:47.266 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.266 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.266 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:47.266 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.645889+0000", 2024-04-24T15:33:47.266 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:47.266 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.267 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.267 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:47.267 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.645894+0000", 2024-04-24T15:33:47.267 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.587e-06 2024-04-24T15:33:47.267 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.267 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.267 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:47.267 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.645903+0000", 2024-04-24T15:33:47.267 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.9090000000000006e-06 2024-04-24T15:33:47.267 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.267 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.267 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:47.267 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.645910+0000", 2024-04-24T15:33:47.267 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.5490000000000003e-06 2024-04-24T15:33:47.267 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.268 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.268 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:47.268 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.646318+0000", 2024-04-24T15:33:47.268 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00040830700000000001 2024-04-24T15:33:47.268 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.268 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.268 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:47.268 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.646337+0000", 2024-04-24T15:33:47.268 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.9108999999999999e-05 2024-04-24T15:33:47.268 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.268 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.268 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:33:47.268 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.648423+0000", 2024-04-24T15:33:47.268 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0020859009999999998 2024-04-24T15:33:47.268 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.269 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.269 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:47.269 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.648461+0000", 2024-04-24T15:33:47.269 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8271e-05 2024-04-24T15:33:47.269 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:47.269 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:47.269 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:47.269 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.269 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.269 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_repop(client.4426.0:75514 4.13 e105/104 4:cfbf367e:::rbd_data.114a858e6641.0000000000000000:head v 105'224, mlcod=0'0)", 2024-04-24T15:33:47.269 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:42.646298+0000", 2024-04-24T15:33:47.269 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 4.603835492, 2024-04-24T15:33:47.269 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.002490969, 2024-04-24T15:33:47.269 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:47.270 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:47.270 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:47.270 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.270 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:47.270 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.646298+0000", 2024-04-24T15:33:47.270 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:47.270 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.270 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.270 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:47.270 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.646298+0000", 2024-04-24T15:33:47.270 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:47.270 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.270 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.270 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:47.271 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.646297+0000", 2024-04-24T15:33:47.271 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:47.271 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.271 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.271 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:47.271 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.646314+0000", 2024-04-24T15:33:47.271 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.6444999999999999e-05 2024-04-24T15:33:47.271 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.271 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.271 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:47.271 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.646322+0000", 2024-04-24T15:33:47.271 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.8289999999999997e-06 2024-04-24T15:33:47.271 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.271 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.271 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:47.272 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.646327+0000", 2024-04-24T15:33:47.272 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.0359999999999997e-06 2024-04-24T15:33:47.272 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.272 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.272 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:47.272 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.646359+0000", 2024-04-24T15:33:47.272 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.2014000000000002e-05 2024-04-24T15:33:47.272 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.272 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.272 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:47.272 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.646383+0000", 2024-04-24T15:33:47.272 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.3898000000000001e-05 2024-04-24T15:33:47.272 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.272 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.272 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:33:47.273 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.648759+0000", 2024-04-24T15:33:47.273 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0023756620000000002 2024-04-24T15:33:47.273 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.273 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.273 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:47.273 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.648789+0000", 2024-04-24T15:33:47.273 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.9536999999999999e-05 2024-04-24T15:33:47.273 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:47.273 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:47.273 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:47.273 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.273 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.273 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "MOSDPGPush(4.12 105/104 [PushOp(4:4c712395:::rbd_data.114ab402c616.0000000000000006:1a, version: 79'279, data_included: [0~655360,720896~131072,917504~131072], data_size: 917504, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(4:4c712395:::rbd_data.114ab402c616.0000000000000006:1a@79'279, size: 2097152, copy_subset: [0~2097152], clone_subset: {}, snapset: 1a=[]:{1a=[1a]}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:2097152, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-04-24T15:33:47.273 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:42.651483+0000", 2024-04-24T15:33:47.273 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 4.5986496859999999, 2024-04-24T15:33:47.274 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.002709309, 2024-04-24T15:33:47.274 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:47.274 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:47.274 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:47.274 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.274 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:47.274 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.651483+0000", 2024-04-24T15:33:47.274 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:47.274 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.274 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.274 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:47.274 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.651483+0000", 2024-04-24T15:33:47.274 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:47.274 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.275 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.275 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:47.275 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.651483+0000", 2024-04-24T15:33:47.275 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:47.275 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.275 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.275 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:47.275 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.652426+0000", 2024-04-24T15:33:47.275 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00094301300000000003 2024-04-24T15:33:47.275 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.275 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.275 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:47.275 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.652442+0000", 2024-04-24T15:33:47.275 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.5563999999999999e-05 2024-04-24T15:33:47.275 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.276 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.276 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:47.276 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.652454+0000", 2024-04-24T15:33:47.276 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.2262999999999999e-05 2024-04-24T15:33:47.276 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.276 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.276 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:47.276 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.652521+0000", 2024-04-24T15:33:47.276 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.7073e-05 2024-04-24T15:33:47.276 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.276 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.276 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:47.276 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.652530+0000", 2024-04-24T15:33:47.276 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.465e-06 2024-04-24T15:33:47.276 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.277 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.277 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:47.277 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.654193+0000", 2024-04-24T15:33:47.277 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0016623740000000001 2024-04-24T15:33:47.277 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:47.277 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:47.277 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:47.277 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.277 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.277 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_repop(osd.4.0:2759 4.18 e105/104 4:1fc87759:::rbd_data.114a8b543e91.0000000000000002:18 v 105'408, mlcod=105'404)", 2024-04-24T15:33:47.277 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:42.674602+0000", 2024-04-24T15:33:47.277 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 4.575530648, 2024-04-24T15:33:47.277 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0043182719999999997, 2024-04-24T15:33:47.277 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:47.278 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:47.278 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:47.278 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.278 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:47.278 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.674602+0000", 2024-04-24T15:33:47.278 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:47.278 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.278 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.278 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:47.278 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.674602+0000", 2024-04-24T15:33:47.278 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:47.278 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.278 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.278 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:47.279 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.674602+0000", 2024-04-24T15:33:47.279 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:47.279 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.279 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.279 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:47.279 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.674609+0000", 2024-04-24T15:33:47.279 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.6889999999999998e-06 2024-04-24T15:33:47.279 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.279 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.279 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:47.279 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.674618+0000", 2024-04-24T15:33:47.279 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.1130000000000007e-06 2024-04-24T15:33:47.279 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.279 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.279 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:47.280 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.674623+0000", 2024-04-24T15:33:47.280 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.3959999999999997e-06 2024-04-24T15:33:47.280 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.280 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.280 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:47.280 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.674657+0000", 2024-04-24T15:33:47.280 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.3847999999999998e-05 2024-04-24T15:33:47.280 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.280 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.280 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:47.280 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.674673+0000", 2024-04-24T15:33:47.280 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.5619000000000001e-05 2024-04-24T15:33:47.280 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.280 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.281 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:33:47.281 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.678897+0000", 2024-04-24T15:33:47.281 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0042239280000000001 2024-04-24T15:33:47.281 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.281 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.281 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:47.281 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.678921+0000", 2024-04-24T15:33:47.281 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.4178000000000001e-05 2024-04-24T15:33:47.281 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:47.281 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:47.281 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:47.281 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.281 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.281 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_repop(osd.4.0:2760 4.18 e105/104 4:1c88e57e:::rbd_data.114ab402c616.000000000000001a:1a v 105'410, mlcod=105'406)", 2024-04-24T15:33:47.281 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:42.682611+0000", 2024-04-24T15:33:47.282 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 4.5675223630000001, 2024-04-24T15:33:47.282 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0050158809999999998, 2024-04-24T15:33:47.282 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:47.282 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:47.282 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:47.282 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.282 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:47.282 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.682611+0000", 2024-04-24T15:33:47.282 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:47.282 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.282 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.282 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:47.282 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.682611+0000", 2024-04-24T15:33:47.282 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:47.283 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.283 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.283 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:47.283 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.682610+0000", 2024-04-24T15:33:47.283 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:47.283 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.283 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.283 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:47.283 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.682616+0000", 2024-04-24T15:33:47.283 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.207e-06 2024-04-24T15:33:47.283 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.283 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.283 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:47.283 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.682624+0000", 2024-04-24T15:33:47.283 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.9410000000000007e-06 2024-04-24T15:33:47.284 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.284 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.284 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:47.284 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.682629+0000", 2024-04-24T15:33:47.284 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.9779999999999996e-06 2024-04-24T15:33:47.284 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.284 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.284 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:47.284 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.682665+0000", 2024-04-24T15:33:47.284 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5581999999999999e-05 2024-04-24T15:33:47.284 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.284 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.284 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:47.284 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.682684+0000", 2024-04-24T15:33:47.285 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.8788999999999999e-05 2024-04-24T15:33:47.285 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.285 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.285 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:33:47.285 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.687594+0000", 2024-04-24T15:33:47.285 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0049105490000000002 2024-04-24T15:33:47.285 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.285 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.285 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:47.285 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.687627+0000", 2024-04-24T15:33:47.285 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.2396e-05 2024-04-24T15:33:47.285 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:47.285 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:47.285 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:47.285 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.286 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.286 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "MOSDPGPush(4.10 105/104 [PushOp(4:0ef81912:::rbd_data.114aee5fd3a9.000000000000000e:1c, version: 81'181, data_included: [0~786432,1310720~9581], data_size: 796013, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(4:0ef81912:::rbd_data.114aee5fd3a9.000000000000000e:1c@81'181, size: 6553600, copy_subset: [0~6553600], clone_subset: {}, snapset: 1c=[]:{1c=[1c]}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:6553600, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-04-24T15:33:47.286 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:42.690941+0000", 2024-04-24T15:33:47.286 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 4.5591924930000003, 2024-04-24T15:33:47.286 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0027029699999999999, 2024-04-24T15:33:47.286 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:47.286 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:47.286 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:47.286 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.286 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:47.286 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.690941+0000", 2024-04-24T15:33:47.286 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:47.286 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.286 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.287 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:47.287 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.690941+0000", 2024-04-24T15:33:47.287 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:47.287 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.287 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.287 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:47.287 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.690940+0000", 2024-04-24T15:33:47.287 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:47.287 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.287 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.287 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:47.287 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.691765+0000", 2024-04-24T15:33:47.287 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00082501799999999995 2024-04-24T15:33:47.287 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.287 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.288 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:47.288 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.691784+0000", 2024-04-24T15:33:47.288 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.9361999999999999e-05 2024-04-24T15:33:47.288 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.288 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.288 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:47.288 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.691795+0000", 2024-04-24T15:33:47.288 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.044e-05 2024-04-24T15:33:47.288 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.288 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.288 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:47.288 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.691874+0000", 2024-04-24T15:33:47.288 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.9474000000000005e-05 2024-04-24T15:33:47.288 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.289 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.289 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:47.289 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.691878+0000", 2024-04-24T15:33:47.289 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0849999999999999e-06 2024-04-24T15:33:47.289 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.289 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.289 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:47.289 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.693644+0000", 2024-04-24T15:33:47.289 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0017652 2024-04-24T15:33:47.289 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:47.289 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:47.289 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:47.289 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.289 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.289 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "MOSDPGPush(4.15 105/104 [PushOp(4:aca5755e:::rbd_data.114a8b543e91.0000000000000009:18, version: 77'224, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(4:aca5755e:::rbd_data.114a8b543e91.0000000000000009:18@77'224, size: 2097152, copy_subset: [0~2097152], clone_subset: {}, snapset: 18=[]:{18=[18]}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:2097152, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to:1048576, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2024-04-24T15:33:47.290 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:42.752086+0000", 2024-04-24T15:33:47.290 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 4.4980467730000004, 2024-04-24T15:33:47.290 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0025164599999999999, 2024-04-24T15:33:47.290 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:47.290 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:47.290 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:47.290 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.290 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:47.290 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.752086+0000", 2024-04-24T15:33:47.290 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:47.290 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.290 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.290 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:47.291 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.752086+0000", 2024-04-24T15:33:47.291 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:47.291 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.291 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.291 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:47.291 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.752086+0000", 2024-04-24T15:33:47.291 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:47.291 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.291 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.291 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:47.291 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.753322+0000", 2024-04-24T15:33:47.291 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0012367719999999999 2024-04-24T15:33:47.291 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.291 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.291 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:47.292 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.753343+0000", 2024-04-24T15:33:47.292 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.0194999999999999e-05 2024-04-24T15:33:47.292 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.292 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.292 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:47.292 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.753354+0000", 2024-04-24T15:33:47.292 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0889e-05 2024-04-24T15:33:47.292 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.292 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.292 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:47.292 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.753410+0000", 2024-04-24T15:33:47.292 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6752999999999997e-05 2024-04-24T15:33:47.292 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.292 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.292 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:47.293 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.753417+0000", 2024-04-24T15:33:47.293 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.5320000000000003e-06 2024-04-24T15:33:47.293 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.293 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.293 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:47.293 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.754603+0000", 2024-04-24T15:33:47.293 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.001185974 2024-04-24T15:33:47.293 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:47.293 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:47.293 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:47.293 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.293 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.293 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "MOSDPGPush(4.15 105/104 [PushOp(4:a939818e:::rbd_data.114a9052fca3.000000000000000b:1e, version: 83'240, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(4:a939818e:::rbd_data.114a9052fca3.000000000000000b:1e@83'240, size: 2097152, copy_subset: [0~2097152], clone_subset: {}, snapset: 1e=[]:{1e=[1e]}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:1048576, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-04-24T15:33:47.293 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:42.758490+0000", 2024-04-24T15:33:47.293 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 4.4916434240000003, 2024-04-24T15:33:47.294 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0024740919999999998, 2024-04-24T15:33:47.294 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:47.294 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:47.294 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:47.294 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.294 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:47.294 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.758490+0000", 2024-04-24T15:33:47.294 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:47.294 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.294 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.294 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:47.294 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.758490+0000", 2024-04-24T15:33:47.294 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:47.294 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.295 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.295 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:47.295 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.758484+0000", 2024-04-24T15:33:47.295 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999943 2024-04-24T15:33:47.295 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.295 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.295 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:47.295 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.759748+0000", 2024-04-24T15:33:47.295 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.001264083 2024-04-24T15:33:47.295 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.295 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.295 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:47.295 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.759763+0000", 2024-04-24T15:33:47.295 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.5221999999999999e-05 2024-04-24T15:33:47.295 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.296 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.296 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:47.296 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.759772+0000", 2024-04-24T15:33:47.296 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.2500000000000006e-06 2024-04-24T15:33:47.296 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.296 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.296 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:47.296 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.759828+0000", 2024-04-24T15:33:47.296 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.5942000000000001e-05 2024-04-24T15:33:47.296 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.296 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.296 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:47.296 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.759832+0000", 2024-04-24T15:33:47.296 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.7870000000000001e-06 2024-04-24T15:33:47.297 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.297 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.297 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:47.297 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.760964+0000", 2024-04-24T15:33:47.297 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.001131359 2024-04-24T15:33:47.297 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:47.297 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:47.297 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:47.297 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.297 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.297 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_repop(osd.4.0:2767 4.1 e105/104 4:827a7463:::rbd_data.114a8191d38d.0000000000000000:23 v 105'299, mlcod=105'295)", 2024-04-24T15:33:47.297 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:42.760819+0000", 2024-04-24T15:33:47.297 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 4.4893143909999997, 2024-04-24T15:33:47.297 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0051269829999999999, 2024-04-24T15:33:47.297 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:47.297 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:47.298 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:47.298 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.298 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:47.298 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.760819+0000", 2024-04-24T15:33:47.298 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:47.298 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.298 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.298 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:47.298 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.760819+0000", 2024-04-24T15:33:47.298 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:47.298 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.298 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.298 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:47.298 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.760818+0000", 2024-04-24T15:33:47.299 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:47.299 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.299 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.299 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:47.299 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.760826+0000", 2024-04-24T15:33:47.299 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7370000000000006e-06 2024-04-24T15:33:47.299 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.299 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.299 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:47.299 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.760837+0000", 2024-04-24T15:33:47.299 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.1516e-05 2024-04-24T15:33:47.299 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.299 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.299 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:47.300 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.760844+0000", 2024-04-24T15:33:47.300 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.1890000000000004e-06 2024-04-24T15:33:47.300 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.300 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.300 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:47.300 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.760881+0000", 2024-04-24T15:33:47.300 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.7642999999999999e-05 2024-04-24T15:33:47.300 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.300 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.300 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:47.300 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.760900+0000", 2024-04-24T15:33:47.300 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.9154e-05 2024-04-24T15:33:47.300 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.300 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.300 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:33:47.301 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.765915+0000", 2024-04-24T15:33:47.301 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0050150150000000003 2024-04-24T15:33:47.301 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.301 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.301 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:47.301 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.765946+0000", 2024-04-24T15:33:47.301 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.0388000000000001e-05 2024-04-24T15:33:47.301 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:47.301 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:47.301 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:47.301 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.301 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.301 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_repop(osd.4.0:2768 4.1 e105/104 4:8506c281:::rbd_data.114a8191d38d.0000000000000012:23 v 105'301, mlcod=105'295)", 2024-04-24T15:33:47.302 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:42.761567+0000", 2024-04-24T15:33:47.302 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 4.4885664470000002, 2024-04-24T15:33:47.302 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0069725309999999997, 2024-04-24T15:33:47.302 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:47.302 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:47.302 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:47.302 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.302 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:47.302 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.761567+0000", 2024-04-24T15:33:47.302 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:47.302 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.302 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.302 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:47.302 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.761567+0000", 2024-04-24T15:33:47.303 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:47.303 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.303 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.303 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:47.303 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.761566+0000", 2024-04-24T15:33:47.303 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:47.303 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.303 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.303 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:47.303 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.761571+0000", 2024-04-24T15:33:47.303 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.809e-06 2024-04-24T15:33:47.303 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.303 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.303 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:47.304 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.761578+0000", 2024-04-24T15:33:47.304 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.0829999999999998e-06 2024-04-24T15:33:47.304 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.304 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.304 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:47.304 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.761582+0000", 2024-04-24T15:33:47.304 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.1699999999999999e-06 2024-04-24T15:33:47.304 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.304 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.304 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:47.304 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.761607+0000", 2024-04-24T15:33:47.304 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.4946000000000001e-05 2024-04-24T15:33:47.304 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.304 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.304 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:47.305 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.761623+0000", 2024-04-24T15:33:47.305 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.5899000000000001e-05 2024-04-24T15:33:47.305 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.305 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.305 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:33:47.305 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.768508+0000", 2024-04-24T15:33:47.305 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0068846209999999996 2024-04-24T15:33:47.305 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.305 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.305 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:47.305 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.768539+0000", 2024-04-24T15:33:47.305 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.1525000000000001e-05 2024-04-24T15:33:47.305 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:47.305 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:47.305 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:47.306 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.306 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.306 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "MOSDPGPush(4.15 105/104 [PushOp(4:a939818e:::rbd_data.114a9052fca3.000000000000000b:1e, version: 83'240, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(4:a939818e:::rbd_data.114a9052fca3.000000000000000b:1e@83'240, size: 2097152, copy_subset: [0~2097152], clone_subset: {}, snapset: 1e=[]:{1e=[1e]}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:2097152, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to:1048576, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2024-04-24T15:33:47.306 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:42.766457+0000", 2024-04-24T15:33:47.306 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 4.4836764310000001, 2024-04-24T15:33:47.306 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00257128, 2024-04-24T15:33:47.306 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:47.306 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:47.306 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:47.306 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.306 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:47.306 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.766457+0000", 2024-04-24T15:33:47.306 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:47.306 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.307 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.307 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:47.307 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.766457+0000", 2024-04-24T15:33:47.307 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:47.307 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.307 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.307 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:47.307 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.766456+0000", 2024-04-24T15:33:47.307 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:47.307 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.307 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.307 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:47.307 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.767975+0000", 2024-04-24T15:33:47.307 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0015188650000000001 2024-04-24T15:33:47.308 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.308 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.308 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:47.308 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.767989+0000", 2024-04-24T15:33:47.308 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3994e-05 2024-04-24T15:33:47.308 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.308 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.308 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:47.308 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.767996+0000", 2024-04-24T15:33:47.308 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.255e-06 2024-04-24T15:33:47.308 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.308 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.308 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:47.308 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.768044+0000", 2024-04-24T15:33:47.309 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.7729999999999999e-05 2024-04-24T15:33:47.309 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.309 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.309 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:47.309 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.768050+0000", 2024-04-24T15:33:47.309 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.9660000000000001e-06 2024-04-24T15:33:47.309 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.309 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.309 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:47.309 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.769028+0000", 2024-04-24T15:33:47.309 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00097793899999999994 2024-04-24T15:33:47.309 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:47.309 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:47.309 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:47.310 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.310 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.310 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "MOSDPGPush(4.15 105/104 [PushOp(4:ad717d5c:::rbd_data.114a9052fca3.0000000000000002:1e, version: 83'243, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(4:ad717d5c:::rbd_data.114a9052fca3.0000000000000002:1e@83'243, size: 2097152, copy_subset: [0~2097152], clone_subset: {}, snapset: 1e=[]:{1e=[1e]}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:1048576, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-04-24T15:33:47.310 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:42.777619+0000", 2024-04-24T15:33:47.310 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 4.4725141199999996, 2024-04-24T15:33:47.310 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0025080139999999998, 2024-04-24T15:33:47.310 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:47.310 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:47.310 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:47.310 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.310 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:47.310 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.777619+0000", 2024-04-24T15:33:47.310 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:47.310 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.311 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.311 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:47.311 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.777619+0000", 2024-04-24T15:33:47.311 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:47.311 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.311 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.311 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:47.311 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.777619+0000", 2024-04-24T15:33:47.311 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:47.311 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.311 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.311 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:47.311 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.779004+0000", 2024-04-24T15:33:47.311 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.001385145 2024-04-24T15:33:47.312 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.312 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.312 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:47.312 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.779017+0000", 2024-04-24T15:33:47.312 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3668e-05 2024-04-24T15:33:47.312 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.312 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.312 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:47.312 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.779025+0000", 2024-04-24T15:33:47.312 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0439999999999996e-06 2024-04-24T15:33:47.312 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.312 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.312 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:47.312 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.779085+0000", 2024-04-24T15:33:47.312 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.9435999999999998e-05 2024-04-24T15:33:47.313 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.313 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.313 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:47.313 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.779093+0000", 2024-04-24T15:33:47.313 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7190000000000001e-06 2024-04-24T15:33:47.313 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.313 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.313 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:47.313 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.780127+0000", 2024-04-24T15:33:47.313 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.001034456 2024-04-24T15:33:47.313 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:47.313 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:47.313 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:47.313 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.314 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.314 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_repop(osd.4.0:2769 4.13 e105/104 4:cfbf367e:::rbd_data.114a858e6641.0000000000000000:17 v 105'226, mlcod=103'223)", 2024-04-24T15:33:47.314 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:42.815674+0000", 2024-04-24T15:33:47.314 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 4.4344595800000004, 2024-04-24T15:33:47.314 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0031768130000000001, 2024-04-24T15:33:47.314 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:47.314 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:47.314 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:47.314 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.314 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:47.314 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.815674+0000", 2024-04-24T15:33:47.314 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:47.314 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.315 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.315 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:47.315 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.815674+0000", 2024-04-24T15:33:47.315 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:47.315 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.315 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.315 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:47.315 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.815673+0000", 2024-04-24T15:33:47.315 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:47.315 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.315 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.315 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:47.315 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.815682+0000", 2024-04-24T15:33:47.315 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.6410000000000008e-06 2024-04-24T15:33:47.315 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.316 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.316 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:47.316 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.815691+0000", 2024-04-24T15:33:47.316 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.4199999999999996e-06 2024-04-24T15:33:47.316 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.316 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.316 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:47.316 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.815697+0000", 2024-04-24T15:33:47.316 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6300000000000003e-06 2024-04-24T15:33:47.316 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.316 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.316 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:47.316 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.815728+0000", 2024-04-24T15:33:47.316 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.1433999999999998e-05 2024-04-24T15:33:47.316 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.317 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.317 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:47.317 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.815747+0000", 2024-04-24T15:33:47.317 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.929e-05 2024-04-24T15:33:47.317 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.317 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.317 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:33:47.317 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.818807+0000", 2024-04-24T15:33:47.317 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0030592309999999999 2024-04-24T15:33:47.317 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.317 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.317 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:47.317 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.818850+0000", 2024-04-24T15:33:47.317 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3819000000000002e-05 2024-04-24T15:33:47.317 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:47.318 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:47.318 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:47.318 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.318 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.318 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_repop(osd.4.0:2770 4.13 e105/104 4:ce8f4ace:::rbd_data.114a9052fca3.0000000000000018:1e v 105'228, mlcod=105'224)", 2024-04-24T15:33:47.318 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:42.821450+0000", 2024-04-24T15:33:47.318 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 4.4286835130000002, 2024-04-24T15:33:47.318 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0025787380000000001, 2024-04-24T15:33:47.318 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:47.318 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:47.318 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:47.318 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.318 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:47.318 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.821450+0000", 2024-04-24T15:33:47.319 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:47.319 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.319 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.319 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:47.319 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.821450+0000", 2024-04-24T15:33:47.319 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:47.319 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.319 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.319 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:47.319 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.821449+0000", 2024-04-24T15:33:47.319 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:47.319 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.319 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.319 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:47.319 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.821456+0000", 2024-04-24T15:33:47.320 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2150000000000004e-06 2024-04-24T15:33:47.320 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.320 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.320 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:47.320 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.821467+0000", 2024-04-24T15:33:47.320 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.1188e-05 2024-04-24T15:33:47.320 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.320 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.320 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:47.320 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.821474+0000", 2024-04-24T15:33:47.320 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.905e-06 2024-04-24T15:33:47.320 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.320 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.320 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:47.320 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.821511+0000", 2024-04-24T15:33:47.321 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.6769e-05 2024-04-24T15:33:47.321 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.321 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.321 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:47.321 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.821537+0000", 2024-04-24T15:33:47.321 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.5782000000000002e-05 2024-04-24T15:33:47.321 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.321 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.321 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:33:47.321 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.824004+0000", 2024-04-24T15:33:47.321 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0024674929999999999 2024-04-24T15:33:47.321 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.321 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.321 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:47.321 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.824028+0000", 2024-04-24T15:33:47.321 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.3966000000000001e-05 2024-04-24T15:33:47.322 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:47.322 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:47.322 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:47.322 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.322 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.322 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_repop(client.4426.0:75515 4.e e106/105 4:765ea430:::rbd_data.114a858e6641.0000000000000001:head v 106'500, mlcod=0'0)", 2024-04-24T15:33:47.322 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:43.648041+0000", 2024-04-24T15:33:47.322 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 3.6020920809999999, 2024-04-24T15:33:47.322 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0062474779999999999, 2024-04-24T15:33:47.322 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:47.322 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:47.322 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:47.322 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.322 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:47.323 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:43.648041+0000", 2024-04-24T15:33:47.323 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:47.323 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.323 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.323 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:47.323 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:43.648041+0000", 2024-04-24T15:33:47.323 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:47.323 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.323 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.323 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:47.323 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:43.648041+0000", 2024-04-24T15:33:47.323 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:47.323 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.323 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.323 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:47.324 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:43.648044+0000", 2024-04-24T15:33:47.324 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5099999999999999e-06 2024-04-24T15:33:47.324 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.324 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.324 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:47.324 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:43.648051+0000", 2024-04-24T15:33:47.324 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.5490000000000003e-06 2024-04-24T15:33:47.324 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.324 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.324 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:47.324 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:43.648057+0000", 2024-04-24T15:33:47.324 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.1859999999999997e-06 2024-04-24T15:33:47.324 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.324 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.324 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:47.324 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:43.648096+0000", 2024-04-24T15:33:47.325 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8717999999999998e-05 2024-04-24T15:33:47.325 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.325 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.325 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:47.325 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:43.648113+0000", 2024-04-24T15:33:47.325 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.6778000000000001e-05 2024-04-24T15:33:47.325 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.325 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.325 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:33:47.325 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:43.654248+0000", 2024-04-24T15:33:47.325 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.006135031 2024-04-24T15:33:47.325 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.325 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.325 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:47.325 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:43.654289+0000", 2024-04-24T15:33:47.325 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0901e-05 2024-04-24T15:33:47.326 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:47.326 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:47.326 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:47.326 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.326 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.326 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_repop(client.4426.0:75520 4.5 e106/105 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:head v 106'552, mlcod=0'0)", 2024-04-24T15:33:47.326 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:43.650116+0000", 2024-04-24T15:33:47.326 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 3.6000173430000002, 2024-04-24T15:33:47.326 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0042652690000000004, 2024-04-24T15:33:47.326 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:47.326 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:47.326 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:47.326 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.326 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:47.327 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:43.650116+0000", 2024-04-24T15:33:47.327 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:47.327 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.327 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.327 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:47.327 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:43.650116+0000", 2024-04-24T15:33:47.327 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:47.327 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.327 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.327 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:47.327 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:43.650115+0000", 2024-04-24T15:33:47.327 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:47.327 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.327 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.327 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:47.327 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:43.650120+0000", 2024-04-24T15:33:47.327 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4900000000000002e-06 2024-04-24T15:33:47.328 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.328 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.328 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:47.328 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:43.650127+0000", 2024-04-24T15:33:47.328 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.9720000000000003e-06 2024-04-24T15:33:47.328 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.328 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.328 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:47.328 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:43.650132+0000", 2024-04-24T15:33:47.328 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.3979999999999998e-06 2024-04-24T15:33:47.328 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.328 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.328 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:47.328 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:43.650167+0000", 2024-04-24T15:33:47.328 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.4391000000000003e-05 2024-04-24T15:33:47.329 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.329 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.329 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:47.329 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:43.650189+0000", 2024-04-24T15:33:47.329 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.2464999999999999e-05 2024-04-24T15:33:47.329 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.329 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.329 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:33:47.329 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:43.654347+0000", 2024-04-24T15:33:47.329 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0041584400000000002 2024-04-24T15:33:47.329 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.329 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.329 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:47.329 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:43.654381+0000", 2024-04-24T15:33:47.329 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.3575999999999998e-05 2024-04-24T15:33:47.329 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:47.329 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:47.330 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:47.330 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.330 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.330 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "MOSDPGPush(4.1f 106/105 [PushOp(4:f9f49183:::rbd_data.114a9052fca3.0000000000000000:1e, version: 83'1363, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(4:f9f49183:::rbd_data.114a9052fca3.0000000000000000:1e@83'1363, size: 2097152, copy_subset: [0~2097152], clone_subset: {}, snapset: 1e=[]:{1e=[1e]}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:2097152, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to:1048576, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2024-04-24T15:33:47.330 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:43.746799+0000", 2024-04-24T15:33:47.330 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 3.5033344130000001, 2024-04-24T15:33:47.330 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0024212169999999998, 2024-04-24T15:33:47.330 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:47.330 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:47.330 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:47.330 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.330 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:47.330 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:43.746799+0000", 2024-04-24T15:33:47.330 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:47.331 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.331 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.331 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:47.331 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:43.746799+0000", 2024-04-24T15:33:47.331 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:47.331 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.331 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.331 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:47.331 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:43.746798+0000", 2024-04-24T15:33:47.331 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:47.331 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.331 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.331 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:47.331 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:43.748143+0000", 2024-04-24T15:33:47.331 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0013450210000000001 2024-04-24T15:33:47.332 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.332 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.332 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:47.332 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:43.748155+0000", 2024-04-24T15:33:47.332 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.2183e-05 2024-04-24T15:33:47.332 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.332 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.332 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:47.332 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:43.748180+0000", 2024-04-24T15:33:47.332 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.4145e-05 2024-04-24T15:33:47.332 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.332 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.332 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:47.332 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:43.748224+0000", 2024-04-24T15:33:47.332 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4752000000000001e-05 2024-04-24T15:33:47.332 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.333 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.333 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:47.333 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:43.748229+0000", 2024-04-24T15:33:47.333 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.357e-06 2024-04-24T15:33:47.333 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:47.333 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:47.333 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:47.333 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:43.749220+0000", 2024-04-24T15:33:47.333 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00099128499999999995 2024-04-24T15:33:47.333 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:47.333 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:47.333 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:47.333 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:47.333 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:47.333 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:47.335 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:47.336+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:47.335 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-24T15:33:47.436 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:33:47.437+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:33:47.459 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:47.459 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:33:47.459 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-04-24T15:33:47.459 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:47.469 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-24T15:33:47.537 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:47.538+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:47.592 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:47.592 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:33:47.592 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-04-24T15:33:47.592 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-04-24T15:33:47.593 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:47.602 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-24T15:33:47.643 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:47.643+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:47.665 INFO:teuthology.orchestra.run.smithi096.stderr:noscrub is unset 2024-04-24T15:33:47.739 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:47.740+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:47.784 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:47.785 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-04-24T15:33:47.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-04-24T15:33:47.785 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-04-24T15:33:47.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.785 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:47384 4.1f 4:fdac2fcc:::rbd_header.114a8b543e91:head [watch reconnect cookie 140672527126976 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-04-24T15:33:47.785 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:09.802333+0000", 2024-04-24T15:33:47.785 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.982130488000003, 2024-04-24T15:33:47.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0093213050000001, 2024-04-24T15:33:47.785 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:47.785 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:47.785 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:47.785 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:47.785 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:47.786 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 47384 2024-04-24T15:33:47.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.786 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:47.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:47.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.802333+0000", 2024-04-24T15:33:47.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:47.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:47.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.802333+0000", 2024-04-24T15:33:47.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:47.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:47.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.802332+0000", 2024-04-24T15:33:47.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999981 2024-04-24T15:33:47.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:47.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.802340+0000", 2024-04-24T15:33:47.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7700000000000007e-06 2024-04-24T15:33:47.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:47.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.802360+0000", 2024-04-24T15:33:47.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9959999999999999e-05 2024-04-24T15:33:47.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:47.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.802371+0000", 2024-04-24T15:33:47.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0473e-05 2024-04-24T15:33:47.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:47.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.802447+0000", 2024-04-24T15:33:47.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6089999999999998e-05 2024-04-24T15:33:47.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op must wait for map", 2024-04-24T15:33:47.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.802456+0000", 2024-04-24T15:33:47.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0599999999999997e-06 2024-04-24T15:33:47.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:47.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.808378+0000", 2024-04-24T15:33:47.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0059224910000000002 2024-04-24T15:33:47.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-04-24T15:33:47.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.808384+0000", 2024-04-24T15:33:47.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7740000000000001e-06 2024-04-24T15:33:47.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:47.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.809812+0000", 2024-04-24T15:33:47.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.001428043 2024-04-24T15:33:47.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:47.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.810082+0000", 2024-04-24T15:33:47.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00026971800000000001 2024-04-24T15:33:47.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:47.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.811655+0000", 2024-04-24T15:33:47.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015726869999999999 2024-04-24T15:33:47.790 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:47.790 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:47.791 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:47.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.791 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:47385 4.1f 4:fdac2fcc:::rbd_header.114a8b543e91:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-04-24T15:33:47.791 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:09.803953+0000", 2024-04-24T15:33:47.791 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.980510574, 2024-04-24T15:33:47.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0080910729999999, 2024-04-24T15:33:47.791 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:47.791 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:47.791 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:47.791 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:47.791 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:47.791 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 47385 2024-04-24T15:33:47.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.792 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:47.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:47.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.803953+0000", 2024-04-24T15:33:47.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:47.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:47.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.803953+0000", 2024-04-24T15:33:47.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:47.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:47.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.803952+0000", 2024-04-24T15:33:47.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999981 2024-04-24T15:33:47.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:47.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.803961+0000", 2024-04-24T15:33:47.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9549999999999998e-06 2024-04-24T15:33:47.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:47.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.803977+0000", 2024-04-24T15:33:47.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6498999999999999e-05 2024-04-24T15:33:47.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:47.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.803985+0000", 2024-04-24T15:33:47.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5000000000000002e-06 2024-04-24T15:33:47.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:47.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.804032+0000", 2024-04-24T15:33:47.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7657999999999997e-05 2024-04-24T15:33:47.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2024-04-24T15:33:47.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.804039+0000", 2024-04-24T15:33:47.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5719999999999999e-06 2024-04-24T15:33:47.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:47.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.808397+0000", 2024-04-24T15:33:47.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0043579860000000003 2024-04-24T15:33:47.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-04-24T15:33:47.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.808401+0000", 2024-04-24T15:33:47.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3089999999999997e-06 2024-04-24T15:33:47.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:47.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.810700+0000", 2024-04-24T15:33:47.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0022984699999999 2024-04-24T15:33:47.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:47.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.810758+0000", 2024-04-24T15:33:47.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8121000000000001e-05 2024-04-24T15:33:47.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:33:47.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.811932+0000", 2024-04-24T15:33:47.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001174542 2024-04-24T15:33:47.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:47.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.811978+0000", 2024-04-24T15:33:47.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5645000000000001e-05 2024-04-24T15:33:47.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:47.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.812044+0000", 2024-04-24T15:33:47.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6575999999999994e-05 2024-04-24T15:33:47.797 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:47.797 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:47.797 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:47.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.797 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48705 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 1851392~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:47.798 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889236+0000", 2024-04-24T15:33:47.798 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.895228033999999, 2024-04-24T15:33:47.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016984302, 2024-04-24T15:33:47.798 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:47.798 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:47.798 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:47.798 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:47.798 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:47.798 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48705 2024-04-24T15:33:47.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.798 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:47.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:47.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889236+0000", 2024-04-24T15:33:47.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:47.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:47.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889236+0000", 2024-04-24T15:33:47.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:47.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:47.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889235+0000", 2024-04-24T15:33:47.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:47.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:47.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889238+0000", 2024-04-24T15:33:47.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.249e-06 2024-04-24T15:33:47.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:47.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889243+0000", 2024-04-24T15:33:47.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6810000000000003e-06 2024-04-24T15:33:47.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:47.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889247+0000", 2024-04-24T15:33:47.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6780000000000002e-06 2024-04-24T15:33:47.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:47.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906008+0000", 2024-04-24T15:33:47.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01676099 2024-04-24T15:33:47.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:47.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906090+0000", 2024-04-24T15:33:47.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1508999999999995e-05 2024-04-24T15:33:47.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:47.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906220+0000", 2024-04-24T15:33:47.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013056999999999999 2024-04-24T15:33:47.802 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:47.802 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:47.802 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:47.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.802 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48706 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 1916928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:47.802 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889251+0000", 2024-04-24T15:33:47.802 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.895212600999997, 2024-04-24T15:33:47.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01718693, 2024-04-24T15:33:47.802 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:47.802 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:47.802 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:47.802 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:47.803 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:47.803 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48706 2024-04-24T15:33:47.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.803 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:47.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:47.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889251+0000", 2024-04-24T15:33:47.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:47.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:47.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889251+0000", 2024-04-24T15:33:47.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:47.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:47.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889251+0000", 2024-04-24T15:33:47.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:47.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:47.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889253+0000", 2024-04-24T15:33:47.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.244e-06 2024-04-24T15:33:47.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:47.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889259+0000", 2024-04-24T15:33:47.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3640000000000004e-06 2024-04-24T15:33:47.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.805 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:47.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889263+0000", 2024-04-24T15:33:47.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3720000000000001e-06 2024-04-24T15:33:47.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.805 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:47.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906258+0000", 2024-04-24T15:33:47.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016995310999999999 2024-04-24T15:33:47.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.805 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:47.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906343+0000", 2024-04-24T15:33:47.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4536000000000005e-05 2024-04-24T15:33:47.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.806 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:47.806 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906438+0000", 2024-04-24T15:33:47.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5502999999999997e-05 2024-04-24T15:33:47.806 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:47.806 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:47.806 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:47.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.806 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48707 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 1982464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:47.806 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889267+0000", 2024-04-24T15:33:47.806 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.895197041000003, 2024-04-24T15:33:47.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017425395, 2024-04-24T15:33:47.807 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:47.807 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:47.807 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:47.807 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:47.807 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:47.807 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48707 2024-04-24T15:33:47.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.807 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:47.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:47.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889267+0000", 2024-04-24T15:33:47.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:47.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:47.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889267+0000", 2024-04-24T15:33:47.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:47.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:47.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889266+0000", 2024-04-24T15:33:47.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:47.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:47.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889275+0000", 2024-04-24T15:33:47.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.568e-06 2024-04-24T15:33:47.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:47.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889281+0000", 2024-04-24T15:33:47.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3779999999999998e-06 2024-04-24T15:33:47.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:47.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889291+0000", 2024-04-24T15:33:47.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2769999999999994e-06 2024-04-24T15:33:47.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:47.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906476+0000", 2024-04-24T15:33:47.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017185277999999998 2024-04-24T15:33:47.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.810 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:47.810 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906567+0000", 2024-04-24T15:33:47.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0902999999999993e-05 2024-04-24T15:33:47.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.810 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.810 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:47.810 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906692+0000", 2024-04-24T15:33:47.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012541700000000001 2024-04-24T15:33:47.810 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:47.810 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:47.810 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:47.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.810 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.810 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48708 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2048000~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:47.810 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889295+0000", 2024-04-24T15:33:47.811 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.895168550000001, 2024-04-24T15:33:47.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017534984999999999, 2024-04-24T15:33:47.811 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:47.811 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:47.811 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:47.811 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:47.811 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:47.811 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48708 2024-04-24T15:33:47.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.811 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:47.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:47.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889295+0000", 2024-04-24T15:33:47.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:47.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:47.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889295+0000", 2024-04-24T15:33:47.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:47.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:47.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889295+0000", 2024-04-24T15:33:47.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:47.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:47.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889297+0000", 2024-04-24T15:33:47.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3479999999999998e-06 2024-04-24T15:33:47.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:47.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889304+0000", 2024-04-24T15:33:47.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3720000000000001e-06 2024-04-24T15:33:47.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:47.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889307+0000", 2024-04-24T15:33:47.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7160000000000001e-06 2024-04-24T15:33:47.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:47.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906711+0000", 2024-04-24T15:33:47.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017403452999999999 2024-04-24T15:33:47.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.814 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:47.814 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906765+0000", 2024-04-24T15:33:47.814 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4219999999999999e-05 2024-04-24T15:33:47.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.814 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:47.814 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906830+0000", 2024-04-24T15:33:47.814 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5277000000000005e-05 2024-04-24T15:33:47.814 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:47.814 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:47.814 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:47.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.814 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48709 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2113536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:47.815 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889312+0000", 2024-04-24T15:33:47.815 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.895152242999998, 2024-04-24T15:33:47.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017772013, 2024-04-24T15:33:47.815 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:47.815 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:47.815 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:47.815 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:47.815 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:47.815 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48709 2024-04-24T15:33:47.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.815 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:47.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:47.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889312+0000", 2024-04-24T15:33:47.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:47.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:47.816 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889312+0000", 2024-04-24T15:33:47.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:47.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:47.816 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889311+0000", 2024-04-24T15:33:47.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:47.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:47.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889314+0000", 2024-04-24T15:33:47.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2759999999999999e-06 2024-04-24T15:33:47.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:47.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889322+0000", 2024-04-24T15:33:47.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8589999999999994e-06 2024-04-24T15:33:47.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:47.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889326+0000", 2024-04-24T15:33:47.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6380000000000002e-06 2024-04-24T15:33:47.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.818 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:47.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906862+0000", 2024-04-24T15:33:47.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017536403999999998 2024-04-24T15:33:47.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.818 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:47.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906949+0000", 2024-04-24T15:33:47.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6217000000000005e-05 2024-04-24T15:33:47.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.818 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:47.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907084+0000", 2024-04-24T15:33:47.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013497100000000001 2024-04-24T15:33:47.819 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:47.819 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:47.819 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:47.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.819 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48710 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2179072~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:47.819 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889332+0000", 2024-04-24T15:33:47.819 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.895131429000003, 2024-04-24T15:33:47.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017944720000000001, 2024-04-24T15:33:47.819 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:47.819 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:47.819 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:47.819 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:47.819 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:47.819 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48710 2024-04-24T15:33:47.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.819 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:47.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.820 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:47.820 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889332+0000", 2024-04-24T15:33:47.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:47.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.820 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:47.820 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889332+0000", 2024-04-24T15:33:47.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:47.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.820 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:47.820 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889332+0000", 2024-04-24T15:33:47.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:47.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:47.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889334+0000", 2024-04-24T15:33:47.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4210000000000002e-06 2024-04-24T15:33:47.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:47.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889340+0000", 2024-04-24T15:33:47.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7209999999999999e-06 2024-04-24T15:33:47.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:47.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889344+0000", 2024-04-24T15:33:47.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.726e-06 2024-04-24T15:33:47.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:47.822 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907119+0000", 2024-04-24T15:33:47.822 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017775551000000001 2024-04-24T15:33:47.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.822 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:47.822 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907190+0000", 2024-04-24T15:33:47.822 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0634000000000002e-05 2024-04-24T15:33:47.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.822 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:47.822 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907277+0000", 2024-04-24T15:33:47.822 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7120999999999995e-05 2024-04-24T15:33:47.822 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:47.823 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:47.823 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:47.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.823 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48711 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2244608~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:47.823 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889348+0000", 2024-04-24T15:33:47.823 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.895115812, 2024-04-24T15:33:47.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018188109000000001, 2024-04-24T15:33:47.823 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:47.823 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:47.823 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:47.823 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:47.823 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:47.823 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48711 2024-04-24T15:33:47.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.824 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:47.824 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.824 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:47.824 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889348+0000", 2024-04-24T15:33:47.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:47.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.824 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.824 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:47.824 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889348+0000", 2024-04-24T15:33:47.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:47.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.824 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.824 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:47.825 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889348+0000", 2024-04-24T15:33:47.825 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:47.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.825 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.825 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:47.825 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889350+0000", 2024-04-24T15:33:47.825 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2330000000000001e-06 2024-04-24T15:33:47.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.825 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.825 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:47.825 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889356+0000", 2024-04-24T15:33:47.825 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8769999999999998e-06 2024-04-24T15:33:47.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.825 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.826 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:47.826 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889359+0000", 2024-04-24T15:33:47.826 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3699999999999999e-06 2024-04-24T15:33:47.826 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.826 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:47.826 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907316+0000", 2024-04-24T15:33:47.826 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017956857999999999 2024-04-24T15:33:47.826 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.826 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:47.826 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907402+0000", 2024-04-24T15:33:47.826 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5927000000000006e-05 2024-04-24T15:33:47.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.827 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.827 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:47.827 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907536+0000", 2024-04-24T15:33:47.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000134258 2024-04-24T15:33:47.827 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:47.827 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:47.827 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:47.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.827 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.827 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48712 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2310144~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:47.827 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889363+0000", 2024-04-24T15:33:47.827 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.895100493000001, 2024-04-24T15:33:47.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018314815000000002, 2024-04-24T15:33:47.827 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:47.828 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:47.828 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:47.828 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:47.828 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:47.828 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48712 2024-04-24T15:33:47.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.828 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:47.828 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.828 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:47.828 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889363+0000", 2024-04-24T15:33:47.828 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:47.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.829 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:47.829 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889363+0000", 2024-04-24T15:33:47.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:47.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.829 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:47.829 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889363+0000", 2024-04-24T15:33:47.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:47.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.829 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:47.829 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889366+0000", 2024-04-24T15:33:47.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6709999999999999e-06 2024-04-24T15:33:47.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.830 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:47.830 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889373+0000", 2024-04-24T15:33:47.830 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1269999999999997e-06 2024-04-24T15:33:47.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.830 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:47.830 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889377+0000", 2024-04-24T15:33:47.830 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1219999999999997e-06 2024-04-24T15:33:47.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.830 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:47.831 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907560+0000", 2024-04-24T15:33:47.831 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018183357000000001 2024-04-24T15:33:47.831 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.831 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.831 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:47.831 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907612+0000", 2024-04-24T15:33:47.831 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2172999999999997e-05 2024-04-24T15:33:47.831 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.831 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.831 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:47.831 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907678+0000", 2024-04-24T15:33:47.831 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5736999999999998e-05 2024-04-24T15:33:47.831 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:47.831 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:47.832 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:47.832 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.832 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.832 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48713 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2375680~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:47.832 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889381+0000", 2024-04-24T15:33:47.832 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.895082600000002, 2024-04-24T15:33:47.832 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018556019, 2024-04-24T15:33:47.832 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:47.832 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:47.832 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:47.832 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:47.832 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:47.832 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48713 2024-04-24T15:33:47.832 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.832 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:47.832 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.833 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:47.833 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889381+0000", 2024-04-24T15:33:47.833 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:47.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.833 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:47.833 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889381+0000", 2024-04-24T15:33:47.833 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:47.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.833 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:47.833 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889381+0000", 2024-04-24T15:33:47.833 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:47.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.834 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:47.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889388+0000", 2024-04-24T15:33:47.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8809999999999998e-06 2024-04-24T15:33:47.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.834 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:47.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889394+0000", 2024-04-24T15:33:47.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0410000000000003e-06 2024-04-24T15:33:47.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.834 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:47.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889397+0000", 2024-04-24T15:33:47.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5939999999999998e-06 2024-04-24T15:33:47.835 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.835 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.835 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:47.835 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907711+0000", 2024-04-24T15:33:47.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018313949999999999 2024-04-24T15:33:47.835 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.835 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.835 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:47.835 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907802+0000", 2024-04-24T15:33:47.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0307000000000001e-05 2024-04-24T15:33:47.835 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.835 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.835 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:47.835 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907937+0000", 2024-04-24T15:33:47.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013561899999999999 2024-04-24T15:33:47.836 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:47.836 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:47.836 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:47.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.836 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.836 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48714 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2441216~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:47.836 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889435+0000", 2024-04-24T15:33:47.836 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.895029252, 2024-04-24T15:33:47.836 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018636040999999999, 2024-04-24T15:33:47.836 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:47.836 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:47.836 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:47.836 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:47.836 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:47.836 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48714 2024-04-24T15:33:47.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.837 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:47.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.837 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:47.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889435+0000", 2024-04-24T15:33:47.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:47.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.837 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:47.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889435+0000", 2024-04-24T15:33:47.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:47.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.837 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:47.838 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889434+0000", 2024-04-24T15:33:47.838 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:47.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.838 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:47.838 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889437+0000", 2024-04-24T15:33:47.838 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.875e-06 2024-04-24T15:33:47.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.838 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:47.838 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889444+0000", 2024-04-24T15:33:47.838 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9870000000000002e-06 2024-04-24T15:33:47.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.839 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:47.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889448+0000", 2024-04-24T15:33:47.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3030000000000001e-06 2024-04-24T15:33:47.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.839 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:47.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907952+0000", 2024-04-24T15:33:47.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018504245999999999 2024-04-24T15:33:47.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.839 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:47.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908007+0000", 2024-04-24T15:33:47.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4579e-05 2024-04-24T15:33:47.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.840 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:47.840 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908071+0000", 2024-04-24T15:33:47.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3615999999999995e-05 2024-04-24T15:33:47.840 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:47.840 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:47.840 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:47.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.840 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48715 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2506752~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:47.840 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889453+0000", 2024-04-24T15:33:47.840 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.895010888000002, 2024-04-24T15:33:47.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018873935000000001, 2024-04-24T15:33:47.840 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:47.840 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:47.840 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:47.840 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:47.841 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:47.841 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48715 2024-04-24T15:33:47.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.841 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:47.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.841 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:47.841 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889453+0000", 2024-04-24T15:33:47.841 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:47.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.841 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:47.841 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889453+0000", 2024-04-24T15:33:47.841 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:47.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.842 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.842 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:47.842 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889452+0000", 2024-04-24T15:33:47.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:47.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.842 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.842 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:47.842 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889459+0000", 2024-04-24T15:33:47.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9269999999999999e-06 2024-04-24T15:33:47.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.842 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.842 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:47.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889468+0000", 2024-04-24T15:33:47.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2390000000000002e-06 2024-04-24T15:33:47.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.843 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:47.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889475+0000", 2024-04-24T15:33:47.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3939999999999998e-06 2024-04-24T15:33:47.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.843 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:47.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908103+0000", 2024-04-24T15:33:47.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018628388999999999 2024-04-24T15:33:47.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.844 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:47.844 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908193+0000", 2024-04-24T15:33:47.844 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9343000000000002e-05 2024-04-24T15:33:47.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.844 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:47.844 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908327+0000", 2024-04-24T15:33:47.844 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013415899999999999 2024-04-24T15:33:47.844 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:47.844 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:47.844 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:47.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.844 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48716 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2572288~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:47.845 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889480+0000", 2024-04-24T15:33:47.845 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.894984037999997, 2024-04-24T15:33:47.845 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018992862999999999, 2024-04-24T15:33:47.845 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:47.845 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:47.845 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:47.845 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:47.845 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:47.845 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48716 2024-04-24T15:33:47.845 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.845 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:47.845 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.845 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:47.845 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889480+0000", 2024-04-24T15:33:47.846 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:47.846 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.846 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.846 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:47.846 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889480+0000", 2024-04-24T15:33:47.846 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:47.846 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.846 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.846 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:47.846 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889479+0000", 2024-04-24T15:33:47.846 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:47.846 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.846 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.846 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:47.847 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889482+0000", 2024-04-24T15:33:47.847 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4729999999999999e-06 2024-04-24T15:33:47.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.847 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.847 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:47.847 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889488+0000", 2024-04-24T15:33:47.847 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1750000000000002e-06 2024-04-24T15:33:47.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.847 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.847 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:47.847 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889501+0000", 2024-04-24T15:33:47.847 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.288e-05 2024-04-24T15:33:47.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.847 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.848 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:47.848 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908345+0000", 2024-04-24T15:33:47.848 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018843835999999999 2024-04-24T15:33:47.848 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.848 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.848 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:47.848 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908395+0000", 2024-04-24T15:33:47.848 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9765999999999998e-05 2024-04-24T15:33:47.848 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.848 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.848 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:47.848 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908473+0000", 2024-04-24T15:33:47.848 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8159999999999997e-05 2024-04-24T15:33:47.848 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:47.849 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:47.849 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:47.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.849 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.849 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48717 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2637824~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:47.849 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889506+0000", 2024-04-24T15:33:47.849 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.894957787000003, 2024-04-24T15:33:47.849 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.019241864000000001, 2024-04-24T15:33:47.849 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:47.849 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:47.849 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:47.849 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:47.849 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:47.849 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48717 2024-04-24T15:33:47.850 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.850 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:47.850 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.850 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:47.850 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889506+0000", 2024-04-24T15:33:47.850 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:47.850 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.850 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.850 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:47.850 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889506+0000", 2024-04-24T15:33:47.850 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:47.850 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.850 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.850 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:47.851 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889506+0000", 2024-04-24T15:33:47.851 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:47.851 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.851 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.851 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:47.851 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889508+0000", 2024-04-24T15:33:47.851 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6900000000000001e-06 2024-04-24T15:33:47.851 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.851 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.851 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:47.851 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889515+0000", 2024-04-24T15:33:47.851 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5409999999999997e-06 2024-04-24T15:33:47.851 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.851 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.852 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:47.852 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889519+0000", 2024-04-24T15:33:47.852 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7129999999999999e-06 2024-04-24T15:33:47.852 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.852 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.852 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:47.852 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908520+0000", 2024-04-24T15:33:47.852 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.019001606000000001 2024-04-24T15:33:47.852 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.852 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.852 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:47.852 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908612+0000", 2024-04-24T15:33:47.852 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1909000000000004e-05 2024-04-24T15:33:47.852 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.852 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.853 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:47.853 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908748+0000", 2024-04-24T15:33:47.853 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000135847 2024-04-24T15:33:47.853 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:47.853 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:47.853 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:47.853 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.853 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.853 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48718 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2703360~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:47.853 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889523+0000", 2024-04-24T15:33:47.853 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.894940794999997, 2024-04-24T15:33:47.853 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.019451725999999999, 2024-04-24T15:33:47.853 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:47.853 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:47.853 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:47.854 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:47.854 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:47.854 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48718 2024-04-24T15:33:47.854 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.854 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:47.854 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.854 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:47.854 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889523+0000", 2024-04-24T15:33:47.854 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:47.854 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.854 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.854 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:47.854 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889523+0000", 2024-04-24T15:33:47.855 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:47.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.855 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.855 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:47.855 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889523+0000", 2024-04-24T15:33:47.855 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:47.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.855 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.855 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:47.855 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889525+0000", 2024-04-24T15:33:47.855 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9970000000000001e-06 2024-04-24T15:33:47.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.855 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.855 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:47.856 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889530+0000", 2024-04-24T15:33:47.856 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7610000000000004e-06 2024-04-24T15:33:47.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.856 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.856 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:47.856 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889534+0000", 2024-04-24T15:33:47.856 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6119999999999999e-06 2024-04-24T15:33:47.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.856 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.856 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:47.856 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908775+0000", 2024-04-24T15:33:47.856 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.019240709000000002 2024-04-24T15:33:47.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.856 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.857 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:47.857 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908856+0000", 2024-04-24T15:33:47.857 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0788e-05 2024-04-24T15:33:47.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.857 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.857 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:47.857 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908975+0000", 2024-04-24T15:33:47.857 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011927800000000001 2024-04-24T15:33:47.857 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:47.857 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:47.857 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:47.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.857 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.857 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:49872 4.1f 4:fe48f4c1:::rbd_data.114ab402c616.0000000000000014:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-04-24T15:33:47.858 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:12.416730+0000", 2024-04-24T15:33:47.858 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 35.367733817000001, 2024-04-24T15:33:47.858 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.020522327999999999, 2024-04-24T15:33:47.858 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:47.858 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:47.858 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:47.858 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:47.858 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:47.858 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 49872 2024-04-24T15:33:47.858 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.858 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:47.858 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.858 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:47.858 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.416730+0000", 2024-04-24T15:33:47.859 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:47.859 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.859 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.859 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:47.859 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.416730+0000", 2024-04-24T15:33:47.859 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:47.859 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.859 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.859 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:47.859 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.416729+0000", 2024-04-24T15:33:47.859 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:33:47.859 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.859 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.859 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:47.860 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.416734+0000", 2024-04-24T15:33:47.860 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0180000000000001e-06 2024-04-24T15:33:47.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.860 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.860 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:47.860 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.416744+0000", 2024-04-24T15:33:47.860 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0069e-05 2024-04-24T15:33:47.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.860 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.860 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:47.860 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.416749+0000", 2024-04-24T15:33:47.860 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7190000000000001e-06 2024-04-24T15:33:47.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.860 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.861 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:47.861 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.416777+0000", 2024-04-24T15:33:47.861 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7532999999999999e-05 2024-04-24T15:33:47.861 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.861 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.861 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:47.861 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.416824+0000", 2024-04-24T15:33:47.861 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7738000000000003e-05 2024-04-24T15:33:47.861 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.861 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.861 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:33:47.861 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.437145+0000", 2024-04-24T15:33:47.861 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.020320416000000001 2024-04-24T15:33:47.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.862 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.862 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:47.862 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.437189+0000", 2024-04-24T15:33:47.862 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4493000000000002e-05 2024-04-24T15:33:47.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.862 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.862 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:47.862 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.437252+0000", 2024-04-24T15:33:47.862 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3110999999999999e-05 2024-04-24T15:33:47.862 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:47.862 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:47.862 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:47.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.863 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.863 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:52545 4.a 4:5331820a:::rbd_data.114ab867d54d.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-04-24T15:33:47.863 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:14.399696+0000", 2024-04-24T15:33:47.863 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.384767414000002, 2024-04-24T15:33:47.863 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016877893000000001, 2024-04-24T15:33:47.863 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:47.863 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:47.863 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:47.863 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:47.863 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:47.863 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 52545 2024-04-24T15:33:47.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.863 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:47.864 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.864 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:47.864 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.399696+0000", 2024-04-24T15:33:47.864 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:47.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.864 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.864 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:47.864 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.399696+0000", 2024-04-24T15:33:47.864 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:47.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.864 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.864 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:47.864 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.399695+0000", 2024-04-24T15:33:47.865 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999981 2024-04-24T15:33:47.865 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.865 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.865 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:47.865 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.399704+0000", 2024-04-24T15:33:47.865 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.098e-06 2024-04-24T15:33:47.865 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.865 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.865 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:47.865 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.399721+0000", 2024-04-24T15:33:47.865 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7088999999999999e-05 2024-04-24T15:33:47.865 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.865 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.865 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:47.865 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.399728+0000", 2024-04-24T15:33:47.866 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5699999999999998e-06 2024-04-24T15:33:47.866 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.866 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.866 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:47.866 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.399775+0000", 2024-04-24T15:33:47.866 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7445999999999999e-05 2024-04-24T15:33:47.866 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.866 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.866 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:47.866 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.399840+0000", 2024-04-24T15:33:47.866 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5492000000000002e-05 2024-04-24T15:33:47.866 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.866 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.867 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:33:47.867 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.416468+0000", 2024-04-24T15:33:47.867 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016627527 2024-04-24T15:33:47.867 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.867 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.867 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:47.867 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.416521+0000", 2024-04-24T15:33:47.867 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2988000000000002e-05 2024-04-24T15:33:47.867 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.867 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.867 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:47.867 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.416574+0000", 2024-04-24T15:33:47.867 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3363999999999999e-05 2024-04-24T15:33:47.867 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:47.867 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:47.868 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:47.868 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.868 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.868 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:66966 4.1f 4:fb3b69af:::rbd_data.114aed92ca6e.000000000000001a:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-04-24T15:33:47.868 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:21.265525+0000", 2024-04-24T15:33:47.868 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.518938873, 2024-04-24T15:33:47.868 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018862027999999999, 2024-04-24T15:33:47.868 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:47.868 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:47.868 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:47.868 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:47.868 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:47.868 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 66966 2024-04-24T15:33:47.868 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.868 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:47.869 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.869 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:47.869 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:21.265525+0000", 2024-04-24T15:33:47.869 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:47.869 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.869 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.869 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:47.869 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:21.265525+0000", 2024-04-24T15:33:47.869 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:47.869 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.869 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.869 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:47.869 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:21.265523+0000", 2024-04-24T15:33:47.869 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999981 2024-04-24T15:33:47.869 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.870 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.870 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:47.870 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:21.265536+0000", 2024-04-24T15:33:47.870 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2489e-05 2024-04-24T15:33:47.870 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.870 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.870 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:47.870 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:21.265550+0000", 2024-04-24T15:33:47.870 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4218e-05 2024-04-24T15:33:47.870 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.870 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.870 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:47.870 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:21.265556+0000", 2024-04-24T15:33:47.870 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.241e-06 2024-04-24T15:33:47.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.871 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.871 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:47.871 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:21.265606+0000", 2024-04-24T15:33:47.871 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9972000000000002e-05 2024-04-24T15:33:47.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.871 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.871 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:47.871 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:21.265670+0000", 2024-04-24T15:33:47.871 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3425999999999999e-05 2024-04-24T15:33:47.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.871 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.871 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:33:47.871 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:21.284297+0000", 2024-04-24T15:33:47.872 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018627517999999999 2024-04-24T15:33:47.872 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.872 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.872 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:47.872 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:21.284339+0000", 2024-04-24T15:33:47.872 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1628000000000002e-05 2024-04-24T15:33:47.872 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.872 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.872 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:47.872 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:21.284387+0000", 2024-04-24T15:33:47.872 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8242000000000003e-05 2024-04-24T15:33:47.872 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:47.872 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:47.872 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:47.872 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.873 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.873 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:75519 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:head [delete] snapc 17=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e105)", 2024-04-24T15:33:47.873 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:42.638138+0000", 2024-04-24T15:33:47.873 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 5.1463261579999999, 2024-04-24T15:33:47.873 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0071697449999999, 2024-04-24T15:33:47.873 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:47.873 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:47.873 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:47.873 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:47.873 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:47.873 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 75519 2024-04-24T15:33:47.873 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.873 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:47.874 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.874 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:47.874 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:42.638138+0000", 2024-04-24T15:33:47.874 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:47.874 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.874 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.874 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:47.874 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:42.638138+0000", 2024-04-24T15:33:47.874 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:47.874 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.874 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.874 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:47.874 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:42.638137+0000", 2024-04-24T15:33:47.874 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999986 2024-04-24T15:33:47.875 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.875 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.875 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:47.875 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:42.638142+0000", 2024-04-24T15:33:47.875 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8989999999999998e-06 2024-04-24T15:33:47.875 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.875 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.875 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:47.875 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:42.638154+0000", 2024-04-24T15:33:47.875 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1572e-05 2024-04-24T15:33:47.875 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.875 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.875 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:47.875 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:42.638160+0000", 2024-04-24T15:33:47.875 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3899999999999998e-06 2024-04-24T15:33:47.876 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.876 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.876 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:47.876 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:42.638203+0000", 2024-04-24T15:33:47.876 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2252000000000001e-05 2024-04-24T15:33:47.876 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.876 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.876 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op must wait for map", 2024-04-24T15:33:47.876 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:42.638209+0000", 2024-04-24T15:33:47.876 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4359999999999999e-06 2024-04-24T15:33:47.876 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.876 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.876 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:47.876 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:42.639589+0000", 2024-04-24T15:33:47.876 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001379903 2024-04-24T15:33:47.877 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.877 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.877 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-04-24T15:33:47.877 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:42.639600+0000", 2024-04-24T15:33:47.877 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0719e-05 2024-04-24T15:33:47.877 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.877 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.877 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:47.877 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:43.643334+0000", 2024-04-24T15:33:47.877 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0037341790000001 2024-04-24T15:33:47.877 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.877 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.877 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:47.877 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:43.643460+0000", 2024-04-24T15:33:47.877 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012588700000000001 2024-04-24T15:33:47.878 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.878 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.878 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 6", 2024-04-24T15:33:47.878 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:43.643622+0000", 2024-04-24T15:33:47.878 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016211400000000001 2024-04-24T15:33:47.878 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.878 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.878 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:33:47.878 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:43.644400+0000", 2024-04-24T15:33:47.878 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000777708 2024-04-24T15:33:47.878 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.878 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.878 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:33:47.878 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:43.645196+0000", 2024-04-24T15:33:47.879 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00079659900000000005 2024-04-24T15:33:47.879 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.879 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.879 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:47.879 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:43.645227+0000", 2024-04-24T15:33:47.879 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1250000000000001e-05 2024-04-24T15:33:47.879 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:47.879 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:47.879 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:47.879 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:43.645307+0000", 2024-04-24T15:33:47.879 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0035000000000002e-05 2024-04-24T15:33:47.879 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:47.879 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:47.879 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:47.879 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:47.879 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:47.880 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:47.881 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-24T15:33:47.882 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:47.881+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:47.981 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:47.981+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:48.043 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:48.044 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:33:48.044 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-04-24T15:33:48.044 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:48.053 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-24T15:33:48.095 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:48.096+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:48.183 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:48.183+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:48.217 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:48.217 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:33:48.217 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-04-24T15:33:48.218 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-04-24T15:33:48.218 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:48.227 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-24T15:33:48.283 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:33:48.285+0000 7f0f45efb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 59992) UID: 0 2024-04-24T15:33:48.384 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:33:48.385+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:33:48.390 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:48.390 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-04-24T15:33:48.391 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-04-24T15:33:48.391 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [] 2024-04-24T15:33:48.391 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:48.400 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-24T15:33:48.485 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:48.485+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:48.524 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:48.524 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:33:48.524 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-04-24T15:33:48.524 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:48.533 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-24T15:33:48.619 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:48.619+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:48.687 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:33:48.687+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:33:48.706 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:48.706 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:33:48.706 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-04-24T15:33:48.706 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-04-24T15:33:48.706 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:48.717 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-24T15:33:48.801 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:48.802+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:48.884 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:48.884 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-04-24T15:33:48.884 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-04-24T15:33:48.885 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-04-24T15:33:48.885 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.885 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:2522 4.6 4:60ceedf0:::rbd_data.114a1f60cb66.000000000000011d:head [set-alloc-hint object_size 262144 write_size 262144,write 110592~2048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-24T15:33:48.885 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:23.088240+0000", 2024-04-24T15:33:48.885 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 85.796146515999993, 2024-04-24T15:33:48.885 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.988232797, 2024-04-24T15:33:48.885 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:48.885 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:48.885 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:48.885 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:48.885 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:48.885 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2522 2024-04-24T15:33:48.885 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.885 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:48.886 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.886 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:48.886 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088240+0000", 2024-04-24T15:33:48.886 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:48.886 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.886 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.886 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:48.886 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088240+0000", 2024-04-24T15:33:48.886 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:48.886 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.886 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.886 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:48.886 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088238+0000", 2024-04-24T15:33:48.886 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999981 2024-04-24T15:33:48.887 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.887 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.887 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:48.887 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088254+0000", 2024-04-24T15:33:48.887 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5562000000000001e-05 2024-04-24T15:33:48.887 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.887 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.887 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:48.887 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088275+0000", 2024-04-24T15:33:48.887 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1256000000000001e-05 2024-04-24T15:33:48.887 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.887 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.887 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:48.887 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088283+0000", 2024-04-24T15:33:48.887 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3219999999999993e-06 2024-04-24T15:33:48.887 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.888 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.888 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:48.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088351+0000", 2024-04-24T15:33:48.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7595000000000006e-05 2024-04-24T15:33:48.888 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.888 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.888 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-04-24T15:33:48.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088362+0000", 2024-04-24T15:33:48.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1764e-05 2024-04-24T15:33:48.888 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.888 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.888 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:48.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.073748+0000", 2024-04-24T15:33:48.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98538541899999998 2024-04-24T15:33:48.888 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:48.889 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.073925+0000", 2024-04-24T15:33:48.889 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017747600000000001 2024-04-24T15:33:48.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-24T15:33:48.889 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.074092+0000", 2024-04-24T15:33:48.889 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000166622 2024-04-24T15:33:48.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:33:48.889 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.074774+0000", 2024-04-24T15:33:48.889 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00068225000000000002 2024-04-24T15:33:48.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.890 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:33:48.890 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.076412+0000", 2024-04-24T15:33:48.890 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001638189 2024-04-24T15:33:48.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.890 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.890 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:48.890 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.076446+0000", 2024-04-24T15:33:48.890 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3587000000000002e-05 2024-04-24T15:33:48.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.890 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.890 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:48.890 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.076473+0000", 2024-04-24T15:33:48.890 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6886999999999999e-05 2024-04-24T15:33:48.891 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:48.891 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:48.891 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:48.891 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.891 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.891 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43121 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 5865472~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:48.891 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094178+0000", 2024-04-24T15:33:48.891 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 41.790208833000001, 2024-04-24T15:33:48.891 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032860598999999997, 2024-04-24T15:33:48.891 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:48.891 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:48.891 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:48.891 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:48.891 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:48.891 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43121 2024-04-24T15:33:48.891 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.892 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:48.892 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.892 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:48.892 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094178+0000", 2024-04-24T15:33:48.892 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:48.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.892 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.892 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:48.892 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094178+0000", 2024-04-24T15:33:48.892 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:48.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.892 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.892 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:48.892 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094178+0000", 2024-04-24T15:33:48.892 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:48.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.893 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.893 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:48.893 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094180+0000", 2024-04-24T15:33:48.893 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9989999999999998e-06 2024-04-24T15:33:48.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.893 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.893 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:48.893 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094185+0000", 2024-04-24T15:33:48.893 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.541e-06 2024-04-24T15:33:48.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.893 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.893 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:48.893 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094189+0000", 2024-04-24T15:33:48.893 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0509999999999998e-06 2024-04-24T15:33:48.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.894 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.894 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:48.894 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.126877+0000", 2024-04-24T15:33:48.894 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032687473000000002 2024-04-24T15:33:48.894 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.894 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.894 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:48.894 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.126952+0000", 2024-04-24T15:33:48.894 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5924e-05 2024-04-24T15:33:48.894 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.894 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.894 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:48.894 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127038+0000", 2024-04-24T15:33:48.894 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5908999999999994e-05 2024-04-24T15:33:48.895 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:48.895 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:48.895 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:48.895 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.895 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.895 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43122 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 6291456~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:48.895 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094193+0000", 2024-04-24T15:33:48.895 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 41.790193588999998, 2024-04-24T15:33:48.895 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032978383999999999, 2024-04-24T15:33:48.895 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:48.895 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:48.896 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:48.896 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:48.896 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:48.896 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43122 2024-04-24T15:33:48.896 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.896 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:48.896 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.896 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:48.896 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094193+0000", 2024-04-24T15:33:48.896 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:48.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.897 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.897 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:48.897 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094193+0000", 2024-04-24T15:33:48.897 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:48.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.897 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.897 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:48.897 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094193+0000", 2024-04-24T15:33:48.897 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:48.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.897 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.897 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:48.897 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094195+0000", 2024-04-24T15:33:48.897 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3269999999999999e-06 2024-04-24T15:33:48.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.898 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.898 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:48.898 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094201+0000", 2024-04-24T15:33:48.898 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9850000000000003e-06 2024-04-24T15:33:48.898 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.898 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.898 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:48.898 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094205+0000", 2024-04-24T15:33:48.898 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8430000000000003e-06 2024-04-24T15:33:48.898 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.898 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.898 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:48.899 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127068+0000", 2024-04-24T15:33:48.899 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032863305000000002 2024-04-24T15:33:48.899 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.899 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.899 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:48.899 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127113+0000", 2024-04-24T15:33:48.899 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4412000000000001e-05 2024-04-24T15:33:48.899 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.899 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.899 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:48.899 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127171+0000", 2024-04-24T15:33:48.899 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8807999999999997e-05 2024-04-24T15:33:48.899 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:48.899 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:48.900 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:48.900 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.900 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.900 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43123 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 6717440~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:48.900 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094209+0000", 2024-04-24T15:33:48.900 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 41.790177423000003, 2024-04-24T15:33:48.900 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033144583999999998, 2024-04-24T15:33:48.900 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:48.900 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:48.900 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:48.900 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:48.900 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:48.900 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43123 2024-04-24T15:33:48.900 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.901 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:48.901 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.901 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:48.901 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094209+0000", 2024-04-24T15:33:48.901 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:48.901 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.901 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.901 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:48.901 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094209+0000", 2024-04-24T15:33:48.901 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:48.901 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.901 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.901 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:48.901 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094209+0000", 2024-04-24T15:33:48.901 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:48.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.902 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.902 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:48.902 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094211+0000", 2024-04-24T15:33:48.902 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2089999999999999e-06 2024-04-24T15:33:48.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.902 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.902 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:48.902 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094217+0000", 2024-04-24T15:33:48.902 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6799999999999998e-06 2024-04-24T15:33:48.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.902 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.902 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:48.902 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094220+0000", 2024-04-24T15:33:48.902 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.444e-06 2024-04-24T15:33:48.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.903 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:48.903 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127190+0000", 2024-04-24T15:33:48.903 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032969471 2024-04-24T15:33:48.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.903 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:48.903 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127269+0000", 2024-04-24T15:33:48.903 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9246999999999995e-05 2024-04-24T15:33:48.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.903 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:48.903 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127354+0000", 2024-04-24T15:33:48.903 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4853999999999998e-05 2024-04-24T15:33:48.904 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:48.904 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:48.904 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:48.904 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.904 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.904 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43124 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 7143424~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:48.904 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094224+0000", 2024-04-24T15:33:48.904 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 41.790162449999997, 2024-04-24T15:33:48.904 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033271955999999998, 2024-04-24T15:33:48.904 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:48.904 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:48.904 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:48.904 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:48.904 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:48.905 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43124 2024-04-24T15:33:48.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.905 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:48.905 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.905 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:48.905 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094224+0000", 2024-04-24T15:33:48.905 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:48.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.905 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.905 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:48.905 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094224+0000", 2024-04-24T15:33:48.905 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:48.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.906 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.906 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:48.906 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094224+0000", 2024-04-24T15:33:48.906 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:48.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.906 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.906 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:48.906 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094226+0000", 2024-04-24T15:33:48.906 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9039999999999999e-06 2024-04-24T15:33:48.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.906 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.906 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:48.906 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094233+0000", 2024-04-24T15:33:48.906 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7970000000000003e-06 2024-04-24T15:33:48.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.907 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:48.907 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094236+0000", 2024-04-24T15:33:48.907 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6339999999999999e-06 2024-04-24T15:33:48.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.907 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:48.907 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127390+0000", 2024-04-24T15:33:48.907 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.03315353 2024-04-24T15:33:48.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.907 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:48.907 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127439+0000", 2024-04-24T15:33:48.908 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8890000000000001e-05 2024-04-24T15:33:48.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.908 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.908 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:48.908 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127496+0000", 2024-04-24T15:33:48.908 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7476999999999999e-05 2024-04-24T15:33:48.908 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:48.908 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:48.908 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:48.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.908 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.908 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43125 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 7569408~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:48.908 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094240+0000", 2024-04-24T15:33:48.908 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 41.790146362999998, 2024-04-24T15:33:48.908 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033446278000000003, 2024-04-24T15:33:48.909 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:48.909 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:48.909 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:48.909 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:48.909 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:48.909 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43125 2024-04-24T15:33:48.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.909 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:48.909 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.909 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:48.909 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094240+0000", 2024-04-24T15:33:48.909 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:48.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.909 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.909 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:48.909 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094240+0000", 2024-04-24T15:33:48.910 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:48.910 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.910 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.910 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:48.910 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094240+0000", 2024-04-24T15:33:48.910 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:48.910 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.910 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.910 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:48.910 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094247+0000", 2024-04-24T15:33:48.910 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6209999999999998e-06 2024-04-24T15:33:48.910 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.910 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.910 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:48.911 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094253+0000", 2024-04-24T15:33:48.911 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0739999999999998e-06 2024-04-24T15:33:48.911 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.911 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.911 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:48.911 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094256+0000", 2024-04-24T15:33:48.911 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7699999999999999e-06 2024-04-24T15:33:48.911 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.911 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.911 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:48.911 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127526+0000", 2024-04-24T15:33:48.911 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033269703999999997 2024-04-24T15:33:48.911 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.911 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.912 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:48.912 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127603+0000", 2024-04-24T15:33:48.912 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6964000000000004e-05 2024-04-24T15:33:48.912 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.912 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.912 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:48.912 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127687+0000", 2024-04-24T15:33:48.912 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3431999999999997e-05 2024-04-24T15:33:48.912 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:48.912 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:48.912 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:48.912 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.912 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.912 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43126 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 7995392~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:48.912 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094261+0000", 2024-04-24T15:33:48.913 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 41.790125860000003, 2024-04-24T15:33:48.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033555201999999999, 2024-04-24T15:33:48.913 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:48.913 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:48.913 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:48.913 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:48.913 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:48.913 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43126 2024-04-24T15:33:48.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.913 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:48.913 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.913 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:48.913 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094261+0000", 2024-04-24T15:33:48.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:48.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.913 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.914 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:48.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094261+0000", 2024-04-24T15:33:48.914 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:48.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.914 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.914 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:48.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094260+0000", 2024-04-24T15:33:48.914 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:48.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.914 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.914 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:48.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094263+0000", 2024-04-24T15:33:48.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2579999999999998e-06 2024-04-24T15:33:48.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.915 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:48.915 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094269+0000", 2024-04-24T15:33:48.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.823e-06 2024-04-24T15:33:48.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.915 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:48.915 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094272+0000", 2024-04-24T15:33:48.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8999999999999999e-06 2024-04-24T15:33:48.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.915 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:48.916 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127720+0000", 2024-04-24T15:33:48.916 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033447521000000001 2024-04-24T15:33:48.916 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.916 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.916 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:48.916 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127765+0000", 2024-04-24T15:33:48.916 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5352000000000002e-05 2024-04-24T15:33:48.916 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.916 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.916 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:48.916 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127816+0000", 2024-04-24T15:33:48.916 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0689999999999997e-05 2024-04-24T15:33:48.916 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:48.916 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:48.917 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:48.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.917 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.917 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63800 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1032192~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:48.917 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555407+0000", 2024-04-24T15:33:48.917 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.328979874000002, 2024-04-24T15:33:48.917 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032809030000000003, 2024-04-24T15:33:48.917 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:48.917 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:48.917 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:48.917 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:48.917 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:48.917 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63800 2024-04-24T15:33:48.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.917 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:48.918 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.918 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:48.918 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555407+0000", 2024-04-24T15:33:48.918 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:48.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.918 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.918 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:48.918 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555407+0000", 2024-04-24T15:33:48.918 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:48.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.918 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.918 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:48.918 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555406+0000", 2024-04-24T15:33:48.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:48.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.919 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:48.919 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555409+0000", 2024-04-24T15:33:48.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.706e-06 2024-04-24T15:33:48.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.919 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:48.919 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555416+0000", 2024-04-24T15:33:48.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1609999999999997e-06 2024-04-24T15:33:48.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.919 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:48.919 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555420+0000", 2024-04-24T15:33:48.920 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0369999999999996e-06 2024-04-24T15:33:48.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.920 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.920 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:48.920 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588088+0000", 2024-04-24T15:33:48.920 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032668228000000001 2024-04-24T15:33:48.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.920 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.920 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:48.920 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588143+0000", 2024-04-24T15:33:48.920 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4852999999999998e-05 2024-04-24T15:33:48.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.920 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.920 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:48.921 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588216+0000", 2024-04-24T15:33:48.921 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2467999999999998e-05 2024-04-24T15:33:48.921 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:48.921 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:48.921 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:48.921 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.921 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.921 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63801 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1114112~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:48.921 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555425+0000", 2024-04-24T15:33:48.921 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.328961967000001, 2024-04-24T15:33:48.921 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032924691999999998, 2024-04-24T15:33:48.921 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:48.921 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:48.922 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:48.922 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:48.922 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:48.922 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63801 2024-04-24T15:33:48.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.922 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:48.922 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.922 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:48.922 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555425+0000", 2024-04-24T15:33:48.922 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:48.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.922 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.922 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:48.922 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555425+0000", 2024-04-24T15:33:48.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:48.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:48.923 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555424+0000", 2024-04-24T15:33:48.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:48.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:48.923 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555427+0000", 2024-04-24T15:33:48.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2350000000000002e-06 2024-04-24T15:33:48.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:48.923 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555432+0000", 2024-04-24T15:33:48.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.434e-06 2024-04-24T15:33:48.924 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.924 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.924 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:48.924 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555435+0000", 2024-04-24T15:33:48.924 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2009999999999999e-06 2024-04-24T15:33:48.924 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.924 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.924 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:48.924 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588237+0000", 2024-04-24T15:33:48.924 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032801508 2024-04-24T15:33:48.924 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.924 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.924 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:48.925 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588281+0000", 2024-04-24T15:33:48.925 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4100999999999997e-05 2024-04-24T15:33:48.925 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.925 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.925 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:48.925 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588349+0000", 2024-04-24T15:33:48.925 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8551000000000001e-05 2024-04-24T15:33:48.925 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:48.925 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:48.925 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:48.925 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.925 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.925 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63802 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1196032~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:48.925 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555439+0000", 2024-04-24T15:33:48.926 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.328947446000001, 2024-04-24T15:33:48.926 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033057744999999999, 2024-04-24T15:33:48.926 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:48.926 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:48.926 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:48.926 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:48.926 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:48.926 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63802 2024-04-24T15:33:48.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.926 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:48.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.926 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:48.926 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555439+0000", 2024-04-24T15:33:48.926 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:48.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.927 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:48.927 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555439+0000", 2024-04-24T15:33:48.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:48.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.927 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:48.927 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555439+0000", 2024-04-24T15:33:48.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:48.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.927 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:48.927 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555445+0000", 2024-04-24T15:33:48.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4019999999999999e-06 2024-04-24T15:33:48.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.928 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:48.928 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555452+0000", 2024-04-24T15:33:48.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8600000000000004e-06 2024-04-24T15:33:48.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.928 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:48.928 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555461+0000", 2024-04-24T15:33:48.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0499999999999997e-06 2024-04-24T15:33:48.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.928 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:48.928 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588378+0000", 2024-04-24T15:33:48.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032916527000000001 2024-04-24T15:33:48.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.928 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:48.929 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588423+0000", 2024-04-24T15:33:48.929 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5092999999999997e-05 2024-04-24T15:33:48.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.929 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.929 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:48.929 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588497+0000", 2024-04-24T15:33:48.929 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4140999999999994e-05 2024-04-24T15:33:48.929 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:48.929 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:48.929 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:48.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.929 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.929 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63803 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1277952~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:48.929 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555466+0000", 2024-04-24T15:33:48.929 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.328920500999999, 2024-04-24T15:33:48.930 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033162297, 2024-04-24T15:33:48.930 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:48.930 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:48.930 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:48.930 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:48.930 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:48.930 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63803 2024-04-24T15:33:48.930 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.930 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:48.930 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.930 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:48.930 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555466+0000", 2024-04-24T15:33:48.930 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:48.930 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.930 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.931 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:48.931 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555466+0000", 2024-04-24T15:33:48.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:48.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.931 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:48.931 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555466+0000", 2024-04-24T15:33:48.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:48.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.931 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:48.931 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555468+0000", 2024-04-24T15:33:48.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4040000000000002e-06 2024-04-24T15:33:48.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.931 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:48.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555475+0000", 2024-04-24T15:33:48.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9480000000000002e-06 2024-04-24T15:33:48.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.932 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.932 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:48.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555479+0000", 2024-04-24T15:33:48.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0289999999999999e-06 2024-04-24T15:33:48.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.932 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.932 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:48.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588516+0000", 2024-04-24T15:33:48.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033037094000000003 2024-04-24T15:33:48.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.932 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.932 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:48.933 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588562+0000", 2024-04-24T15:33:48.933 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6215999999999999e-05 2024-04-24T15:33:48.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.933 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.933 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:48.933 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588628+0000", 2024-04-24T15:33:48.933 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5971999999999998e-05 2024-04-24T15:33:48.933 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:48.933 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:48.933 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:48.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.933 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.933 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63804 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1359872~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:48.933 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555483+0000", 2024-04-24T15:33:48.933 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.328903334, 2024-04-24T15:33:48.933 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033287068000000003, 2024-04-24T15:33:48.934 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:48.934 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:48.934 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:48.934 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:48.934 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:48.934 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63804 2024-04-24T15:33:48.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.934 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:48.934 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.934 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:48.934 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555483+0000", 2024-04-24T15:33:48.934 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:48.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.935 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:48.935 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555483+0000", 2024-04-24T15:33:48.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:48.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.935 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:48.935 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555483+0000", 2024-04-24T15:33:48.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:48.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.935 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:48.935 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555485+0000", 2024-04-24T15:33:48.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0710000000000002e-06 2024-04-24T15:33:48.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.936 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:48.936 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555498+0000", 2024-04-24T15:33:48.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2748999999999999e-05 2024-04-24T15:33:48.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.936 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:48.936 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555502+0000", 2024-04-24T15:33:48.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0219999999999998e-06 2024-04-24T15:33:48.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.936 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:48.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588657+0000", 2024-04-24T15:33:48.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033154683999999997 2024-04-24T15:33:48.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:48.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588702+0000", 2024-04-24T15:33:48.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5809000000000001e-05 2024-04-24T15:33:48.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:48.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588770+0000", 2024-04-24T15:33:48.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8056e-05 2024-04-24T15:33:48.937 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:48.937 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:48.937 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:48.938 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.938 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.938 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63805 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1441792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:48.938 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555508+0000", 2024-04-24T15:33:48.938 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.328878485000001, 2024-04-24T15:33:48.938 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033375952, 2024-04-24T15:33:48.938 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:48.938 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:48.938 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:48.938 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:48.938 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:48.938 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63805 2024-04-24T15:33:48.938 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.938 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:48.938 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.939 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:48.939 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555508+0000", 2024-04-24T15:33:48.939 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:48.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.939 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.939 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:48.939 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555508+0000", 2024-04-24T15:33:48.939 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:48.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.939 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.939 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:48.939 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555508+0000", 2024-04-24T15:33:48.939 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:48.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.940 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:48.940 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555510+0000", 2024-04-24T15:33:48.940 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.424e-06 2024-04-24T15:33:48.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.940 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:48.940 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555516+0000", 2024-04-24T15:33:48.940 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3730000000000001e-06 2024-04-24T15:33:48.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.940 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:48.940 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555519+0000", 2024-04-24T15:33:48.940 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.039e-06 2024-04-24T15:33:48.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.941 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:48.941 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588789+0000", 2024-04-24T15:33:48.941 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0332702 2024-04-24T15:33:48.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.941 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:48.941 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588833+0000", 2024-04-24T15:33:48.941 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4660999999999998e-05 2024-04-24T15:33:48.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.941 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:48.942 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588884+0000", 2024-04-24T15:33:48.942 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0661999999999997e-05 2024-04-24T15:33:48.942 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:48.942 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:48.942 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:48.942 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.942 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.942 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63806 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1523712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:48.942 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555523+0000", 2024-04-24T15:33:48.942 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.328863903999999, 2024-04-24T15:33:48.942 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033479653999999998, 2024-04-24T15:33:48.942 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:48.942 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:48.942 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:48.942 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:48.943 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:48.943 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63806 2024-04-24T15:33:48.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.943 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:48.943 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.943 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:48.943 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555523+0000", 2024-04-24T15:33:48.943 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:48.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.943 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.943 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:48.943 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555523+0000", 2024-04-24T15:33:48.943 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:48.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.943 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.943 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:48.944 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555522+0000", 2024-04-24T15:33:48.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:48.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.944 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:48.944 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555524+0000", 2024-04-24T15:33:48.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0700000000000001e-06 2024-04-24T15:33:48.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.944 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:48.944 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555530+0000", 2024-04-24T15:33:48.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5450000000000003e-06 2024-04-24T15:33:48.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.944 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:48.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555533+0000", 2024-04-24T15:33:48.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2200000000000001e-06 2024-04-24T15:33:48.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.945 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.945 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:48.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588908+0000", 2024-04-24T15:33:48.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033374556 2024-04-24T15:33:48.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.945 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.945 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:48.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588950+0000", 2024-04-24T15:33:48.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2361999999999999e-05 2024-04-24T15:33:48.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.945 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.946 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:48.946 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589002+0000", 2024-04-24T15:33:48.946 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2224e-05 2024-04-24T15:33:48.946 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:48.946 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:48.946 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:48.946 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.946 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.946 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63807 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1605632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:48.946 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555537+0000", 2024-04-24T15:33:48.946 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.328849388999998, 2024-04-24T15:33:48.946 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033577227000000001, 2024-04-24T15:33:48.946 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:48.946 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:48.947 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:48.947 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:48.947 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:48.947 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63807 2024-04-24T15:33:48.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.947 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:48.947 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.947 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:48.947 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555537+0000", 2024-04-24T15:33:48.947 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:48.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.947 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.947 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:48.947 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555537+0000", 2024-04-24T15:33:48.947 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:48.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.948 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:48.948 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555537+0000", 2024-04-24T15:33:48.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:48.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.948 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:48.948 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555539+0000", 2024-04-24T15:33:48.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9769999999999999e-06 2024-04-24T15:33:48.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.948 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:48.948 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555544+0000", 2024-04-24T15:33:48.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.468e-06 2024-04-24T15:33:48.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:48.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555548+0000", 2024-04-24T15:33:48.949 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4319999999999999e-06 2024-04-24T15:33:48.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:48.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589021+0000", 2024-04-24T15:33:48.949 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033473655999999997 2024-04-24T15:33:48.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:48.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589064+0000", 2024-04-24T15:33:48.949 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2397000000000001e-05 2024-04-24T15:33:48.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.950 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.950 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:48.950 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589114+0000", 2024-04-24T15:33:48.950 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0611e-05 2024-04-24T15:33:48.950 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:48.950 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:48.950 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:48.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.950 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.950 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63808 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1687552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:48.950 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555552+0000", 2024-04-24T15:33:48.950 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.328834697000001, 2024-04-24T15:33:48.950 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033686019999999997, 2024-04-24T15:33:48.950 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:48.951 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:48.951 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:48.951 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:48.951 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:48.951 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63808 2024-04-24T15:33:48.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.951 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:48.951 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.951 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:48.951 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555552+0000", 2024-04-24T15:33:48.951 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:48.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.951 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.951 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:48.951 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555552+0000", 2024-04-24T15:33:48.952 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:48.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.952 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:48.952 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555552+0000", 2024-04-24T15:33:48.952 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:48.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.952 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:48.952 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555554+0000", 2024-04-24T15:33:48.952 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2170000000000001e-06 2024-04-24T15:33:48.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.952 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:48.952 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555562+0000", 2024-04-24T15:33:48.952 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8779999999999998e-06 2024-04-24T15:33:48.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.953 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:48.953 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555565+0000", 2024-04-24T15:33:48.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4230000000000001e-06 2024-04-24T15:33:48.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.953 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:48.953 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589143+0000", 2024-04-24T15:33:48.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033577382000000003 2024-04-24T15:33:48.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.953 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:48.953 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589185+0000", 2024-04-24T15:33:48.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2206999999999998e-05 2024-04-24T15:33:48.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.954 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.954 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:48.954 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589238+0000", 2024-04-24T15:33:48.954 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3211000000000003e-05 2024-04-24T15:33:48.954 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:48.954 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:48.954 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:48.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.954 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.954 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63809 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1769472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:48.954 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555569+0000", 2024-04-24T15:33:48.954 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.328817155999999, 2024-04-24T15:33:48.954 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.03378068, 2024-04-24T15:33:48.954 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:48.955 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:48.955 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:48.955 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:48.955 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:48.955 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63809 2024-04-24T15:33:48.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.955 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:48.955 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.955 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:48.955 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555569+0000", 2024-04-24T15:33:48.955 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:48.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.955 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.955 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:48.955 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555569+0000", 2024-04-24T15:33:48.956 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:48.956 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.956 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.956 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:48.956 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555569+0000", 2024-04-24T15:33:48.956 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:48.956 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.956 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.956 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:48.956 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555571+0000", 2024-04-24T15:33:48.956 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0889999999999998e-06 2024-04-24T15:33:48.956 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.956 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.956 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:48.956 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555577+0000", 2024-04-24T15:33:48.957 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4149999999999998e-06 2024-04-24T15:33:48.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.957 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.957 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:48.957 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555580+0000", 2024-04-24T15:33:48.957 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2399999999999999e-06 2024-04-24T15:33:48.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.957 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.957 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:48.957 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589258+0000", 2024-04-24T15:33:48.957 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033677817999999998 2024-04-24T15:33:48.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.957 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.957 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:48.957 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589300+0000", 2024-04-24T15:33:48.957 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2317999999999997e-05 2024-04-24T15:33:48.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.958 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.958 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:48.958 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589350+0000", 2024-04-24T15:33:48.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0111000000000002e-05 2024-04-24T15:33:48.958 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:48.958 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:48.958 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:48.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.958 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.958 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63810 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1851392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:48.958 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555584+0000", 2024-04-24T15:33:48.958 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.328802686, 2024-04-24T15:33:48.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033886251999999999, 2024-04-24T15:33:48.958 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:48.959 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:48.959 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:48.959 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:48.959 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:48.959 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63810 2024-04-24T15:33:48.959 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.959 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:48.959 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.959 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:48.959 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555584+0000", 2024-04-24T15:33:48.959 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:48.959 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.959 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.959 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:48.959 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555584+0000", 2024-04-24T15:33:48.959 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:48.960 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.960 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.960 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:48.960 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555584+0000", 2024-04-24T15:33:48.960 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:48.960 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.960 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.960 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:48.960 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555586+0000", 2024-04-24T15:33:48.960 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0899999999999999e-06 2024-04-24T15:33:48.960 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.960 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.960 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:48.960 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555591+0000", 2024-04-24T15:33:48.960 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5260000000000001e-06 2024-04-24T15:33:48.960 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.961 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.961 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:48.961 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555595+0000", 2024-04-24T15:33:48.961 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.41e-06 2024-04-24T15:33:48.961 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.961 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.961 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:48.961 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589377+0000", 2024-04-24T15:33:48.961 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033781868 2024-04-24T15:33:48.961 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.961 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.961 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:48.961 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589418+0000", 2024-04-24T15:33:48.961 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1461000000000002e-05 2024-04-24T15:33:48.961 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.961 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.962 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:48.962 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589470+0000", 2024-04-24T15:33:48.962 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2194999999999997e-05 2024-04-24T15:33:48.962 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:48.962 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:48.962 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:48.962 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.962 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.962 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63811 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1933312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:48.962 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555599+0000", 2024-04-24T15:33:48.962 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.328787877, 2024-04-24T15:33:48.962 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033987493000000001, 2024-04-24T15:33:48.962 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:48.962 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:48.962 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:48.963 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:48.963 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:48.963 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63811 2024-04-24T15:33:48.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.963 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:48.963 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.963 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:48.963 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555599+0000", 2024-04-24T15:33:48.963 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:48.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.963 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.963 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:48.963 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555599+0000", 2024-04-24T15:33:48.963 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:48.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.964 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.964 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:48.964 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555598+0000", 2024-04-24T15:33:48.964 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:48.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.964 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.964 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:48.964 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555600+0000", 2024-04-24T15:33:48.964 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.04e-06 2024-04-24T15:33:48.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.964 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.964 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:48.964 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555606+0000", 2024-04-24T15:33:48.964 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4199999999999998e-06 2024-04-24T15:33:48.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.965 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.965 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:48.965 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555616+0000", 2024-04-24T15:33:48.965 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0054000000000001e-05 2024-04-24T15:33:48.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.965 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.965 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:48.965 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589493+0000", 2024-04-24T15:33:48.965 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033877009999999999 2024-04-24T15:33:48.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.965 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.965 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:48.965 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589536+0000", 2024-04-24T15:33:48.965 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2855999999999998e-05 2024-04-24T15:33:48.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.966 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.966 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:48.966 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589586+0000", 2024-04-24T15:33:48.966 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0420999999999997e-05 2024-04-24T15:33:48.966 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:48.966 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:48.966 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:48.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.966 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.966 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63812 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 2015232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:48.966 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555620+0000", 2024-04-24T15:33:48.966 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.328766239, 2024-04-24T15:33:48.967 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034081696000000002, 2024-04-24T15:33:48.967 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:48.967 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:48.967 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:48.967 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:48.967 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:48.967 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63812 2024-04-24T15:33:48.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.967 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:48.967 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.967 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:48.967 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555620+0000", 2024-04-24T15:33:48.967 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:48.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.967 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.967 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:48.968 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555620+0000", 2024-04-24T15:33:48.968 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:48.968 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.968 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.968 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:48.968 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555620+0000", 2024-04-24T15:33:48.968 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:48.968 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.968 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.968 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:48.968 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555625+0000", 2024-04-24T15:33:48.968 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2859999999999999e-06 2024-04-24T15:33:48.968 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.968 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.968 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:48.969 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555633+0000", 2024-04-24T15:33:48.969 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8879999999999997e-06 2024-04-24T15:33:48.969 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.969 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.969 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:48.969 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555637+0000", 2024-04-24T15:33:48.969 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9500000000000003e-06 2024-04-24T15:33:48.969 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.969 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.969 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:48.969 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589604+0000", 2024-04-24T15:33:48.969 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033966478000000001 2024-04-24T15:33:48.969 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.969 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.969 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:48.969 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589650+0000", 2024-04-24T15:33:48.970 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6721000000000002e-05 2024-04-24T15:33:48.970 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:48.970 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:48.970 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:48.970 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589702+0000", 2024-04-24T15:33:48.970 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1709000000000002e-05 2024-04-24T15:33:48.970 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:48.970 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:48.970 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:48.970 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:48.970 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:48.970 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:48.972 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-24T15:33:48.972 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:33:48.973+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:33:49.072 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:33:49.074+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:33:49.136 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:49.136 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:33:49.136 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-04-24T15:33:49.136 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:49.146 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-24T15:33:49.179 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-24T15:33:49.187 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:33:49.188+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:33:49.273 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:33:49.275+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:33:49.314 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:49.314 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:33:49.314 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-04-24T15:33:49.314 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-04-24T15:33:49.314 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:49.325 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-24T15:33:49.329 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-04-24T15:33:49.348 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-04-24T15:33:49.367 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-04-24T15:33:49.374 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:49.376+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:49.386 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-04-24T15:33:49.405 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-04-24T15:33:49.424 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-04-24T15:33:49.443 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-04-24T15:33:49.455 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:49.455 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:49.462 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-04-24T15:33:49.463 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T15:33:49.464 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T15:33:49.464 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T15:33:49.464 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T15:33:49.464 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T15:33:49.464 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T15:33:49.464 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T15:33:49.464 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T15:33:49.466 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-24T15:33:49.475 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:33:49.476+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:33:49.575 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:33:49.577+0000 7f0f45efb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 59992) UID: 0 2024-04-24T15:33:49.600 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:49.600 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:49.612 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-24T15:33:49.680 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-24T15:33:49.696 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:49.698+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:49.777 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:49.779+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:49.781 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:49.781 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:49.791 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-24T15:33:49.879 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:49.880+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:49.955 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:49.955 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:49.965 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-24T15:33:49.980 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:33:49.981+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:33:50.081 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:50.082+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:50.088 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:50.088 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:50.097 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-24T15:33:50.182 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:33:50.183+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:33:50.219 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:50.220 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:50.229 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-24T15:33:50.283 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:33:50.284+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:33:50.352 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:50.352 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:50.363 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-24T15:33:50.384 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:50.385+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:50.485 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:50.486+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:50.487 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:50.487 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:50.497 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-24T15:33:50.586 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:50.587+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:50.620 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:50.620 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:50.630 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-24T15:33:50.669 INFO:teuthology.orchestra.run.smithi096.stderr:nodeep-scrub is unset 2024-04-24T15:33:50.687 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:33:50.688+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:33:50.763 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:50.763 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:50.773 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-24T15:33:50.790 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:33:50.791+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:33:50.891 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:50.892+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:50.937 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:50.937 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:50.947 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-24T15:33:50.991 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:50.993+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:51.069 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:51.069 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:51.079 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-24T15:33:51.092 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:51.094+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:51.193 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:51.195+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:51.202 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:51.203 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:51.213 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-24T15:33:51.294 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:33:51.295+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:33:51.337 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:51.338 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:51.347 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-24T15:33:51.395 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:51.396+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:51.470 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:51.470 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:51.476 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-24T15:33:51.479 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-24T15:33:51.496 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:51.497+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:51.597 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:51.599+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:51.620 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-04-24T15:33:51.639 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-04-24T15:33:51.650 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:51.650 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-04-24T15:33:51.650 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-04-24T15:33:51.650 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-04-24T15:33:51.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.650 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:47384 4.1f 4:fdac2fcc:::rbd_header.114a8b543e91:head [watch reconnect cookie 140672527126976 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-04-24T15:33:51.650 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:09.802333+0000", 2024-04-24T15:33:51.651 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 41.847829785000002, 2024-04-24T15:33:51.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0093213050000001, 2024-04-24T15:33:51.651 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:51.651 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:51.651 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:51.651 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:51.651 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:51.651 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 47384 2024-04-24T15:33:51.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.651 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:51.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.651 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:51.651 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.802333+0000", 2024-04-24T15:33:51.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:51.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:51.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.802333+0000", 2024-04-24T15:33:51.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:51.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:51.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.802332+0000", 2024-04-24T15:33:51.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999981 2024-04-24T15:33:51.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:51.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.802340+0000", 2024-04-24T15:33:51.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7700000000000007e-06 2024-04-24T15:33:51.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:51.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.802360+0000", 2024-04-24T15:33:51.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9959999999999999e-05 2024-04-24T15:33:51.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:51.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.802371+0000", 2024-04-24T15:33:51.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0473e-05 2024-04-24T15:33:51.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:51.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.802447+0000", 2024-04-24T15:33:51.654 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6089999999999998e-05 2024-04-24T15:33:51.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.654 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op must wait for map", 2024-04-24T15:33:51.654 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.802456+0000", 2024-04-24T15:33:51.654 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0599999999999997e-06 2024-04-24T15:33:51.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.654 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:51.654 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.808378+0000", 2024-04-24T15:33:51.654 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0059224910000000002 2024-04-24T15:33:51.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.654 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-04-24T15:33:51.655 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.808384+0000", 2024-04-24T15:33:51.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7740000000000001e-06 2024-04-24T15:33:51.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.655 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:51.655 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.809812+0000", 2024-04-24T15:33:51.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.001428043 2024-04-24T15:33:51.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.655 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:51.655 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.810082+0000", 2024-04-24T15:33:51.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00026971800000000001 2024-04-24T15:33:51.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.655 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:51.656 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.811655+0000", 2024-04-24T15:33:51.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015726869999999999 2024-04-24T15:33:51.656 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:51.656 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:51.656 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:51.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.656 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:47385 4.1f 4:fdac2fcc:::rbd_header.114a8b543e91:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-04-24T15:33:51.656 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:09.803953+0000", 2024-04-24T15:33:51.656 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 41.846209870999999, 2024-04-24T15:33:51.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0080910729999999, 2024-04-24T15:33:51.656 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:51.656 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:51.656 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:51.656 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:51.656 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:51.657 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 47385 2024-04-24T15:33:51.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.657 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:51.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:51.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.803953+0000", 2024-04-24T15:33:51.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:51.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:51.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.803953+0000", 2024-04-24T15:33:51.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:51.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:51.658 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.803952+0000", 2024-04-24T15:33:51.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999981 2024-04-24T15:33:51.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:51.658 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.803961+0000", 2024-04-24T15:33:51.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9549999999999998e-06 2024-04-24T15:33:51.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:51.658 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.803977+0000", 2024-04-24T15:33:51.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6498999999999999e-05 2024-04-24T15:33:51.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.659 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:51.659 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.803985+0000", 2024-04-24T15:33:51.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5000000000000002e-06 2024-04-24T15:33:51.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.659 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:51.659 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.804032+0000", 2024-04-24T15:33:51.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7657999999999997e-05 2024-04-24T15:33:51.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.659 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2024-04-24T15:33:51.659 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.804039+0000", 2024-04-24T15:33:51.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5719999999999999e-06 2024-04-24T15:33:51.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.660 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:51.660 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.808397+0000", 2024-04-24T15:33:51.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0043579860000000003 2024-04-24T15:33:51.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.660 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-04-24T15:33:51.660 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:09.808401+0000", 2024-04-24T15:33:51.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3089999999999997e-06 2024-04-24T15:33:51.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.660 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:51.660 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.810700+0000", 2024-04-24T15:33:51.661 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0022984699999999 2024-04-24T15:33:51.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:51.661 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.810758+0000", 2024-04-24T15:33:51.661 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8121000000000001e-05 2024-04-24T15:33:51.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:33:51.661 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.811932+0000", 2024-04-24T15:33:51.661 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001174542 2024-04-24T15:33:51.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:51.661 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.811978+0000", 2024-04-24T15:33:51.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5645000000000001e-05 2024-04-24T15:33:51.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.662 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:51.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.812044+0000", 2024-04-24T15:33:51.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6575999999999994e-05 2024-04-24T15:33:51.662 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:51.662 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:51.662 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:51.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.662 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48705 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 1851392~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:51.662 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889236+0000", 2024-04-24T15:33:51.662 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 40.760927330999998, 2024-04-24T15:33:51.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016984302, 2024-04-24T15:33:51.663 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:51.663 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:51.663 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:51.663 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:51.663 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:51.663 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48705 2024-04-24T15:33:51.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.663 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:51.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.663 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:51.663 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889236+0000", 2024-04-24T15:33:51.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:51.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.664 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:51.664 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889236+0000", 2024-04-24T15:33:51.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:51.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.664 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:51.664 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889235+0000", 2024-04-24T15:33:51.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:51.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.664 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:51.664 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889238+0000", 2024-04-24T15:33:51.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.249e-06 2024-04-24T15:33:51.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:51.665 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889243+0000", 2024-04-24T15:33:51.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6810000000000003e-06 2024-04-24T15:33:51.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:51.665 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889247+0000", 2024-04-24T15:33:51.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6780000000000002e-06 2024-04-24T15:33:51.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:51.665 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906008+0000", 2024-04-24T15:33:51.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01676099 2024-04-24T15:33:51.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.666 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:51.666 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906090+0000", 2024-04-24T15:33:51.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1508999999999995e-05 2024-04-24T15:33:51.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.666 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:51.666 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906220+0000", 2024-04-24T15:33:51.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013056999999999999 2024-04-24T15:33:51.666 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:51.666 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:51.667 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:51.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.667 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48706 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 1916928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:51.667 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889251+0000", 2024-04-24T15:33:51.667 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 40.760911898000003, 2024-04-24T15:33:51.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01718693, 2024-04-24T15:33:51.667 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:51.667 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:51.667 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:51.667 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:51.667 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:51.667 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48706 2024-04-24T15:33:51.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.667 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:51.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.668 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:51.668 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889251+0000", 2024-04-24T15:33:51.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:51.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.668 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:51.668 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889251+0000", 2024-04-24T15:33:51.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:51.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.668 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:51.668 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889251+0000", 2024-04-24T15:33:51.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:51.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:51.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889253+0000", 2024-04-24T15:33:51.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.244e-06 2024-04-24T15:33:51.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:51.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889259+0000", 2024-04-24T15:33:51.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3640000000000004e-06 2024-04-24T15:33:51.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:51.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889263+0000", 2024-04-24T15:33:51.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3720000000000001e-06 2024-04-24T15:33:51.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:51.670 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906258+0000", 2024-04-24T15:33:51.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016995310999999999 2024-04-24T15:33:51.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:51.670 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906343+0000", 2024-04-24T15:33:51.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4536000000000005e-05 2024-04-24T15:33:51.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:51.670 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906438+0000", 2024-04-24T15:33:51.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5502999999999997e-05 2024-04-24T15:33:51.670 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:51.670 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:51.671 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:51.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.671 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48707 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 1982464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:51.671 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889267+0000", 2024-04-24T15:33:51.671 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 40.760896338000002, 2024-04-24T15:33:51.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017425395, 2024-04-24T15:33:51.671 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:51.671 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:51.671 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:51.671 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:51.671 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:51.671 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48707 2024-04-24T15:33:51.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.671 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:51.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.672 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:51.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889267+0000", 2024-04-24T15:33:51.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:51.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.672 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:51.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889267+0000", 2024-04-24T15:33:51.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:51.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.672 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:51.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889266+0000", 2024-04-24T15:33:51.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:51.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.673 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:51.673 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889275+0000", 2024-04-24T15:33:51.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.568e-06 2024-04-24T15:33:51.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.673 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:51.673 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889281+0000", 2024-04-24T15:33:51.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3779999999999998e-06 2024-04-24T15:33:51.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.673 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:51.673 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889291+0000", 2024-04-24T15:33:51.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2769999999999994e-06 2024-04-24T15:33:51.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.673 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:51.674 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906476+0000", 2024-04-24T15:33:51.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017185277999999998 2024-04-24T15:33:51.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.674 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:51.674 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906567+0000", 2024-04-24T15:33:51.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0902999999999993e-05 2024-04-24T15:33:51.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.674 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:51.674 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906692+0000", 2024-04-24T15:33:51.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012541700000000001 2024-04-24T15:33:51.674 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:51.675 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:51.675 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:51.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.675 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48708 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2048000~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:51.675 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889295+0000", 2024-04-24T15:33:51.675 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 40.760867847, 2024-04-24T15:33:51.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017534984999999999, 2024-04-24T15:33:51.675 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:51.675 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:51.675 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:51.675 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:51.675 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:51.675 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48708 2024-04-24T15:33:51.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.676 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:51.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:51.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889295+0000", 2024-04-24T15:33:51.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:51.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:51.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889295+0000", 2024-04-24T15:33:51.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:51.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:51.677 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889295+0000", 2024-04-24T15:33:51.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:51.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.677 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:51.677 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889297+0000", 2024-04-24T15:33:51.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3479999999999998e-06 2024-04-24T15:33:51.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.677 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:51.677 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889304+0000", 2024-04-24T15:33:51.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3720000000000001e-06 2024-04-24T15:33:51.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:51.678 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889307+0000", 2024-04-24T15:33:51.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7160000000000001e-06 2024-04-24T15:33:51.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:51.678 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906711+0000", 2024-04-24T15:33:51.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017403452999999999 2024-04-24T15:33:51.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:51.678 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906765+0000", 2024-04-24T15:33:51.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4219999999999999e-05 2024-04-24T15:33:51.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.679 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:51.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906830+0000", 2024-04-24T15:33:51.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5277000000000005e-05 2024-04-24T15:33:51.679 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:51.679 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:51.679 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:51.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.679 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48709 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2113536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:51.679 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889312+0000", 2024-04-24T15:33:51.679 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 40.760851539999997, 2024-04-24T15:33:51.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017772013, 2024-04-24T15:33:51.679 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:51.680 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:51.680 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:51.680 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:51.680 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:51.680 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48709 2024-04-24T15:33:51.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.680 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:51.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.680 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:51.680 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889312+0000", 2024-04-24T15:33:51.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:51.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.680 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:51.680 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889312+0000", 2024-04-24T15:33:51.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:51.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:51.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889311+0000", 2024-04-24T15:33:51.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:51.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:51.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889314+0000", 2024-04-24T15:33:51.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2759999999999999e-06 2024-04-24T15:33:51.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:51.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889322+0000", 2024-04-24T15:33:51.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8589999999999994e-06 2024-04-24T15:33:51.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.682 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.682 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:51.682 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889326+0000", 2024-04-24T15:33:51.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6380000000000002e-06 2024-04-24T15:33:51.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.682 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.682 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:51.682 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906862+0000", 2024-04-24T15:33:51.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017536403999999998 2024-04-24T15:33:51.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.682 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.682 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:51.682 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.906949+0000", 2024-04-24T15:33:51.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6217000000000005e-05 2024-04-24T15:33:51.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.683 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:51.683 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907084+0000", 2024-04-24T15:33:51.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013497100000000001 2024-04-24T15:33:51.683 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:51.683 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:51.683 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:51.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.683 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48710 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2179072~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:51.683 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889332+0000", 2024-04-24T15:33:51.683 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 40.760830726000002, 2024-04-24T15:33:51.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017944720000000001, 2024-04-24T15:33:51.684 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:51.684 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:51.684 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:51.684 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:51.684 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:51.684 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48710 2024-04-24T15:33:51.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.684 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:51.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:51.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889332+0000", 2024-04-24T15:33:51.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:51.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:51.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889332+0000", 2024-04-24T15:33:51.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:51.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:51.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889332+0000", 2024-04-24T15:33:51.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:51.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:51.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889334+0000", 2024-04-24T15:33:51.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4210000000000002e-06 2024-04-24T15:33:51.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:51.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889340+0000", 2024-04-24T15:33:51.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7209999999999999e-06 2024-04-24T15:33:51.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:51.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889344+0000", 2024-04-24T15:33:51.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.726e-06 2024-04-24T15:33:51.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:51.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907119+0000", 2024-04-24T15:33:51.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017775551000000001 2024-04-24T15:33:51.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:51.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907190+0000", 2024-04-24T15:33:51.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0634000000000002e-05 2024-04-24T15:33:51.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.687 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:51.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907277+0000", 2024-04-24T15:33:51.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7120999999999995e-05 2024-04-24T15:33:51.687 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:51.687 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:51.687 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:51.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.687 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48711 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2244608~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:51.687 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889348+0000", 2024-04-24T15:33:51.687 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 40.760815108999999, 2024-04-24T15:33:51.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018188109000000001, 2024-04-24T15:33:51.688 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:51.688 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:51.688 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:51.688 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:51.688 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:51.688 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48711 2024-04-24T15:33:51.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.688 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:51.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:51.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889348+0000", 2024-04-24T15:33:51.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:51.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:51.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889348+0000", 2024-04-24T15:33:51.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:51.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:51.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889348+0000", 2024-04-24T15:33:51.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:51.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:51.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889350+0000", 2024-04-24T15:33:51.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2330000000000001e-06 2024-04-24T15:33:51.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:51.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889356+0000", 2024-04-24T15:33:51.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8769999999999998e-06 2024-04-24T15:33:51.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:51.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889359+0000", 2024-04-24T15:33:51.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3699999999999999e-06 2024-04-24T15:33:51.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:51.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907316+0000", 2024-04-24T15:33:51.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017956857999999999 2024-04-24T15:33:51.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:51.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907402+0000", 2024-04-24T15:33:51.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5927000000000006e-05 2024-04-24T15:33:51.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:51.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907536+0000", 2024-04-24T15:33:51.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000134258 2024-04-24T15:33:51.691 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:51.691 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:51.691 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:51.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.691 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48712 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2310144~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:51.691 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889363+0000", 2024-04-24T15:33:51.691 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 40.76079979, 2024-04-24T15:33:51.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018314815000000002, 2024-04-24T15:33:51.692 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:51.692 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:51.692 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:51.692 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:51.692 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:51.692 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48712 2024-04-24T15:33:51.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.692 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:51.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.692 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:51.692 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889363+0000", 2024-04-24T15:33:51.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:51.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:51.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889363+0000", 2024-04-24T15:33:51.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:51.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:51.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889363+0000", 2024-04-24T15:33:51.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:51.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:51.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889366+0000", 2024-04-24T15:33:51.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6709999999999999e-06 2024-04-24T15:33:51.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.694 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:51.694 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889373+0000", 2024-04-24T15:33:51.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1269999999999997e-06 2024-04-24T15:33:51.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.694 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:51.694 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889377+0000", 2024-04-24T15:33:51.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1219999999999997e-06 2024-04-24T15:33:51.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.694 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:51.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907560+0000", 2024-04-24T15:33:51.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018183357000000001 2024-04-24T15:33:51.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:51.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907612+0000", 2024-04-24T15:33:51.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2172999999999997e-05 2024-04-24T15:33:51.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:51.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907678+0000", 2024-04-24T15:33:51.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5736999999999998e-05 2024-04-24T15:33:51.695 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:51.695 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:51.695 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:51.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.696 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48713 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2375680~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:51.696 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889381+0000", 2024-04-24T15:33:51.696 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 40.760781897000001, 2024-04-24T15:33:51.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018556019, 2024-04-24T15:33:51.696 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:51.696 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:51.696 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:51.696 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:51.696 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:51.696 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48713 2024-04-24T15:33:51.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.696 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:51.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.696 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:51.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889381+0000", 2024-04-24T15:33:51.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:51.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.697 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:51.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889381+0000", 2024-04-24T15:33:51.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:51.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.697 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:51.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889381+0000", 2024-04-24T15:33:51.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:51.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.698 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:51.698 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889388+0000", 2024-04-24T15:33:51.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8809999999999998e-06 2024-04-24T15:33:51.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.698 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:51.698 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889394+0000", 2024-04-24T15:33:51.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0410000000000003e-06 2024-04-24T15:33:51.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.698 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:51.698 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889397+0000", 2024-04-24T15:33:51.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5939999999999998e-06 2024-04-24T15:33:51.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:51.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907711+0000", 2024-04-24T15:33:51.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018313949999999999 2024-04-24T15:33:51.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:51.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907802+0000", 2024-04-24T15:33:51.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0307000000000001e-05 2024-04-24T15:33:51.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:51.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907937+0000", 2024-04-24T15:33:51.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013561899999999999 2024-04-24T15:33:51.699 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:51.700 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:51.700 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:51.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.700 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48714 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2441216~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:51.700 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889435+0000", 2024-04-24T15:33:51.700 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 40.760728549, 2024-04-24T15:33:51.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018636040999999999, 2024-04-24T15:33:51.700 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:51.700 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:51.700 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:51.700 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:51.700 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:51.700 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48714 2024-04-24T15:33:51.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.701 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:51.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:51.701 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889435+0000", 2024-04-24T15:33:51.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:51.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:51.701 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889435+0000", 2024-04-24T15:33:51.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:51.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:51.701 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889434+0000", 2024-04-24T15:33:51.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:51.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.702 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:51.702 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889437+0000", 2024-04-24T15:33:51.702 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.875e-06 2024-04-24T15:33:51.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.702 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:51.702 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889444+0000", 2024-04-24T15:33:51.702 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9870000000000002e-06 2024-04-24T15:33:51.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.702 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:51.702 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889448+0000", 2024-04-24T15:33:51.703 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3030000000000001e-06 2024-04-24T15:33:51.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.703 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:51.703 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.907952+0000", 2024-04-24T15:33:51.703 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018504245999999999 2024-04-24T15:33:51.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.703 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:51.703 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908007+0000", 2024-04-24T15:33:51.703 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4579e-05 2024-04-24T15:33:51.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.703 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:51.703 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908071+0000", 2024-04-24T15:33:51.703 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3615999999999995e-05 2024-04-24T15:33:51.704 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:51.704 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:51.704 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:51.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.704 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.704 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48715 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2506752~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:51.704 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889453+0000", 2024-04-24T15:33:51.704 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 40.760710185000001, 2024-04-24T15:33:51.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018873935000000001, 2024-04-24T15:33:51.704 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:51.704 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:51.704 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:51.704 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:51.704 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:51.704 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48715 2024-04-24T15:33:51.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.705 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:51.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:51.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889453+0000", 2024-04-24T15:33:51.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:51.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:51.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889453+0000", 2024-04-24T15:33:51.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:51.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:51.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889452+0000", 2024-04-24T15:33:51.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:51.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.706 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:51.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889459+0000", 2024-04-24T15:33:51.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9269999999999999e-06 2024-04-24T15:33:51.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.706 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:51.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889468+0000", 2024-04-24T15:33:51.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2390000000000002e-06 2024-04-24T15:33:51.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.707 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:51.707 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889475+0000", 2024-04-24T15:33:51.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3939999999999998e-06 2024-04-24T15:33:51.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.707 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:51.707 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908103+0000", 2024-04-24T15:33:51.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018628388999999999 2024-04-24T15:33:51.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.707 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:51.707 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908193+0000", 2024-04-24T15:33:51.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9343000000000002e-05 2024-04-24T15:33:51.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:51.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908327+0000", 2024-04-24T15:33:51.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013415899999999999 2024-04-24T15:33:51.708 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:51.708 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:51.708 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:51.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.708 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48716 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2572288~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:51.708 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889480+0000", 2024-04-24T15:33:51.708 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 40.760683335000003, 2024-04-24T15:33:51.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018992862999999999, 2024-04-24T15:33:51.708 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:51.708 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:51.709 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:51.709 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:51.709 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:51.709 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48716 2024-04-24T15:33:51.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.709 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:51.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:51.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889480+0000", 2024-04-24T15:33:51.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:51.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:51.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889480+0000", 2024-04-24T15:33:51.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:51.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:51.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889479+0000", 2024-04-24T15:33:51.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:51.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:51.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889482+0000", 2024-04-24T15:33:51.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4729999999999999e-06 2024-04-24T15:33:51.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:51.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889488+0000", 2024-04-24T15:33:51.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1750000000000002e-06 2024-04-24T15:33:51.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:51.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889501+0000", 2024-04-24T15:33:51.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.288e-05 2024-04-24T15:33:51.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:51.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908345+0000", 2024-04-24T15:33:51.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018843835999999999 2024-04-24T15:33:51.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:51.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908395+0000", 2024-04-24T15:33:51.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9765999999999998e-05 2024-04-24T15:33:51.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:51.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908473+0000", 2024-04-24T15:33:51.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8159999999999997e-05 2024-04-24T15:33:51.712 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:51.712 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:51.712 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:51.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.712 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48717 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2637824~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:51.713 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889506+0000", 2024-04-24T15:33:51.713 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 40.760657084000002, 2024-04-24T15:33:51.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.019241864000000001, 2024-04-24T15:33:51.713 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:51.713 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:51.713 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:51.713 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:51.713 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:51.713 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48717 2024-04-24T15:33:51.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.713 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:51.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.713 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:51.713 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889506+0000", 2024-04-24T15:33:51.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:51.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:51.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889506+0000", 2024-04-24T15:33:51.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:51.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:51.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889506+0000", 2024-04-24T15:33:51.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:51.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:51.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889508+0000", 2024-04-24T15:33:51.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6900000000000001e-06 2024-04-24T15:33:51.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:51.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889515+0000", 2024-04-24T15:33:51.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5409999999999997e-06 2024-04-24T15:33:51.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:51.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889519+0000", 2024-04-24T15:33:51.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7129999999999999e-06 2024-04-24T15:33:51.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.716 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:51.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908520+0000", 2024-04-24T15:33:51.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.019001606000000001 2024-04-24T15:33:51.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.716 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:51.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908612+0000", 2024-04-24T15:33:51.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1909000000000004e-05 2024-04-24T15:33:51.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.716 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:51.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908748+0000", 2024-04-24T15:33:51.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000135847 2024-04-24T15:33:51.717 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:51.717 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:51.717 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:51.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.717 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:48718 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:17 [read 2703360~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:51.717 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:10.889523+0000", 2024-04-24T15:33:51.717 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 40.760640092000003, 2024-04-24T15:33:51.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.019451725999999999, 2024-04-24T15:33:51.717 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:51.717 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:51.717 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:51.717 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:51.717 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:51.718 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48718 2024-04-24T15:33:51.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.718 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:51.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:51.718 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889523+0000", 2024-04-24T15:33:51.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:51.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:51.718 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889523+0000", 2024-04-24T15:33:51.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:51.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:51.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889523+0000", 2024-04-24T15:33:51.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:51.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:51.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889525+0000", 2024-04-24T15:33:51.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9970000000000001e-06 2024-04-24T15:33:51.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:51.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889530+0000", 2024-04-24T15:33:51.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7610000000000004e-06 2024-04-24T15:33:51.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:51.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.889534+0000", 2024-04-24T15:33:51.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6119999999999999e-06 2024-04-24T15:33:51.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:51.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908775+0000", 2024-04-24T15:33:51.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.019240709000000002 2024-04-24T15:33:51.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:51.721 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908856+0000", 2024-04-24T15:33:51.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0788e-05 2024-04-24T15:33:51.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.721 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:51.721 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:10.908975+0000", 2024-04-24T15:33:51.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011927800000000001 2024-04-24T15:33:51.721 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:51.721 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:51.721 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:51.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.721 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:49872 4.1f 4:fe48f4c1:::rbd_data.114ab402c616.0000000000000014:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-04-24T15:33:51.721 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:12.416730+0000", 2024-04-24T15:33:51.721 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 39.233433114, 2024-04-24T15:33:51.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.020522327999999999, 2024-04-24T15:33:51.722 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:51.722 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:51.722 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:51.722 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:51.722 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:51.722 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 49872 2024-04-24T15:33:51.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.722 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:51.722 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.722 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:51.722 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.416730+0000", 2024-04-24T15:33:51.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:51.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:51.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.416730+0000", 2024-04-24T15:33:51.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:51.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:51.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.416729+0000", 2024-04-24T15:33:51.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.999999 2024-04-24T15:33:51.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:51.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.416734+0000", 2024-04-24T15:33:51.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0180000000000001e-06 2024-04-24T15:33:51.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.724 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:51.724 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.416744+0000", 2024-04-24T15:33:51.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0069e-05 2024-04-24T15:33:51.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.724 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:51.724 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.416749+0000", 2024-04-24T15:33:51.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7190000000000001e-06 2024-04-24T15:33:51.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.724 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:51.725 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.416777+0000", 2024-04-24T15:33:51.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7532999999999999e-05 2024-04-24T15:33:51.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.725 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:51.725 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.416824+0000", 2024-04-24T15:33:51.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7738000000000003e-05 2024-04-24T15:33:51.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.725 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:33:51.725 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.437145+0000", 2024-04-24T15:33:51.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.020320416000000001 2024-04-24T15:33:51.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.726 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:51.726 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.437189+0000", 2024-04-24T15:33:51.726 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4493000000000002e-05 2024-04-24T15:33:51.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.726 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.726 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:51.726 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:12.437252+0000", 2024-04-24T15:33:51.726 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3110999999999999e-05 2024-04-24T15:33:51.726 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:51.726 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:51.726 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:51.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.726 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.726 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:52545 4.a 4:5331820a:::rbd_data.114ab867d54d.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-04-24T15:33:51.727 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:14.399696+0000", 2024-04-24T15:33:51.727 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.250466711000001, 2024-04-24T15:33:51.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016877893000000001, 2024-04-24T15:33:51.727 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:51.727 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:51.727 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:51.727 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:51.727 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:51.727 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 52545 2024-04-24T15:33:51.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.727 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:51.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:51.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.399696+0000", 2024-04-24T15:33:51.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:51.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:51.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.399696+0000", 2024-04-24T15:33:51.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:51.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:51.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.399695+0000", 2024-04-24T15:33:51.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999981 2024-04-24T15:33:51.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:51.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.399704+0000", 2024-04-24T15:33:51.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.098e-06 2024-04-24T15:33:51.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:51.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.399721+0000", 2024-04-24T15:33:51.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7088999999999999e-05 2024-04-24T15:33:51.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:51.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.399728+0000", 2024-04-24T15:33:51.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5699999999999998e-06 2024-04-24T15:33:51.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:51.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.399775+0000", 2024-04-24T15:33:51.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7445999999999999e-05 2024-04-24T15:33:51.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.730 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:51.730 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.399840+0000", 2024-04-24T15:33:51.730 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5492000000000002e-05 2024-04-24T15:33:51.730 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.730 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.730 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:33:51.730 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.416468+0000", 2024-04-24T15:33:51.730 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016627527 2024-04-24T15:33:51.730 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.730 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.730 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:51.730 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.416521+0000", 2024-04-24T15:33:51.730 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2988000000000002e-05 2024-04-24T15:33:51.730 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.731 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.731 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:51.731 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:14.416574+0000", 2024-04-24T15:33:51.731 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3363999999999999e-05 2024-04-24T15:33:51.731 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:51.731 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:51.731 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:51.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.731 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.731 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:66966 4.1f 4:fb3b69af:::rbd_data.114aed92ca6e.000000000000001a:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-04-24T15:33:51.731 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:21.265525+0000", 2024-04-24T15:33:51.731 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 30.384638169999999, 2024-04-24T15:33:51.731 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018862027999999999, 2024-04-24T15:33:51.731 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:51.731 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:51.732 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:51.732 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:51.732 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:51.732 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 66966 2024-04-24T15:33:51.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.732 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:51.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:51.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:21.265525+0000", 2024-04-24T15:33:51.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:51.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:51.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:21.265525+0000", 2024-04-24T15:33:51.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:51.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:51.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:21.265523+0000", 2024-04-24T15:33:51.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999981 2024-04-24T15:33:51.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:51.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:21.265536+0000", 2024-04-24T15:33:51.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2489e-05 2024-04-24T15:33:51.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:51.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:21.265550+0000", 2024-04-24T15:33:51.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4218e-05 2024-04-24T15:33:51.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.734 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:51.734 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:21.265556+0000", 2024-04-24T15:33:51.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.241e-06 2024-04-24T15:33:51.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.734 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.734 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:51.734 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:21.265606+0000", 2024-04-24T15:33:51.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9972000000000002e-05 2024-04-24T15:33:51.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.734 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.734 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:51.734 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:21.265670+0000", 2024-04-24T15:33:51.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3425999999999999e-05 2024-04-24T15:33:51.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.735 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:33:51.735 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:21.284297+0000", 2024-04-24T15:33:51.735 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018627517999999999 2024-04-24T15:33:51.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.735 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:51.735 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:21.284339+0000", 2024-04-24T15:33:51.735 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1628000000000002e-05 2024-04-24T15:33:51.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.735 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:51.735 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:21.284387+0000", 2024-04-24T15:33:51.735 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8242000000000003e-05 2024-04-24T15:33:51.735 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:51.736 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:51.736 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:51.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.736 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.736 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:75519 4.a 4:573fa540:::rbd_data.114a858e6641.0000000000000005:head [delete] snapc 17=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e105)", 2024-04-24T15:33:51.736 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:42.638138+0000", 2024-04-24T15:33:51.736 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.0120254549999999, 2024-04-24T15:33:51.736 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0071697449999999, 2024-04-24T15:33:51.736 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:51.736 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:51.736 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:51.736 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:51.736 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:51.736 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 75519 2024-04-24T15:33:51.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.736 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:51.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:51.737 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:42.638138+0000", 2024-04-24T15:33:51.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:51.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:51.737 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:42.638138+0000", 2024-04-24T15:33:51.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:51.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:51.737 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:42.638137+0000", 2024-04-24T15:33:51.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999986 2024-04-24T15:33:51.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.738 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:51.738 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:42.638142+0000", 2024-04-24T15:33:51.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8989999999999998e-06 2024-04-24T15:33:51.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.738 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.738 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:51.738 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:42.638154+0000", 2024-04-24T15:33:51.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1572e-05 2024-04-24T15:33:51.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.738 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.738 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:51.738 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:42.638160+0000", 2024-04-24T15:33:51.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3899999999999998e-06 2024-04-24T15:33:51.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:51.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:42.638203+0000", 2024-04-24T15:33:51.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2252000000000001e-05 2024-04-24T15:33:51.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op must wait for map", 2024-04-24T15:33:51.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:42.638209+0000", 2024-04-24T15:33:51.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4359999999999999e-06 2024-04-24T15:33:51.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:51.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:42.639589+0000", 2024-04-24T15:33:51.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001379903 2024-04-24T15:33:51.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-04-24T15:33:51.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:42.639600+0000", 2024-04-24T15:33:51.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0719e-05 2024-04-24T15:33:51.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:51.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:43.643334+0000", 2024-04-24T15:33:51.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0037341790000001 2024-04-24T15:33:51.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:51.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:43.643460+0000", 2024-04-24T15:33:51.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012588700000000001 2024-04-24T15:33:51.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 6", 2024-04-24T15:33:51.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:43.643622+0000", 2024-04-24T15:33:51.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016211400000000001 2024-04-24T15:33:51.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:33:51.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:43.644400+0000", 2024-04-24T15:33:51.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000777708 2024-04-24T15:33:51.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.742 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:33:51.742 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:43.645196+0000", 2024-04-24T15:33:51.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00079659900000000005 2024-04-24T15:33:51.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.742 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:51.742 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:43.645227+0000", 2024-04-24T15:33:51.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1250000000000001e-05 2024-04-24T15:33:51.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:51.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:51.742 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:51.742 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:43.645307+0000", 2024-04-24T15:33:51.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0035000000000002e-05 2024-04-24T15:33:51.743 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:51.743 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:51.743 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:51.743 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:51.743 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:51.743 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:51.743 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 6, 4] out_osds: [1, 5, 7] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 7, 6] 2024-04-24T15:33:51.743 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-24T15:33:51.743 INFO:tasks.thrashosds.thrasher:Adding osd 7 2024-04-24T15:33:51.743 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 7 2024-04-24T15:33:51.744 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-04-24T15:33:51.745 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-24T15:33:51.746 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-04-24T15:33:51.746 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-04-24T15:33:51.746 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-04-24T15:33:51.746 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-04-24T15:33:51.747 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:51.746+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:51.762 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-04-24T15:33:51.764 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T15:33:51.765 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T15:33:51.765 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T15:33:51.765 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T15:33:51.765 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T15:33:51.765 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T15:33:51.765 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T15:33:51.765 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T15:33:51.846 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:51.847+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:51.896 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:51.896 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:33:51.897 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-04-24T15:33:51.897 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:51.907 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-24T15:33:51.948 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:33:51.949+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:33:52.048 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:52.050+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:52.073 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:52.073 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:33:52.073 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-04-24T15:33:52.073 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-04-24T15:33:52.073 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:52.083 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-24T15:33:52.110 INFO:tasks.ceph.mon.a.smithi096.stderr:2024-04-24T15:33:52.111+0000 7fa88ddc8700 -1 mon.a@0(leader).osd e114 definitely_dead 0 2024-04-24T15:33:52.149 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:33:52.150+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:33:52.205 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:52.205 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-04-24T15:33:52.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-04-24T15:33:52.205 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [] 2024-04-24T15:33:52.206 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:52.215 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-24T15:33:52.257 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:33:52.258+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:33:52.350 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:52.352+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:52.379 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:52.380 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:33:52.380 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-04-24T15:33:52.380 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:52.389 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-24T15:33:52.452 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:52.453+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:52.512 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:52.512 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:33:52.512 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-04-24T15:33:52.512 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-04-24T15:33:52.512 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:52.522 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-24T15:33:52.553 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:33:52.555+0000 7f0f45efb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 59992) UID: 0 2024-04-24T15:33:52.653 INFO:teuthology.orchestra.run.smithi096.stderr:marked in osd.7. 2024-04-24T15:33:52.654 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:33:52.655+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:33:52.655 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:52.655 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-04-24T15:33:52.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-04-24T15:33:52.656 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-04-24T15:33:52.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.656 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:2522 4.6 4:60ceedf0:::rbd_data.114a1f60cb66.000000000000011d:head [set-alloc-hint object_size 262144 write_size 262144,write 110592~2048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-24T15:33:52.656 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:23.088240+0000", 2024-04-24T15:33:52.656 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 89.566785637999999, 2024-04-24T15:33:52.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.988232797, 2024-04-24T15:33:52.656 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:52.656 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:52.656 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:52.656 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:52.656 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:52.656 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2522 2024-04-24T15:33:52.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.657 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:52.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:52.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088240+0000", 2024-04-24T15:33:52.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:52.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:52.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088240+0000", 2024-04-24T15:33:52.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:52.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:52.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088238+0000", 2024-04-24T15:33:52.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999981 2024-04-24T15:33:52.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:52.658 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088254+0000", 2024-04-24T15:33:52.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5562000000000001e-05 2024-04-24T15:33:52.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:52.658 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088275+0000", 2024-04-24T15:33:52.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1256000000000001e-05 2024-04-24T15:33:52.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:52.659 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088283+0000", 2024-04-24T15:33:52.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3219999999999993e-06 2024-04-24T15:33:52.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.659 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:52.659 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088351+0000", 2024-04-24T15:33:52.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7595000000000006e-05 2024-04-24T15:33:52.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.659 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-04-24T15:33:52.659 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088362+0000", 2024-04-24T15:33:52.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1764e-05 2024-04-24T15:33:52.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.659 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:52.660 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.073748+0000", 2024-04-24T15:33:52.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98538541899999998 2024-04-24T15:33:52.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.660 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:52.660 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.073925+0000", 2024-04-24T15:33:52.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017747600000000001 2024-04-24T15:33:52.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.660 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-24T15:33:52.660 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.074092+0000", 2024-04-24T15:33:52.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000166622 2024-04-24T15:33:52.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.660 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:33:52.661 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.074774+0000", 2024-04-24T15:33:52.661 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00068225000000000002 2024-04-24T15:33:52.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:33:52.661 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.076412+0000", 2024-04-24T15:33:52.661 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001638189 2024-04-24T15:33:52.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:52.661 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.076446+0000", 2024-04-24T15:33:52.661 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3587000000000002e-05 2024-04-24T15:33:52.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:52.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.076473+0000", 2024-04-24T15:33:52.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6886999999999999e-05 2024-04-24T15:33:52.662 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:52.662 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:52.662 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:52.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.662 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43121 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 5865472~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:52.662 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094178+0000", 2024-04-24T15:33:52.662 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 45.560847955, 2024-04-24T15:33:52.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032860598999999997, 2024-04-24T15:33:52.662 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:52.662 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:52.663 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:52.663 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:52.663 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:52.663 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43121 2024-04-24T15:33:52.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.663 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:52.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.663 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:52.663 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094178+0000", 2024-04-24T15:33:52.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:52.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.663 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:52.663 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094178+0000", 2024-04-24T15:33:52.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:52.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.664 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:52.664 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094178+0000", 2024-04-24T15:33:52.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:52.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.664 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:52.664 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094180+0000", 2024-04-24T15:33:52.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9989999999999998e-06 2024-04-24T15:33:52.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.664 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:52.664 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094185+0000", 2024-04-24T15:33:52.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.541e-06 2024-04-24T15:33:52.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:52.665 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094189+0000", 2024-04-24T15:33:52.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0509999999999998e-06 2024-04-24T15:33:52.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:52.665 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.126877+0000", 2024-04-24T15:33:52.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032687473000000002 2024-04-24T15:33:52.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.666 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:52.666 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.126952+0000", 2024-04-24T15:33:52.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5924e-05 2024-04-24T15:33:52.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.666 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:52.666 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127038+0000", 2024-04-24T15:33:52.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5908999999999994e-05 2024-04-24T15:33:52.666 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:52.666 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:52.666 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:52.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.666 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43122 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 6291456~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:52.666 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094193+0000", 2024-04-24T15:33:52.667 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 45.560832711000003, 2024-04-24T15:33:52.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032978383999999999, 2024-04-24T15:33:52.667 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:52.667 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:52.667 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:52.667 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:52.667 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:52.667 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43122 2024-04-24T15:33:52.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.667 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:52.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.667 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:52.667 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094193+0000", 2024-04-24T15:33:52.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:52.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.668 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:52.668 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094193+0000", 2024-04-24T15:33:52.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:52.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.668 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:52.668 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094193+0000", 2024-04-24T15:33:52.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:52.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.668 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:52.668 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094195+0000", 2024-04-24T15:33:52.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3269999999999999e-06 2024-04-24T15:33:52.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:52.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094201+0000", 2024-04-24T15:33:52.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9850000000000003e-06 2024-04-24T15:33:52.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:52.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094205+0000", 2024-04-24T15:33:52.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8430000000000003e-06 2024-04-24T15:33:52.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:52.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127068+0000", 2024-04-24T15:33:52.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032863305000000002 2024-04-24T15:33:52.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:52.670 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127113+0000", 2024-04-24T15:33:52.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4412000000000001e-05 2024-04-24T15:33:52.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:52.670 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127171+0000", 2024-04-24T15:33:52.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8807999999999997e-05 2024-04-24T15:33:52.670 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:52.670 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:52.670 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:52.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.671 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43123 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 6717440~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:52.671 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094209+0000", 2024-04-24T15:33:52.671 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 45.560816545000002, 2024-04-24T15:33:52.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033144583999999998, 2024-04-24T15:33:52.671 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:52.671 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:52.671 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:52.671 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:52.671 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:52.671 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43123 2024-04-24T15:33:52.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.671 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:52.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.671 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:52.671 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094209+0000", 2024-04-24T15:33:52.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:52.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.672 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:52.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094209+0000", 2024-04-24T15:33:52.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:52.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.672 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:52.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094209+0000", 2024-04-24T15:33:52.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:52.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.672 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:52.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094211+0000", 2024-04-24T15:33:52.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2089999999999999e-06 2024-04-24T15:33:52.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.673 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:52.673 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094217+0000", 2024-04-24T15:33:52.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6799999999999998e-06 2024-04-24T15:33:52.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.673 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:52.673 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094220+0000", 2024-04-24T15:33:52.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.444e-06 2024-04-24T15:33:52.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.673 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:52.673 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127190+0000", 2024-04-24T15:33:52.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032969471 2024-04-24T15:33:52.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.674 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:52.674 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127269+0000", 2024-04-24T15:33:52.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9246999999999995e-05 2024-04-24T15:33:52.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.674 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:52.674 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127354+0000", 2024-04-24T15:33:52.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4853999999999998e-05 2024-04-24T15:33:52.674 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:52.674 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:52.674 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:52.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.675 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43124 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 7143424~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:52.675 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094224+0000", 2024-04-24T15:33:52.675 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 45.560801572000003, 2024-04-24T15:33:52.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033271955999999998, 2024-04-24T15:33:52.675 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:52.675 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:52.675 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:52.675 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:52.675 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:52.675 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43124 2024-04-24T15:33:52.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.675 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:52.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:52.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094224+0000", 2024-04-24T15:33:52.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:52.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:52.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094224+0000", 2024-04-24T15:33:52.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:52.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:52.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094224+0000", 2024-04-24T15:33:52.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:52.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.677 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:52.677 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094226+0000", 2024-04-24T15:33:52.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9039999999999999e-06 2024-04-24T15:33:52.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.677 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:52.677 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094233+0000", 2024-04-24T15:33:52.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7970000000000003e-06 2024-04-24T15:33:52.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.677 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:52.677 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094236+0000", 2024-04-24T15:33:52.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6339999999999999e-06 2024-04-24T15:33:52.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:52.678 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127390+0000", 2024-04-24T15:33:52.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.03315353 2024-04-24T15:33:52.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:52.678 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127439+0000", 2024-04-24T15:33:52.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8890000000000001e-05 2024-04-24T15:33:52.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:52.678 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127496+0000", 2024-04-24T15:33:52.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7476999999999999e-05 2024-04-24T15:33:52.679 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:52.679 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:52.679 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:52.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.679 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43125 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 7569408~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:52.679 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094240+0000", 2024-04-24T15:33:52.679 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 45.560785484999997, 2024-04-24T15:33:52.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033446278000000003, 2024-04-24T15:33:52.679 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:52.679 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:52.679 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:52.679 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:52.679 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:52.679 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43125 2024-04-24T15:33:52.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.680 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:52.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.680 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:52.680 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094240+0000", 2024-04-24T15:33:52.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:52.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.680 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:52.680 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094240+0000", 2024-04-24T15:33:52.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:52.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.680 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:52.680 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094240+0000", 2024-04-24T15:33:52.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:52.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:52.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094247+0000", 2024-04-24T15:33:52.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6209999999999998e-06 2024-04-24T15:33:52.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:52.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094253+0000", 2024-04-24T15:33:52.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0739999999999998e-06 2024-04-24T15:33:52.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:52.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094256+0000", 2024-04-24T15:33:52.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7699999999999999e-06 2024-04-24T15:33:52.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.682 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:52.682 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127526+0000", 2024-04-24T15:33:52.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033269703999999997 2024-04-24T15:33:52.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.682 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.682 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:52.682 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127603+0000", 2024-04-24T15:33:52.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6964000000000004e-05 2024-04-24T15:33:52.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.682 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.682 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:52.682 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127687+0000", 2024-04-24T15:33:52.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3431999999999997e-05 2024-04-24T15:33:52.682 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:52.683 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:52.683 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:52.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.683 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43126 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 7995392~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:52.683 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094261+0000", 2024-04-24T15:33:52.683 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 45.560764982000002, 2024-04-24T15:33:52.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033555201999999999, 2024-04-24T15:33:52.683 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:52.683 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:52.683 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:52.683 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:52.683 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:52.683 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43126 2024-04-24T15:33:52.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.683 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:52.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:52.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094261+0000", 2024-04-24T15:33:52.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:52.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:52.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094261+0000", 2024-04-24T15:33:52.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:52.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:52.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094260+0000", 2024-04-24T15:33:52.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:52.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:52.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094263+0000", 2024-04-24T15:33:52.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2579999999999998e-06 2024-04-24T15:33:52.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:52.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094269+0000", 2024-04-24T15:33:52.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.823e-06 2024-04-24T15:33:52.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:52.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094272+0000", 2024-04-24T15:33:52.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8999999999999999e-06 2024-04-24T15:33:52.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:52.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127720+0000", 2024-04-24T15:33:52.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033447521000000001 2024-04-24T15:33:52.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:52.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127765+0000", 2024-04-24T15:33:52.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5352000000000002e-05 2024-04-24T15:33:52.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:52.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127816+0000", 2024-04-24T15:33:52.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0689999999999997e-05 2024-04-24T15:33:52.687 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:52.687 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:52.687 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:52.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.687 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63800 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1032192~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:52.687 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555407+0000", 2024-04-24T15:33:52.687 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.099618995999997, 2024-04-24T15:33:52.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032809030000000003, 2024-04-24T15:33:52.687 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:52.687 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:52.687 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:52.687 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:52.688 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:52.688 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63800 2024-04-24T15:33:52.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.688 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:52.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:52.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555407+0000", 2024-04-24T15:33:52.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:52.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:52.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555407+0000", 2024-04-24T15:33:52.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:52.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:52.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555406+0000", 2024-04-24T15:33:52.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:52.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:52.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555409+0000", 2024-04-24T15:33:52.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.706e-06 2024-04-24T15:33:52.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:52.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555416+0000", 2024-04-24T15:33:52.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1609999999999997e-06 2024-04-24T15:33:52.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:52.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555420+0000", 2024-04-24T15:33:52.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0369999999999996e-06 2024-04-24T15:33:52.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:52.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588088+0000", 2024-04-24T15:33:52.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032668228000000001 2024-04-24T15:33:52.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:52.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588143+0000", 2024-04-24T15:33:52.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4852999999999998e-05 2024-04-24T15:33:52.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:52.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588216+0000", 2024-04-24T15:33:52.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2467999999999998e-05 2024-04-24T15:33:52.691 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:52.691 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:52.691 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:52.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.691 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63801 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1114112~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:52.691 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555425+0000", 2024-04-24T15:33:52.691 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.099601088999997, 2024-04-24T15:33:52.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032924691999999998, 2024-04-24T15:33:52.691 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:52.691 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:52.691 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:52.691 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:52.692 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:52.692 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63801 2024-04-24T15:33:52.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.692 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:52.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.692 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:52.692 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555425+0000", 2024-04-24T15:33:52.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:52.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.692 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:52.692 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555425+0000", 2024-04-24T15:33:52.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:52.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.692 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:52.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555424+0000", 2024-04-24T15:33:52.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:52.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:52.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555427+0000", 2024-04-24T15:33:52.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2350000000000002e-06 2024-04-24T15:33:52.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:52.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555432+0000", 2024-04-24T15:33:52.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.434e-06 2024-04-24T15:33:52.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:52.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555435+0000", 2024-04-24T15:33:52.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2009999999999999e-06 2024-04-24T15:33:52.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.694 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:52.694 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588237+0000", 2024-04-24T15:33:52.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032801508 2024-04-24T15:33:52.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.694 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:52.694 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588281+0000", 2024-04-24T15:33:52.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4100999999999997e-05 2024-04-24T15:33:52.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.694 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:52.694 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588349+0000", 2024-04-24T15:33:52.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8551000000000001e-05 2024-04-24T15:33:52.695 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:52.695 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:52.695 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:52.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.695 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63802 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1196032~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:52.695 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555439+0000", 2024-04-24T15:33:52.695 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.099586567999999, 2024-04-24T15:33:52.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033057744999999999, 2024-04-24T15:33:52.695 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:52.695 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:52.695 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:52.695 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:52.696 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:52.696 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63802 2024-04-24T15:33:52.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.696 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:52.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.696 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:52.696 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555439+0000", 2024-04-24T15:33:52.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:52.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.696 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:52.696 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555439+0000", 2024-04-24T15:33:52.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:52.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.697 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:52.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555439+0000", 2024-04-24T15:33:52.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:52.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.697 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:52.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555445+0000", 2024-04-24T15:33:52.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4019999999999999e-06 2024-04-24T15:33:52.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.697 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:52.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555452+0000", 2024-04-24T15:33:52.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8600000000000004e-06 2024-04-24T15:33:52.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.697 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:52.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555461+0000", 2024-04-24T15:33:52.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0499999999999997e-06 2024-04-24T15:33:52.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.698 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:52.698 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588378+0000", 2024-04-24T15:33:52.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032916527000000001 2024-04-24T15:33:52.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.698 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:52.698 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588423+0000", 2024-04-24T15:33:52.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5092999999999997e-05 2024-04-24T15:33:52.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.698 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:52.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588497+0000", 2024-04-24T15:33:52.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4140999999999994e-05 2024-04-24T15:33:52.699 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:52.699 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:52.699 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:52.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.699 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63803 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1277952~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:52.699 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555466+0000", 2024-04-24T15:33:52.699 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.099559622999998, 2024-04-24T15:33:52.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033162297, 2024-04-24T15:33:52.699 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:52.699 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:52.699 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:52.699 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:52.699 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:52.700 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63803 2024-04-24T15:33:52.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.700 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:52.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:52.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555466+0000", 2024-04-24T15:33:52.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:52.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:52.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555466+0000", 2024-04-24T15:33:52.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:52.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:52.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555466+0000", 2024-04-24T15:33:52.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:52.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:52.701 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555468+0000", 2024-04-24T15:33:52.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4040000000000002e-06 2024-04-24T15:33:52.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:52.701 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555475+0000", 2024-04-24T15:33:52.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9480000000000002e-06 2024-04-24T15:33:52.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:52.701 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555479+0000", 2024-04-24T15:33:52.702 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0289999999999999e-06 2024-04-24T15:33:52.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.702 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:52.702 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588516+0000", 2024-04-24T15:33:52.702 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033037094000000003 2024-04-24T15:33:52.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.702 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:52.702 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588562+0000", 2024-04-24T15:33:52.702 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6215999999999999e-05 2024-04-24T15:33:52.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.702 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:52.703 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588628+0000", 2024-04-24T15:33:52.703 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5971999999999998e-05 2024-04-24T15:33:52.703 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:52.703 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:52.703 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:52.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.703 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63804 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1359872~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:52.703 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555483+0000", 2024-04-24T15:33:52.703 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.099542456000002, 2024-04-24T15:33:52.703 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033287068000000003, 2024-04-24T15:33:52.703 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:52.703 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:52.703 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:52.703 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:52.704 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:52.704 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63804 2024-04-24T15:33:52.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.704 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:52.704 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.704 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:52.704 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555483+0000", 2024-04-24T15:33:52.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:52.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.704 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.704 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:52.704 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555483+0000", 2024-04-24T15:33:52.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:52.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:52.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555483+0000", 2024-04-24T15:33:52.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:52.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:52.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555485+0000", 2024-04-24T15:33:52.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0710000000000002e-06 2024-04-24T15:33:52.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:52.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555498+0000", 2024-04-24T15:33:52.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2748999999999999e-05 2024-04-24T15:33:52.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.706 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:52.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555502+0000", 2024-04-24T15:33:52.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0219999999999998e-06 2024-04-24T15:33:52.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.706 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:52.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588657+0000", 2024-04-24T15:33:52.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033154683999999997 2024-04-24T15:33:52.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.706 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:52.707 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588702+0000", 2024-04-24T15:33:52.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5809000000000001e-05 2024-04-24T15:33:52.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.707 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:52.707 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588770+0000", 2024-04-24T15:33:52.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8056e-05 2024-04-24T15:33:52.707 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:52.707 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:52.707 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:52.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.707 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63805 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1441792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:52.707 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555508+0000", 2024-04-24T15:33:52.707 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.099517607000003, 2024-04-24T15:33:52.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033375952, 2024-04-24T15:33:52.708 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:52.708 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:52.708 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:52.708 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:52.708 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:52.708 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63805 2024-04-24T15:33:52.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.708 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:52.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:52.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555508+0000", 2024-04-24T15:33:52.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:52.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:52.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555508+0000", 2024-04-24T15:33:52.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:52.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:52.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555508+0000", 2024-04-24T15:33:52.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:52.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:52.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555510+0000", 2024-04-24T15:33:52.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.424e-06 2024-04-24T15:33:52.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:52.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555516+0000", 2024-04-24T15:33:52.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3730000000000001e-06 2024-04-24T15:33:52.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:52.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555519+0000", 2024-04-24T15:33:52.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.039e-06 2024-04-24T15:33:52.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:52.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588789+0000", 2024-04-24T15:33:52.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0332702 2024-04-24T15:33:52.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:52.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588833+0000", 2024-04-24T15:33:52.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4660999999999998e-05 2024-04-24T15:33:52.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:52.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588884+0000", 2024-04-24T15:33:52.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0661999999999997e-05 2024-04-24T15:33:52.711 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:52.711 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:52.711 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:52.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.711 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63806 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1523712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:52.711 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555523+0000", 2024-04-24T15:33:52.711 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.099503026000001, 2024-04-24T15:33:52.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033479653999999998, 2024-04-24T15:33:52.712 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:52.712 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:52.712 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:52.712 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:52.712 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:52.712 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63806 2024-04-24T15:33:52.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.712 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:52.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:52.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555523+0000", 2024-04-24T15:33:52.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:52.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.713 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:52.713 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555523+0000", 2024-04-24T15:33:52.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:52.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.713 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:52.713 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555522+0000", 2024-04-24T15:33:52.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:52.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.713 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:52.713 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555524+0000", 2024-04-24T15:33:52.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0700000000000001e-06 2024-04-24T15:33:52.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:52.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555530+0000", 2024-04-24T15:33:52.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5450000000000003e-06 2024-04-24T15:33:52.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:52.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555533+0000", 2024-04-24T15:33:52.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2200000000000001e-06 2024-04-24T15:33:52.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:52.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588908+0000", 2024-04-24T15:33:52.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033374556 2024-04-24T15:33:52.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:52.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588950+0000", 2024-04-24T15:33:52.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2361999999999999e-05 2024-04-24T15:33:52.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:52.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589002+0000", 2024-04-24T15:33:52.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2224e-05 2024-04-24T15:33:52.715 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:52.715 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:52.715 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:52.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.716 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63807 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1605632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:52.716 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555537+0000", 2024-04-24T15:33:52.716 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.099488510999997, 2024-04-24T15:33:52.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033577227000000001, 2024-04-24T15:33:52.716 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:52.716 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:52.716 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:52.716 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:52.716 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:52.716 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63807 2024-04-24T15:33:52.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.716 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:52.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.716 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:52.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555537+0000", 2024-04-24T15:33:52.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:52.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:52.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555537+0000", 2024-04-24T15:33:52.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:52.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:52.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555537+0000", 2024-04-24T15:33:52.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:52.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:52.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555539+0000", 2024-04-24T15:33:52.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9769999999999999e-06 2024-04-24T15:33:52.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:52.718 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555544+0000", 2024-04-24T15:33:52.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.468e-06 2024-04-24T15:33:52.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:52.718 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555548+0000", 2024-04-24T15:33:52.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4319999999999999e-06 2024-04-24T15:33:52.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:52.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589021+0000", 2024-04-24T15:33:52.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033473655999999997 2024-04-24T15:33:52.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:52.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589064+0000", 2024-04-24T15:33:52.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2397000000000001e-05 2024-04-24T15:33:52.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:52.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589114+0000", 2024-04-24T15:33:52.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0611e-05 2024-04-24T15:33:52.719 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:52.719 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:52.720 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:52.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.720 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63808 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1687552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:52.720 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555552+0000", 2024-04-24T15:33:52.720 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.099473819000004, 2024-04-24T15:33:52.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033686019999999997, 2024-04-24T15:33:52.720 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:52.720 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:52.720 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:52.720 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:52.720 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:52.720 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63808 2024-04-24T15:33:52.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.721 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:52.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.721 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:52.721 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555552+0000", 2024-04-24T15:33:52.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:52.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.721 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:52.721 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555552+0000", 2024-04-24T15:33:52.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:52.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.721 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:52.722 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555552+0000", 2024-04-24T15:33:52.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:52.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.722 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.722 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:52.722 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555554+0000", 2024-04-24T15:33:52.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2170000000000001e-06 2024-04-24T15:33:52.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.722 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.722 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:52.722 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555562+0000", 2024-04-24T15:33:52.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8779999999999998e-06 2024-04-24T15:33:52.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.722 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:52.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555565+0000", 2024-04-24T15:33:52.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4230000000000001e-06 2024-04-24T15:33:52.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:52.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589143+0000", 2024-04-24T15:33:52.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033577382000000003 2024-04-24T15:33:52.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:52.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589185+0000", 2024-04-24T15:33:52.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2206999999999998e-05 2024-04-24T15:33:52.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.724 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:52.724 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589238+0000", 2024-04-24T15:33:52.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3211000000000003e-05 2024-04-24T15:33:52.724 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:52.724 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:52.724 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:52.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.724 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63809 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1769472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:52.724 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555569+0000", 2024-04-24T15:33:52.724 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.099456277999998, 2024-04-24T15:33:52.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.03378068, 2024-04-24T15:33:52.724 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:52.724 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:52.724 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:52.724 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:52.725 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:52.725 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63809 2024-04-24T15:33:52.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.725 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:52.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.725 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:52.725 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555569+0000", 2024-04-24T15:33:52.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:52.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.725 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:52.725 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555569+0000", 2024-04-24T15:33:52.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:52.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.726 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.726 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:52.726 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555569+0000", 2024-04-24T15:33:52.726 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:52.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.726 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.726 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:52.726 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555571+0000", 2024-04-24T15:33:52.726 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0889999999999998e-06 2024-04-24T15:33:52.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.726 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.726 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:52.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555577+0000", 2024-04-24T15:33:52.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4149999999999998e-06 2024-04-24T15:33:52.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:52.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555580+0000", 2024-04-24T15:33:52.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2399999999999999e-06 2024-04-24T15:33:52.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:52.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589258+0000", 2024-04-24T15:33:52.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033677817999999998 2024-04-24T15:33:52.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:52.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589300+0000", 2024-04-24T15:33:52.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2317999999999997e-05 2024-04-24T15:33:52.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:52.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589350+0000", 2024-04-24T15:33:52.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0111000000000002e-05 2024-04-24T15:33:52.728 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:52.728 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:52.728 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:52.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.728 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63810 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1851392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:52.728 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555584+0000", 2024-04-24T15:33:52.729 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.099441808000002, 2024-04-24T15:33:52.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033886251999999999, 2024-04-24T15:33:52.729 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:52.729 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:52.729 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:52.729 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:52.729 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:52.729 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63810 2024-04-24T15:33:52.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.729 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:52.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:52.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555584+0000", 2024-04-24T15:33:52.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:52.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.730 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.730 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:52.730 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555584+0000", 2024-04-24T15:33:52.730 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:52.730 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.730 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.730 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:52.730 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555584+0000", 2024-04-24T15:33:52.730 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:52.730 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.730 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.730 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:52.730 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555586+0000", 2024-04-24T15:33:52.730 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0899999999999999e-06 2024-04-24T15:33:52.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.731 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.731 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:52.731 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555591+0000", 2024-04-24T15:33:52.731 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5260000000000001e-06 2024-04-24T15:33:52.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.731 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.731 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:52.731 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555595+0000", 2024-04-24T15:33:52.731 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.41e-06 2024-04-24T15:33:52.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.731 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.731 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:52.731 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589377+0000", 2024-04-24T15:33:52.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033781868 2024-04-24T15:33:52.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:52.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589418+0000", 2024-04-24T15:33:52.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1461000000000002e-05 2024-04-24T15:33:52.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:52.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589470+0000", 2024-04-24T15:33:52.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2194999999999997e-05 2024-04-24T15:33:52.732 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:52.732 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:52.732 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:52.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.733 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63811 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1933312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:52.733 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555599+0000", 2024-04-24T15:33:52.733 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.099426999000002, 2024-04-24T15:33:52.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033987493000000001, 2024-04-24T15:33:52.733 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:52.733 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:52.733 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:52.733 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:52.733 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:52.733 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63811 2024-04-24T15:33:52.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.733 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:52.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.734 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:52.734 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555599+0000", 2024-04-24T15:33:52.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:52.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.734 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.734 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:52.734 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555599+0000", 2024-04-24T15:33:52.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:52.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.734 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.734 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:52.734 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555598+0000", 2024-04-24T15:33:52.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:52.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.735 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:52.735 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555600+0000", 2024-04-24T15:33:52.735 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.04e-06 2024-04-24T15:33:52.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.735 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:52.735 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555606+0000", 2024-04-24T15:33:52.735 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4199999999999998e-06 2024-04-24T15:33:52.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.735 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:52.735 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555616+0000", 2024-04-24T15:33:52.735 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0054000000000001e-05 2024-04-24T15:33:52.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.736 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.736 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:52.736 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589493+0000", 2024-04-24T15:33:52.736 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033877009999999999 2024-04-24T15:33:52.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.736 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.736 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:52.736 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589536+0000", 2024-04-24T15:33:52.736 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2855999999999998e-05 2024-04-24T15:33:52.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.736 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.736 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:52.737 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589586+0000", 2024-04-24T15:33:52.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0420999999999997e-05 2024-04-24T15:33:52.737 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:52.737 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:52.737 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:52.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.737 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63812 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 2015232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:52.737 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555620+0000", 2024-04-24T15:33:52.737 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.099405361000002, 2024-04-24T15:33:52.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034081696000000002, 2024-04-24T15:33:52.737 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:52.737 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:52.737 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:52.738 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:52.738 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:52.738 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63812 2024-04-24T15:33:52.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.738 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:52.738 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.738 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:52.738 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555620+0000", 2024-04-24T15:33:52.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:52.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.738 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.738 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:52.738 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555620+0000", 2024-04-24T15:33:52.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:52.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:52.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555620+0000", 2024-04-24T15:33:52.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:52.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:52.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555625+0000", 2024-04-24T15:33:52.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2859999999999999e-06 2024-04-24T15:33:52.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:52.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555633+0000", 2024-04-24T15:33:52.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8879999999999997e-06 2024-04-24T15:33:52.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:52.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555637+0000", 2024-04-24T15:33:52.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9500000000000003e-06 2024-04-24T15:33:52.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:52.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589604+0000", 2024-04-24T15:33:52.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033966478000000001 2024-04-24T15:33:52.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:52.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589650+0000", 2024-04-24T15:33:52.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6721000000000002e-05 2024-04-24T15:33:52.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:52.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589702+0000", 2024-04-24T15:33:52.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1709000000000002e-05 2024-04-24T15:33:52.741 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:52.741 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:52.741 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:52.741 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:52.741 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:52.741 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:52.742 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-24T15:33:52.743 INFO:tasks.thrashosds.thrasher:Added osd 7 2024-04-24T15:33:52.744 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-24T15:33:52.785 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:52.786+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:52.856 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:33:52.858+0000 7f0f45efb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 59992) UID: 0 2024-04-24T15:33:52.920 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:52.920 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-04-24T15:33:52.920 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.920 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:76630 4.14 4.4bd147b4 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e115)", 2024-04-24T15:33:52.920 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:52.636576+0000", 2024-04-24T15:33:52.920 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.28470695400000001, 2024-04-24T15:33:52.920 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28473070099999997, 2024-04-24T15:33:52.920 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:52.920 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for peered", 2024-04-24T15:33:52.921 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:52.921 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:52.921 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:52.921 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 76630 2024-04-24T15:33:52.921 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.921 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:52.922 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.922 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:52.922 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:52.636576+0000", 2024-04-24T15:33:52.922 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:52.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.922 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:52.923 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:52.636576+0000", 2024-04-24T15:33:52.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:52.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:52.923 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:52.636574+0000", 2024-04-24T15:33:52.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999981 2024-04-24T15:33:52.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:52.923 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:52.636585+0000", 2024-04-24T15:33:52.924 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0994e-05 2024-04-24T15:33:52.924 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.924 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.924 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:52.924 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:52.636605+0000", 2024-04-24T15:33:52.924 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0707e-05 2024-04-24T15:33:52.924 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.924 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.924 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:52.924 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:52.636614+0000", 2024-04-24T15:33:52.924 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9379999999999998e-06 2024-04-24T15:33:52.925 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.925 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.925 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:52.925 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:52.636665+0000", 2024-04-24T15:33:52.925 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1107999999999999e-05 2024-04-24T15:33:52.925 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.925 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.925 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op must wait for map", 2024-04-24T15:33:52.925 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:52.636670+0000", 2024-04-24T15:33:52.925 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7230000000000004e-06 2024-04-24T15:33:52.925 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.926 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:52.926 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:52.645741+0000", 2024-04-24T15:33:52.926 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0090709199999999997 2024-04-24T15:33:52.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.926 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-04-24T15:33:52.926 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:52.645751+0000", 2024-04-24T15:33:52.926 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6390000000000004e-06 2024-04-24T15:33:52.926 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:52.927 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:52.927 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:52.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.927 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:76631 4.14 4.4bd147b4 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e115)", 2024-04-24T15:33:52.927 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:52.643648+0000", 2024-04-24T15:33:52.927 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.27763483300000003, 2024-04-24T15:33:52.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27771189400000001, 2024-04-24T15:33:52.927 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:52.927 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for peered", 2024-04-24T15:33:52.927 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:52.927 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:52.927 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:52.927 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 76631 2024-04-24T15:33:52.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.928 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:52.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.928 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:52.928 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:52.643648+0000", 2024-04-24T15:33:52.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:52.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.928 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:52.928 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:52.643648+0000", 2024-04-24T15:33:52.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:52.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.928 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:52.929 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:52.643646+0000", 2024-04-24T15:33:52.929 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999986 2024-04-24T15:33:52.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.929 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.929 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:52.929 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:52.643652+0000", 2024-04-24T15:33:52.929 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0360000000000003e-06 2024-04-24T15:33:52.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.929 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.929 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:52.929 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:52.643663+0000", 2024-04-24T15:33:52.929 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0681e-05 2024-04-24T15:33:52.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.929 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.930 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:52.930 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:52.643671+0000", 2024-04-24T15:33:52.930 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.605e-06 2024-04-24T15:33:52.930 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.930 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.930 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:52.930 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:52.643738+0000", 2024-04-24T15:33:52.930 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7706000000000005e-05 2024-04-24T15:33:52.930 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.930 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.930 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2024-04-24T15:33:52.930 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:52.643745+0000", 2024-04-24T15:33:52.930 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.883e-06 2024-04-24T15:33:52.930 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.931 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:52.931 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:52.645769+0000", 2024-04-24T15:33:52.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020239500000000001 2024-04-24T15:33:52.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:52.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:52.931 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-04-24T15:33:52.931 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:52.645775+0000", 2024-04-24T15:33:52.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2450000000000003e-06 2024-04-24T15:33:52.931 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:52.931 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:52.931 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:52.931 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:52.931 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-24T15:33:52.932 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 2 2024-04-24T15:33:52.932 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:52.934 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-24T15:33:52.957 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:33:52.959+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:33:53.058 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:53.059+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:53.059 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:53.059 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:33:53.059 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-04-24T15:33:53.059 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-04-24T15:33:53.059 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:53.069 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-24T15:33:53.160 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:53.161+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:53.232 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:53.232 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-04-24T15:33:53.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-04-24T15:33:53.232 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-04-24T15:33:53.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.232 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14043 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2383872~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:53.233 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892347+0000", 2024-04-24T15:33:53.233 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 79.339786188999994, 2024-04-24T15:33:53.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038537279000000001, 2024-04-24T15:33:53.233 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:53.233 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:53.233 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:53.233 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:53.233 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:53.233 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14043 2024-04-24T15:33:53.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.233 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:53.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:53.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892347+0000", 2024-04-24T15:33:53.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:53.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:53.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892347+0000", 2024-04-24T15:33:53.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:53.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:53.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892347+0000", 2024-04-24T15:33:53.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:53.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:53.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892349+0000", 2024-04-24T15:33:53.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3400000000000001e-06 2024-04-24T15:33:53.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:53.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892357+0000", 2024-04-24T15:33:53.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1540000000000002e-06 2024-04-24T15:33:53.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:53.236 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892360+0000", 2024-04-24T15:33:53.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5829999999999998e-06 2024-04-24T15:33:53.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.236 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:53.236 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930795+0000", 2024-04-24T15:33:53.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038434590999999997 2024-04-24T15:33:53.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.236 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:53.236 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930836+0000", 2024-04-24T15:33:53.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0827000000000001e-05 2024-04-24T15:33:53.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.236 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:53.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930885+0000", 2024-04-24T15:33:53.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9029000000000001e-05 2024-04-24T15:33:53.237 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:53.237 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:53.237 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:53.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.237 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14044 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2424832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:53.238 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892372+0000", 2024-04-24T15:33:53.238 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 79.339761545000002, 2024-04-24T15:33:53.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038624890000000002, 2024-04-24T15:33:53.238 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:53.238 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:53.238 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:53.239 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:53.239 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:53.239 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14044 2024-04-24T15:33:53.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.239 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:53.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:53.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892372+0000", 2024-04-24T15:33:53.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:53.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:53.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892372+0000", 2024-04-24T15:33:53.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:53.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:53.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892372+0000", 2024-04-24T15:33:53.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:53.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:53.241 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892374+0000", 2024-04-24T15:33:53.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4899999999999999e-06 2024-04-24T15:33:53.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:53.241 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892380+0000", 2024-04-24T15:33:53.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.631e-06 2024-04-24T15:33:53.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:53.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892383+0000", 2024-04-24T15:33:53.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7400000000000002e-06 2024-04-24T15:33:53.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:53.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930905+0000", 2024-04-24T15:33:53.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038521396999999999 2024-04-24T15:33:53.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:53.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930949+0000", 2024-04-24T15:33:53.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4153999999999998e-05 2024-04-24T15:33:53.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:53.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930997+0000", 2024-04-24T15:33:53.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7998999999999999e-05 2024-04-24T15:33:53.243 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:53.243 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:53.243 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:53.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.244 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14045 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2465792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:53.244 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892387+0000", 2024-04-24T15:33:53.244 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 79.339746278000007, 2024-04-24T15:33:53.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038720933999999999, 2024-04-24T15:33:53.244 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:53.244 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:53.244 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:53.244 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:53.244 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:53.244 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14045 2024-04-24T15:33:53.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.245 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:53.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:53.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892387+0000", 2024-04-24T15:33:53.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:53.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:53.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892387+0000", 2024-04-24T15:33:53.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:53.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:53.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892387+0000", 2024-04-24T15:33:53.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:53.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.246 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:53.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892392+0000", 2024-04-24T15:33:53.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0039999999999999e-06 2024-04-24T15:33:53.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.246 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:53.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892398+0000", 2024-04-24T15:33:53.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0900000000000001e-06 2024-04-24T15:33:53.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:53.247 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892402+0000", 2024-04-24T15:33:53.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.399e-06 2024-04-24T15:33:53.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:53.247 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931018+0000", 2024-04-24T15:33:53.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038616551999999998 2024-04-24T15:33:53.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:53.247 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931059+0000", 2024-04-24T15:33:53.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1205000000000003e-05 2024-04-24T15:33:53.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:53.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931108+0000", 2024-04-24T15:33:53.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8996000000000003e-05 2024-04-24T15:33:53.248 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:53.248 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:53.248 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:53.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.248 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14046 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2506752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:53.248 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892406+0000", 2024-04-24T15:33:53.248 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 79.339727687999996, 2024-04-24T15:33:53.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038814507999999998, 2024-04-24T15:33:53.249 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:53.249 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:53.249 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:53.249 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:53.249 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:53.249 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14046 2024-04-24T15:33:53.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.249 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:53.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:53.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892406+0000", 2024-04-24T15:33:53.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:53.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:53.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892406+0000", 2024-04-24T15:33:53.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:53.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:53.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892406+0000", 2024-04-24T15:33:53.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:53.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:53.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892413+0000", 2024-04-24T15:33:53.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7449999999999995e-06 2024-04-24T15:33:53.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:53.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892419+0000", 2024-04-24T15:33:53.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5790000000000003e-06 2024-04-24T15:33:53.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:53.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892423+0000", 2024-04-24T15:33:53.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8199999999999998e-06 2024-04-24T15:33:53.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:53.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931128+0000", 2024-04-24T15:33:53.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038705502000000003 2024-04-24T15:33:53.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:53.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931172+0000", 2024-04-24T15:33:53.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4066999999999997e-05 2024-04-24T15:33:53.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:53.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931220+0000", 2024-04-24T15:33:53.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8167999999999997e-05 2024-04-24T15:33:53.252 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:53.252 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:53.252 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:53.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.253 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14047 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2547712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:53.253 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892427+0000", 2024-04-24T15:33:53.253 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 79.339706458999999, 2024-04-24T15:33:53.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038904506999999998, 2024-04-24T15:33:53.253 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:53.253 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:53.253 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:53.253 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:53.253 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:53.253 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14047 2024-04-24T15:33:53.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.253 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:53.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:53.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892427+0000", 2024-04-24T15:33:53.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:53.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:53.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892427+0000", 2024-04-24T15:33:53.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:53.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:53.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892427+0000", 2024-04-24T15:33:53.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:53.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:53.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892429+0000", 2024-04-24T15:33:53.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3099999999999999e-06 2024-04-24T15:33:53.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:53.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892435+0000", 2024-04-24T15:33:53.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5520000000000004e-06 2024-04-24T15:33:53.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:53.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892438+0000", 2024-04-24T15:33:53.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4240000000000002e-06 2024-04-24T15:33:53.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:53.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931242+0000", 2024-04-24T15:33:53.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038804162000000003 2024-04-24T15:33:53.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:53.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931284+0000", 2024-04-24T15:33:53.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1414e-05 2024-04-24T15:33:53.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:53.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931332+0000", 2024-04-24T15:33:53.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8024e-05 2024-04-24T15:33:53.256 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:53.256 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:53.256 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:53.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.257 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14048 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2588672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:53.257 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892442+0000", 2024-04-24T15:33:53.257 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 79.339691496, 2024-04-24T15:33:53.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039003132000000003, 2024-04-24T15:33:53.257 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:53.257 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:53.257 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:53.257 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:53.257 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:53.257 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14048 2024-04-24T15:33:53.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.257 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:53.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:53.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892442+0000", 2024-04-24T15:33:53.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:53.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:53.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892442+0000", 2024-04-24T15:33:53.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:53.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:53.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892442+0000", 2024-04-24T15:33:53.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:53.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.259 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:53.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892444+0000", 2024-04-24T15:33:53.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1169999999999998e-06 2024-04-24T15:33:53.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.259 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:53.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892449+0000", 2024-04-24T15:33:53.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1599999999999997e-06 2024-04-24T15:33:53.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.259 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:53.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892453+0000", 2024-04-24T15:33:53.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5930000000000002e-06 2024-04-24T15:33:53.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.259 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:53.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931352+0000", 2024-04-24T15:33:53.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038899361 2024-04-24T15:33:53.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:53.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931398+0000", 2024-04-24T15:33:53.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6315e-05 2024-04-24T15:33:53.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:53.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931445+0000", 2024-04-24T15:33:53.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6950000000000003e-05 2024-04-24T15:33:53.260 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:53.260 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:53.261 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:53.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.261 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14049 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2629632~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:53.261 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892458+0000", 2024-04-24T15:33:53.261 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 79.339676076000003, 2024-04-24T15:33:53.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039103322000000003, 2024-04-24T15:33:53.261 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:53.261 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:53.261 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:53.261 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:53.261 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:53.261 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14049 2024-04-24T15:33:53.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.261 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:53.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:53.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892458+0000", 2024-04-24T15:33:53.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:53.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:53.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892458+0000", 2024-04-24T15:33:53.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:53.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:53.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892457+0000", 2024-04-24T15:33:53.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:53.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:53.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892459+0000", 2024-04-24T15:33:53.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.305e-06 2024-04-24T15:33:53.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:53.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892465+0000", 2024-04-24T15:33:53.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4360000000000001e-06 2024-04-24T15:33:53.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:53.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892474+0000", 2024-04-24T15:33:53.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7639999999999994e-06 2024-04-24T15:33:53.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:53.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931466+0000", 2024-04-24T15:33:53.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038992644 2024-04-24T15:33:53.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:53.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931512+0000", 2024-04-24T15:33:53.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5200000000000001e-05 2024-04-24T15:33:53.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:53.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931561+0000", 2024-04-24T15:33:53.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9357999999999997e-05 2024-04-24T15:33:53.265 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:53.265 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:53.265 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:53.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.265 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14050 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2670592~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:53.265 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892478+0000", 2024-04-24T15:33:53.265 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 79.339655488999995, 2024-04-24T15:33:53.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039195102000000002, 2024-04-24T15:33:53.265 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:53.265 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:53.265 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:53.265 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:53.265 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:53.266 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14050 2024-04-24T15:33:53.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.266 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:53.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:53.266 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892478+0000", 2024-04-24T15:33:53.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:53.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:53.266 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892478+0000", 2024-04-24T15:33:53.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:53.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:53.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892478+0000", 2024-04-24T15:33:53.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:53.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:53.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892480+0000", 2024-04-24T15:33:53.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3300000000000001e-06 2024-04-24T15:33:53.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:53.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892486+0000", 2024-04-24T15:33:53.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9200000000000001e-06 2024-04-24T15:33:53.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:53.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892494+0000", 2024-04-24T15:33:53.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5159999999999997e-06 2024-04-24T15:33:53.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:53.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931581+0000", 2024-04-24T15:33:53.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039087894999999998 2024-04-24T15:33:53.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:53.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931626+0000", 2024-04-24T15:33:53.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4943e-05 2024-04-24T15:33:53.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:53.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931673+0000", 2024-04-24T15:33:53.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6906000000000001e-05 2024-04-24T15:33:53.269 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:53.269 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:53.269 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:53.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.269 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.269 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14051 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2711552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:53.269 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892507+0000", 2024-04-24T15:33:53.269 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 79.339626789999997, 2024-04-24T15:33:53.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039274393999999997, 2024-04-24T15:33:53.269 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:53.269 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:53.269 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:53.270 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:53.270 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:53.270 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14051 2024-04-24T15:33:53.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.270 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:53.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.270 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:53.270 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892507+0000", 2024-04-24T15:33:53.270 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:53.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.270 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:53.270 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892507+0000", 2024-04-24T15:33:53.270 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:53.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.271 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:53.271 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892506+0000", 2024-04-24T15:33:53.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:53.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.271 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:53.271 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892509+0000", 2024-04-24T15:33:53.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8760000000000001e-06 2024-04-24T15:33:53.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.271 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:53.271 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892515+0000", 2024-04-24T15:33:53.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1840000000000004e-06 2024-04-24T15:33:53.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:53.272 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892519+0000", 2024-04-24T15:33:53.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7570000000000002e-06 2024-04-24T15:33:53.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:53.272 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931694+0000", 2024-04-24T15:33:53.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039174960000000002 2024-04-24T15:33:53.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:53.272 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931734+0000", 2024-04-24T15:33:53.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0284000000000003e-05 2024-04-24T15:33:53.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.273 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.273 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:53.273 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931781+0000", 2024-04-24T15:33:53.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6813000000000001e-05 2024-04-24T15:33:53.273 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:53.273 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:53.274 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:53.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.274 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.274 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14052 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2752512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:53.274 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892524+0000", 2024-04-24T15:33:53.274 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 79.339609643000003, 2024-04-24T15:33:53.274 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039371122000000001, 2024-04-24T15:33:53.274 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:53.274 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:53.274 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:53.274 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:53.274 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:53.274 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14052 2024-04-24T15:33:53.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.275 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:53.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.275 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:53.275 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892524+0000", 2024-04-24T15:33:53.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:53.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:53.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892524+0000", 2024-04-24T15:33:53.276 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:53.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:53.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892524+0000", 2024-04-24T15:33:53.276 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:53.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.277 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.277 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:53.277 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892526+0000", 2024-04-24T15:33:53.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3630000000000001e-06 2024-04-24T15:33:53.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.277 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.277 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:53.277 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892538+0000", 2024-04-24T15:33:53.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1677e-05 2024-04-24T15:33:53.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.277 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.277 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:53.277 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892541+0000", 2024-04-24T15:33:53.278 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.822e-06 2024-04-24T15:33:53.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.278 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:53.278 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931802+0000", 2024-04-24T15:33:53.278 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039260911000000003 2024-04-24T15:33:53.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.278 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:53.278 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931847+0000", 2024-04-24T15:33:53.278 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4450999999999998e-05 2024-04-24T15:33:53.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.278 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:53.279 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931895+0000", 2024-04-24T15:33:53.279 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8257999999999998e-05 2024-04-24T15:33:53.279 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:53.279 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:53.279 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:53.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.279 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.279 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14053 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2793472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:53.279 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892546+0000", 2024-04-24T15:33:53.279 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 79.339587664999996, 2024-04-24T15:33:53.279 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039457712999999998, 2024-04-24T15:33:53.279 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:53.279 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:53.279 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:53.279 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:53.280 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:53.280 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14053 2024-04-24T15:33:53.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.280 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:53.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.280 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:53.280 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892546+0000", 2024-04-24T15:33:53.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:53.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.280 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:53.280 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892546+0000", 2024-04-24T15:33:53.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:53.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:53.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892546+0000", 2024-04-24T15:33:53.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:53.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:53.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892548+0000", 2024-04-24T15:33:53.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3120000000000001e-06 2024-04-24T15:33:53.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:53.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892553+0000", 2024-04-24T15:33:53.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2320000000000001e-06 2024-04-24T15:33:53.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.282 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.282 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:53.282 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892559+0000", 2024-04-24T15:33:53.282 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5480000000000001e-06 2024-04-24T15:33:53.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.282 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.282 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:53.282 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931916+0000", 2024-04-24T15:33:53.282 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039357180999999998 2024-04-24T15:33:53.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.282 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.282 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:53.282 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931956+0000", 2024-04-24T15:33:53.282 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0565000000000002e-05 2024-04-24T15:33:53.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.283 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:53.283 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932004+0000", 2024-04-24T15:33:53.283 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7224000000000001e-05 2024-04-24T15:33:53.283 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:53.283 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:53.283 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:53.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.283 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14054 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2834432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:53.283 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892563+0000", 2024-04-24T15:33:53.283 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 79.339571012999997, 2024-04-24T15:33:53.283 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039552406999999998, 2024-04-24T15:33:53.284 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:53.284 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:53.284 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:53.284 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:53.284 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:53.284 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14054 2024-04-24T15:33:53.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.284 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:53.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.284 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:53.284 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892563+0000", 2024-04-24T15:33:53.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:53.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.284 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:53.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892563+0000", 2024-04-24T15:33:53.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:53.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:53.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892562+0000", 2024-04-24T15:33:53.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:53.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:53.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892564+0000", 2024-04-24T15:33:53.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2029999999999999e-06 2024-04-24T15:33:53.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:53.286 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892570+0000", 2024-04-24T15:33:53.286 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1460000000000004e-06 2024-04-24T15:33:53.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:53.286 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892573+0000", 2024-04-24T15:33:53.286 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2030000000000001e-06 2024-04-24T15:33:53.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:53.286 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932024+0000", 2024-04-24T15:33:53.286 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039451696000000001 2024-04-24T15:33:53.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.287 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:53.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932068+0000", 2024-04-24T15:33:53.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3970000000000001e-05 2024-04-24T15:33:53.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.287 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:53.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932115+0000", 2024-04-24T15:33:53.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6569000000000001e-05 2024-04-24T15:33:53.287 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:53.287 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:53.287 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:53.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.287 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14055 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2875392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:53.288 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892591+0000", 2024-04-24T15:33:53.288 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 79.339543039999995, 2024-04-24T15:33:53.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039634309, 2024-04-24T15:33:53.288 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:53.288 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:53.288 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:53.288 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:53.288 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:53.288 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14055 2024-04-24T15:33:53.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.288 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:53.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.288 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:53.288 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892591+0000", 2024-04-24T15:33:53.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:53.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:53.289 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892591+0000", 2024-04-24T15:33:53.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:53.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:53.289 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892590+0000", 2024-04-24T15:33:53.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:53.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:53.289 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892592+0000", 2024-04-24T15:33:53.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7519999999999999e-06 2024-04-24T15:33:53.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:53.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892596+0000", 2024-04-24T15:33:53.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.608e-06 2024-04-24T15:33:53.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:53.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892598+0000", 2024-04-24T15:33:53.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.531e-06 2024-04-24T15:33:53.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:53.291 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932136+0000", 2024-04-24T15:33:53.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039537668999999998 2024-04-24T15:33:53.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:53.291 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932177+0000", 2024-04-24T15:33:53.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0880000000000002e-05 2024-04-24T15:33:53.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:53.291 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932225+0000", 2024-04-24T15:33:53.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8223999999999998e-05 2024-04-24T15:33:53.291 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:53.291 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:53.292 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:53.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.292 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14056 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2916352~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:53.292 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892600+0000", 2024-04-24T15:33:53.292 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 79.339533205999999, 2024-04-24T15:33:53.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039736447000000001, 2024-04-24T15:33:53.292 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:53.292 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:53.292 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:53.292 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:53.292 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:53.292 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14056 2024-04-24T15:33:53.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.293 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:53.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.293 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:53.293 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892600+0000", 2024-04-24T15:33:53.293 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:53.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.293 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:53.293 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892600+0000", 2024-04-24T15:33:53.293 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:53.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.294 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.294 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:53.294 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892600+0000", 2024-04-24T15:33:53.294 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967296 2024-04-24T15:33:53.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.294 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:53.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892602+0000", 2024-04-24T15:33:53.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.576e-06 2024-04-24T15:33:53.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:53.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892605+0000", 2024-04-24T15:33:53.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.574e-06 2024-04-24T15:33:53.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:53.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892608+0000", 2024-04-24T15:33:53.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3800000000000001e-06 2024-04-24T15:33:53.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.297 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:53.297 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932245+0000", 2024-04-24T15:33:53.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039637378000000001 2024-04-24T15:33:53.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.298 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.298 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:53.298 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932290+0000", 2024-04-24T15:33:53.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4734999999999998e-05 2024-04-24T15:33:53.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.298 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.299 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:53.299 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932337+0000", 2024-04-24T15:33:53.299 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7033000000000002e-05 2024-04-24T15:33:53.299 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:53.299 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:53.299 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:53.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.300 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.300 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14063 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 3203072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:53.300 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.894564+0000", 2024-04-24T15:33:53.300 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 79.337569615000007, 2024-04-24T15:33:53.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038554995000000002, 2024-04-24T15:33:53.301 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:53.301 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:53.301 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:53.301 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:53.301 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:53.301 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14063 2024-04-24T15:33:53.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.302 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:53.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.302 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:53.302 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894564+0000", 2024-04-24T15:33:53.302 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:53.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.303 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:53.303 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894564+0000", 2024-04-24T15:33:53.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:53.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.303 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:53.304 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894564+0000", 2024-04-24T15:33:53.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967296 2024-04-24T15:33:53.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.304 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:53.304 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894565+0000", 2024-04-24T15:33:53.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2640000000000001e-06 2024-04-24T15:33:53.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.305 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:53.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894569+0000", 2024-04-24T15:33:53.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1710000000000004e-06 2024-04-24T15:33:53.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.306 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:53.306 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894573+0000", 2024-04-24T15:33:53.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4410000000000002e-06 2024-04-24T15:33:53.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.306 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:53.307 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933030+0000", 2024-04-24T15:33:53.307 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038457709999999999 2024-04-24T15:33:53.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.307 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.307 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:53.307 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933072+0000", 2024-04-24T15:33:53.308 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.193e-05 2024-04-24T15:33:53.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.308 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:53.308 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933119+0000", 2024-04-24T15:33:53.308 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6715000000000003e-05 2024-04-24T15:33:53.308 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:53.309 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:53.309 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:53.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.309 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14064 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 3244032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:53.309 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.894591+0000", 2024-04-24T15:33:53.310 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 79.337542972999998, 2024-04-24T15:33:53.310 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038639806999999998, 2024-04-24T15:33:53.310 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:53.313 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:53.314 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:53.314 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:53.314 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:53.314 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14064 2024-04-24T15:33:53.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.314 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:53.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.315 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:53.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894591+0000", 2024-04-24T15:33:53.315 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:53.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.316 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:53.316 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894591+0000", 2024-04-24T15:33:53.316 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:53.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.316 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.317 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:53.317 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894590+0000", 2024-04-24T15:33:53.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:53.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.317 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:53.317 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894593+0000", 2024-04-24T15:33:53.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4719999999999998e-06 2024-04-24T15:33:53.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.318 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.318 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:53.318 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894598+0000", 2024-04-24T15:33:53.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.57e-06 2024-04-24T15:33:53.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.318 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.318 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:53.319 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894602+0000", 2024-04-24T15:33:53.319 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.614e-06 2024-04-24T15:33:53.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.319 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:53.319 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933139+0000", 2024-04-24T15:33:53.319 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038536929999999997 2024-04-24T15:33:53.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.320 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:53.320 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933183+0000", 2024-04-24T15:33:53.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4217e-05 2024-04-24T15:33:53.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.320 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:53.320 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933230+0000", 2024-04-24T15:33:53.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7434e-05 2024-04-24T15:33:53.321 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:53.321 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:53.321 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:53.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.321 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.321 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14065 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 3284992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:53.321 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.894606+0000", 2024-04-24T15:33:53.321 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 79.337527590999997, 2024-04-24T15:33:53.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038734126000000001, 2024-04-24T15:33:53.322 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:53.322 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:53.322 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:53.322 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:53.322 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:53.322 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14065 2024-04-24T15:33:53.322 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.322 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:53.323 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.323 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:53.323 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894606+0000", 2024-04-24T15:33:53.323 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:53.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.323 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.323 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:53.323 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894606+0000", 2024-04-24T15:33:53.323 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:53.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.323 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.324 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:53.324 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894606+0000", 2024-04-24T15:33:53.324 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:53.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.324 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.324 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:53.324 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894608+0000", 2024-04-24T15:33:53.324 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0880000000000002e-06 2024-04-24T15:33:53.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.324 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.324 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:53.324 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894614+0000", 2024-04-24T15:33:53.324 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7830000000000004e-06 2024-04-24T15:33:53.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.324 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.325 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:53.325 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894621+0000", 2024-04-24T15:33:53.325 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5809999999999999e-06 2024-04-24T15:33:53.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.325 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:53.325 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933251+0000", 2024-04-24T15:33:53.325 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038629995 2024-04-24T15:33:53.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.325 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:53.325 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933292+0000", 2024-04-24T15:33:53.325 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0380000000000003e-05 2024-04-24T15:33:53.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.325 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:53.326 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933340+0000", 2024-04-24T15:33:53.326 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8662999999999998e-05 2024-04-24T15:33:53.326 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:53.326 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:53.326 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:53.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.326 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.326 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14066 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 3325952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:53.326 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.894625+0000", 2024-04-24T15:33:53.326 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 79.337508151999998, 2024-04-24T15:33:53.326 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038825539999999999, 2024-04-24T15:33:53.326 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:53.326 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:53.327 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:53.327 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:53.327 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:53.327 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14066 2024-04-24T15:33:53.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.327 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:53.327 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.327 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:53.327 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894625+0000", 2024-04-24T15:33:53.327 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:53.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.327 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.327 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:53.327 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894625+0000", 2024-04-24T15:33:53.327 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:53.328 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.328 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.328 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:53.328 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894625+0000", 2024-04-24T15:33:53.328 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:53.328 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.328 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.328 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:53.328 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894627+0000", 2024-04-24T15:33:53.328 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.092e-06 2024-04-24T15:33:53.328 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.328 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.328 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:53.328 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894635+0000", 2024-04-24T15:33:53.328 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6660000000000008e-06 2024-04-24T15:33:53.329 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.329 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.329 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:53.329 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894639+0000", 2024-04-24T15:33:53.329 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.799e-06 2024-04-24T15:33:53.329 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.329 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.329 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:53.329 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933360+0000", 2024-04-24T15:33:53.329 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038721569999999997 2024-04-24T15:33:53.329 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.329 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.329 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:53.329 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933403+0000", 2024-04-24T15:33:53.329 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2605999999999998e-05 2024-04-24T15:33:53.329 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.330 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.330 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:53.330 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933451+0000", 2024-04-24T15:33:53.330 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8149999999999998e-05 2024-04-24T15:33:53.330 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:53.330 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:53.330 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:53.330 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.330 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.330 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14067 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 3366912~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:53.330 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.894644+0000", 2024-04-24T15:33:53.330 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 79.337490094000003, 2024-04-24T15:33:53.331 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038926068000000001, 2024-04-24T15:33:53.331 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:53.331 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:53.331 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:53.331 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:53.331 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:53.331 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14067 2024-04-24T15:33:53.331 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.331 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:53.331 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.331 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:53.331 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894644+0000", 2024-04-24T15:33:53.331 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:53.331 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.331 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.332 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:53.332 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894644+0000", 2024-04-24T15:33:53.332 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:53.332 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.332 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.332 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:53.332 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894643+0000", 2024-04-24T15:33:53.332 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:53.332 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.332 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.332 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:53.332 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894646+0000", 2024-04-24T15:33:53.332 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3379999999999999e-06 2024-04-24T15:33:53.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.333 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.333 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:53.333 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894651+0000", 2024-04-24T15:33:53.333 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.485e-06 2024-04-24T15:33:53.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.333 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.333 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:53.333 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894655+0000", 2024-04-24T15:33:53.333 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.709e-06 2024-04-24T15:33:53.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.333 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.333 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:53.333 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933472+0000", 2024-04-24T15:33:53.333 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038816992000000002 2024-04-24T15:33:53.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.334 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:53.334 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933521+0000", 2024-04-24T15:33:53.334 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8841999999999998e-05 2024-04-24T15:33:53.334 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.334 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:53.334 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933570+0000", 2024-04-24T15:33:53.334 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9063000000000001e-05 2024-04-24T15:33:53.334 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:53.334 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:53.334 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:53.334 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.335 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.335 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:30240 4.0 4:059b194c:::rbd_object_map.114a2e5eebc:head [read 0~43] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-04-24T15:33:53.335 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:47.244473+0000", 2024-04-24T15:33:53.335 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 65.987660907999995, 2024-04-24T15:33:53.335 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4146627299999999, 2024-04-24T15:33:53.335 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:53.335 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:53.335 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:53.335 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:53.335 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:53.335 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30240 2024-04-24T15:33:53.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.335 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:53.336 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.336 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:53.336 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.244473+0000", 2024-04-24T15:33:53.336 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:53.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.336 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.336 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:53.336 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.244473+0000", 2024-04-24T15:33:53.336 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:53.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.336 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.336 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:53.336 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.244471+0000", 2024-04-24T15:33:53.336 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999986 2024-04-24T15:33:53.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.337 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.337 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:53.337 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.244477+0000", 2024-04-24T15:33:53.337 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7819999999999999e-06 2024-04-24T15:33:53.337 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.337 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.337 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:53.337 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.244487+0000", 2024-04-24T15:33:53.337 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0247e-05 2024-04-24T15:33:53.337 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.337 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.337 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:53.337 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.244499+0000", 2024-04-24T15:33:53.337 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1324e-05 2024-04-24T15:33:53.337 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.337 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.338 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:53.338 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.244545+0000", 2024-04-24T15:33:53.338 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6186000000000001e-05 2024-04-24T15:33:53.338 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.338 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.338 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-04-24T15:33:53.338 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.244553+0000", 2024-04-24T15:33:53.338 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5380000000000002e-06 2024-04-24T15:33:53.338 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.338 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.338 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:53.339 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.631596+0000", 2024-04-24T15:33:53.339 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.38704289600000003 2024-04-24T15:33:53.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.339 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.339 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-04-24T15:33:53.339 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.631606+0000", 2024-04-24T15:33:53.339 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6390000000000004e-06 2024-04-24T15:33:53.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.339 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.339 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:53.339 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:48.658831+0000", 2024-04-24T15:33:53.339 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0272250540000001 2024-04-24T15:33:53.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.339 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.339 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:53.340 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:48.658991+0000", 2024-04-24T15:33:53.340 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015994800000000001 2024-04-24T15:33:53.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:53.340 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:53.340 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:53.340 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:48.659135+0000", 2024-04-24T15:33:53.340 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000144456 2024-04-24T15:33:53.340 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:53.340 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:53.340 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:53.340 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:53.340 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:53.340 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:53.342 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-24T15:33:53.342 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:53.343+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:53.443 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:33:53.444+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:33:53.477 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:53.478 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:33:53.478 INFO:teuthology.orchestra.run.smithi151.stdout: "num_ops": 0 2024-04-24T15:33:53.478 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:53.488 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-24T15:33:53.544 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:53.545+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:53.612 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:53.612 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:33:53.612 INFO:teuthology.orchestra.run.smithi151.stdout: "complaint_time": 30, 2024-04-24T15:33:53.612 INFO:teuthology.orchestra.run.smithi151.stdout: "num_blocked_ops": 0 2024-04-24T15:33:53.612 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:53.621 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-24T15:33:53.645 INFO:teuthology.orchestra.run.smithi096.stderr:noscrub is set 2024-04-24T15:33:53.662 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:53.664+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:53.746 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:53.747+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:53.781 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-24T15:33:53.787 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:53.787 INFO:teuthology.orchestra.run.smithi151.stdout: "size": 20, 2024-04-24T15:33:53.787 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 600, 2024-04-24T15:33:53.787 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [ 2024-04-24T15:33:53.787 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.787 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:39041 4.1 4:82c1a902:::rbd_data.114abe589119.000000000000032c:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e68)", 2024-04-24T15:33:53.787 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:01.760552+0000", 2024-04-24T15:33:53.787 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 52.026270269999998, 2024-04-24T15:33:53.787 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.87570748300000001, 2024-04-24T15:33:53.787 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:53.787 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "reached pg", 2024-04-24T15:33:53.788 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:53.788 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:53.788 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:53.788 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 39041 2024-04-24T15:33:53.788 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.788 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:53.788 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.788 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:53.788 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760552+0000", 2024-04-24T15:33:53.788 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:53.788 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.788 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.788 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:53.788 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760552+0000", 2024-04-24T15:33:53.789 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:53.789 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.789 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.789 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:53.789 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760550+0000", 2024-04-24T15:33:53.789 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999986 2024-04-24T15:33:53.789 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.789 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.789 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:53.789 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760557+0000", 2024-04-24T15:33:53.789 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1420000000000004e-06 2024-04-24T15:33:53.789 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.789 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.789 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:53.789 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760572+0000", 2024-04-24T15:33:53.790 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.483e-05 2024-04-24T15:33:53.790 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.790 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.790 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:53.790 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760578+0000", 2024-04-24T15:33:53.790 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.2820000000000002e-06 2024-04-24T15:33:53.790 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.790 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.790 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:53.790 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760628+0000", 2024-04-24T15:33:53.790 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.0040000000000002e-05 2024-04-24T15:33:53.790 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.790 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.791 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op must wait for map", 2024-04-24T15:33:53.791 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760637+0000", 2024-04-24T15:33:53.791 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.6899999999999998e-06 2024-04-24T15:33:53.791 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.791 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.791 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:53.791 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.770562+0000", 2024-04-24T15:33:53.791 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0099252739999999996 2024-04-24T15:33:53.791 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.791 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.791 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for peered", 2024-04-24T15:33:53.791 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.770572+0000", 2024-04-24T15:33:53.791 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0105999999999999e-05 2024-04-24T15:33:53.791 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.791 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.792 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:53.792 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:02.636170+0000", 2024-04-24T15:33:53.792 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.86559722299999997 2024-04-24T15:33:53.792 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.792 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.792 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:53.792 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:02.636259+0000", 2024-04-24T15:33:53.792 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.9446999999999999e-05 2024-04-24T15:33:53.792 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:53.792 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:53.792 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:53.792 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.792 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.792 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48675 4.d 4:b465d49b:::rbd_data.114a858e6641.0000000000000002:17 [read 2768896~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:53.793 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.926403+0000", 2024-04-24T15:33:53.793 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 42.860418699999997, 2024-04-24T15:33:53.793 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035742807000000001, 2024-04-24T15:33:53.793 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:53.793 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:53.793 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:53.793 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:53.793 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:53.793 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48675 2024-04-24T15:33:53.793 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.793 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:53.793 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.793 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:53.793 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926403+0000", 2024-04-24T15:33:53.794 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:53.794 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.794 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.794 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:53.794 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926403+0000", 2024-04-24T15:33:53.794 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:53.794 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.794 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.794 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:53.794 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926403+0000", 2024-04-24T15:33:53.794 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:53.794 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.794 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.794 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:53.795 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926404+0000", 2024-04-24T15:33:53.795 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.424e-06 2024-04-24T15:33:53.795 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.795 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.795 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:53.795 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926413+0000", 2024-04-24T15:33:53.795 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.6850000000000007e-06 2024-04-24T15:33:53.795 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.795 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.795 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:53.795 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926417+0000", 2024-04-24T15:33:53.795 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.7859999999999998e-06 2024-04-24T15:33:53.795 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.795 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.795 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:53.796 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.961986+0000", 2024-04-24T15:33:53.796 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035568862 2024-04-24T15:33:53.796 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.796 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.796 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:53.796 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.962063+0000", 2024-04-24T15:33:53.796 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7257999999999998e-05 2024-04-24T15:33:53.796 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.796 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.796 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:53.796 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.962146+0000", 2024-04-24T15:33:53.796 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3094000000000001e-05 2024-04-24T15:33:53.796 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:53.797 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:53.797 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:53.797 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.797 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.797 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48676 4.d 4:b465d49b:::rbd_data.114a858e6641.0000000000000002:17 [read 2834432~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:53.797 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.926421+0000", 2024-04-24T15:33:53.797 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 42.860401156999998, 2024-04-24T15:33:53.797 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035829002999999998, 2024-04-24T15:33:53.797 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:53.797 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:53.797 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:53.797 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:53.797 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:53.797 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48676 2024-04-24T15:33:53.798 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.798 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:53.798 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.798 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:53.798 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926421+0000", 2024-04-24T15:33:53.798 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:53.798 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.798 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.798 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:53.798 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926421+0000", 2024-04-24T15:33:53.798 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:53.798 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.798 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.798 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:53.799 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926420+0000", 2024-04-24T15:33:53.799 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:53.799 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.799 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.799 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:53.799 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926423+0000", 2024-04-24T15:33:53.799 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.356e-06 2024-04-24T15:33:53.799 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.799 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.799 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:53.799 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926428+0000", 2024-04-24T15:33:53.799 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.3340000000000001e-06 2024-04-24T15:33:53.799 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.799 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.800 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:53.800 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926431+0000", 2024-04-24T15:33:53.800 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.2899999999999998e-06 2024-04-24T15:33:53.800 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.800 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.800 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:53.800 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.962156+0000", 2024-04-24T15:33:53.800 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035725159999999999 2024-04-24T15:33:53.800 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.800 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.800 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:53.800 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.962197+0000", 2024-04-24T15:33:53.800 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0457999999999998e-05 2024-04-24T15:33:53.800 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.801 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.801 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:53.801 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.962250+0000", 2024-04-24T15:33:53.801 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.2812000000000002e-05 2024-04-24T15:33:53.801 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:53.801 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:53.801 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:53.801 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.801 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.801 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48704 4.d 4:b59d5735:::rbd_data.114a858e6641.0000000000000004:17 [read 2703360~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:53.801 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928541+0000", 2024-04-24T15:33:53.801 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 42.858281187000003, 2024-04-24T15:33:53.801 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035763016000000002, 2024-04-24T15:33:53.802 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:53.802 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:53.802 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:53.802 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:53.802 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:53.802 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48704 2024-04-24T15:33:53.802 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.802 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:53.802 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.802 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:53.802 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928541+0000", 2024-04-24T15:33:53.802 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:53.802 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.803 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.803 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:53.803 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928541+0000", 2024-04-24T15:33:53.803 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:53.803 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.803 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.803 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:53.803 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928540+0000", 2024-04-24T15:33:53.803 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:53.803 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.803 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.803 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:53.803 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928542+0000", 2024-04-24T15:33:53.803 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.37e-06 2024-04-24T15:33:53.804 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.804 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.804 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:53.804 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928545+0000", 2024-04-24T15:33:53.804 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8229999999999999e-06 2024-04-24T15:33:53.804 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.804 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.804 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:53.804 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928547+0000", 2024-04-24T15:33:53.804 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.9360000000000002e-06 2024-04-24T15:33:53.804 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.804 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.804 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:53.804 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964219+0000", 2024-04-24T15:33:53.805 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035672813999999997 2024-04-24T15:33:53.805 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.805 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.805 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:53.805 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964259+0000", 2024-04-24T15:33:53.805 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0070000000000001e-05 2024-04-24T15:33:53.805 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.805 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.805 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:53.805 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964304+0000", 2024-04-24T15:33:53.805 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4229e-05 2024-04-24T15:33:53.805 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:53.805 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:53.805 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:53.805 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.806 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.806 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48719 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 1851392~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:53.806 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928549+0000", 2024-04-24T15:33:53.806 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 42.85827304, 2024-04-24T15:33:53.806 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035890645999999998, 2024-04-24T15:33:53.806 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:53.806 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:53.806 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:53.806 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:53.806 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:53.806 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48719 2024-04-24T15:33:53.806 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.806 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:53.807 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.807 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:53.807 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928549+0000", 2024-04-24T15:33:53.807 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:53.807 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.807 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.807 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:53.807 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928549+0000", 2024-04-24T15:33:53.807 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:53.807 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.807 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.807 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:53.807 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928549+0000", 2024-04-24T15:33:53.807 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:53.808 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.808 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.808 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:53.808 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928550+0000", 2024-04-24T15:33:53.808 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3829999999999999e-06 2024-04-24T15:33:53.808 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.808 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.808 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:53.808 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928553+0000", 2024-04-24T15:33:53.808 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7369999999999998e-06 2024-04-24T15:33:53.808 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.808 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.808 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:53.808 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928555+0000", 2024-04-24T15:33:53.809 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.8920000000000001e-06 2024-04-24T15:33:53.809 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.809 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.809 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:53.809 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964240+0000", 2024-04-24T15:33:53.809 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035685537000000003 2024-04-24T15:33:53.809 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.809 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.809 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:53.809 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964318+0000", 2024-04-24T15:33:53.809 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7838999999999998e-05 2024-04-24T15:33:53.809 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.809 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.809 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:53.809 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964439+0000", 2024-04-24T15:33:53.810 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000121564 2024-04-24T15:33:53.810 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:53.810 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:53.810 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:53.810 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.810 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.810 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48720 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 1916928~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:53.810 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928557+0000", 2024-04-24T15:33:53.810 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 42.858265195000001, 2024-04-24T15:33:53.810 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036005219999999998, 2024-04-24T15:33:53.810 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:53.810 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:53.810 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:53.811 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:53.811 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:53.811 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48720 2024-04-24T15:33:53.811 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.811 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:53.811 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.811 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:53.811 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928557+0000", 2024-04-24T15:33:53.811 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:53.811 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.811 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.811 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:53.811 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928557+0000", 2024-04-24T15:33:53.811 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:53.811 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.812 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.812 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:53.812 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928556+0000", 2024-04-24T15:33:53.812 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:53.812 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.812 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.812 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:53.812 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928558+0000", 2024-04-24T15:33:53.812 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.13e-06 2024-04-24T15:33:53.812 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.812 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.812 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:53.812 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928561+0000", 2024-04-24T15:33:53.812 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5530000000000001e-06 2024-04-24T15:33:53.812 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.813 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.813 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:53.813 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928563+0000", 2024-04-24T15:33:53.813 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.8810000000000001e-06 2024-04-24T15:33:53.813 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.813 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.813 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:53.813 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964450+0000", 2024-04-24T15:33:53.813 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035887040000000002 2024-04-24T15:33:53.813 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.813 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.813 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:53.813 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964494+0000", 2024-04-24T15:33:53.813 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3699000000000003e-05 2024-04-24T15:33:53.813 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.813 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.814 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:53.814 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964562+0000", 2024-04-24T15:33:53.814 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.8149000000000001e-05 2024-04-24T15:33:53.814 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:53.814 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:53.814 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:53.814 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.814 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.814 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48721 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 1982464~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:53.814 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928565+0000", 2024-04-24T15:33:53.814 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 42.858256670999999, 2024-04-24T15:33:53.814 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036233466999999998, 2024-04-24T15:33:53.815 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:53.815 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:53.815 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:53.815 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:53.815 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:53.815 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48721 2024-04-24T15:33:53.815 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.815 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:53.815 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.815 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:53.815 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928565+0000", 2024-04-24T15:33:53.815 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:53.815 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.815 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.815 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:53.815 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928565+0000", 2024-04-24T15:33:53.816 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:53.816 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.816 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.816 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:53.816 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928565+0000", 2024-04-24T15:33:53.816 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:53.816 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.816 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.816 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:53.816 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928566+0000", 2024-04-24T15:33:53.816 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.336e-06 2024-04-24T15:33:53.816 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.816 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.816 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:53.816 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928569+0000", 2024-04-24T15:33:53.817 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.9189999999999999e-06 2024-04-24T15:33:53.817 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.817 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.817 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:53.817 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928571+0000", 2024-04-24T15:33:53.817 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.1909999999999999e-06 2024-04-24T15:33:53.817 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.817 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.817 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:53.817 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964592+0000", 2024-04-24T15:33:53.817 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036020120000000003 2024-04-24T15:33:53.817 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.817 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.817 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:53.817 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964673+0000", 2024-04-24T15:33:53.818 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.1254000000000005e-05 2024-04-24T15:33:53.818 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.818 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.818 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:53.818 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964799+0000", 2024-04-24T15:33:53.818 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00012588700000000001 2024-04-24T15:33:53.818 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:53.818 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:53.818 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:53.818 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.818 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.818 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48722 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2048000~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:53.818 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928586+0000", 2024-04-24T15:33:53.818 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 42.858235546000003, 2024-04-24T15:33:53.819 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036331347999999999, 2024-04-24T15:33:53.819 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:53.819 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:53.819 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:53.819 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:53.819 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:53.819 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48722 2024-04-24T15:33:53.819 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.819 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:53.819 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.819 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:53.819 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928586+0000", 2024-04-24T15:33:53.819 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:53.819 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.819 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.820 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:53.820 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928586+0000", 2024-04-24T15:33:53.820 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:53.820 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.820 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.820 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:53.820 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928586+0000", 2024-04-24T15:33:53.820 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:53.820 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.820 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.820 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:53.820 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928587+0000", 2024-04-24T15:33:53.820 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3519999999999999e-06 2024-04-24T15:33:53.820 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.820 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.821 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:53.821 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928590+0000", 2024-04-24T15:33:53.821 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.9699999999999999e-06 2024-04-24T15:33:53.821 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.821 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.821 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:53.821 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928592+0000", 2024-04-24T15:33:53.821 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.097e-06 2024-04-24T15:33:53.821 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.821 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.821 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:53.821 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964810+0000", 2024-04-24T15:33:53.821 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036217249 2024-04-24T15:33:53.821 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.822 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.822 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:53.822 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964851+0000", 2024-04-24T15:33:53.822 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.1151e-05 2024-04-24T15:33:53.822 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.822 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.822 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:53.822 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964918+0000", 2024-04-24T15:33:53.822 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.6898999999999998e-05 2024-04-24T15:33:53.822 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:53.822 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:53.822 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:53.822 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.823 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.823 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48723 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2113536~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:53.823 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928598+0000", 2024-04-24T15:33:53.823 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 42.858224157000002, 2024-04-24T15:33:53.823 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036552125999999997, 2024-04-24T15:33:53.823 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:53.823 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:53.823 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:53.823 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:53.823 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:53.823 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48723 2024-04-24T15:33:53.823 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.823 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:53.823 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.823 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:53.824 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928598+0000", 2024-04-24T15:33:53.824 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:53.824 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.824 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.824 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:53.824 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928598+0000", 2024-04-24T15:33:53.824 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:53.824 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.824 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.824 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:53.824 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928597+0000", 2024-04-24T15:33:53.824 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:53.824 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.825 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.825 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:53.825 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928599+0000", 2024-04-24T15:33:53.825 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.621e-06 2024-04-24T15:33:53.825 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.825 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.825 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:53.825 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928602+0000", 2024-04-24T15:33:53.825 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.9629999999999998e-06 2024-04-24T15:33:53.825 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.825 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.825 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:53.825 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928607+0000", 2024-04-24T15:33:53.825 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8940000000000001e-06 2024-04-24T15:33:53.825 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.825 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.826 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:53.826 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964947+0000", 2024-04-24T15:33:53.826 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036340439000000002 2024-04-24T15:33:53.826 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.826 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.826 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:53.826 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965028+0000", 2024-04-24T15:33:53.826 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.1054e-05 2024-04-24T15:33:53.826 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.826 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.826 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:53.826 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965150+0000", 2024-04-24T15:33:53.826 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00012154000000000001 2024-04-24T15:33:53.826 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:53.827 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:53.827 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:53.827 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.827 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.827 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48724 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2179072~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:53.827 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928609+0000", 2024-04-24T15:33:53.827 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 42.858212782000003, 2024-04-24T15:33:53.827 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036679614999999999, 2024-04-24T15:33:53.827 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:53.827 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:53.827 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:53.827 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:53.827 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:53.827 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48724 2024-04-24T15:33:53.828 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.828 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:53.828 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.828 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:53.828 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928609+0000", 2024-04-24T15:33:53.828 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:53.828 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.828 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.828 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:53.828 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928609+0000", 2024-04-24T15:33:53.828 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:53.828 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.828 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.828 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:53.829 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928609+0000", 2024-04-24T15:33:53.829 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:53.829 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.829 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.829 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:53.829 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928610+0000", 2024-04-24T15:33:53.829 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3030000000000001e-06 2024-04-24T15:33:53.829 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.829 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.829 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:53.829 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928613+0000", 2024-04-24T15:33:53.829 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.762e-06 2024-04-24T15:33:53.829 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.829 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.830 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:53.830 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928615+0000", 2024-04-24T15:33:53.830 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.832e-06 2024-04-24T15:33:53.830 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.830 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.830 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:53.830 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965174+0000", 2024-04-24T15:33:53.830 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036559633000000001 2024-04-24T15:33:53.830 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.830 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.830 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:53.830 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965221+0000", 2024-04-24T15:33:53.830 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6239999999999998e-05 2024-04-24T15:33:53.830 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.831 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.831 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:53.831 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965289+0000", 2024-04-24T15:33:53.831 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.8053999999999996e-05 2024-04-24T15:33:53.831 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:53.831 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:53.831 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:53.831 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.831 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.831 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48725 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2244608~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:53.831 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928617+0000", 2024-04-24T15:33:53.831 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 42.858205011999999, 2024-04-24T15:33:53.831 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036902659999999997, 2024-04-24T15:33:53.832 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:53.832 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:53.832 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:53.832 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:53.832 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:53.832 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48725 2024-04-24T15:33:53.832 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.832 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:53.832 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.832 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:53.832 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928617+0000", 2024-04-24T15:33:53.832 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:53.832 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.832 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.833 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:53.833 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928617+0000", 2024-04-24T15:33:53.833 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:53.833 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.833 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.833 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:53.833 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928617+0000", 2024-04-24T15:33:53.833 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:53.833 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.833 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.833 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:53.833 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928618+0000", 2024-04-24T15:33:53.833 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.206e-06 2024-04-24T15:33:53.833 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.833 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.834 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:53.834 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928621+0000", 2024-04-24T15:33:53.834 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5130000000000001e-06 2024-04-24T15:33:53.834 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.834 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.834 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:53.834 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928623+0000", 2024-04-24T15:33:53.834 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.9640000000000002e-06 2024-04-24T15:33:53.834 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.834 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.834 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:53.834 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965318+0000", 2024-04-24T15:33:53.834 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036694953000000002 2024-04-24T15:33:53.834 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.835 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.835 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:53.835 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965398+0000", 2024-04-24T15:33:53.835 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.9986000000000003e-05 2024-04-24T15:33:53.835 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.835 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.835 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:53.835 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965520+0000", 2024-04-24T15:33:53.835 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000121247 2024-04-24T15:33:53.835 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:53.835 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:53.835 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:53.835 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.835 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.836 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48726 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2310144~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:53.836 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928625+0000", 2024-04-24T15:33:53.836 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 42.858196464999999, 2024-04-24T15:33:53.836 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037015521000000003, 2024-04-24T15:33:53.836 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:53.836 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:53.836 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:53.836 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:53.836 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:53.836 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48726 2024-04-24T15:33:53.836 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.836 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:53.836 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.836 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:53.836 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928625+0000", 2024-04-24T15:33:53.837 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:53.837 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.837 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.837 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:53.837 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928625+0000", 2024-04-24T15:33:53.837 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:53.837 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.837 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.837 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:53.837 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928625+0000", 2024-04-24T15:33:53.837 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:53.837 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.837 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.837 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:53.838 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928627+0000", 2024-04-24T15:33:53.838 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3179999999999999e-06 2024-04-24T15:33:53.838 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.838 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.838 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:53.838 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928629+0000", 2024-04-24T15:33:53.838 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.819e-06 2024-04-24T15:33:53.838 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.838 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.838 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:53.838 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928634+0000", 2024-04-24T15:33:53.838 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6999999999999999e-06 2024-04-24T15:33:53.838 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.838 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.839 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:53.839 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965531+0000", 2024-04-24T15:33:53.839 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036897040999999998 2024-04-24T15:33:53.839 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.839 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.839 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:53.839 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965573+0000", 2024-04-24T15:33:53.839 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.189e-05 2024-04-24T15:33:53.839 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.839 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.839 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:53.839 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965641+0000", 2024-04-24T15:33:53.839 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.7959000000000004e-05 2024-04-24T15:33:53.839 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:53.840 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:53.840 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:53.840 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.840 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.840 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48727 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2375680~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:53.840 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928637+0000", 2024-04-24T15:33:53.840 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 42.858185104, 2024-04-24T15:33:53.840 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037235129999999998, 2024-04-24T15:33:53.840 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:53.840 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:53.840 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:53.840 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:53.840 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:53.840 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48727 2024-04-24T15:33:53.841 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.841 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:53.841 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.841 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:53.841 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928637+0000", 2024-04-24T15:33:53.841 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:53.841 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.841 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.841 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:53.841 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928637+0000", 2024-04-24T15:33:53.841 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:53.841 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.841 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.842 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:53.842 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928637+0000", 2024-04-24T15:33:53.842 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:53.842 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.842 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.842 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:53.842 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928638+0000", 2024-04-24T15:33:53.842 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.4279999999999999e-06 2024-04-24T15:33:53.842 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.842 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.842 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:53.842 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928641+0000", 2024-04-24T15:33:53.842 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.909e-06 2024-04-24T15:33:53.842 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.843 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.843 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:53.843 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928643+0000", 2024-04-24T15:33:53.843 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.9269999999999999e-06 2024-04-24T15:33:53.843 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.843 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.843 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:53.843 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965671+0000", 2024-04-24T15:33:53.843 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037028393 2024-04-24T15:33:53.843 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.843 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.843 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:53.843 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965752+0000", 2024-04-24T15:33:53.843 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0921999999999995e-05 2024-04-24T15:33:53.844 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.844 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.844 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:53.844 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965872+0000", 2024-04-24T15:33:53.844 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000119759 2024-04-24T15:33:53.844 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:53.844 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:53.844 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:53.844 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.844 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.844 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48728 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2441216~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:53.844 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928645+0000", 2024-04-24T15:33:53.844 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 42.858176921000002, 2024-04-24T15:33:53.844 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037346852999999999, 2024-04-24T15:33:53.845 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:53.845 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:53.845 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:53.845 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:53.845 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:53.845 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48728 2024-04-24T15:33:53.845 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.845 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:53.845 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.845 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:53.845 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928645+0000", 2024-04-24T15:33:53.845 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:53.845 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.845 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.846 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:53.846 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928645+0000", 2024-04-24T15:33:53.846 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:53.846 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.846 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.846 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:53.846 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928645+0000", 2024-04-24T15:33:53.846 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:53.846 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.846 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.846 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:53.846 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928646+0000", 2024-04-24T15:33:53.846 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.291e-06 2024-04-24T15:33:53.846 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.847 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.847 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:53.847 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928649+0000", 2024-04-24T15:33:53.847 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8509999999999999e-06 2024-04-24T15:33:53.847 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.847 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.847 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:53.847 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928651+0000", 2024-04-24T15:33:53.847 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.809e-06 2024-04-24T15:33:53.847 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.847 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.847 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:53.847 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965882+0000", 2024-04-24T15:33:53.847 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037231715999999998 2024-04-24T15:33:53.848 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.848 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.848 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:53.848 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965925+0000", 2024-04-24T15:33:53.848 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2493999999999997e-05 2024-04-24T15:33:53.848 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.848 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.848 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:53.848 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965992+0000", 2024-04-24T15:33:53.848 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.6907000000000002e-05 2024-04-24T15:33:53.848 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:53.848 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:53.848 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:53.848 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.849 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.849 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48729 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2506752~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:53.849 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928653+0000", 2024-04-24T15:33:53.849 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 42.858169046, 2024-04-24T15:33:53.849 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037576904000000001, 2024-04-24T15:33:53.849 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:53.849 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:53.849 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:53.849 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:53.849 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:53.849 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48729 2024-04-24T15:33:53.849 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.849 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:53.849 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.850 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:53.850 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928653+0000", 2024-04-24T15:33:53.850 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:53.850 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.850 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.850 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:53.850 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928653+0000", 2024-04-24T15:33:53.850 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:53.850 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.850 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.850 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:53.850 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928653+0000", 2024-04-24T15:33:53.850 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:53.850 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.851 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.851 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:53.851 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928654+0000", 2024-04-24T15:33:53.851 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3209999999999999e-06 2024-04-24T15:33:53.851 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.851 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.851 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:53.851 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928657+0000", 2024-04-24T15:33:53.851 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8559999999999998e-06 2024-04-24T15:33:53.851 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.851 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.851 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:53.851 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928659+0000", 2024-04-24T15:33:53.851 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7080000000000002e-06 2024-04-24T15:33:53.852 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.852 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.852 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:53.852 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966021+0000", 2024-04-24T15:33:53.852 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037361296000000002 2024-04-24T15:33:53.852 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.852 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.852 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:53.852 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966102+0000", 2024-04-24T15:33:53.852 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0939999999999994e-05 2024-04-24T15:33:53.852 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.852 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.852 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:53.853 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966230+0000", 2024-04-24T15:33:53.853 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00012799999999999999 2024-04-24T15:33:53.853 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:53.853 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:53.853 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:53.853 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.853 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.853 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48730 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2572288~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:53.853 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928662+0000", 2024-04-24T15:33:53.853 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 42.858159958000002, 2024-04-24T15:33:53.853 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037687246000000001, 2024-04-24T15:33:53.853 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:53.853 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:53.853 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:53.854 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:53.854 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:53.854 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48730 2024-04-24T15:33:53.854 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.854 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:53.854 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.854 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:53.854 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928662+0000", 2024-04-24T15:33:53.854 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:53.854 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.854 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.854 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:53.858 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928662+0000", 2024-04-24T15:33:53.858 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:53.858 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.858 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.858 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:53.858 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928662+0000", 2024-04-24T15:33:53.858 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:53.858 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.858 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.858 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:53.859 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928663+0000", 2024-04-24T15:33:53.859 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.381e-06 2024-04-24T15:33:53.859 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.859 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.859 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:53.859 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928666+0000", 2024-04-24T15:33:53.859 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.796e-06 2024-04-24T15:33:53.859 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.859 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.859 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:53.859 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928668+0000", 2024-04-24T15:33:53.859 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.8920000000000001e-06 2024-04-24T15:33:53.859 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.860 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.860 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:53.860 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966240+0000", 2024-04-24T15:33:53.860 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037572886999999999 2024-04-24T15:33:53.860 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.860 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.860 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:53.860 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966281+0000", 2024-04-24T15:33:53.860 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0450000000000001e-05 2024-04-24T15:33:53.860 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.860 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.860 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:53.860 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966349+0000", 2024-04-24T15:33:53.861 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.8215999999999999e-05 2024-04-24T15:33:53.861 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:53.861 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:53.861 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:53.861 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.861 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.861 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48731 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2637824~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:53.861 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928747+0000", 2024-04-24T15:33:53.861 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 42.858074952999999, 2024-04-24T15:33:53.861 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037833602000000001, 2024-04-24T15:33:53.861 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:53.861 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:53.861 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:53.861 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:53.862 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:53.862 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48731 2024-04-24T15:33:53.862 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.862 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:53.862 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.862 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:53.862 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928747+0000", 2024-04-24T15:33:53.862 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:53.862 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.862 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.862 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:53.862 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928747+0000", 2024-04-24T15:33:53.863 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:53.863 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.863 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.863 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:53.863 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928670+0000", 2024-04-24T15:33:53.863 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999228 2024-04-24T15:33:53.863 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.863 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.863 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:53.863 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928750+0000", 2024-04-24T15:33:53.863 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0527999999999999e-05 2024-04-24T15:33:53.863 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.863 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.863 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:53.864 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928757+0000", 2024-04-24T15:33:53.864 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.9419999999999998e-06 2024-04-24T15:33:53.864 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.864 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.864 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:53.864 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928762+0000", 2024-04-24T15:33:53.864 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4240000000000003e-06 2024-04-24T15:33:53.864 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.864 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.864 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:53.864 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966379+0000", 2024-04-24T15:33:53.864 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037617267000000003 2024-04-24T15:33:53.864 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.864 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.864 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:53.865 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966461+0000", 2024-04-24T15:33:53.865 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.2720999999999996e-05 2024-04-24T15:33:53.865 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.865 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.865 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:53.865 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966581+0000", 2024-04-24T15:33:53.865 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000119014 2024-04-24T15:33:53.865 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:53.865 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:53.865 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:53.865 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.865 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.865 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48732 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2703360~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:53.865 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928766+0000", 2024-04-24T15:33:53.865 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 42.858055469, 2024-04-24T15:33:53.866 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037930924999999997, 2024-04-24T15:33:53.866 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:53.866 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:53.866 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:53.866 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:53.866 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:53.866 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48732 2024-04-24T15:33:53.866 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.866 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:53.866 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.866 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:53.866 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928766+0000", 2024-04-24T15:33:53.866 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:53.867 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.867 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.867 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:53.867 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928766+0000", 2024-04-24T15:33:53.867 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:53.867 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.867 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.867 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:53.867 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928766+0000", 2024-04-24T15:33:53.867 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:53.867 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.867 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.867 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:53.867 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928768+0000", 2024-04-24T15:33:53.867 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.3420000000000002e-06 2024-04-24T15:33:53.868 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.868 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.868 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:53.868 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928783+0000", 2024-04-24T15:33:53.868 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.4813e-05 2024-04-24T15:33:53.868 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.868 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.868 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:53.868 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928787+0000", 2024-04-24T15:33:53.868 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.1999999999999996e-06 2024-04-24T15:33:53.868 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.868 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.868 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:53.868 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966592+0000", 2024-04-24T15:33:53.868 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037804484999999999 2024-04-24T15:33:53.869 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.869 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.869 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:53.869 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966632+0000", 2024-04-24T15:33:53.869 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0014e-05 2024-04-24T15:33:53.869 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.869 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.869 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:53.869 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966697+0000", 2024-04-24T15:33:53.869 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.5489999999999998e-05 2024-04-24T15:33:53.869 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:53.869 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:53.869 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:53.869 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.869 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.870 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48813 4.d 4:b39400aa:::rbd_data.114a8b543e91.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:53.870 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.936015+0000", 2024-04-24T15:33:53.870 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 42.850807134999997, 2024-04-24T15:33:53.870 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.039312294999999997, 2024-04-24T15:33:53.870 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:53.870 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:53.870 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:53.870 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:53.870 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:53.870 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48813 2024-04-24T15:33:53.870 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.870 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:53.870 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.871 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:53.871 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.936015+0000", 2024-04-24T15:33:53.871 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:53.871 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.871 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.871 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:53.871 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.936015+0000", 2024-04-24T15:33:53.871 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:53.871 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.871 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.871 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:53.871 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.936014+0000", 2024-04-24T15:33:53.871 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:53.871 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.871 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.872 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:53.872 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.942618+0000", 2024-04-24T15:33:53.872 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0066038499999999997 2024-04-24T15:33:53.872 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.872 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.872 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:53.872 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.942629+0000", 2024-04-24T15:33:53.872 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0606e-05 2024-04-24T15:33:53.872 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.872 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.872 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:53.872 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.942634+0000", 2024-04-24T15:33:53.872 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.2000000000000002e-06 2024-04-24T15:33:53.872 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.873 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.873 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:53.873 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.969006+0000", 2024-04-24T15:33:53.873 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.026371564 2024-04-24T15:33:53.873 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.873 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.873 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:53.873 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.969126+0000", 2024-04-24T15:33:53.873 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000120085 2024-04-24T15:33:53.873 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.873 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.873 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 2", 2024-04-24T15:33:53.873 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.969469+0000", 2024-04-24T15:33:53.873 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00034295400000000002 2024-04-24T15:33:53.874 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.874 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.874 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:33:53.874 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.973118+0000", 2024-04-24T15:33:53.874 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0036489700000000001 2024-04-24T15:33:53.874 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.874 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.874 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:33:53.874 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.975250+0000", 2024-04-24T15:33:53.874 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0021327030000000001 2024-04-24T15:33:53.874 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.874 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.874 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:33:53.875 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.975293+0000", 2024-04-24T15:33:53.875 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2951999999999998e-05 2024-04-24T15:33:53.875 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.875 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.875 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:53.875 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.975327+0000", 2024-04-24T15:33:53.875 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.3840000000000001e-05 2024-04-24T15:33:53.875 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:53.875 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:53.875 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:53.875 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.875 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.875 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:75520 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:head [delete] snapc 17=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e105)", 2024-04-24T15:33:53.875 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:42.638168+0000", 2024-04-24T15:33:53.876 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 11.148653512999999, 2024-04-24T15:33:53.876 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0190266560000001, 2024-04-24T15:33:53.876 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:53.876 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:53.876 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:53.876 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:53.876 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:53.876 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 75520 2024-04-24T15:33:53.876 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.876 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:53.876 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.876 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:53.876 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.638168+0000", 2024-04-24T15:33:53.876 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:53.876 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.877 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.877 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:53.877 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.638168+0000", 2024-04-24T15:33:53.877 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:53.877 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.877 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.877 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:53.877 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.638167+0000", 2024-04-24T15:33:53.877 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999986 2024-04-24T15:33:53.877 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.877 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.877 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:53.877 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.638174+0000", 2024-04-24T15:33:53.877 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.861e-06 2024-04-24T15:33:53.877 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.878 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.878 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:53.878 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.638186+0000", 2024-04-24T15:33:53.878 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.2109e-05 2024-04-24T15:33:53.878 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.878 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.878 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:53.878 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.638194+0000", 2024-04-24T15:33:53.878 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.8040000000000002e-06 2024-04-24T15:33:53.878 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.878 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.878 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:53.878 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.638682+0000", 2024-04-24T15:33:53.879 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00048809599999999998 2024-04-24T15:33:53.879 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.879 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.879 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for peered", 2024-04-24T15:33:53.879 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.638692+0000", 2024-04-24T15:33:53.879 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.8970000000000003e-06 2024-04-24T15:33:53.879 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.879 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.879 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:53.879 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:43.649868+0000", 2024-04-24T15:33:53.879 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.011176641 2024-04-24T15:33:53.879 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.879 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.879 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:53.879 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:43.649960+0000", 2024-04-24T15:33:53.880 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.1632999999999995e-05 2024-04-24T15:33:53.880 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.880 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.880 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 6", 2024-04-24T15:33:53.880 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:43.650060+0000", 2024-04-24T15:33:53.880 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.9562999999999995e-05 2024-04-24T15:33:53.880 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.880 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.880 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:33:53.880 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:43.654465+0000", 2024-04-24T15:33:53.880 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0044056620000000003 2024-04-24T15:33:53.880 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.880 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.880 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:33:53.880 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:43.656963+0000", 2024-04-24T15:33:53.881 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.002497829 2024-04-24T15:33:53.881 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.881 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.881 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:33:53.881 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:43.657083+0000", 2024-04-24T15:33:53.881 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000120338 2024-04-24T15:33:53.881 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:53.881 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:53.881 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:53.881 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:43.657195+0000", 2024-04-24T15:33:53.881 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000111624 2024-04-24T15:33:53.881 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:53.881 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:53.881 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:53.881 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:53.882 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:53.882 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:53.883 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-24T15:33:53.884 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:33:53.884+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:33:53.920 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-04-24T15:33:53.938 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-04-24T15:33:53.955 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-04-24T15:33:53.973 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-04-24T15:33:53.984 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:33:53.985+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:33:53.992 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-04-24T15:33:54.010 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-04-24T15:33:54.028 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-04-24T15:33:54.048 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-04-24T15:33:54.049 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T15:33:54.049 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T15:33:54.049 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T15:33:54.049 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T15:33:54.049 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T15:33:54.049 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T15:33:54.049 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T15:33:54.049 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T15:33:54.051 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:54.051 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:54.061 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-24T15:33:54.102 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:54.104+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:54.185 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:54.187+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:54.225 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:54.225 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:54.235 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-24T15:33:54.286 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:33:54.287+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:33:54.358 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:54.358 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:54.367 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-24T15:33:54.410 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:33:54.412+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:33:54.488 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:54.489+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:54.491 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:54.491 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:54.500 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-24T15:33:54.589 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:54.591+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:54.666 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:54.666 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:54.676 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-24T15:33:54.690 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:54.692+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:54.791 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:33:54.792+0000 7f0f45efb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 59992) UID: 0 2024-04-24T15:33:54.852 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:54.852 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:54.863 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-24T15:33:54.904 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:54.906+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:54.992 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:54.993+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:55.044 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:55.044 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:55.055 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-24T15:33:55.093 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:55.094+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:55.194 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:55.195+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:55.201 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:55.202 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:55.215 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-24T15:33:55.294 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:33:55.295+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:33:55.351 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:55.351 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:55.362 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-24T15:33:55.406 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:55.407+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:55.494 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:55.494 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:55.495 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:55.497+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:55.505 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-24T15:33:55.596 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:55.598+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:55.658 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-24T15:33:55.676 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:55.676 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:55.690 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-24T15:33:55.698 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:55.699+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:55.799 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:33:55.800+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:33:55.872 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:55.872 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:55.884 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-24T15:33:55.900 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:33:55.902+0000 7f0f45efb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 59992) UID: 0 2024-04-24T15:33:56.001 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:33:56.003+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:33:56.032 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:56.032 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:56.044 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-24T15:33:56.060 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-24T15:33:56.102 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:33:56.104+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:33:56.170 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:56.171 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:56.181 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-24T15:33:56.224 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:33:56.226+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:33:56.230 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-04-24T15:33:56.249 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-04-24T15:33:56.267 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-04-24T15:33:56.286 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-04-24T15:33:56.304 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:56.306+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:56.305 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-04-24T15:33:56.310 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:56.311 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-04-24T15:33:56.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-04-24T15:33:56.311 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [] 2024-04-24T15:33:56.311 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:56.321 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-24T15:33:56.324 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-04-24T15:33:56.345 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-04-24T15:33:56.365 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-04-24T15:33:56.366 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T15:33:56.366 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T15:33:56.367 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T15:33:56.367 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T15:33:56.367 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T15:33:56.367 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T15:33:56.367 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T15:33:56.367 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T15:33:56.417 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:56.419+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:56.451 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:56.451 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:33:56.451 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-04-24T15:33:56.451 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:56.461 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-24T15:33:56.506 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:33:56.508+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:33:56.607 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:56.608+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:56.622 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:56.623 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:33:56.623 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-04-24T15:33:56.623 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-04-24T15:33:56.623 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:56.633 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-24T15:33:56.698 INFO:teuthology.orchestra.run.smithi096.stderr:nodeep-scrub is set 2024-04-24T15:33:56.708 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:56.710+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:56.769 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:56.769 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-04-24T15:33:56.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-04-24T15:33:56.769 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-04-24T15:33:56.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.770 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:2522 4.6 4:60ceedf0:::rbd_data.114a1f60cb66.000000000000011d:head [set-alloc-hint object_size 262144 write_size 262144,write 110592~2048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-24T15:33:56.770 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:23.088240+0000", 2024-04-24T15:33:56.770 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 93.680780460999998, 2024-04-24T15:33:56.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.988232797, 2024-04-24T15:33:56.770 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:56.770 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:56.770 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:56.770 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:56.770 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:56.770 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2522 2024-04-24T15:33:56.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.770 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:56.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:56.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088240+0000", 2024-04-24T15:33:56.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:56.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:56.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088240+0000", 2024-04-24T15:33:56.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:56.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:56.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088238+0000", 2024-04-24T15:33:56.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999981 2024-04-24T15:33:56.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:56.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088254+0000", 2024-04-24T15:33:56.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5562000000000001e-05 2024-04-24T15:33:56.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:56.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088275+0000", 2024-04-24T15:33:56.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1256000000000001e-05 2024-04-24T15:33:56.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:56.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088283+0000", 2024-04-24T15:33:56.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3219999999999993e-06 2024-04-24T15:33:56.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:56.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088351+0000", 2024-04-24T15:33:56.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7595000000000006e-05 2024-04-24T15:33:56.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-04-24T15:33:56.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088362+0000", 2024-04-24T15:33:56.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1764e-05 2024-04-24T15:33:56.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:56.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.073748+0000", 2024-04-24T15:33:56.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98538541899999998 2024-04-24T15:33:56.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:56.774 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.073925+0000", 2024-04-24T15:33:56.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017747600000000001 2024-04-24T15:33:56.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-24T15:33:56.774 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.074092+0000", 2024-04-24T15:33:56.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000166622 2024-04-24T15:33:56.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:33:56.774 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.074774+0000", 2024-04-24T15:33:56.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00068225000000000002 2024-04-24T15:33:56.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:33:56.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.076412+0000", 2024-04-24T15:33:56.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001638189 2024-04-24T15:33:56.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:33:56.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.076446+0000", 2024-04-24T15:33:56.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3587000000000002e-05 2024-04-24T15:33:56.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:56.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.076473+0000", 2024-04-24T15:33:56.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6886999999999999e-05 2024-04-24T15:33:56.776 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:56.776 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:56.776 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:56.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.776 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43121 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 5865472~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:56.776 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094178+0000", 2024-04-24T15:33:56.776 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 49.674842777999999, 2024-04-24T15:33:56.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032860598999999997, 2024-04-24T15:33:56.776 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:56.776 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:56.776 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:56.776 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:56.776 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:56.776 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43121 2024-04-24T15:33:56.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.777 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:56.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:56.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094178+0000", 2024-04-24T15:33:56.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:56.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:56.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094178+0000", 2024-04-24T15:33:56.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:56.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:56.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094178+0000", 2024-04-24T15:33:56.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:56.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:56.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094180+0000", 2024-04-24T15:33:56.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9989999999999998e-06 2024-04-24T15:33:56.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:56.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094185+0000", 2024-04-24T15:33:56.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.541e-06 2024-04-24T15:33:56.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:56.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094189+0000", 2024-04-24T15:33:56.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0509999999999998e-06 2024-04-24T15:33:56.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:56.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.126877+0000", 2024-04-24T15:33:56.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032687473000000002 2024-04-24T15:33:56.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:56.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.126952+0000", 2024-04-24T15:33:56.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5924e-05 2024-04-24T15:33:56.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:56.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127038+0000", 2024-04-24T15:33:56.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5908999999999994e-05 2024-04-24T15:33:56.780 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:56.780 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:56.780 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:56.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.780 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43122 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 6291456~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:56.780 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094193+0000", 2024-04-24T15:33:56.780 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 49.674827534000002, 2024-04-24T15:33:56.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032978383999999999, 2024-04-24T15:33:56.780 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:56.780 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:56.780 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:56.781 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:56.781 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:56.781 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43122 2024-04-24T15:33:56.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.781 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:56.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:56.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094193+0000", 2024-04-24T15:33:56.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:56.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:56.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094193+0000", 2024-04-24T15:33:56.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:56.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:56.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094193+0000", 2024-04-24T15:33:56.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:56.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:56.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094195+0000", 2024-04-24T15:33:56.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3269999999999999e-06 2024-04-24T15:33:56.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:56.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094201+0000", 2024-04-24T15:33:56.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9850000000000003e-06 2024-04-24T15:33:56.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:56.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094205+0000", 2024-04-24T15:33:56.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8430000000000003e-06 2024-04-24T15:33:56.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:56.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127068+0000", 2024-04-24T15:33:56.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032863305000000002 2024-04-24T15:33:56.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:56.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127113+0000", 2024-04-24T15:33:56.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4412000000000001e-05 2024-04-24T15:33:56.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:56.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127171+0000", 2024-04-24T15:33:56.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8807999999999997e-05 2024-04-24T15:33:56.784 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:56.784 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:56.784 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:56.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.784 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43123 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 6717440~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:56.784 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094209+0000", 2024-04-24T15:33:56.784 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 49.674811368, 2024-04-24T15:33:56.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033144583999999998, 2024-04-24T15:33:56.785 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:56.785 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:56.785 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:56.785 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:56.785 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:56.785 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43123 2024-04-24T15:33:56.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.785 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:56.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:56.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094209+0000", 2024-04-24T15:33:56.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:56.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:56.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094209+0000", 2024-04-24T15:33:56.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:56.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:56.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094209+0000", 2024-04-24T15:33:56.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:56.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:56.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094211+0000", 2024-04-24T15:33:56.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2089999999999999e-06 2024-04-24T15:33:56.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:56.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094217+0000", 2024-04-24T15:33:56.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6799999999999998e-06 2024-04-24T15:33:56.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:56.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094220+0000", 2024-04-24T15:33:56.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.444e-06 2024-04-24T15:33:56.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:56.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127190+0000", 2024-04-24T15:33:56.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032969471 2024-04-24T15:33:56.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:56.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127269+0000", 2024-04-24T15:33:56.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9246999999999995e-05 2024-04-24T15:33:56.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:56.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127354+0000", 2024-04-24T15:33:56.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4853999999999998e-05 2024-04-24T15:33:56.788 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:56.788 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:56.788 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:56.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.788 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43124 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 7143424~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:56.789 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094224+0000", 2024-04-24T15:33:56.789 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 49.674796395000001, 2024-04-24T15:33:56.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033271955999999998, 2024-04-24T15:33:56.789 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:56.789 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:56.789 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:56.789 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:56.789 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:56.789 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43124 2024-04-24T15:33:56.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.789 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:56.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:56.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094224+0000", 2024-04-24T15:33:56.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:56.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:56.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094224+0000", 2024-04-24T15:33:56.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:56.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:56.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094224+0000", 2024-04-24T15:33:56.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:56.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:56.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094226+0000", 2024-04-24T15:33:56.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9039999999999999e-06 2024-04-24T15:33:56.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:56.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094233+0000", 2024-04-24T15:33:56.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7970000000000003e-06 2024-04-24T15:33:56.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:56.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094236+0000", 2024-04-24T15:33:56.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6339999999999999e-06 2024-04-24T15:33:56.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:56.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127390+0000", 2024-04-24T15:33:56.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.03315353 2024-04-24T15:33:56.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:56.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127439+0000", 2024-04-24T15:33:56.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8890000000000001e-05 2024-04-24T15:33:56.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:56.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127496+0000", 2024-04-24T15:33:56.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7476999999999999e-05 2024-04-24T15:33:56.792 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:56.793 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:56.793 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:56.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.793 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43125 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 7569408~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:56.793 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094240+0000", 2024-04-24T15:33:56.793 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 49.674780308000003, 2024-04-24T15:33:56.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033446278000000003, 2024-04-24T15:33:56.793 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:56.793 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:56.793 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:56.793 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:56.793 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:56.793 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43125 2024-04-24T15:33:56.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.794 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:56.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:56.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094240+0000", 2024-04-24T15:33:56.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:56.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:56.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094240+0000", 2024-04-24T15:33:56.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:56.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:56.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094240+0000", 2024-04-24T15:33:56.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:56.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:56.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094247+0000", 2024-04-24T15:33:56.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6209999999999998e-06 2024-04-24T15:33:56.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:56.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094253+0000", 2024-04-24T15:33:56.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0739999999999998e-06 2024-04-24T15:33:56.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:56.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094256+0000", 2024-04-24T15:33:56.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7699999999999999e-06 2024-04-24T15:33:56.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:56.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127526+0000", 2024-04-24T15:33:56.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033269703999999997 2024-04-24T15:33:56.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:56.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127603+0000", 2024-04-24T15:33:56.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6964000000000004e-05 2024-04-24T15:33:56.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:56.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127687+0000", 2024-04-24T15:33:56.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3431999999999997e-05 2024-04-24T15:33:56.796 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:56.797 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:56.797 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:56.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.797 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43126 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 7995392~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:33:56.797 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094261+0000", 2024-04-24T15:33:56.797 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 49.674759805000001, 2024-04-24T15:33:56.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033555201999999999, 2024-04-24T15:33:56.797 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:56.797 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:56.797 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:56.797 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:56.797 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:56.797 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43126 2024-04-24T15:33:56.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.798 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:56.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:56.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094261+0000", 2024-04-24T15:33:56.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:56.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:56.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094261+0000", 2024-04-24T15:33:56.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:56.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:56.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094260+0000", 2024-04-24T15:33:56.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:56.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:56.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094263+0000", 2024-04-24T15:33:56.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2579999999999998e-06 2024-04-24T15:33:56.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:56.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094269+0000", 2024-04-24T15:33:56.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.823e-06 2024-04-24T15:33:56.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:56.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094272+0000", 2024-04-24T15:33:56.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8999999999999999e-06 2024-04-24T15:33:56.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:56.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127720+0000", 2024-04-24T15:33:56.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033447521000000001 2024-04-24T15:33:56.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:56.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127765+0000", 2024-04-24T15:33:56.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5352000000000002e-05 2024-04-24T15:33:56.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:56.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127816+0000", 2024-04-24T15:33:56.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0689999999999997e-05 2024-04-24T15:33:56.801 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:56.801 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:56.801 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:56.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.801 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63800 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1032192~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:56.801 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555407+0000", 2024-04-24T15:33:56.801 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.213613819000003, 2024-04-24T15:33:56.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032809030000000003, 2024-04-24T15:33:56.802 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:56.802 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:56.802 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:56.802 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:56.802 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:56.802 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63800 2024-04-24T15:33:56.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.802 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:56.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.802 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:56.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555407+0000", 2024-04-24T15:33:56.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:56.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:56.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555407+0000", 2024-04-24T15:33:56.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:56.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:56.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555406+0000", 2024-04-24T15:33:56.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:56.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:56.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555409+0000", 2024-04-24T15:33:56.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.706e-06 2024-04-24T15:33:56.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:56.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555416+0000", 2024-04-24T15:33:56.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1609999999999997e-06 2024-04-24T15:33:56.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:56.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555420+0000", 2024-04-24T15:33:56.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0369999999999996e-06 2024-04-24T15:33:56.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:56.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588088+0000", 2024-04-24T15:33:56.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032668228000000001 2024-04-24T15:33:56.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.805 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:56.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588143+0000", 2024-04-24T15:33:56.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4852999999999998e-05 2024-04-24T15:33:56.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.805 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:56.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588216+0000", 2024-04-24T15:33:56.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2467999999999998e-05 2024-04-24T15:33:56.805 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:56.805 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:56.805 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:56.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.805 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63801 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1114112~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:56.806 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555425+0000", 2024-04-24T15:33:56.806 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.213595912000002, 2024-04-24T15:33:56.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032924691999999998, 2024-04-24T15:33:56.806 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:56.806 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:56.806 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:56.806 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:56.806 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:56.806 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63801 2024-04-24T15:33:56.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.806 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:56.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.806 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:56.806 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555425+0000", 2024-04-24T15:33:56.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:56.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:56.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555425+0000", 2024-04-24T15:33:56.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:56.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:56.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555424+0000", 2024-04-24T15:33:56.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:56.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:56.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555427+0000", 2024-04-24T15:33:56.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2350000000000002e-06 2024-04-24T15:33:56.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:56.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555432+0000", 2024-04-24T15:33:56.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.434e-06 2024-04-24T15:33:56.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:56.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555435+0000", 2024-04-24T15:33:56.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2009999999999999e-06 2024-04-24T15:33:56.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:56.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588237+0000", 2024-04-24T15:33:56.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032801508 2024-04-24T15:33:56.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:56.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588281+0000", 2024-04-24T15:33:56.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4100999999999997e-05 2024-04-24T15:33:56.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:56.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588349+0000", 2024-04-24T15:33:56.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8551000000000001e-05 2024-04-24T15:33:56.809 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:56.810 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:56.810 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:56.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.810 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.810 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63802 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1196032~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:56.810 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555439+0000", 2024-04-24T15:33:56.810 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.213581390999998, 2024-04-24T15:33:56.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033057744999999999, 2024-04-24T15:33:56.810 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:56.810 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:56.810 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:56.810 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:56.810 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:56.810 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63802 2024-04-24T15:33:56.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.811 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:56.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:56.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555439+0000", 2024-04-24T15:33:56.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:56.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:56.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555439+0000", 2024-04-24T15:33:56.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:56.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:56.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555439+0000", 2024-04-24T15:33:56.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:56.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:56.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555445+0000", 2024-04-24T15:33:56.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4019999999999999e-06 2024-04-24T15:33:56.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:56.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555452+0000", 2024-04-24T15:33:56.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8600000000000004e-06 2024-04-24T15:33:56.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:56.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555461+0000", 2024-04-24T15:33:56.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0499999999999997e-06 2024-04-24T15:33:56.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:56.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588378+0000", 2024-04-24T15:33:56.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032916527000000001 2024-04-24T15:33:56.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:56.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588423+0000", 2024-04-24T15:33:56.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5092999999999997e-05 2024-04-24T15:33:56.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.814 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:56.814 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588497+0000", 2024-04-24T15:33:56.814 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4140999999999994e-05 2024-04-24T15:33:56.814 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:56.814 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:56.814 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:56.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.814 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63803 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1277952~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:56.814 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555466+0000", 2024-04-24T15:33:56.814 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.213554446000003, 2024-04-24T15:33:56.814 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033162297, 2024-04-24T15:33:56.814 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:56.815 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:56.815 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:56.815 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:56.815 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:56.815 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63803 2024-04-24T15:33:56.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.815 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:56.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:56.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555466+0000", 2024-04-24T15:33:56.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:56.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:56.816 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555466+0000", 2024-04-24T15:33:56.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:56.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:56.816 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555466+0000", 2024-04-24T15:33:56.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:56.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:56.816 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555468+0000", 2024-04-24T15:33:56.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4040000000000002e-06 2024-04-24T15:33:56.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:56.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555475+0000", 2024-04-24T15:33:56.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9480000000000002e-06 2024-04-24T15:33:56.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:56.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555479+0000", 2024-04-24T15:33:56.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0289999999999999e-06 2024-04-24T15:33:56.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:56.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588516+0000", 2024-04-24T15:33:56.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033037094000000003 2024-04-24T15:33:56.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.818 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:56.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588562+0000", 2024-04-24T15:33:56.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6215999999999999e-05 2024-04-24T15:33:56.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.818 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:56.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588628+0000", 2024-04-24T15:33:56.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5971999999999998e-05 2024-04-24T15:33:56.818 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:56.818 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:56.818 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:56.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.819 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63804 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1359872~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:56.819 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555483+0000", 2024-04-24T15:33:56.819 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.213537279000001, 2024-04-24T15:33:56.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033287068000000003, 2024-04-24T15:33:56.819 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:56.819 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:56.819 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:56.819 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:56.819 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:56.819 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63804 2024-04-24T15:33:56.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.819 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:56.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.819 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:56.820 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555483+0000", 2024-04-24T15:33:56.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:56.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.820 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:56.820 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555483+0000", 2024-04-24T15:33:56.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:56.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.820 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:56.820 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555483+0000", 2024-04-24T15:33:56.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:56.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:56.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555485+0000", 2024-04-24T15:33:56.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0710000000000002e-06 2024-04-24T15:33:56.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:56.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555498+0000", 2024-04-24T15:33:56.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2748999999999999e-05 2024-04-24T15:33:56.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:56.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555502+0000", 2024-04-24T15:33:56.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0219999999999998e-06 2024-04-24T15:33:56.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.822 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:56.822 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588657+0000", 2024-04-24T15:33:56.822 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033154683999999997 2024-04-24T15:33:56.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.822 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:56.822 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588702+0000", 2024-04-24T15:33:56.822 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5809000000000001e-05 2024-04-24T15:33:56.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.822 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:56.822 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588770+0000", 2024-04-24T15:33:56.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8056e-05 2024-04-24T15:33:56.823 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:56.823 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:56.823 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:56.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.823 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63805 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1441792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:56.823 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555508+0000", 2024-04-24T15:33:56.823 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.213512430000002, 2024-04-24T15:33:56.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033375952, 2024-04-24T15:33:56.823 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:56.823 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:56.823 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:56.823 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:56.824 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:56.824 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63805 2024-04-24T15:33:56.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.824 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:56.824 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.824 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:56.824 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555508+0000", 2024-04-24T15:33:56.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:56.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.824 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.824 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:56.824 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555508+0000", 2024-04-24T15:33:56.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:56.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.825 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.825 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:56.825 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555508+0000", 2024-04-24T15:33:56.825 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:56.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.825 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.825 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:56.825 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555510+0000", 2024-04-24T15:33:56.825 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.424e-06 2024-04-24T15:33:56.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.825 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.825 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:56.825 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555516+0000", 2024-04-24T15:33:56.825 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3730000000000001e-06 2024-04-24T15:33:56.826 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.826 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:56.826 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555519+0000", 2024-04-24T15:33:56.826 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.039e-06 2024-04-24T15:33:56.826 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.826 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:56.826 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588789+0000", 2024-04-24T15:33:56.826 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0332702 2024-04-24T15:33:56.826 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.826 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:56.827 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588833+0000", 2024-04-24T15:33:56.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4660999999999998e-05 2024-04-24T15:33:56.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.827 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.827 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:56.827 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588884+0000", 2024-04-24T15:33:56.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0661999999999997e-05 2024-04-24T15:33:56.827 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:56.827 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:56.827 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:56.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.827 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.827 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63806 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1523712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:56.827 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555523+0000", 2024-04-24T15:33:56.828 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.213497848999999, 2024-04-24T15:33:56.828 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033479653999999998, 2024-04-24T15:33:56.828 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:56.828 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:56.828 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:56.828 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:56.828 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:56.828 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63806 2024-04-24T15:33:56.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.828 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:56.828 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.828 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:56.828 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555523+0000", 2024-04-24T15:33:56.828 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:56.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.829 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:56.829 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555523+0000", 2024-04-24T15:33:56.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:56.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.829 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:56.829 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555522+0000", 2024-04-24T15:33:56.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:56.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.829 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:56.830 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555524+0000", 2024-04-24T15:33:56.830 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0700000000000001e-06 2024-04-24T15:33:56.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.830 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:56.830 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555530+0000", 2024-04-24T15:33:56.830 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5450000000000003e-06 2024-04-24T15:33:56.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.830 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:56.830 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555533+0000", 2024-04-24T15:33:56.830 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2200000000000001e-06 2024-04-24T15:33:56.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.831 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.831 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:56.831 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588908+0000", 2024-04-24T15:33:56.831 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033374556 2024-04-24T15:33:56.831 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.831 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.831 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:56.831 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588950+0000", 2024-04-24T15:33:56.831 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2361999999999999e-05 2024-04-24T15:33:56.831 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.831 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.831 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:56.831 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589002+0000", 2024-04-24T15:33:56.831 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2224e-05 2024-04-24T15:33:56.831 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:56.832 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:56.832 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:56.832 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.832 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.832 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63807 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1605632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:56.832 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555537+0000", 2024-04-24T15:33:56.832 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.213483334000003, 2024-04-24T15:33:56.832 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033577227000000001, 2024-04-24T15:33:56.832 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:56.832 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:56.832 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:56.832 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:56.832 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:56.832 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63807 2024-04-24T15:33:56.832 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.832 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:56.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.833 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:56.833 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555537+0000", 2024-04-24T15:33:56.833 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:56.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.833 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:56.833 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555537+0000", 2024-04-24T15:33:56.833 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:56.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.833 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:56.833 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555537+0000", 2024-04-24T15:33:56.833 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:56.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.834 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:56.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555539+0000", 2024-04-24T15:33:56.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9769999999999999e-06 2024-04-24T15:33:56.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.834 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:56.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555544+0000", 2024-04-24T15:33:56.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.468e-06 2024-04-24T15:33:56.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.834 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:56.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555548+0000", 2024-04-24T15:33:56.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4319999999999999e-06 2024-04-24T15:33:56.835 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.835 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.835 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:56.835 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589021+0000", 2024-04-24T15:33:56.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033473655999999997 2024-04-24T15:33:56.835 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.835 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.835 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:56.835 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589064+0000", 2024-04-24T15:33:56.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2397000000000001e-05 2024-04-24T15:33:56.835 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.835 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.835 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:56.836 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589114+0000", 2024-04-24T15:33:56.836 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0611e-05 2024-04-24T15:33:56.836 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:56.836 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:56.836 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:56.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.836 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.836 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63808 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1687552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:56.836 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555552+0000", 2024-04-24T15:33:56.836 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.213468642000002, 2024-04-24T15:33:56.836 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033686019999999997, 2024-04-24T15:33:56.836 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:56.836 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:56.836 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:56.837 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:56.837 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:56.837 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63808 2024-04-24T15:33:56.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.837 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:56.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.837 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:56.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555552+0000", 2024-04-24T15:33:56.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:56.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.837 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:56.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555552+0000", 2024-04-24T15:33:56.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:56.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.838 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:56.838 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555552+0000", 2024-04-24T15:33:56.838 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:56.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.838 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:56.838 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555554+0000", 2024-04-24T15:33:56.838 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2170000000000001e-06 2024-04-24T15:33:56.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.838 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:56.838 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555562+0000", 2024-04-24T15:33:56.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8779999999999998e-06 2024-04-24T15:33:56.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.839 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:56.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555565+0000", 2024-04-24T15:33:56.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4230000000000001e-06 2024-04-24T15:33:56.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.839 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:56.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589143+0000", 2024-04-24T15:33:56.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033577382000000003 2024-04-24T15:33:56.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.840 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:56.840 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589185+0000", 2024-04-24T15:33:56.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2206999999999998e-05 2024-04-24T15:33:56.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.840 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:56.840 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589238+0000", 2024-04-24T15:33:56.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3211000000000003e-05 2024-04-24T15:33:56.840 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:56.840 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:56.840 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:56.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.840 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63809 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1769472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:56.841 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555569+0000", 2024-04-24T15:33:56.841 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.213451100999997, 2024-04-24T15:33:56.841 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.03378068, 2024-04-24T15:33:56.841 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:56.841 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:56.841 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:56.841 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:56.841 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:56.841 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63809 2024-04-24T15:33:56.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.841 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:56.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.841 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:56.841 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555569+0000", 2024-04-24T15:33:56.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:56.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.842 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.842 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:56.842 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555569+0000", 2024-04-24T15:33:56.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:56.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.842 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.842 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:56.842 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555569+0000", 2024-04-24T15:33:56.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:56.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.842 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.843 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:56.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555571+0000", 2024-04-24T15:33:56.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0889999999999998e-06 2024-04-24T15:33:56.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.843 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:56.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555577+0000", 2024-04-24T15:33:56.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4149999999999998e-06 2024-04-24T15:33:56.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.843 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:56.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555580+0000", 2024-04-24T15:33:56.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2399999999999999e-06 2024-04-24T15:33:56.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.844 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:56.844 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589258+0000", 2024-04-24T15:33:56.844 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033677817999999998 2024-04-24T15:33:56.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.844 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:56.844 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589300+0000", 2024-04-24T15:33:56.844 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2317999999999997e-05 2024-04-24T15:33:56.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.844 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:56.844 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589350+0000", 2024-04-24T15:33:56.845 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0111000000000002e-05 2024-04-24T15:33:56.845 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:56.845 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:56.845 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:56.845 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.845 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.845 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63810 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1851392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:56.845 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555584+0000", 2024-04-24T15:33:56.845 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.213436631, 2024-04-24T15:33:56.845 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033886251999999999, 2024-04-24T15:33:56.845 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:56.845 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:56.845 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:56.846 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:56.846 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:56.846 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63810 2024-04-24T15:33:56.846 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.846 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:56.846 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.846 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:56.846 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555584+0000", 2024-04-24T15:33:56.846 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:56.846 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.846 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.846 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:56.846 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555584+0000", 2024-04-24T15:33:56.846 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:56.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.847 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.847 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:56.847 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555584+0000", 2024-04-24T15:33:56.847 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:56.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.847 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.847 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:56.847 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555586+0000", 2024-04-24T15:33:56.847 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0899999999999999e-06 2024-04-24T15:33:56.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.847 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.847 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:56.847 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555591+0000", 2024-04-24T15:33:56.848 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5260000000000001e-06 2024-04-24T15:33:56.848 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.848 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.848 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:56.848 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555595+0000", 2024-04-24T15:33:56.848 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.41e-06 2024-04-24T15:33:56.848 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.848 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.848 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:56.848 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589377+0000", 2024-04-24T15:33:56.848 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033781868 2024-04-24T15:33:56.848 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.848 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.848 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:56.848 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589418+0000", 2024-04-24T15:33:56.849 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1461000000000002e-05 2024-04-24T15:33:56.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.849 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.849 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:56.849 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589470+0000", 2024-04-24T15:33:56.849 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2194999999999997e-05 2024-04-24T15:33:56.849 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:56.849 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:56.849 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:56.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.849 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.849 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63811 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1933312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:56.849 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555599+0000", 2024-04-24T15:33:56.849 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.213421822000001, 2024-04-24T15:33:56.850 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033987493000000001, 2024-04-24T15:33:56.850 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:56.850 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:56.850 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:56.850 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:56.850 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:56.850 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63811 2024-04-24T15:33:56.850 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.850 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:56.850 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.850 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:56.850 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555599+0000", 2024-04-24T15:33:56.850 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:56.850 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.851 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.851 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:56.851 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555599+0000", 2024-04-24T15:33:56.851 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:56.851 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.851 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.851 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:56.851 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555598+0000", 2024-04-24T15:33:56.851 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:56.851 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.851 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.851 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:56.851 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555600+0000", 2024-04-24T15:33:56.851 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.04e-06 2024-04-24T15:33:56.852 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.852 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.852 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:56.852 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555606+0000", 2024-04-24T15:33:56.852 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4199999999999998e-06 2024-04-24T15:33:56.852 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.852 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.852 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:56.852 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555616+0000", 2024-04-24T15:33:56.852 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0054000000000001e-05 2024-04-24T15:33:56.852 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.852 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.852 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:56.852 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589493+0000", 2024-04-24T15:33:56.852 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033877009999999999 2024-04-24T15:33:56.853 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.853 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.853 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:56.853 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589536+0000", 2024-04-24T15:33:56.853 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2855999999999998e-05 2024-04-24T15:33:56.853 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.853 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.853 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:56.853 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589586+0000", 2024-04-24T15:33:56.853 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0420999999999997e-05 2024-04-24T15:33:56.853 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:56.853 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:56.853 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:56.853 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.854 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.854 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63812 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 2015232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:33:56.854 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555620+0000", 2024-04-24T15:33:56.854 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.213400184000001, 2024-04-24T15:33:56.854 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034081696000000002, 2024-04-24T15:33:56.854 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:56.854 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:56.854 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:56.854 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:56.854 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:56.854 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63812 2024-04-24T15:33:56.854 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.854 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:56.854 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.855 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:56.855 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555620+0000", 2024-04-24T15:33:56.855 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:56.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.855 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.855 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:56.855 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555620+0000", 2024-04-24T15:33:56.855 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:56.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.855 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.855 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:56.855 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555620+0000", 2024-04-24T15:33:56.855 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:56.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.856 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.856 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:56.856 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555625+0000", 2024-04-24T15:33:56.856 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2859999999999999e-06 2024-04-24T15:33:56.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.856 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.856 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:56.856 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555633+0000", 2024-04-24T15:33:56.856 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8879999999999997e-06 2024-04-24T15:33:56.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.856 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.856 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:56.856 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555637+0000", 2024-04-24T15:33:56.856 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9500000000000003e-06 2024-04-24T15:33:56.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.857 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.857 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:56.857 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589604+0000", 2024-04-24T15:33:56.857 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033966478000000001 2024-04-24T15:33:56.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.857 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.857 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:56.857 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589650+0000", 2024-04-24T15:33:56.857 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6721000000000002e-05 2024-04-24T15:33:56.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:56.857 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:56.857 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:56.857 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589702+0000", 2024-04-24T15:33:56.857 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1709000000000002e-05 2024-04-24T15:33:56.858 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:56.858 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:56.858 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:56.858 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:56.858 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:56.858 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:56.859 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-24T15:33:56.860 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:56.860+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:56.960 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:33:56.961+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:33:57.023 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:57.024 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:33:57.024 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-04-24T15:33:57.024 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:57.033 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-24T15:33:57.075 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:57.076+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:57.161 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:33:57.163+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:33:57.197 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:57.197 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:33:57.197 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-04-24T15:33:57.197 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-04-24T15:33:57.197 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:57.206 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-24T15:33:57.262 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:57.264+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:57.363 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:57.365+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:57.371 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:33:57.371 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-04-24T15:33:57.371 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-04-24T15:33:57.371 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-04-24T15:33:57.371 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.371 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14043 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2383872~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:57.371 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892347+0000", 2024-04-24T15:33:57.371 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 83.478670937000004, 2024-04-24T15:33:57.371 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038537279000000001, 2024-04-24T15:33:57.371 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:57.371 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:57.371 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:57.371 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:57.372 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:57.372 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14043 2024-04-24T15:33:57.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.372 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:57.372 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.372 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:57.372 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892347+0000", 2024-04-24T15:33:57.372 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:57.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.372 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.372 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:57.372 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892347+0000", 2024-04-24T15:33:57.372 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:57.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.372 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.373 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:57.373 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892347+0000", 2024-04-24T15:33:57.373 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:57.373 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.373 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.373 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:57.373 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892349+0000", 2024-04-24T15:33:57.373 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3400000000000001e-06 2024-04-24T15:33:57.373 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.373 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.373 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:57.373 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892357+0000", 2024-04-24T15:33:57.373 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1540000000000002e-06 2024-04-24T15:33:57.373 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.374 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.374 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:57.374 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892360+0000", 2024-04-24T15:33:57.374 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5829999999999998e-06 2024-04-24T15:33:57.374 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.374 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.374 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:57.374 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930795+0000", 2024-04-24T15:33:57.374 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038434590999999997 2024-04-24T15:33:57.374 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.374 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.374 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:57.374 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930836+0000", 2024-04-24T15:33:57.375 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0827000000000001e-05 2024-04-24T15:33:57.375 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.375 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.375 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:57.375 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930885+0000", 2024-04-24T15:33:57.375 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9029000000000001e-05 2024-04-24T15:33:57.375 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:57.375 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:57.375 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:57.375 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.375 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.375 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14044 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2424832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:57.375 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892372+0000", 2024-04-24T15:33:57.376 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 83.478646292999997, 2024-04-24T15:33:57.376 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038624890000000002, 2024-04-24T15:33:57.376 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:57.376 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:57.376 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:57.376 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:57.376 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:57.376 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14044 2024-04-24T15:33:57.376 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.376 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:57.376 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.376 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:57.376 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892372+0000", 2024-04-24T15:33:57.376 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:57.376 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.377 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.377 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:57.377 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892372+0000", 2024-04-24T15:33:57.377 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:57.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.377 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.377 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:57.377 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892372+0000", 2024-04-24T15:33:57.377 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:57.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.377 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.377 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:57.377 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892374+0000", 2024-04-24T15:33:57.377 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4899999999999999e-06 2024-04-24T15:33:57.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.378 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.378 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:57.378 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892380+0000", 2024-04-24T15:33:57.378 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.631e-06 2024-04-24T15:33:57.378 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.378 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.378 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:57.378 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892383+0000", 2024-04-24T15:33:57.378 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7400000000000002e-06 2024-04-24T15:33:57.378 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.378 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.378 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:57.378 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930905+0000", 2024-04-24T15:33:57.379 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038521396999999999 2024-04-24T15:33:57.379 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.379 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.379 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:57.379 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930949+0000", 2024-04-24T15:33:57.379 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4153999999999998e-05 2024-04-24T15:33:57.379 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.379 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.379 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:57.379 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930997+0000", 2024-04-24T15:33:57.379 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7998999999999999e-05 2024-04-24T15:33:57.379 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:57.379 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:57.379 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:57.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.380 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.380 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14045 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2465792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:57.380 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892387+0000", 2024-04-24T15:33:57.380 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 83.478631026000002, 2024-04-24T15:33:57.380 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038720933999999999, 2024-04-24T15:33:57.380 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:57.380 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:57.380 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:57.380 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:57.380 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:57.380 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14045 2024-04-24T15:33:57.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.380 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:57.380 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.381 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:57.381 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892387+0000", 2024-04-24T15:33:57.381 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:57.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.381 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:57.381 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892387+0000", 2024-04-24T15:33:57.381 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:57.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.381 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:57.381 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892387+0000", 2024-04-24T15:33:57.381 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:57.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.382 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:57.382 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892392+0000", 2024-04-24T15:33:57.382 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0039999999999999e-06 2024-04-24T15:33:57.382 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.382 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.382 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:57.382 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892398+0000", 2024-04-24T15:33:57.382 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0900000000000001e-06 2024-04-24T15:33:57.382 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.382 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.382 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:57.382 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892402+0000", 2024-04-24T15:33:57.382 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.399e-06 2024-04-24T15:33:57.382 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.383 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.383 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:57.383 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931018+0000", 2024-04-24T15:33:57.383 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038616551999999998 2024-04-24T15:33:57.383 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.383 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.383 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:57.383 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931059+0000", 2024-04-24T15:33:57.383 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1205000000000003e-05 2024-04-24T15:33:57.383 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.383 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.383 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:57.383 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931108+0000", 2024-04-24T15:33:57.383 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8996000000000003e-05 2024-04-24T15:33:57.384 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:57.384 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:57.384 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:57.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.384 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.384 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14046 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2506752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:57.384 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892406+0000", 2024-04-24T15:33:57.384 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 83.478612436000006, 2024-04-24T15:33:57.384 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038814507999999998, 2024-04-24T15:33:57.384 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:57.384 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:57.384 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:57.384 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:57.384 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:57.384 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14046 2024-04-24T15:33:57.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.385 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:57.385 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.385 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:57.385 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892406+0000", 2024-04-24T15:33:57.385 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:57.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.385 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.385 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:57.385 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892406+0000", 2024-04-24T15:33:57.385 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:57.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.385 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.385 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:57.385 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892406+0000", 2024-04-24T15:33:57.385 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:57.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.386 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.386 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:57.386 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892413+0000", 2024-04-24T15:33:57.386 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7449999999999995e-06 2024-04-24T15:33:57.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.386 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.386 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:57.386 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892419+0000", 2024-04-24T15:33:57.386 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5790000000000003e-06 2024-04-24T15:33:57.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.386 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.386 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:57.387 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892423+0000", 2024-04-24T15:33:57.387 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8199999999999998e-06 2024-04-24T15:33:57.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.387 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.387 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:57.387 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931128+0000", 2024-04-24T15:33:57.387 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038705502000000003 2024-04-24T15:33:57.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.387 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.387 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:57.387 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931172+0000", 2024-04-24T15:33:57.387 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4066999999999997e-05 2024-04-24T15:33:57.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.387 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.388 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:57.388 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931220+0000", 2024-04-24T15:33:57.388 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8167999999999997e-05 2024-04-24T15:33:57.388 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:57.388 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:57.388 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:57.388 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.388 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.388 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14047 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2547712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:57.388 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892427+0000", 2024-04-24T15:33:57.388 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 83.478591206999994, 2024-04-24T15:33:57.388 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038904506999999998, 2024-04-24T15:33:57.388 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:57.388 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:57.388 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:57.389 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:57.389 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:57.389 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14047 2024-04-24T15:33:57.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.389 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:57.389 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.389 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:57.389 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892427+0000", 2024-04-24T15:33:57.389 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:57.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.389 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.389 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:57.389 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892427+0000", 2024-04-24T15:33:57.389 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:57.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.389 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.390 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:57.390 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892427+0000", 2024-04-24T15:33:57.390 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:57.390 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.390 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.390 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:57.390 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892429+0000", 2024-04-24T15:33:57.390 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3099999999999999e-06 2024-04-24T15:33:57.390 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.390 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.390 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:57.390 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892435+0000", 2024-04-24T15:33:57.390 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5520000000000004e-06 2024-04-24T15:33:57.390 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.391 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.391 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:57.391 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892438+0000", 2024-04-24T15:33:57.391 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4240000000000002e-06 2024-04-24T15:33:57.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.391 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.391 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:57.391 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931242+0000", 2024-04-24T15:33:57.391 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038804162000000003 2024-04-24T15:33:57.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.391 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.391 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:57.391 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931284+0000", 2024-04-24T15:33:57.391 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1414e-05 2024-04-24T15:33:57.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.392 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.392 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:57.392 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931332+0000", 2024-04-24T15:33:57.392 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8024e-05 2024-04-24T15:33:57.392 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:57.392 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:57.392 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:57.392 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.392 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.392 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14048 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2588672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:57.392 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892442+0000", 2024-04-24T15:33:57.392 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 83.478576243999996, 2024-04-24T15:33:57.392 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039003132000000003, 2024-04-24T15:33:57.392 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:57.392 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:57.393 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:57.393 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:57.393 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:57.393 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14048 2024-04-24T15:33:57.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.393 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:57.393 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.393 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:57.393 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892442+0000", 2024-04-24T15:33:57.393 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:57.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.393 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.393 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:57.393 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892442+0000", 2024-04-24T15:33:57.394 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:57.394 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.394 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.394 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:57.394 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892442+0000", 2024-04-24T15:33:57.394 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:57.394 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.394 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.394 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:57.394 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892444+0000", 2024-04-24T15:33:57.394 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1169999999999998e-06 2024-04-24T15:33:57.394 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.394 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.395 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:57.395 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892449+0000", 2024-04-24T15:33:57.395 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1599999999999997e-06 2024-04-24T15:33:57.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.395 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.395 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:57.395 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892453+0000", 2024-04-24T15:33:57.395 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5930000000000002e-06 2024-04-24T15:33:57.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.395 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.395 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:57.395 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931352+0000", 2024-04-24T15:33:57.395 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038899361 2024-04-24T15:33:57.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.395 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.395 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:57.396 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931398+0000", 2024-04-24T15:33:57.396 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6315e-05 2024-04-24T15:33:57.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.396 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.396 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:57.396 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931445+0000", 2024-04-24T15:33:57.396 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6950000000000003e-05 2024-04-24T15:33:57.396 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:57.396 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:57.396 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:57.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.396 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.396 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14049 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2629632~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:57.396 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892458+0000", 2024-04-24T15:33:57.396 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 83.478560823999999, 2024-04-24T15:33:57.397 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039103322000000003, 2024-04-24T15:33:57.397 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:57.397 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:57.397 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:57.397 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:57.397 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:57.397 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14049 2024-04-24T15:33:57.397 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.397 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:57.397 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.397 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:57.397 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892458+0000", 2024-04-24T15:33:57.397 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:57.397 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.397 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.398 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:57.398 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892458+0000", 2024-04-24T15:33:57.398 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:57.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.398 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.398 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:57.398 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892457+0000", 2024-04-24T15:33:57.398 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:57.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.398 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.398 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:57.398 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892459+0000", 2024-04-24T15:33:57.398 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.305e-06 2024-04-24T15:33:57.399 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.399 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.399 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:57.399 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892465+0000", 2024-04-24T15:33:57.399 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4360000000000001e-06 2024-04-24T15:33:57.399 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.399 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.399 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:57.399 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892474+0000", 2024-04-24T15:33:57.399 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7639999999999994e-06 2024-04-24T15:33:57.399 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.399 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.399 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:57.399 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931466+0000", 2024-04-24T15:33:57.400 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038992644 2024-04-24T15:33:57.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.400 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.400 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:57.400 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931512+0000", 2024-04-24T15:33:57.400 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5200000000000001e-05 2024-04-24T15:33:57.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.400 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.400 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:57.400 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931561+0000", 2024-04-24T15:33:57.400 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9357999999999997e-05 2024-04-24T15:33:57.400 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:57.400 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:57.400 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:57.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.401 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.401 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14050 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2670592~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:57.401 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892478+0000", 2024-04-24T15:33:57.401 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 83.478540237000004, 2024-04-24T15:33:57.401 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039195102000000002, 2024-04-24T15:33:57.401 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:57.401 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:57.401 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:57.401 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:57.401 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:57.401 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14050 2024-04-24T15:33:57.401 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.401 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:57.401 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.402 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:57.402 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892478+0000", 2024-04-24T15:33:57.402 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:57.402 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.402 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.402 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:57.402 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892478+0000", 2024-04-24T15:33:57.402 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:57.402 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.402 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.402 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:57.402 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892478+0000", 2024-04-24T15:33:57.402 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:57.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.403 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.403 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:57.403 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892480+0000", 2024-04-24T15:33:57.403 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3300000000000001e-06 2024-04-24T15:33:57.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.403 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.403 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:57.403 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892486+0000", 2024-04-24T15:33:57.403 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9200000000000001e-06 2024-04-24T15:33:57.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.403 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.403 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:57.403 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892494+0000", 2024-04-24T15:33:57.404 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5159999999999997e-06 2024-04-24T15:33:57.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.404 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.404 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:57.404 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931581+0000", 2024-04-24T15:33:57.404 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039087894999999998 2024-04-24T15:33:57.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.404 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.404 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:57.404 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931626+0000", 2024-04-24T15:33:57.404 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4943e-05 2024-04-24T15:33:57.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.404 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.404 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:57.404 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931673+0000", 2024-04-24T15:33:57.405 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6906000000000001e-05 2024-04-24T15:33:57.405 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:57.405 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:57.405 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:57.405 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.405 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.405 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14051 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2711552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:57.405 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892507+0000", 2024-04-24T15:33:57.405 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 83.478511538000006, 2024-04-24T15:33:57.405 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039274393999999997, 2024-04-24T15:33:57.405 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:57.405 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:57.405 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:57.405 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:57.406 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:57.406 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14051 2024-04-24T15:33:57.406 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.406 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:57.406 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.406 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:57.406 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892507+0000", 2024-04-24T15:33:57.406 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:57.406 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.406 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.406 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:57.406 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892507+0000", 2024-04-24T15:33:57.406 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:57.406 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.407 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.407 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:57.407 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892506+0000", 2024-04-24T15:33:57.407 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:57.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.407 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.407 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:57.407 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892509+0000", 2024-04-24T15:33:57.407 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8760000000000001e-06 2024-04-24T15:33:57.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.407 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.407 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:57.407 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892515+0000", 2024-04-24T15:33:57.407 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1840000000000004e-06 2024-04-24T15:33:57.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.408 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.408 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:57.408 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892519+0000", 2024-04-24T15:33:57.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7570000000000002e-06 2024-04-24T15:33:57.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.408 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.408 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:57.408 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931694+0000", 2024-04-24T15:33:57.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039174960000000002 2024-04-24T15:33:57.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.408 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.408 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:57.408 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931734+0000", 2024-04-24T15:33:57.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0284000000000003e-05 2024-04-24T15:33:57.409 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.409 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.409 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:57.409 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931781+0000", 2024-04-24T15:33:57.409 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6813000000000001e-05 2024-04-24T15:33:57.409 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:57.409 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:57.409 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:57.409 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.409 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.409 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14052 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2752512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:57.409 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892524+0000", 2024-04-24T15:33:57.409 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 83.478494390999998, 2024-04-24T15:33:57.409 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039371122000000001, 2024-04-24T15:33:57.410 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:57.410 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:57.410 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:57.410 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:57.410 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:57.410 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14052 2024-04-24T15:33:57.410 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.410 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:57.410 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.410 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:57.410 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892524+0000", 2024-04-24T15:33:57.410 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:57.410 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.410 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:57.411 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892524+0000", 2024-04-24T15:33:57.411 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:57.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:57.411 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892524+0000", 2024-04-24T15:33:57.411 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:57.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:57.411 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892526+0000", 2024-04-24T15:33:57.411 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3630000000000001e-06 2024-04-24T15:33:57.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:57.412 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892538+0000", 2024-04-24T15:33:57.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1677e-05 2024-04-24T15:33:57.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:57.412 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892541+0000", 2024-04-24T15:33:57.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.822e-06 2024-04-24T15:33:57.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:57.412 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931802+0000", 2024-04-24T15:33:57.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039260911000000003 2024-04-24T15:33:57.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.413 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.413 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:57.413 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931847+0000", 2024-04-24T15:33:57.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4450999999999998e-05 2024-04-24T15:33:57.413 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.413 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.413 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:57.413 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931895+0000", 2024-04-24T15:33:57.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8257999999999998e-05 2024-04-24T15:33:57.413 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:57.413 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:57.413 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:57.413 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.413 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.414 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14053 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2793472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:57.414 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892546+0000", 2024-04-24T15:33:57.414 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 83.478472413000006, 2024-04-24T15:33:57.414 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039457712999999998, 2024-04-24T15:33:57.414 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:57.414 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:57.414 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:57.414 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:57.414 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:57.414 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14053 2024-04-24T15:33:57.414 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.414 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:57.414 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.414 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:57.414 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892546+0000", 2024-04-24T15:33:57.415 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:57.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.415 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.415 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:57.415 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892546+0000", 2024-04-24T15:33:57.415 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:57.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.415 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.415 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:57.415 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892546+0000", 2024-04-24T15:33:57.415 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:57.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.415 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.416 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:57.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892548+0000", 2024-04-24T15:33:57.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3120000000000001e-06 2024-04-24T15:33:57.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.416 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.416 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:57.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892553+0000", 2024-04-24T15:33:57.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2320000000000001e-06 2024-04-24T15:33:57.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.416 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.416 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:57.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892559+0000", 2024-04-24T15:33:57.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5480000000000001e-06 2024-04-24T15:33:57.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.416 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.417 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:57.417 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931916+0000", 2024-04-24T15:33:57.417 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039357180999999998 2024-04-24T15:33:57.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.417 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.417 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:57.417 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931956+0000", 2024-04-24T15:33:57.417 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0565000000000002e-05 2024-04-24T15:33:57.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.417 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.417 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:57.417 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932004+0000", 2024-04-24T15:33:57.417 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7224000000000001e-05 2024-04-24T15:33:57.417 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:57.417 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:57.418 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:57.418 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.418 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.418 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14054 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2834432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:57.418 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892563+0000", 2024-04-24T15:33:57.418 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 83.478455761000006, 2024-04-24T15:33:57.418 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039552406999999998, 2024-04-24T15:33:57.418 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:57.418 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:57.418 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:57.418 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:57.418 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:57.418 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14054 2024-04-24T15:33:57.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.419 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:57.419 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.419 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:57.419 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892563+0000", 2024-04-24T15:33:57.419 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:57.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.419 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.419 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:57.419 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892563+0000", 2024-04-24T15:33:57.419 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:57.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.419 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.419 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:57.420 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892562+0000", 2024-04-24T15:33:57.420 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:57.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.420 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:57.420 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892564+0000", 2024-04-24T15:33:57.420 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2029999999999999e-06 2024-04-24T15:33:57.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.420 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:57.420 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892570+0000", 2024-04-24T15:33:57.420 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1460000000000004e-06 2024-04-24T15:33:57.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.421 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:57.421 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892573+0000", 2024-04-24T15:33:57.421 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2030000000000001e-06 2024-04-24T15:33:57.421 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.421 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.421 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:57.421 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932024+0000", 2024-04-24T15:33:57.421 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039451696000000001 2024-04-24T15:33:57.421 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.421 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.421 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:57.421 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932068+0000", 2024-04-24T15:33:57.421 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3970000000000001e-05 2024-04-24T15:33:57.421 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.421 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.422 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:57.422 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932115+0000", 2024-04-24T15:33:57.422 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6569000000000001e-05 2024-04-24T15:33:57.422 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:57.422 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:57.422 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:57.422 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.422 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.422 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14055 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2875392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:57.422 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892591+0000", 2024-04-24T15:33:57.422 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 83.478427788000005, 2024-04-24T15:33:57.422 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039634309, 2024-04-24T15:33:57.422 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:57.422 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:57.422 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:57.423 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:57.423 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:57.423 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14055 2024-04-24T15:33:57.423 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.423 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:57.423 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.423 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:57.423 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892591+0000", 2024-04-24T15:33:57.423 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:57.423 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.423 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.423 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:57.423 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892591+0000", 2024-04-24T15:33:57.423 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:57.423 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.424 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.424 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:57.424 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892590+0000", 2024-04-24T15:33:57.424 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:57.424 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.424 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.424 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:57.424 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892592+0000", 2024-04-24T15:33:57.424 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7519999999999999e-06 2024-04-24T15:33:57.424 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.424 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.424 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:57.424 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892596+0000", 2024-04-24T15:33:57.424 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.608e-06 2024-04-24T15:33:57.424 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.425 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.425 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:57.425 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892598+0000", 2024-04-24T15:33:57.425 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.531e-06 2024-04-24T15:33:57.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.425 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.425 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:57.425 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932136+0000", 2024-04-24T15:33:57.425 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039537668999999998 2024-04-24T15:33:57.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.425 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.425 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:57.425 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932177+0000", 2024-04-24T15:33:57.425 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0880000000000002e-05 2024-04-24T15:33:57.426 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.426 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.426 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:57.426 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932225+0000", 2024-04-24T15:33:57.426 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8223999999999998e-05 2024-04-24T15:33:57.426 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:57.426 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:57.426 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:57.426 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.426 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.426 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14056 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2916352~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:57.426 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892600+0000", 2024-04-24T15:33:57.426 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 83.478417953999994, 2024-04-24T15:33:57.427 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039736447000000001, 2024-04-24T15:33:57.427 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:57.427 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:57.427 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:57.427 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:57.427 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:57.427 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14056 2024-04-24T15:33:57.427 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.427 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:57.427 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.427 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:57.427 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892600+0000", 2024-04-24T15:33:57.427 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:57.427 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.428 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.428 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:57.428 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892600+0000", 2024-04-24T15:33:57.428 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:57.428 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.428 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.428 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:57.428 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892600+0000", 2024-04-24T15:33:57.428 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967296 2024-04-24T15:33:57.428 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.428 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.428 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:57.428 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892602+0000", 2024-04-24T15:33:57.428 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.576e-06 2024-04-24T15:33:57.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.429 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:57.429 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892605+0000", 2024-04-24T15:33:57.429 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.574e-06 2024-04-24T15:33:57.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.429 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:57.429 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892608+0000", 2024-04-24T15:33:57.429 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3800000000000001e-06 2024-04-24T15:33:57.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.429 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:57.429 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932245+0000", 2024-04-24T15:33:57.429 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039637378000000001 2024-04-24T15:33:57.430 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.430 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.430 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:57.430 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932290+0000", 2024-04-24T15:33:57.430 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4734999999999998e-05 2024-04-24T15:33:57.430 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.430 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.430 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:57.430 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932337+0000", 2024-04-24T15:33:57.430 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7033000000000002e-05 2024-04-24T15:33:57.430 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:57.430 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:57.430 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:57.430 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.431 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.431 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14063 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 3203072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:57.431 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.894564+0000", 2024-04-24T15:33:57.431 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 83.476454363000002, 2024-04-24T15:33:57.431 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038554995000000002, 2024-04-24T15:33:57.431 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:57.431 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:57.431 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:57.431 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:57.431 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:57.431 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14063 2024-04-24T15:33:57.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.431 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:57.431 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.431 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:57.432 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894564+0000", 2024-04-24T15:33:57.432 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:57.432 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.432 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.432 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:57.432 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894564+0000", 2024-04-24T15:33:57.432 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:57.432 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.432 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.432 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:57.432 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894564+0000", 2024-04-24T15:33:57.432 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967296 2024-04-24T15:33:57.432 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.432 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.432 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:57.433 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894565+0000", 2024-04-24T15:33:57.433 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2640000000000001e-06 2024-04-24T15:33:57.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.433 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.433 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:57.433 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894569+0000", 2024-04-24T15:33:57.433 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1710000000000004e-06 2024-04-24T15:33:57.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.433 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.433 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:57.433 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894573+0000", 2024-04-24T15:33:57.433 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4410000000000002e-06 2024-04-24T15:33:57.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.433 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.433 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:57.434 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933030+0000", 2024-04-24T15:33:57.434 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038457709999999999 2024-04-24T15:33:57.434 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.434 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.434 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:57.434 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933072+0000", 2024-04-24T15:33:57.434 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.193e-05 2024-04-24T15:33:57.434 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.434 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.434 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:57.434 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933119+0000", 2024-04-24T15:33:57.434 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6715000000000003e-05 2024-04-24T15:33:57.434 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:57.434 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:57.435 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:57.435 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.435 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.435 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14064 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 3244032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:57.435 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.894591+0000", 2024-04-24T15:33:57.435 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 83.476427720999993, 2024-04-24T15:33:57.435 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038639806999999998, 2024-04-24T15:33:57.435 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:57.435 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:57.435 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:57.435 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:57.435 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:57.435 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14064 2024-04-24T15:33:57.435 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.435 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:57.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.436 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:57.436 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894591+0000", 2024-04-24T15:33:57.436 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:57.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.436 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:57.436 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894591+0000", 2024-04-24T15:33:57.436 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:57.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.436 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:57.436 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894590+0000", 2024-04-24T15:33:57.436 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:57.437 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.437 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.437 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:57.437 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894593+0000", 2024-04-24T15:33:57.437 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4719999999999998e-06 2024-04-24T15:33:57.437 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.437 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.437 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:57.437 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894598+0000", 2024-04-24T15:33:57.437 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.57e-06 2024-04-24T15:33:57.437 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.437 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.437 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:57.437 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894602+0000", 2024-04-24T15:33:57.437 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.614e-06 2024-04-24T15:33:57.438 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.438 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.438 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:57.438 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933139+0000", 2024-04-24T15:33:57.438 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038536929999999997 2024-04-24T15:33:57.438 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.438 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.438 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:57.438 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933183+0000", 2024-04-24T15:33:57.438 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4217e-05 2024-04-24T15:33:57.438 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.438 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.438 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:57.439 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933230+0000", 2024-04-24T15:33:57.439 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7434e-05 2024-04-24T15:33:57.439 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:57.439 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:57.439 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:57.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.439 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14065 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 3284992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:57.439 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.894606+0000", 2024-04-24T15:33:57.439 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 83.476412339000007, 2024-04-24T15:33:57.439 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038734126000000001, 2024-04-24T15:33:57.439 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:57.439 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:57.439 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:57.440 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:57.440 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:57.440 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14065 2024-04-24T15:33:57.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.440 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:57.440 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.440 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:57.440 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894606+0000", 2024-04-24T15:33:57.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:57.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.440 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.440 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:57.440 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894606+0000", 2024-04-24T15:33:57.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:57.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.441 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:57.441 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894606+0000", 2024-04-24T15:33:57.441 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:57.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.441 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:57.441 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894608+0000", 2024-04-24T15:33:57.441 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0880000000000002e-06 2024-04-24T15:33:57.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.441 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:57.441 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894614+0000", 2024-04-24T15:33:57.441 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7830000000000004e-06 2024-04-24T15:33:57.442 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.442 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.442 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:57.442 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894621+0000", 2024-04-24T15:33:57.442 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5809999999999999e-06 2024-04-24T15:33:57.442 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.442 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.442 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:57.442 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933251+0000", 2024-04-24T15:33:57.442 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038629995 2024-04-24T15:33:57.442 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.442 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.442 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:57.443 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933292+0000", 2024-04-24T15:33:57.443 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0380000000000003e-05 2024-04-24T15:33:57.443 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.443 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.443 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:57.443 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933340+0000", 2024-04-24T15:33:57.443 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8662999999999998e-05 2024-04-24T15:33:57.443 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:57.443 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:57.443 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:57.443 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.443 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.443 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14066 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 3325952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:57.443 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.894625+0000", 2024-04-24T15:33:57.443 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 83.476392899999993, 2024-04-24T15:33:57.444 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038825539999999999, 2024-04-24T15:33:57.444 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:57.444 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:57.444 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:57.444 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:57.444 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:57.444 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14066 2024-04-24T15:33:57.444 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.444 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:57.444 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.444 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:57.444 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894625+0000", 2024-04-24T15:33:57.444 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:57.444 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.444 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.445 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:57.445 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894625+0000", 2024-04-24T15:33:57.445 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:57.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.445 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.445 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:57.445 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894625+0000", 2024-04-24T15:33:57.445 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:57.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.445 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.445 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:57.445 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894627+0000", 2024-04-24T15:33:57.445 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.092e-06 2024-04-24T15:33:57.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.446 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.446 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:57.446 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894635+0000", 2024-04-24T15:33:57.446 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6660000000000008e-06 2024-04-24T15:33:57.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.446 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.446 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:57.446 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894639+0000", 2024-04-24T15:33:57.446 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.799e-06 2024-04-24T15:33:57.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.446 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.447 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:57.447 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933360+0000", 2024-04-24T15:33:57.447 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038721569999999997 2024-04-24T15:33:57.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.447 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:57.447 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933403+0000", 2024-04-24T15:33:57.447 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2605999999999998e-05 2024-04-24T15:33:57.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.447 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:57.447 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933451+0000", 2024-04-24T15:33:57.447 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8149999999999998e-05 2024-04-24T15:33:57.447 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:57.447 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:57.448 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:57.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.448 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.448 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14067 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 3366912~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:57.448 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.894644+0000", 2024-04-24T15:33:57.448 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 83.476374841999998, 2024-04-24T15:33:57.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038926068000000001, 2024-04-24T15:33:57.448 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:57.448 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:57.448 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:57.448 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:57.448 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:57.448 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14067 2024-04-24T15:33:57.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.448 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:57.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.449 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:57.449 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894644+0000", 2024-04-24T15:33:57.449 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:57.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.449 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:57.449 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894644+0000", 2024-04-24T15:33:57.449 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:57.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.449 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:57.449 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894643+0000", 2024-04-24T15:33:57.449 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:57.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.450 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.450 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:57.450 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894646+0000", 2024-04-24T15:33:57.450 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3379999999999999e-06 2024-04-24T15:33:57.450 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.450 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.450 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:57.450 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894651+0000", 2024-04-24T15:33:57.450 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.485e-06 2024-04-24T15:33:57.450 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.450 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.450 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:57.450 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894655+0000", 2024-04-24T15:33:57.450 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.709e-06 2024-04-24T15:33:57.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.451 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.451 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:57.451 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933472+0000", 2024-04-24T15:33:57.451 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038816992000000002 2024-04-24T15:33:57.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.451 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.451 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:57.451 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933521+0000", 2024-04-24T15:33:57.451 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8841999999999998e-05 2024-04-24T15:33:57.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.451 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.451 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:57.451 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933570+0000", 2024-04-24T15:33:57.451 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9063000000000001e-05 2024-04-24T15:33:57.451 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:57.452 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:57.452 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:57.452 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.452 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.452 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:30240 4.0 4:059b194c:::rbd_object_map.114a2e5eebc:head [read 0~43] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-04-24T15:33:57.452 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:47.244473+0000", 2024-04-24T15:33:57.452 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 70.126545656000005, 2024-04-24T15:33:57.452 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4146627299999999, 2024-04-24T15:33:57.452 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:33:57.452 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:33:57.452 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:33:57.452 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:33:57.452 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:57.452 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30240 2024-04-24T15:33:57.452 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.453 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:33:57.453 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.453 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:33:57.453 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.244473+0000", 2024-04-24T15:33:57.453 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:57.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.453 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.453 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:33:57.453 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.244473+0000", 2024-04-24T15:33:57.453 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:33:57.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.453 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.453 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:33:57.453 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.244471+0000", 2024-04-24T15:33:57.453 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999986 2024-04-24T15:33:57.454 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.454 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.454 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:33:57.454 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.244477+0000", 2024-04-24T15:33:57.454 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7819999999999999e-06 2024-04-24T15:33:57.454 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.454 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.454 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:33:57.454 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.244487+0000", 2024-04-24T15:33:57.454 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0247e-05 2024-04-24T15:33:57.454 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.454 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.454 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:33:57.454 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.244499+0000", 2024-04-24T15:33:57.454 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1324e-05 2024-04-24T15:33:57.455 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.455 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.455 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:57.455 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.244545+0000", 2024-04-24T15:33:57.455 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6186000000000001e-05 2024-04-24T15:33:57.455 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.455 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.455 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-04-24T15:33:57.455 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.244553+0000", 2024-04-24T15:33:57.455 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5380000000000002e-06 2024-04-24T15:33:57.455 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.455 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.455 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:57.455 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.631596+0000", 2024-04-24T15:33:57.455 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.38704289600000003 2024-04-24T15:33:57.456 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.456 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.456 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-04-24T15:33:57.456 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.631606+0000", 2024-04-24T15:33:57.456 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6390000000000004e-06 2024-04-24T15:33:57.456 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.456 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.456 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:33:57.456 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:48.658831+0000", 2024-04-24T15:33:57.456 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0272250540000001 2024-04-24T15:33:57.456 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.456 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.456 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:33:57.456 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:48.658991+0000", 2024-04-24T15:33:57.456 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015994800000000001 2024-04-24T15:33:57.457 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:33:57.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:33:57.457 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:33:57.457 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:48.659135+0000", 2024-04-24T15:33:57.457 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000144456 2024-04-24T15:33:57.457 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:57.457 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:57.457 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:57.457 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:33:57.457 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:33:57.457 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:33:57.458 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-24T15:33:57.464 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:57.465+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:57.565 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:57.566+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:57.594 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:57.594 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:33:57.594 INFO:teuthology.orchestra.run.smithi151.stdout: "num_ops": 0 2024-04-24T15:33:57.594 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:57.604 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-24T15:33:57.689 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:57.691+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:57.744 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 6, 4, 7] out_osds: [1, 5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 7, 6] 2024-04-24T15:33:57.744 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-24T15:33:57.744 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-04-24T15:33:57.744 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 3.0 2024-04-24T15:33:57.773 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:57.773 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:33:57.773 INFO:teuthology.orchestra.run.smithi151.stdout: "complaint_time": 30, 2024-04-24T15:33:57.773 INFO:teuthology.orchestra.run.smithi151.stdout: "num_blocked_ops": 0 2024-04-24T15:33:57.773 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:57.783 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-24T15:33:57.789 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:57.790+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:57.866 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:33:57.867+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:33:57.949 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:57.950 INFO:teuthology.orchestra.run.smithi151.stdout: "size": 20, 2024-04-24T15:33:57.950 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 600, 2024-04-24T15:33:57.950 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [ 2024-04-24T15:33:57.950 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.950 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:39041 4.1 4:82c1a902:::rbd_data.114abe589119.000000000000032c:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e68)", 2024-04-24T15:33:57.950 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:01.760552+0000", 2024-04-24T15:33:57.950 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 56.189155147000001, 2024-04-24T15:33:57.950 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.87570748300000001, 2024-04-24T15:33:57.950 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:57.950 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "reached pg", 2024-04-24T15:33:57.951 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:57.951 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:57.951 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:57.951 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 39041 2024-04-24T15:33:57.951 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.951 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:57.951 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.951 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:57.951 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760552+0000", 2024-04-24T15:33:57.951 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:57.951 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.951 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.953 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:57.953 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760552+0000", 2024-04-24T15:33:57.953 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:57.953 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.953 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.953 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:57.954 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760550+0000", 2024-04-24T15:33:57.954 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999986 2024-04-24T15:33:57.954 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.954 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.954 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:57.954 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760557+0000", 2024-04-24T15:33:57.954 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1420000000000004e-06 2024-04-24T15:33:57.954 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.954 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.954 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:57.954 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760572+0000", 2024-04-24T15:33:57.954 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.483e-05 2024-04-24T15:33:57.954 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.955 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.955 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:57.955 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760578+0000", 2024-04-24T15:33:57.955 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.2820000000000002e-06 2024-04-24T15:33:57.955 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.955 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.955 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:57.955 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760628+0000", 2024-04-24T15:33:57.955 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.0040000000000002e-05 2024-04-24T15:33:57.955 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.955 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.955 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op must wait for map", 2024-04-24T15:33:57.955 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760637+0000", 2024-04-24T15:33:57.955 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.6899999999999998e-06 2024-04-24T15:33:57.956 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.956 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.956 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:57.956 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.770562+0000", 2024-04-24T15:33:57.956 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0099252739999999996 2024-04-24T15:33:57.956 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.956 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.956 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for peered", 2024-04-24T15:33:57.956 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.770572+0000", 2024-04-24T15:33:57.956 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0105999999999999e-05 2024-04-24T15:33:57.956 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.956 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.956 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:57.956 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:02.636170+0000", 2024-04-24T15:33:57.957 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.86559722299999997 2024-04-24T15:33:57.957 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.957 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.957 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:57.957 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:02.636259+0000", 2024-04-24T15:33:57.957 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.9446999999999999e-05 2024-04-24T15:33:57.957 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:57.957 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:57.957 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:57.957 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.957 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.957 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48675 4.d 4:b465d49b:::rbd_data.114a858e6641.0000000000000002:17 [read 2768896~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:57.957 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.926403+0000", 2024-04-24T15:33:57.957 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 47.023303577, 2024-04-24T15:33:57.957 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035742807000000001, 2024-04-24T15:33:57.958 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:57.958 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:57.958 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:57.958 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:57.958 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:57.958 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48675 2024-04-24T15:33:57.958 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.958 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:57.958 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.958 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:57.958 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926403+0000", 2024-04-24T15:33:57.958 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:57.958 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.959 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.959 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:57.959 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926403+0000", 2024-04-24T15:33:57.959 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:57.959 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.959 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.959 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:57.959 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926403+0000", 2024-04-24T15:33:57.959 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:57.959 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.959 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.959 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:57.959 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926404+0000", 2024-04-24T15:33:57.959 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.424e-06 2024-04-24T15:33:57.959 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.960 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.960 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:57.960 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926413+0000", 2024-04-24T15:33:57.960 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.6850000000000007e-06 2024-04-24T15:33:57.960 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.960 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.960 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:57.960 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926417+0000", 2024-04-24T15:33:57.960 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.7859999999999998e-06 2024-04-24T15:33:57.960 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.960 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.960 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:57.960 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.961986+0000", 2024-04-24T15:33:57.960 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035568862 2024-04-24T15:33:57.961 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.961 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.961 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:57.961 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.962063+0000", 2024-04-24T15:33:57.961 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7257999999999998e-05 2024-04-24T15:33:57.961 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.961 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.961 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:57.961 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.962146+0000", 2024-04-24T15:33:57.961 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3094000000000001e-05 2024-04-24T15:33:57.961 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:57.961 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:57.961 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:57.961 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.962 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.962 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48676 4.d 4:b465d49b:::rbd_data.114a858e6641.0000000000000002:17 [read 2834432~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:57.962 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.926421+0000", 2024-04-24T15:33:57.962 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 47.023286034000002, 2024-04-24T15:33:57.962 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035829002999999998, 2024-04-24T15:33:57.962 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:57.962 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:57.962 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:57.962 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:57.962 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:57.962 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48676 2024-04-24T15:33:57.962 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.963 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:57.963 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.963 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:57.963 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926421+0000", 2024-04-24T15:33:57.963 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:57.963 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.963 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.963 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:57.963 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926421+0000", 2024-04-24T15:33:57.963 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:57.963 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.963 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.963 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:57.963 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926420+0000", 2024-04-24T15:33:57.963 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:57.964 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.964 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.964 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:57.964 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926423+0000", 2024-04-24T15:33:57.964 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.356e-06 2024-04-24T15:33:57.964 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.964 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.964 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:57.964 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926428+0000", 2024-04-24T15:33:57.964 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.3340000000000001e-06 2024-04-24T15:33:57.964 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.964 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.964 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:57.965 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926431+0000", 2024-04-24T15:33:57.965 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.2899999999999998e-06 2024-04-24T15:33:57.965 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.965 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.965 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:57.965 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.962156+0000", 2024-04-24T15:33:57.965 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035725159999999999 2024-04-24T15:33:57.965 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.965 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.965 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:57.965 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.962197+0000", 2024-04-24T15:33:57.965 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0457999999999998e-05 2024-04-24T15:33:57.965 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.966 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.966 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:57.966 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.962250+0000", 2024-04-24T15:33:57.966 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.2812000000000002e-05 2024-04-24T15:33:57.966 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:57.966 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:57.966 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:57.966 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.966 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.966 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48704 4.d 4:b59d5735:::rbd_data.114a858e6641.0000000000000004:17 [read 2703360~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:57.966 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928541+0000", 2024-04-24T15:33:57.966 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 47.021166063999999, 2024-04-24T15:33:57.966 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035763016000000002, 2024-04-24T15:33:57.967 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:57.967 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:57.967 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:57.967 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:57.967 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:57.967 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48704 2024-04-24T15:33:57.967 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.967 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:57.967 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.967 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:57.967 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928541+0000", 2024-04-24T15:33:57.967 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:57.967 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.967 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.967 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:57.967 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928541+0000", 2024-04-24T15:33:57.968 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:57.968 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.968 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.968 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:57.968 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928540+0000", 2024-04-24T15:33:57.968 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:57.968 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.968 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.968 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:57.968 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928542+0000", 2024-04-24T15:33:57.968 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.37e-06 2024-04-24T15:33:57.968 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.968 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.968 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:57.969 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928545+0000", 2024-04-24T15:33:57.969 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8229999999999999e-06 2024-04-24T15:33:57.969 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.969 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.969 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:57.969 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928547+0000", 2024-04-24T15:33:57.969 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.9360000000000002e-06 2024-04-24T15:33:57.969 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.969 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.969 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:57.969 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964219+0000", 2024-04-24T15:33:57.969 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035672813999999997 2024-04-24T15:33:57.969 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.969 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.969 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:57.969 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964259+0000", 2024-04-24T15:33:57.970 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0070000000000001e-05 2024-04-24T15:33:57.970 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.970 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.970 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:57.970 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964304+0000", 2024-04-24T15:33:57.970 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4229e-05 2024-04-24T15:33:57.970 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:57.970 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:57.970 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:57.970 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.970 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.970 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48719 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 1851392~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:57.970 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928549+0000", 2024-04-24T15:33:57.970 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 47.021157916999996, 2024-04-24T15:33:57.971 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035890645999999998, 2024-04-24T15:33:57.971 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:57.971 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:57.971 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:57.971 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:57.971 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:57.971 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48719 2024-04-24T15:33:57.971 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.971 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:57.971 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.971 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:57.971 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928549+0000", 2024-04-24T15:33:57.971 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:57.971 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.972 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.972 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:57.972 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928549+0000", 2024-04-24T15:33:57.972 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:57.972 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.972 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.972 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:57.972 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928549+0000", 2024-04-24T15:33:57.972 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:57.972 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.972 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.972 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:57.972 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928550+0000", 2024-04-24T15:33:57.972 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3829999999999999e-06 2024-04-24T15:33:57.972 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.973 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.973 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:57.973 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928553+0000", 2024-04-24T15:33:57.973 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7369999999999998e-06 2024-04-24T15:33:57.973 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.973 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.973 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:57.973 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928555+0000", 2024-04-24T15:33:57.973 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.8920000000000001e-06 2024-04-24T15:33:57.973 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.973 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.973 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:57.973 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964240+0000", 2024-04-24T15:33:57.973 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035685537000000003 2024-04-24T15:33:57.974 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.974 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.974 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:57.974 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964318+0000", 2024-04-24T15:33:57.974 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7838999999999998e-05 2024-04-24T15:33:57.974 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.974 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.974 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:57.974 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964439+0000", 2024-04-24T15:33:57.974 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000121564 2024-04-24T15:33:57.974 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:57.974 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:57.974 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:57.974 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.975 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.975 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48720 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 1916928~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:57.975 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928557+0000", 2024-04-24T15:33:57.975 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 47.021150071999998, 2024-04-24T15:33:57.975 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036005219999999998, 2024-04-24T15:33:57.975 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:57.975 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:57.975 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:57.975 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:57.975 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:57.975 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48720 2024-04-24T15:33:57.975 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.975 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:57.975 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.976 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:57.976 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928557+0000", 2024-04-24T15:33:57.976 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:57.976 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.976 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.976 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:57.976 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928557+0000", 2024-04-24T15:33:57.976 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:57.976 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.976 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.976 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:57.976 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928556+0000", 2024-04-24T15:33:57.976 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:57.976 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.976 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.977 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:57.977 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928558+0000", 2024-04-24T15:33:57.977 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.13e-06 2024-04-24T15:33:57.977 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.977 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.977 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:57.977 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928561+0000", 2024-04-24T15:33:57.977 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5530000000000001e-06 2024-04-24T15:33:57.977 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.977 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.977 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:57.977 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928563+0000", 2024-04-24T15:33:57.977 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.8810000000000001e-06 2024-04-24T15:33:57.977 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.977 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.978 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:57.978 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964450+0000", 2024-04-24T15:33:57.978 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035887040000000002 2024-04-24T15:33:57.978 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.978 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.978 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:57.978 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964494+0000", 2024-04-24T15:33:57.978 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3699000000000003e-05 2024-04-24T15:33:57.978 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.978 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.978 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:57.978 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964562+0000", 2024-04-24T15:33:57.978 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.8149000000000001e-05 2024-04-24T15:33:57.979 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:57.979 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:57.979 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:57.979 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.979 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.979 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48721 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 1982464~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:57.979 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928565+0000", 2024-04-24T15:33:57.979 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 47.021141548000003, 2024-04-24T15:33:57.979 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036233466999999998, 2024-04-24T15:33:57.979 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:57.979 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:57.979 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:57.979 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:57.979 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:57.980 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48721 2024-04-24T15:33:57.980 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.980 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:57.980 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.980 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:57.980 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928565+0000", 2024-04-24T15:33:57.980 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:57.980 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.980 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.980 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:57.980 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928565+0000", 2024-04-24T15:33:57.980 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:57.980 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.980 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.981 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:57.981 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928565+0000", 2024-04-24T15:33:57.981 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:57.981 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.981 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.981 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:57.981 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928566+0000", 2024-04-24T15:33:57.981 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.336e-06 2024-04-24T15:33:57.981 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.981 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.981 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:57.981 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928569+0000", 2024-04-24T15:33:57.981 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.9189999999999999e-06 2024-04-24T15:33:57.981 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.981 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.982 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:57.982 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928571+0000", 2024-04-24T15:33:57.982 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.1909999999999999e-06 2024-04-24T15:33:57.982 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.982 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.982 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:57.982 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964592+0000", 2024-04-24T15:33:57.982 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036020120000000003 2024-04-24T15:33:57.982 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.982 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.982 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:57.982 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964673+0000", 2024-04-24T15:33:57.982 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.1254000000000005e-05 2024-04-24T15:33:57.983 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.983 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.983 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:57.983 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964799+0000", 2024-04-24T15:33:57.983 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00012588700000000001 2024-04-24T15:33:57.983 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:57.983 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:57.983 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:57.983 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.983 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.983 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48722 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2048000~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:57.983 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928586+0000", 2024-04-24T15:33:57.983 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 47.021120422999999, 2024-04-24T15:33:57.983 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036331347999999999, 2024-04-24T15:33:57.984 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:57.984 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:57.984 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:57.984 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:57.984 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:57.984 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48722 2024-04-24T15:33:57.984 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.984 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:57.984 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.984 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:57.984 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928586+0000", 2024-04-24T15:33:57.984 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:57.984 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.984 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.984 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:57.984 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928586+0000", 2024-04-24T15:33:57.985 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:57.985 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.985 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.985 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:57.985 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928586+0000", 2024-04-24T15:33:57.985 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:57.985 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.985 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.985 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:57.985 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928587+0000", 2024-04-24T15:33:57.985 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3519999999999999e-06 2024-04-24T15:33:57.985 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.985 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.985 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:57.985 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928590+0000", 2024-04-24T15:33:57.985 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.9699999999999999e-06 2024-04-24T15:33:57.986 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.986 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.986 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:57.986 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928592+0000", 2024-04-24T15:33:57.986 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.097e-06 2024-04-24T15:33:57.986 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.986 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.986 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:57.986 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964810+0000", 2024-04-24T15:33:57.986 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036217249 2024-04-24T15:33:57.986 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.986 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.986 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:57.986 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964851+0000", 2024-04-24T15:33:57.987 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.1151e-05 2024-04-24T15:33:57.987 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.987 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.987 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:57.987 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964918+0000", 2024-04-24T15:33:57.987 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.6898999999999998e-05 2024-04-24T15:33:57.987 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:57.987 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:57.987 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:57.987 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.987 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.987 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48723 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2113536~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:57.987 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928598+0000", 2024-04-24T15:33:57.987 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 47.021109033999998, 2024-04-24T15:33:57.988 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036552125999999997, 2024-04-24T15:33:57.988 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:57.988 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:57.988 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:57.988 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:57.988 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:57.988 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48723 2024-04-24T15:33:57.988 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.988 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:57.988 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.988 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:57.988 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928598+0000", 2024-04-24T15:33:57.988 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:57.988 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.988 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.989 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:57.989 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928598+0000", 2024-04-24T15:33:57.989 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:57.989 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.989 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.989 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:57.989 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928597+0000", 2024-04-24T15:33:57.989 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:57.989 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.989 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.989 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:57.989 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928599+0000", 2024-04-24T15:33:57.989 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.621e-06 2024-04-24T15:33:57.989 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.990 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.990 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:57.990 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928602+0000", 2024-04-24T15:33:57.990 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.9629999999999998e-06 2024-04-24T15:33:57.990 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.990 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.990 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:57.990 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928607+0000", 2024-04-24T15:33:57.990 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8940000000000001e-06 2024-04-24T15:33:57.990 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.990 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.990 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:57.990 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964947+0000", 2024-04-24T15:33:57.990 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036340439000000002 2024-04-24T15:33:57.991 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.991 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.991 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:57.991 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965028+0000", 2024-04-24T15:33:57.991 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.1054e-05 2024-04-24T15:33:57.991 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.991 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.991 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:57.991 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965150+0000", 2024-04-24T15:33:57.991 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00012154000000000001 2024-04-24T15:33:57.991 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:57.991 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:57.991 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:57.991 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.991 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.992 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48724 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2179072~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:57.992 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928609+0000", 2024-04-24T15:33:57.992 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 47.021097658999999, 2024-04-24T15:33:57.992 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036679614999999999, 2024-04-24T15:33:57.992 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:57.992 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:57.992 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:57.992 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:57.992 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:57.992 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48724 2024-04-24T15:33:57.992 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.992 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:57.992 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.993 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:57.993 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928609+0000", 2024-04-24T15:33:57.993 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:57.993 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.993 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.993 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:57.993 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928609+0000", 2024-04-24T15:33:57.993 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:57.993 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.993 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.993 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:57.993 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928609+0000", 2024-04-24T15:33:57.993 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:57.993 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.994 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.994 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:57.994 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928610+0000", 2024-04-24T15:33:57.994 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3030000000000001e-06 2024-04-24T15:33:57.994 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.994 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.994 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:57.994 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928613+0000", 2024-04-24T15:33:57.994 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.762e-06 2024-04-24T15:33:57.994 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.994 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.994 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:57.994 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928615+0000", 2024-04-24T15:33:57.994 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.832e-06 2024-04-24T15:33:57.995 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.995 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.995 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:57.995 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965174+0000", 2024-04-24T15:33:57.995 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036559633000000001 2024-04-24T15:33:57.995 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.995 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.995 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:57.995 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965221+0000", 2024-04-24T15:33:57.995 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6239999999999998e-05 2024-04-24T15:33:57.995 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.995 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.995 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:57.996 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965289+0000", 2024-04-24T15:33:57.996 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.8053999999999996e-05 2024-04-24T15:33:57.996 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:57.996 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:57.996 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:57.996 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.996 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.996 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48725 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2244608~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:57.996 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928617+0000", 2024-04-24T15:33:57.996 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 47.021089889000002, 2024-04-24T15:33:57.996 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036902659999999997, 2024-04-24T15:33:57.996 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:57.996 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:57.996 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:57.997 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:57.997 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:57.997 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48725 2024-04-24T15:33:57.997 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.997 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:57.997 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.997 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:57.997 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928617+0000", 2024-04-24T15:33:57.997 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:57.997 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.997 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.997 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:57.997 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928617+0000", 2024-04-24T15:33:57.997 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:57.997 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.998 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.998 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:57.998 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928617+0000", 2024-04-24T15:33:57.998 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:57.998 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.998 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.998 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:57.998 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928618+0000", 2024-04-24T15:33:57.998 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.206e-06 2024-04-24T15:33:57.998 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.998 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.998 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:57.998 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928621+0000", 2024-04-24T15:33:57.999 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5130000000000001e-06 2024-04-24T15:33:57.999 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.999 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.999 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:57.999 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928623+0000", 2024-04-24T15:33:57.999 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.9640000000000002e-06 2024-04-24T15:33:57.999 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.999 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.999 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:57.999 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965318+0000", 2024-04-24T15:33:57.999 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036694953000000002 2024-04-24T15:33:57.999 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:57.999 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:57.999 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:57.999 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965398+0000", 2024-04-24T15:33:58.000 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.9986000000000003e-05 2024-04-24T15:33:58.000 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.000 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.000 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:58.000 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965520+0000", 2024-04-24T15:33:58.000 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000121247 2024-04-24T15:33:58.000 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:58.000 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:58.000 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:58.000 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.000 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.000 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48726 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2310144~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:58.000 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928625+0000", 2024-04-24T15:33:58.001 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 47.021081342000002, 2024-04-24T15:33:58.001 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037015521000000003, 2024-04-24T15:33:58.001 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:58.001 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:58.001 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:58.001 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:58.001 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:58.001 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48726 2024-04-24T15:33:58.001 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.001 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:58.001 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.001 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:58.001 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928625+0000", 2024-04-24T15:33:58.001 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:58.001 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.002 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.002 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:58.002 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928625+0000", 2024-04-24T15:33:58.002 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:58.002 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.002 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.002 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:58.002 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928625+0000", 2024-04-24T15:33:58.002 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:58.002 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.002 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.002 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:58.002 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928627+0000", 2024-04-24T15:33:58.003 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3179999999999999e-06 2024-04-24T15:33:58.003 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.003 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.003 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:58.003 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928629+0000", 2024-04-24T15:33:58.003 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.819e-06 2024-04-24T15:33:58.003 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.003 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.003 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:58.003 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928634+0000", 2024-04-24T15:33:58.003 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6999999999999999e-06 2024-04-24T15:33:58.003 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.003 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.003 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:58.004 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965531+0000", 2024-04-24T15:33:58.004 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036897040999999998 2024-04-24T15:33:58.004 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.004 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.004 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:58.004 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965573+0000", 2024-04-24T15:33:58.004 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.189e-05 2024-04-24T15:33:58.004 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.004 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.004 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:58.004 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965641+0000", 2024-04-24T15:33:58.004 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.7959000000000004e-05 2024-04-24T15:33:58.004 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:58.004 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:58.004 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:58.005 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.005 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.005 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48727 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2375680~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:58.005 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928637+0000", 2024-04-24T15:33:58.005 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 47.021069980999997, 2024-04-24T15:33:58.005 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037235129999999998, 2024-04-24T15:33:58.005 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:58.005 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:58.005 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:58.005 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:58.005 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:58.005 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48727 2024-04-24T15:33:58.005 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.005 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:58.005 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.006 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:58.006 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928637+0000", 2024-04-24T15:33:58.006 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:58.006 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.006 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.006 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:58.006 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928637+0000", 2024-04-24T15:33:58.006 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:58.006 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.006 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.006 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:58.006 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928637+0000", 2024-04-24T15:33:58.006 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:58.006 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.007 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.007 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:58.007 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928638+0000", 2024-04-24T15:33:58.007 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.4279999999999999e-06 2024-04-24T15:33:58.007 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.007 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.007 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:58.007 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928641+0000", 2024-04-24T15:33:58.007 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.909e-06 2024-04-24T15:33:58.007 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.007 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.007 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:58.007 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928643+0000", 2024-04-24T15:33:58.007 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.9269999999999999e-06 2024-04-24T15:33:58.008 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.008 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.008 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:58.008 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965671+0000", 2024-04-24T15:33:58.008 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037028393 2024-04-24T15:33:58.008 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.008 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.008 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:58.008 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965752+0000", 2024-04-24T15:33:58.008 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0921999999999995e-05 2024-04-24T15:33:58.008 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.008 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.008 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:58.008 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965872+0000", 2024-04-24T15:33:58.008 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000119759 2024-04-24T15:33:58.009 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:58.009 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:58.009 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:58.009 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.009 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.009 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48728 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2441216~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:58.009 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928645+0000", 2024-04-24T15:33:58.009 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 47.021061797999998, 2024-04-24T15:33:58.009 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037346852999999999, 2024-04-24T15:33:58.009 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:58.009 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:58.009 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:58.009 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:58.009 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:58.009 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48728 2024-04-24T15:33:58.009 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.010 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:58.010 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.010 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:58.010 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928645+0000", 2024-04-24T15:33:58.010 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:58.010 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.010 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.010 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:58.010 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928645+0000", 2024-04-24T15:33:58.010 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:58.010 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.010 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.010 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:58.010 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928645+0000", 2024-04-24T15:33:58.011 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:58.011 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.011 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.011 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:58.011 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928646+0000", 2024-04-24T15:33:58.011 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.291e-06 2024-04-24T15:33:58.011 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.011 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.011 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:58.011 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928649+0000", 2024-04-24T15:33:58.011 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8509999999999999e-06 2024-04-24T15:33:58.011 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.011 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.011 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:58.011 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928651+0000", 2024-04-24T15:33:58.011 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.809e-06 2024-04-24T15:33:58.012 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.012 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.012 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:58.012 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965882+0000", 2024-04-24T15:33:58.012 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037231715999999998 2024-04-24T15:33:58.012 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.012 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.012 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:58.012 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965925+0000", 2024-04-24T15:33:58.012 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2493999999999997e-05 2024-04-24T15:33:58.012 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.012 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.012 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:58.012 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965992+0000", 2024-04-24T15:33:58.012 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.6907000000000002e-05 2024-04-24T15:33:58.013 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:58.013 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:58.013 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:58.013 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.013 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.013 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48729 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2506752~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:58.013 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928653+0000", 2024-04-24T15:33:58.013 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 47.021053922999997, 2024-04-24T15:33:58.013 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037576904000000001, 2024-04-24T15:33:58.013 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:58.013 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:58.013 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:58.013 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:58.013 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:58.013 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48729 2024-04-24T15:33:58.013 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.014 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:58.014 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.014 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:58.014 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928653+0000", 2024-04-24T15:33:58.014 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:58.014 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.014 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.014 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:58.014 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928653+0000", 2024-04-24T15:33:58.014 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:58.014 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.014 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.014 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:58.014 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928653+0000", 2024-04-24T15:33:58.015 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:58.015 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.015 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.015 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:58.015 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928654+0000", 2024-04-24T15:33:58.015 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3209999999999999e-06 2024-04-24T15:33:58.015 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.015 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.015 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:58.015 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928657+0000", 2024-04-24T15:33:58.015 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8559999999999998e-06 2024-04-24T15:33:58.015 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.015 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.015 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:58.015 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928659+0000", 2024-04-24T15:33:58.016 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7080000000000002e-06 2024-04-24T15:33:58.016 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.016 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.016 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:58.016 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966021+0000", 2024-04-24T15:33:58.016 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037361296000000002 2024-04-24T15:33:58.016 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.016 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.016 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:58.016 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966102+0000", 2024-04-24T15:33:58.016 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0939999999999994e-05 2024-04-24T15:33:58.016 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.016 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.016 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:58.016 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966230+0000", 2024-04-24T15:33:58.017 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00012799999999999999 2024-04-24T15:33:58.017 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:58.017 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:58.017 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:58.017 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.017 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.017 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48730 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2572288~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:58.017 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928662+0000", 2024-04-24T15:33:58.017 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 47.021044834999998, 2024-04-24T15:33:58.017 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037687246000000001, 2024-04-24T15:33:58.017 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:58.017 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:58.017 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:58.017 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:58.017 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:58.017 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48730 2024-04-24T15:33:58.018 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.018 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:58.018 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.018 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:58.018 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928662+0000", 2024-04-24T15:33:58.018 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:58.018 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.018 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.018 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:58.018 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928662+0000", 2024-04-24T15:33:58.018 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:58.018 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.018 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.018 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:58.019 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928662+0000", 2024-04-24T15:33:58.019 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:58.019 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.019 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.019 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:58.019 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928663+0000", 2024-04-24T15:33:58.019 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.381e-06 2024-04-24T15:33:58.019 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.019 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.019 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:58.019 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928666+0000", 2024-04-24T15:33:58.019 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.796e-06 2024-04-24T15:33:58.019 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.019 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.019 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:58.020 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928668+0000", 2024-04-24T15:33:58.020 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.8920000000000001e-06 2024-04-24T15:33:58.020 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.020 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.020 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:58.020 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966240+0000", 2024-04-24T15:33:58.020 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037572886999999999 2024-04-24T15:33:58.020 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.020 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.020 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:58.020 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966281+0000", 2024-04-24T15:33:58.020 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0450000000000001e-05 2024-04-24T15:33:58.020 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.020 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.020 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:58.021 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966349+0000", 2024-04-24T15:33:58.021 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.8215999999999999e-05 2024-04-24T15:33:58.021 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:58.021 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:58.021 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:58.021 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.021 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.021 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48731 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2637824~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:58.021 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928747+0000", 2024-04-24T15:33:58.021 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 47.020959830000002, 2024-04-24T15:33:58.021 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037833602000000001, 2024-04-24T15:33:58.021 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:58.021 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:58.021 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:58.022 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:58.022 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:58.022 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48731 2024-04-24T15:33:58.022 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.022 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:58.022 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.022 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:58.022 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928747+0000", 2024-04-24T15:33:58.022 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:58.022 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.022 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.022 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:58.022 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928747+0000", 2024-04-24T15:33:58.022 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:58.023 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.023 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.023 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:58.023 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928670+0000", 2024-04-24T15:33:58.023 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999228 2024-04-24T15:33:58.023 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.023 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.023 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:58.023 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928750+0000", 2024-04-24T15:33:58.023 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0527999999999999e-05 2024-04-24T15:33:58.023 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.023 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.023 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:58.023 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928757+0000", 2024-04-24T15:33:58.023 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.9419999999999998e-06 2024-04-24T15:33:58.024 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.024 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.024 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:58.024 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928762+0000", 2024-04-24T15:33:58.024 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4240000000000003e-06 2024-04-24T15:33:58.024 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.024 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.024 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:58.024 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966379+0000", 2024-04-24T15:33:58.024 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037617267000000003 2024-04-24T15:33:58.024 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.024 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.024 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:58.024 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966461+0000", 2024-04-24T15:33:58.024 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.2720999999999996e-05 2024-04-24T15:33:58.024 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.025 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.025 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:58.025 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966581+0000", 2024-04-24T15:33:58.025 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000119014 2024-04-24T15:33:58.025 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:58.025 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:58.025 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:58.025 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.025 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.025 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48732 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2703360~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:58.025 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928766+0000", 2024-04-24T15:33:58.025 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 47.020940346000003, 2024-04-24T15:33:58.025 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037930924999999997, 2024-04-24T15:33:58.025 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:58.025 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:58.026 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:58.026 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:58.026 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:58.026 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48732 2024-04-24T15:33:58.026 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.026 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:58.026 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.026 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:58.026 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928766+0000", 2024-04-24T15:33:58.026 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:58.026 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.026 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.026 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:58.026 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928766+0000", 2024-04-24T15:33:58.027 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:58.027 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.027 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.027 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:58.027 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928766+0000", 2024-04-24T15:33:58.027 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:58.027 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.027 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.027 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:58.027 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928768+0000", 2024-04-24T15:33:58.027 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.3420000000000002e-06 2024-04-24T15:33:58.027 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.027 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.027 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:58.027 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928783+0000", 2024-04-24T15:33:58.028 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.4813e-05 2024-04-24T15:33:58.028 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.028 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.028 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:58.028 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928787+0000", 2024-04-24T15:33:58.028 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.1999999999999996e-06 2024-04-24T15:33:58.028 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.028 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.028 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:58.028 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966592+0000", 2024-04-24T15:33:58.028 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037804484999999999 2024-04-24T15:33:58.028 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.028 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.028 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:58.029 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966632+0000", 2024-04-24T15:33:58.029 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0014e-05 2024-04-24T15:33:58.029 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.029 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.029 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:58.029 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966697+0000", 2024-04-24T15:33:58.029 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.5489999999999998e-05 2024-04-24T15:33:58.029 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:58.029 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:58.029 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:58.029 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.029 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.029 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48813 4.d 4:b39400aa:::rbd_data.114a8b543e91.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:33:58.029 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.936015+0000", 2024-04-24T15:33:58.029 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 47.013692012, 2024-04-24T15:33:58.030 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.039312294999999997, 2024-04-24T15:33:58.030 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:58.030 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:58.030 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:58.030 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:58.030 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:58.030 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48813 2024-04-24T15:33:58.030 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.030 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:58.030 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.030 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:58.030 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.936015+0000", 2024-04-24T15:33:58.030 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:58.031 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.031 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.031 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:58.031 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.936015+0000", 2024-04-24T15:33:58.031 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:58.031 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.031 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.031 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:58.031 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.936014+0000", 2024-04-24T15:33:58.031 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:58.031 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.031 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.031 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:58.031 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.942618+0000", 2024-04-24T15:33:58.031 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0066038499999999997 2024-04-24T15:33:58.032 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.032 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.032 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:58.032 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.942629+0000", 2024-04-24T15:33:58.032 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0606e-05 2024-04-24T15:33:58.032 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.032 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.032 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:58.032 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.942634+0000", 2024-04-24T15:33:58.032 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.2000000000000002e-06 2024-04-24T15:33:58.032 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.032 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.032 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:58.032 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.969006+0000", 2024-04-24T15:33:58.033 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.026371564 2024-04-24T15:33:58.033 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.033 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.033 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:58.033 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.969126+0000", 2024-04-24T15:33:58.033 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000120085 2024-04-24T15:33:58.033 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.033 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.033 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 2", 2024-04-24T15:33:58.033 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.969469+0000", 2024-04-24T15:33:58.033 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00034295400000000002 2024-04-24T15:33:58.033 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.033 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.033 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:33:58.034 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.973118+0000", 2024-04-24T15:33:58.034 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0036489700000000001 2024-04-24T15:33:58.034 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.034 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.034 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:33:58.034 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.975250+0000", 2024-04-24T15:33:58.034 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0021327030000000001 2024-04-24T15:33:58.034 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.034 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.034 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:33:58.034 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.975293+0000", 2024-04-24T15:33:58.034 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2951999999999998e-05 2024-04-24T15:33:58.034 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.034 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.035 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:58.035 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.975327+0000", 2024-04-24T15:33:58.035 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.3840000000000001e-05 2024-04-24T15:33:58.035 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:58.035 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:58.035 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:58.035 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.035 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.035 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:75520 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:head [delete] snapc 17=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e105)", 2024-04-24T15:33:58.035 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:42.638168+0000", 2024-04-24T15:33:58.035 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 15.311538390000001, 2024-04-24T15:33:58.035 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0190266560000001, 2024-04-24T15:33:58.035 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:58.035 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:33:58.035 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:58.036 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:58.036 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:58.036 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 75520 2024-04-24T15:33:58.036 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.036 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:58.036 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.036 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:58.036 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.638168+0000", 2024-04-24T15:33:58.036 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:58.036 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.036 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.036 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:58.036 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.638168+0000", 2024-04-24T15:33:58.036 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:58.037 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.037 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.037 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:58.037 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.638167+0000", 2024-04-24T15:33:58.037 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999986 2024-04-24T15:33:58.037 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.037 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.037 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:58.037 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.638174+0000", 2024-04-24T15:33:58.037 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.861e-06 2024-04-24T15:33:58.037 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.037 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.037 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:58.037 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.638186+0000", 2024-04-24T15:33:58.038 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.2109e-05 2024-04-24T15:33:58.038 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.038 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.038 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:58.038 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.638194+0000", 2024-04-24T15:33:58.038 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.8040000000000002e-06 2024-04-24T15:33:58.038 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.038 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.038 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:58.038 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.638682+0000", 2024-04-24T15:33:58.038 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00048809599999999998 2024-04-24T15:33:58.038 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.038 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.038 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for peered", 2024-04-24T15:33:58.039 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.638692+0000", 2024-04-24T15:33:58.039 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.8970000000000003e-06 2024-04-24T15:33:58.039 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.039 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.039 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:58.039 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:43.649868+0000", 2024-04-24T15:33:58.039 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.011176641 2024-04-24T15:33:58.039 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.039 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.039 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:58.039 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:43.649960+0000", 2024-04-24T15:33:58.039 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.1632999999999995e-05 2024-04-24T15:33:58.039 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.039 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.040 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 6", 2024-04-24T15:33:58.040 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:43.650060+0000", 2024-04-24T15:33:58.040 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.9562999999999995e-05 2024-04-24T15:33:58.040 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.040 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.040 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:33:58.040 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:43.654465+0000", 2024-04-24T15:33:58.040 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0044056620000000003 2024-04-24T15:33:58.040 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.040 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.040 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:33:58.040 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:43.656963+0000", 2024-04-24T15:33:58.040 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.002497829 2024-04-24T15:33:58.040 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.041 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.041 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:33:58.041 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:43.657083+0000", 2024-04-24T15:33:58.041 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000120338 2024-04-24T15:33:58.041 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.041 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.041 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:58.041 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:43.657195+0000", 2024-04-24T15:33:58.041 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000111624 2024-04-24T15:33:58.041 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:58.041 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:58.041 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:58.041 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:58.041 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:58.042 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:58.043 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-24T15:33:58.044 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:58.044+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:58.060 INFO:teuthology.orchestra.run.smithi096.stdout:moved 0 / 90 (0%) 2024-04-24T15:33:58.060 INFO:teuthology.orchestra.run.smithi096.stdout:avg 15 2024-04-24T15:33:58.060 INFO:teuthology.orchestra.run.smithi096.stdout:stddev 2.38048 -> 2.38048 (expected baseline 3.53553) 2024-04-24T15:33:58.060 INFO:teuthology.orchestra.run.smithi096.stdout:min osd.2 with 10 -> 10 pgs (0.666667 -> 0.666667 * mean) 2024-04-24T15:33:58.060 INFO:teuthology.orchestra.run.smithi096.stdout:max osd.3 with 17 -> 17 pgs (1.13333 -> 1.13333 * mean) 2024-04-24T15:33:58.060 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:33:58.060 INFO:teuthology.orchestra.run.smithi096.stdout:oload 110 2024-04-24T15:33:58.060 INFO:teuthology.orchestra.run.smithi096.stdout:max_change 3 2024-04-24T15:33:58.060 INFO:teuthology.orchestra.run.smithi096.stdout:max_change_osds 4 2024-04-24T15:33:58.061 INFO:teuthology.orchestra.run.smithi096.stdout:average_utilization 0.0032 2024-04-24T15:33:58.061 INFO:teuthology.orchestra.run.smithi096.stdout:overload_utilization 0.0036 2024-04-24T15:33:58.063 INFO:teuthology.orchestra.run.smithi096.stderr:no change 2024-04-24T15:33:58.074 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '1000', 'type': 'reweight-by-utilization'} 2024-04-24T15:33:58.074 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 0.05 2024-04-24T15:33:58.143 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:58.145+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:58.207 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:58.207 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:33:58.207 INFO:teuthology.orchestra.run.smithi151.stdout: "num_ops": 0 2024-04-24T15:33:58.207 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:58.216 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-24T15:33:58.245 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:33:58.246+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:33:58.346 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:33:58.347+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:33:58.377 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-24T15:33:58.378 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:58.378 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:33:58.378 INFO:teuthology.orchestra.run.smithi151.stdout: "complaint_time": 30, 2024-04-24T15:33:58.378 INFO:teuthology.orchestra.run.smithi151.stdout: "num_blocked_ops": 0 2024-04-24T15:33:58.379 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:58.388 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-24T15:33:58.431 INFO:teuthology.orchestra.run.smithi096.stdout:moved 0 / 90 (0%) 2024-04-24T15:33:58.431 INFO:teuthology.orchestra.run.smithi096.stdout:avg 15 2024-04-24T15:33:58.432 INFO:teuthology.orchestra.run.smithi096.stdout:stddev 2.38048 -> 2.38048 (expected baseline 3.53553) 2024-04-24T15:33:58.432 INFO:teuthology.orchestra.run.smithi096.stdout:min osd.2 with 10 -> 10 pgs (0.666667 -> 0.666667 * mean) 2024-04-24T15:33:58.432 INFO:teuthology.orchestra.run.smithi096.stdout:max osd.3 with 17 -> 17 pgs (1.13333 -> 1.13333 * mean) 2024-04-24T15:33:58.432 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:33:58.432 INFO:teuthology.orchestra.run.smithi096.stdout:oload 1000 2024-04-24T15:33:58.432 INFO:teuthology.orchestra.run.smithi096.stdout:max_change 0.05 2024-04-24T15:33:58.432 INFO:teuthology.orchestra.run.smithi096.stdout:max_change_osds 4 2024-04-24T15:33:58.432 INFO:teuthology.orchestra.run.smithi096.stdout:average_utilization 0.0032 2024-04-24T15:33:58.432 INFO:teuthology.orchestra.run.smithi096.stdout:overload_utilization 0.0325 2024-04-24T15:33:58.434 INFO:teuthology.orchestra.run.smithi096.stderr:no change 2024-04-24T15:33:58.446 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-04-24T15:33:58.446 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 1000 3.0 2024-04-24T15:33:58.447 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:58.448+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:58.512 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:33:58.512 INFO:teuthology.orchestra.run.smithi151.stdout: "size": 20, 2024-04-24T15:33:58.512 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 600, 2024-04-24T15:33:58.512 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [ 2024-04-24T15:33:58.512 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.513 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23613 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 14688256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:58.513 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.792522+0000", 2024-04-24T15:33:58.513 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 83.719923093000006, 2024-04-24T15:33:58.513 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040507356000000001, 2024-04-24T15:33:58.513 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:58.513 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:58.513 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:58.513 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:58.513 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:58.513 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23613 2024-04-24T15:33:58.513 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.513 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:58.513 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.513 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:58.513 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792522+0000", 2024-04-24T15:33:58.513 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:58.514 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.514 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.514 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:58.514 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792522+0000", 2024-04-24T15:33:58.514 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:58.514 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.514 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.514 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:58.514 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792522+0000", 2024-04-24T15:33:58.514 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:33:58.514 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.514 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.514 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:58.514 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792523+0000", 2024-04-24T15:33:58.515 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.576e-06 2024-04-24T15:33:58.515 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.515 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.515 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:58.515 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792526+0000", 2024-04-24T15:33:58.515 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7439999999999999e-06 2024-04-24T15:33:58.515 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.515 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.515 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:58.515 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792528+0000", 2024-04-24T15:33:58.515 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.0600000000000002e-06 2024-04-24T15:33:58.515 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.515 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.515 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:58.515 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.832864+0000", 2024-04-24T15:33:58.516 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040335742000000001 2024-04-24T15:33:58.516 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.516 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.516 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:58.516 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.832953+0000", 2024-04-24T15:33:58.516 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.9281000000000001e-05 2024-04-24T15:33:58.516 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.516 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.516 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:58.516 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.833029+0000", 2024-04-24T15:33:58.516 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.6259000000000003e-05 2024-04-24T15:33:58.516 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:58.516 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:58.516 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:58.516 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.516 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.517 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23614 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 14729216~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:58.517 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.792530+0000", 2024-04-24T15:33:58.517 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 83.719914829000004, 2024-04-24T15:33:58.517 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040633957999999998, 2024-04-24T15:33:58.517 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:58.517 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:58.517 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:58.517 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:58.517 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:58.517 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23614 2024-04-24T15:33:58.517 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.517 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:58.517 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.517 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:58.517 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792530+0000", 2024-04-24T15:33:58.517 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:58.518 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.518 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.518 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:58.518 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792530+0000", 2024-04-24T15:33:58.518 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:58.518 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.518 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.518 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:58.518 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792530+0000", 2024-04-24T15:33:58.518 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:58.518 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.518 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.518 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:58.518 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792531+0000", 2024-04-24T15:33:58.519 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.46e-06 2024-04-24T15:33:58.519 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.519 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.519 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:58.519 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792534+0000", 2024-04-24T15:33:58.519 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8959999999999999e-06 2024-04-24T15:33:58.519 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.519 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.519 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:58.519 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792536+0000", 2024-04-24T15:33:58.519 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.841e-06 2024-04-24T15:33:58.519 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.519 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.519 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:58.520 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.833060+0000", 2024-04-24T15:33:58.520 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040524125000000001 2024-04-24T15:33:58.520 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.520 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.520 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:58.520 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.833108+0000", 2024-04-24T15:33:58.520 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.7547999999999999e-05 2024-04-24T15:33:58.520 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.520 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.520 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:58.520 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.833164+0000", 2024-04-24T15:33:58.520 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6297e-05 2024-04-24T15:33:58.520 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:58.520 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:58.521 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:58.521 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.521 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.521 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23623 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15097856~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:58.521 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.793887+0000", 2024-04-24T15:33:58.521 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 83.718557868000005, 2024-04-24T15:33:58.521 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040432784999999999, 2024-04-24T15:33:58.521 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:58.521 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:58.521 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:58.521 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:58.521 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:58.521 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23623 2024-04-24T15:33:58.522 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.522 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:58.522 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.522 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:58.522 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793887+0000", 2024-04-24T15:33:58.522 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:58.522 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.522 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.522 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:58.522 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793887+0000", 2024-04-24T15:33:58.522 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:58.522 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.522 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.522 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:58.523 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793887+0000", 2024-04-24T15:33:58.523 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:33:58.523 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.523 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.523 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:58.523 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793888+0000", 2024-04-24T15:33:58.523 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3659999999999999e-06 2024-04-24T15:33:58.523 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.523 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.523 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:58.523 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793891+0000", 2024-04-24T15:33:58.523 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8389999999999998e-06 2024-04-24T15:33:58.523 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.523 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.524 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:58.524 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793893+0000", 2024-04-24T15:33:58.524 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.9549999999999999e-06 2024-04-24T15:33:58.524 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.524 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.524 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:58.524 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834239+0000", 2024-04-24T15:33:58.524 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040345487999999999 2024-04-24T15:33:58.524 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.524 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.524 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:58.524 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834280+0000", 2024-04-24T15:33:58.524 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.1878000000000001e-05 2024-04-24T15:33:58.524 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.524 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.525 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:58.525 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834320+0000", 2024-04-24T15:33:58.525 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9480000000000001e-05 2024-04-24T15:33:58.525 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:58.525 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:58.525 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:58.525 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.525 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.525 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23624 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15138816~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:58.525 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.794053+0000", 2024-04-24T15:33:58.525 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 83.718391920000002, 2024-04-24T15:33:58.525 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040370801999999997, 2024-04-24T15:33:58.525 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:58.525 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:58.526 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:58.526 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:58.526 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:58.526 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23624 2024-04-24T15:33:58.526 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.526 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:58.526 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.526 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:58.526 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794053+0000", 2024-04-24T15:33:58.526 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:58.526 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.526 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.526 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:58.526 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794053+0000", 2024-04-24T15:33:58.527 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:58.527 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.527 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.527 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:58.527 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793895+0000", 2024-04-24T15:33:58.527 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998422 2024-04-24T15:33:58.527 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.527 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.527 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:58.527 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794057+0000", 2024-04-24T15:33:58.527 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00016149400000000001 2024-04-24T15:33:58.527 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.527 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.527 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:58.528 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794065+0000", 2024-04-24T15:33:58.528 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.5259999999999993e-06 2024-04-24T15:33:58.528 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.528 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.528 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:58.528 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794071+0000", 2024-04-24T15:33:58.528 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6149999999999996e-06 2024-04-24T15:33:58.528 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.528 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.528 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:58.528 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834341+0000", 2024-04-24T15:33:58.528 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040270366000000002 2024-04-24T15:33:58.528 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.528 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.528 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:58.529 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834381+0000", 2024-04-24T15:33:58.529 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0102999999999999e-05 2024-04-24T15:33:58.529 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.529 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.529 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:58.529 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834424+0000", 2024-04-24T15:33:58.529 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2713000000000003e-05 2024-04-24T15:33:58.529 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:58.529 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:58.529 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:58.529 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.529 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.529 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23625 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15179776~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:58.529 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.794160+0000", 2024-04-24T15:33:58.529 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 83.718285038000005, 2024-04-24T15:33:58.529 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040413004000000002, 2024-04-24T15:33:58.530 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:58.530 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:58.530 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:58.530 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:58.530 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:58.530 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23625 2024-04-24T15:33:58.530 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.530 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:58.530 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.530 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:58.530 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794160+0000", 2024-04-24T15:33:58.530 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:58.530 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.530 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.530 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:58.531 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794160+0000", 2024-04-24T15:33:58.531 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:58.531 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.531 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.531 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:58.531 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794086+0000", 2024-04-24T15:33:58.531 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999256 2024-04-24T15:33:58.531 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.531 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.531 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:58.531 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794163+0000", 2024-04-24T15:33:58.531 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7896000000000001e-05 2024-04-24T15:33:58.531 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.531 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.531 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:58.531 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794171+0000", 2024-04-24T15:33:58.532 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.9500000000000001e-06 2024-04-24T15:33:58.532 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.532 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.532 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:58.532 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794179+0000", 2024-04-24T15:33:58.532 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7970000000000001e-06 2024-04-24T15:33:58.532 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.532 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.532 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:58.532 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834451+0000", 2024-04-24T15:33:58.532 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040271492999999998 2024-04-24T15:33:58.532 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.532 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.532 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:58.532 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834519+0000", 2024-04-24T15:33:58.533 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.7984000000000005e-05 2024-04-24T15:33:58.533 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.533 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.533 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:58.533 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834573+0000", 2024-04-24T15:33:58.533 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.4339999999999998e-05 2024-04-24T15:33:58.533 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:58.533 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:58.533 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:58.533 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.533 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.533 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23629 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15343616~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:58.533 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.794748+0000", 2024-04-24T15:33:58.533 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 83.717697236000006, 2024-04-24T15:33:58.533 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040419718, 2024-04-24T15:33:58.533 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:58.534 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:58.534 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:58.534 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:58.534 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:58.534 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23629 2024-04-24T15:33:58.534 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.534 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:58.534 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.534 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:58.534 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794748+0000", 2024-04-24T15:33:58.534 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:58.534 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.534 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.534 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:58.535 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794748+0000", 2024-04-24T15:33:58.535 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:58.535 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.535 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.535 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:58.535 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794673+0000", 2024-04-24T15:33:58.535 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999251 2024-04-24T15:33:58.535 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.535 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.535 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:58.535 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794751+0000", 2024-04-24T15:33:58.535 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.8480999999999996e-05 2024-04-24T15:33:58.535 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.535 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.535 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:58.536 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794760+0000", 2024-04-24T15:33:58.536 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3429999999999996e-06 2024-04-24T15:33:58.536 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.536 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.536 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:58.536 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794764+0000", 2024-04-24T15:33:58.536 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5600000000000004e-06 2024-04-24T15:33:58.536 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.536 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.536 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:58.536 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835040+0000", 2024-04-24T15:33:58.536 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040275188000000003 2024-04-24T15:33:58.536 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.536 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.536 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:58.537 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835105+0000", 2024-04-24T15:33:58.537 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.5992000000000001e-05 2024-04-24T15:33:58.537 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.537 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.537 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:58.537 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835167+0000", 2024-04-24T15:33:58.537 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.1981000000000002e-05 2024-04-24T15:33:58.537 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:58.537 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:58.537 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:58.537 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.537 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.537 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23630 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15384576~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:58.537 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.794839+0000", 2024-04-24T15:33:58.537 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 83.717606167, 2024-04-24T15:33:58.537 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040523894999999997, 2024-04-24T15:33:58.538 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:58.538 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:58.538 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:58.538 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:58.538 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:58.538 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23630 2024-04-24T15:33:58.538 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.538 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:58.538 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.538 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:58.538 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794839+0000", 2024-04-24T15:33:58.538 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:58.538 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.538 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.539 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:58.539 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794839+0000", 2024-04-24T15:33:58.539 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:58.539 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.539 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.539 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:58.539 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794768+0000", 2024-04-24T15:33:58.539 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999294 2024-04-24T15:33:58.539 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.539 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.539 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:58.539 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794841+0000", 2024-04-24T15:33:58.539 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2793000000000006e-05 2024-04-24T15:33:58.539 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.540 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.540 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:58.540 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794846+0000", 2024-04-24T15:33:58.540 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.7230000000000004e-06 2024-04-24T15:33:58.540 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.540 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.540 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:58.540 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794848+0000", 2024-04-24T15:33:58.540 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7659999999999999e-06 2024-04-24T15:33:58.540 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.540 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.540 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:58.540 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835223+0000", 2024-04-24T15:33:58.540 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040374275000000001 2024-04-24T15:33:58.540 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.541 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.541 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:58.541 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835295+0000", 2024-04-24T15:33:58.541 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2633999999999996e-05 2024-04-24T15:33:58.541 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.541 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.541 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:58.541 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835363+0000", 2024-04-24T15:33:58.541 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.7355999999999997e-05 2024-04-24T15:33:58.541 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:58.541 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:58.541 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:58.541 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.541 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.541 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23631 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15425536~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:58.542 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.794956+0000", 2024-04-24T15:33:58.542 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 83.717489326000006, 2024-04-24T15:33:58.542 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040562383, 2024-04-24T15:33:58.542 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:58.542 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:58.542 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:58.542 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:58.542 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:58.542 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23631 2024-04-24T15:33:58.542 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.542 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:58.542 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.542 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:58.542 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794956+0000", 2024-04-24T15:33:58.543 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:58.543 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.543 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.543 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:58.543 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794956+0000", 2024-04-24T15:33:58.543 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:58.543 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.543 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.543 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:58.543 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794859+0000", 2024-04-24T15:33:58.543 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999032 2024-04-24T15:33:58.543 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.543 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.543 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:58.543 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794959+0000", 2024-04-24T15:33:58.544 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000100207 2024-04-24T15:33:58.544 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.544 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.544 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:58.544 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794967+0000", 2024-04-24T15:33:58.544 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7640000000000005e-06 2024-04-24T15:33:58.544 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.544 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.544 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:58.544 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794971+0000", 2024-04-24T15:33:58.544 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.335e-06 2024-04-24T15:33:58.544 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.544 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.544 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:58.544 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835388+0000", 2024-04-24T15:33:58.545 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040417340000000003 2024-04-24T15:33:58.545 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.545 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.545 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:58.545 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835459+0000", 2024-04-24T15:33:58.545 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.0118999999999997e-05 2024-04-24T15:33:58.545 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.545 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.545 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:58.545 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835518+0000", 2024-04-24T15:33:58.545 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.9438999999999998e-05 2024-04-24T15:33:58.545 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:58.545 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:58.545 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:58.545 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.545 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.546 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23632 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15466496~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:58.546 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795060+0000", 2024-04-24T15:33:58.546 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 83.717384616000004, 2024-04-24T15:33:58.546 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040667993, 2024-04-24T15:33:58.546 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:58.546 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:58.546 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:58.546 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:58.546 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:58.546 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23632 2024-04-24T15:33:58.546 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.546 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:58.546 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.546 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:58.546 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795060+0000", 2024-04-24T15:33:58.547 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:58.547 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.547 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.547 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:58.547 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795060+0000", 2024-04-24T15:33:58.547 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:58.547 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.547 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.547 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:58.547 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794976+0000", 2024-04-24T15:33:58.547 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999156 2024-04-24T15:33:58.547 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.547 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.547 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:58.547 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795064+0000", 2024-04-24T15:33:58.547 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.7709999999999999e-05 2024-04-24T15:33:58.548 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.548 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.548 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:58.548 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795072+0000", 2024-04-24T15:33:58.548 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3769999999999996e-06 2024-04-24T15:33:58.548 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.548 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.548 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:58.548 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795078+0000", 2024-04-24T15:33:58.548 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.5079999999999996e-06 2024-04-24T15:33:58.548 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.548 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.548 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:58.548 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835541+0000", 2024-04-24T15:33:58.548 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040463440000000003 2024-04-24T15:33:58.548 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.549 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.549 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:58.549 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835657+0000", 2024-04-24T15:33:58.549 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00011572499999999999 2024-04-24T15:33:58.549 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.549 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.549 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:58.549 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835728+0000", 2024-04-24T15:33:58.549 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1402000000000005e-05 2024-04-24T15:33:58.549 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:58.549 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:58.549 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:58.549 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.549 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.549 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23633 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15507456~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:58.550 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795134+0000", 2024-04-24T15:33:58.550 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 83.717310960999995, 2024-04-24T15:33:58.550 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040748591000000001, 2024-04-24T15:33:58.550 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:58.550 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:58.550 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:58.550 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:58.550 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:58.550 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23633 2024-04-24T15:33:58.550 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.550 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:58.550 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.550 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:58.550 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795134+0000", 2024-04-24T15:33:58.551 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:58.551 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.551 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.551 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:58.551 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795134+0000", 2024-04-24T15:33:58.551 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:58.551 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.551 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.551 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:58.551 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795083+0000", 2024-04-24T15:33:58.551 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999949 2024-04-24T15:33:58.551 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.551 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.551 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:58.551 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795138+0000", 2024-04-24T15:33:58.552 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.4472000000000003e-05 2024-04-24T15:33:58.552 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.552 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.552 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:58.552 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795145+0000", 2024-04-24T15:33:58.552 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2779999999999996e-06 2024-04-24T15:33:58.552 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.552 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.552 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:58.552 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795149+0000", 2024-04-24T15:33:58.552 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.568e-06 2024-04-24T15:33:58.552 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.552 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.552 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:58.552 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835757+0000", 2024-04-24T15:33:58.553 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040608894999999999 2024-04-24T15:33:58.553 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.553 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.553 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:58.553 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835829+0000", 2024-04-24T15:33:58.553 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1879999999999996e-05 2024-04-24T15:33:58.553 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.553 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.553 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:58.553 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835883+0000", 2024-04-24T15:33:58.553 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.3346000000000001e-05 2024-04-24T15:33:58.553 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:58.553 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:58.553 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:58.553 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.554 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.554 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23634 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15548416~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:58.554 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795271+0000", 2024-04-24T15:33:58.554 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 83.717173927000005, 2024-04-24T15:33:58.554 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040755845999999998, 2024-04-24T15:33:58.554 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:58.554 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:58.554 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:58.554 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:58.554 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:58.554 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23634 2024-04-24T15:33:58.554 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.554 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:58.554 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.554 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:58.554 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795271+0000", 2024-04-24T15:33:58.555 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:58.555 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.555 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.555 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:58.555 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795271+0000", 2024-04-24T15:33:58.555 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:58.555 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.555 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.555 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:58.555 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795152+0000", 2024-04-24T15:33:58.555 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998813 2024-04-24T15:33:58.555 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.555 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.555 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:58.556 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795276+0000", 2024-04-24T15:33:58.556 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00012401799999999999 2024-04-24T15:33:58.556 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.556 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.556 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:58.556 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795281+0000", 2024-04-24T15:33:58.556 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8860000000000003e-06 2024-04-24T15:33:58.556 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.556 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.556 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:58.556 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795284+0000", 2024-04-24T15:33:58.556 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.0450000000000001e-06 2024-04-24T15:33:58.556 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.556 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.556 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:58.557 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835905+0000", 2024-04-24T15:33:58.557 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040621035999999999 2024-04-24T15:33:58.557 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.557 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.557 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:58.557 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835971+0000", 2024-04-24T15:33:58.557 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.5314000000000005e-05 2024-04-24T15:33:58.557 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.557 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.557 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:58.557 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836027+0000", 2024-04-24T15:33:58.557 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6305999999999999e-05 2024-04-24T15:33:58.557 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:58.557 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:58.557 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:58.557 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.558 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.558 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23635 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15589376~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:58.558 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795382+0000", 2024-04-24T15:33:58.558 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 83.717063154000002, 2024-04-24T15:33:58.558 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040805363999999997, 2024-04-24T15:33:58.558 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:58.558 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:58.558 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:58.558 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:58.558 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:58.558 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23635 2024-04-24T15:33:58.558 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.558 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:58.558 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.558 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:58.559 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795382+0000", 2024-04-24T15:33:58.559 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:58.559 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.559 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.559 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:58.559 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795382+0000", 2024-04-24T15:33:58.559 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:58.559 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.559 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.559 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:58.559 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795288+0000", 2024-04-24T15:33:58.559 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999056 2024-04-24T15:33:58.559 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.559 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.559 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:58.559 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795385+0000", 2024-04-24T15:33:58.560 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.7288999999999996e-05 2024-04-24T15:33:58.560 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.560 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.560 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:58.560 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795392+0000", 2024-04-24T15:33:58.560 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.452e-06 2024-04-24T15:33:58.560 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.560 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.560 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:58.560 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795397+0000", 2024-04-24T15:33:58.560 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2860000000000001e-06 2024-04-24T15:33:58.560 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.560 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.560 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:58.560 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836054+0000", 2024-04-24T15:33:58.560 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040657581999999998 2024-04-24T15:33:58.561 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.561 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.561 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:58.561 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836124+0000", 2024-04-24T15:33:58.561 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.0020999999999999e-05 2024-04-24T15:33:58.561 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.561 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.561 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:58.561 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836187+0000", 2024-04-24T15:33:58.561 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.3009999999999995e-05 2024-04-24T15:33:58.561 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:58.561 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:58.561 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:58.561 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.562 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.562 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23636 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15630336~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:58.562 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795610+0000", 2024-04-24T15:33:58.562 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 83.716834796000001, 2024-04-24T15:33:58.562 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040730942999999999, 2024-04-24T15:33:58.562 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:58.562 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:58.562 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:58.562 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:58.562 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:58.562 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23636 2024-04-24T15:33:58.562 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.562 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:58.562 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.563 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:58.563 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795610+0000", 2024-04-24T15:33:58.563 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:58.563 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.563 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.563 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:58.563 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795610+0000", 2024-04-24T15:33:58.563 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:58.563 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.563 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.563 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:58.563 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795401+0000", 2024-04-24T15:33:58.563 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9997911 2024-04-24T15:33:58.563 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.563 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.563 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:58.564 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795614+0000", 2024-04-24T15:33:58.564 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00021228600000000001 2024-04-24T15:33:58.564 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.564 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.564 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:58.564 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795622+0000", 2024-04-24T15:33:58.564 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3920000000000003e-06 2024-04-24T15:33:58.564 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.564 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.564 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:58.564 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795627+0000", 2024-04-24T15:33:58.564 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8199999999999996e-06 2024-04-24T15:33:58.564 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.564 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.564 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:58.565 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836211+0000", 2024-04-24T15:33:58.565 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040583803000000002 2024-04-24T15:33:58.565 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.565 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.565 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:58.565 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836284+0000", 2024-04-24T15:33:58.565 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.3617999999999999e-05 2024-04-24T15:33:58.565 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.565 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.565 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:58.565 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836341+0000", 2024-04-24T15:33:58.565 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6959999999999997e-05 2024-04-24T15:33:58.565 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:58.565 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:58.565 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:58.566 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.566 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.566 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23637 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15671296~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:58.566 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795699+0000", 2024-04-24T15:33:58.566 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 83.716745629000002, 2024-04-24T15:33:58.566 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040775556999999997, 2024-04-24T15:33:58.566 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:58.566 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:58.566 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:58.566 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:58.566 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:58.566 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23637 2024-04-24T15:33:58.566 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.566 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:58.567 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.567 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:58.567 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795699+0000", 2024-04-24T15:33:58.567 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:58.567 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.567 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.567 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:58.567 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795699+0000", 2024-04-24T15:33:58.567 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:58.567 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.567 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.567 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:58.567 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795632+0000", 2024-04-24T15:33:58.567 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999323 2024-04-24T15:33:58.567 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.567 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.568 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:58.568 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795704+0000", 2024-04-24T15:33:58.568 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2428000000000005e-05 2024-04-24T15:33:58.568 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.568 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.568 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:58.568 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795715+0000", 2024-04-24T15:33:58.568 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.1532000000000001e-05 2024-04-24T15:33:58.568 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.568 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.568 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:58.568 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795721+0000", 2024-04-24T15:33:58.568 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.8200000000000002e-06 2024-04-24T15:33:58.568 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.568 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.569 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:58.569 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836364+0000", 2024-04-24T15:33:58.569 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040642997 2024-04-24T15:33:58.569 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.569 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.569 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:58.569 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836435+0000", 2024-04-24T15:33:58.569 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.0821000000000005e-05 2024-04-24T15:33:58.569 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.569 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.569 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:58.569 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836475+0000", 2024-04-24T15:33:58.569 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9796999999999999e-05 2024-04-24T15:33:58.569 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:58.569 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:58.570 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:58.570 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.570 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.570 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23638 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15712256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:58.570 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795932+0000", 2024-04-24T15:33:58.570 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 83.716512598999998, 2024-04-24T15:33:58.570 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040668682999999997, 2024-04-24T15:33:58.570 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:58.570 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:58.570 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:58.570 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:58.570 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:58.570 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23638 2024-04-24T15:33:58.570 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.570 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:58.571 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.571 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:58.571 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795932+0000", 2024-04-24T15:33:58.571 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:58.571 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.571 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.571 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:58.571 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795932+0000", 2024-04-24T15:33:58.571 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:58.571 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.571 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.571 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:58.571 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795728+0000", 2024-04-24T15:33:58.571 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9997959 2024-04-24T15:33:58.571 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.572 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.572 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:58.572 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795936+0000", 2024-04-24T15:33:58.572 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000208128 2024-04-24T15:33:58.572 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.572 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.572 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:58.572 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795945+0000", 2024-04-24T15:33:58.572 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.7110000000000001e-06 2024-04-24T15:33:58.572 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.572 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.572 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:58.572 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795957+0000", 2024-04-24T15:33:58.572 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.1800999999999999e-05 2024-04-24T15:33:58.572 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.573 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.573 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:58.573 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836505+0000", 2024-04-24T15:33:58.573 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040547622999999998 2024-04-24T15:33:58.573 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.573 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.573 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:58.573 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836561+0000", 2024-04-24T15:33:58.573 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6598999999999998e-05 2024-04-24T15:33:58.573 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.573 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.573 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:58.573 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836601+0000", 2024-04-24T15:33:58.573 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9907000000000003e-05 2024-04-24T15:33:58.573 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:58.574 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:58.574 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:58.574 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.574 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.574 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23639 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15753216~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:58.574 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.796071+0000", 2024-04-24T15:33:58.574 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 83.716374337999994, 2024-04-24T15:33:58.574 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040648199000000003, 2024-04-24T15:33:58.574 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:58.574 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:58.574 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:58.574 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:58.574 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:58.574 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23639 2024-04-24T15:33:58.575 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.575 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:58.575 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.575 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:58.575 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796071+0000", 2024-04-24T15:33:58.575 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:58.575 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.575 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.575 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:58.575 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796071+0000", 2024-04-24T15:33:58.575 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:58.575 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.575 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.575 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:58.575 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795967+0000", 2024-04-24T15:33:58.575 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998965 2024-04-24T15:33:58.576 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.576 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.576 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:58.576 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796074+0000", 2024-04-24T15:33:58.576 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000107013 2024-04-24T15:33:58.576 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.576 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.576 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:58.576 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796081+0000", 2024-04-24T15:33:58.576 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2590000000000003e-06 2024-04-24T15:33:58.576 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.576 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.576 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:58.576 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796085+0000", 2024-04-24T15:33:58.576 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5269999999999999e-06 2024-04-24T15:33:58.577 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.577 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.577 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:58.577 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836624+0000", 2024-04-24T15:33:58.577 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040539324000000002 2024-04-24T15:33:58.577 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.577 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.577 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:58.577 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836667+0000", 2024-04-24T15:33:58.577 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2562999999999999e-05 2024-04-24T15:33:58.577 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.577 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.577 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:58.577 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836719+0000", 2024-04-24T15:33:58.578 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.2015999999999998e-05 2024-04-24T15:33:58.578 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:58.578 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:58.578 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:58.578 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.578 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.578 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23640 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15794176~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:58.578 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.796492+0000", 2024-04-24T15:33:58.578 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 83.715952580000007, 2024-04-24T15:33:58.578 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040420000999999997, 2024-04-24T15:33:58.578 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:58.578 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:58.578 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:58.578 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:58.579 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:58.579 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23640 2024-04-24T15:33:58.579 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.579 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:58.579 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.579 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:58.579 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796492+0000", 2024-04-24T15:33:58.579 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:58.579 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.579 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.579 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:58.579 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796492+0000", 2024-04-24T15:33:58.579 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:58.579 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.580 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.580 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:58.580 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796088+0000", 2024-04-24T15:33:58.580 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9995961 2024-04-24T15:33:58.580 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.580 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.580 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:58.580 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796496+0000", 2024-04-24T15:33:58.580 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00040785700000000002 2024-04-24T15:33:58.580 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.580 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.580 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:58.580 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796503+0000", 2024-04-24T15:33:58.580 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1049999999999997e-06 2024-04-24T15:33:58.580 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.581 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.581 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:58.581 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796508+0000", 2024-04-24T15:33:58.581 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.155e-06 2024-04-24T15:33:58.581 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.581 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.581 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:58.581 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836756+0000", 2024-04-24T15:33:58.581 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040248850000000003 2024-04-24T15:33:58.581 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.581 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.581 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:58.581 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836841+0000", 2024-04-24T15:33:58.581 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.4735999999999996e-05 2024-04-24T15:33:58.582 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.582 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.582 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:58.582 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836912+0000", 2024-04-24T15:33:58.582 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1229999999999994e-05 2024-04-24T15:33:58.582 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:58.582 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:58.582 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:58.582 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.582 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.582 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23641 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15835136~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:58.582 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.796658+0000", 2024-04-24T15:33:58.582 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 83.715786503999993, 2024-04-24T15:33:58.583 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040420706000000001, 2024-04-24T15:33:58.583 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:58.583 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:58.583 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:58.583 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:58.583 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:58.583 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23641 2024-04-24T15:33:58.583 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.583 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:58.583 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.583 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:58.583 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796658+0000", 2024-04-24T15:33:58.583 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:58.583 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.584 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.584 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:58.584 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796658+0000", 2024-04-24T15:33:58.584 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:58.584 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.584 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.584 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:58.584 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796512+0000", 2024-04-24T15:33:58.584 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998531 2024-04-24T15:33:58.584 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.584 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.584 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:58.584 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796669+0000", 2024-04-24T15:33:58.584 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000157577 2024-04-24T15:33:58.585 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.585 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.585 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:58.585 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796677+0000", 2024-04-24T15:33:58.585 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7800000000000001e-06 2024-04-24T15:33:58.585 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.585 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.585 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:58.585 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796681+0000", 2024-04-24T15:33:58.585 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.721e-06 2024-04-24T15:33:58.585 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.585 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.585 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:58.585 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836941+0000", 2024-04-24T15:33:58.585 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040259907999999997 2024-04-24T15:33:58.586 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.586 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.586 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:58.586 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837017+0000", 2024-04-24T15:33:58.586 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.6071999999999999e-05 2024-04-24T15:33:58.586 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.586 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.586 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:58.586 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837079+0000", 2024-04-24T15:33:58.586 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.2358000000000002e-05 2024-04-24T15:33:58.586 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:58.586 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:58.586 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:58.586 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.587 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.587 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23645 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15998976~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:58.587 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.797360+0000", 2024-04-24T15:33:58.587 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 83.715085005999995, 2024-04-24T15:33:58.587 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040387645999999999, 2024-04-24T15:33:58.587 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:58.587 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:58.587 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:58.587 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:58.587 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:58.587 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23645 2024-04-24T15:33:58.587 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.587 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:58.587 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.588 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:58.588 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797360+0000", 2024-04-24T15:33:58.588 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:58.588 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.588 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.588 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:58.588 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797360+0000", 2024-04-24T15:33:58.588 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:58.588 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.588 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.588 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:58.588 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797288+0000", 2024-04-24T15:33:58.588 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999928 2024-04-24T15:33:58.588 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.588 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.589 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:58.589 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797364+0000", 2024-04-24T15:33:58.589 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.5347999999999998e-05 2024-04-24T15:33:58.589 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.589 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.589 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:58.589 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797372+0000", 2024-04-24T15:33:58.589 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.2409999999999995e-06 2024-04-24T15:33:58.589 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.589 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.589 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:58.589 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797376+0000", 2024-04-24T15:33:58.589 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.515e-06 2024-04-24T15:33:58.589 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.589 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.590 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:58.590 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837586+0000", 2024-04-24T15:33:58.590 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040210047999999998 2024-04-24T15:33:58.590 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.590 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.590 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:58.590 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837672+0000", 2024-04-24T15:33:58.590 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.5997999999999999e-05 2024-04-24T15:33:58.590 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.590 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.590 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:58.590 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837748+0000", 2024-04-24T15:33:58.590 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.5328999999999997e-05 2024-04-24T15:33:58.591 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:58.591 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:58.591 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:58.591 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.591 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.591 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23646 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 16039936~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:33:58.591 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.797498+0000", 2024-04-24T15:33:58.591 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 83.714946905000005, 2024-04-24T15:33:58.591 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040433139, 2024-04-24T15:33:58.591 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:33:58.591 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:33:58.591 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:33:58.591 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:33:58.591 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:33:58.591 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23646 2024-04-24T15:33:58.591 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.592 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:33:58.592 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.592 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:33:58.592 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797498+0000", 2024-04-24T15:33:58.592 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:58.592 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.592 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.592 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:33:58.592 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797498+0000", 2024-04-24T15:33:58.592 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:33:58.592 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.592 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.592 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:33:58.592 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797381+0000", 2024-04-24T15:33:58.593 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998832 2024-04-24T15:33:58.593 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.593 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.593 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:33:58.593 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797501+0000", 2024-04-24T15:33:58.593 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000119759 2024-04-24T15:33:58.593 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.593 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.593 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:33:58.593 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797510+0000", 2024-04-24T15:33:58.593 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.4770000000000003e-06 2024-04-24T15:33:58.593 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.593 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.593 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:33:58.593 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797515+0000", 2024-04-24T15:33:58.594 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8330000000000001e-06 2024-04-24T15:33:58.594 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.594 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.594 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:33:58.594 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837777+0000", 2024-04-24T15:33:58.594 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040262502999999998 2024-04-24T15:33:58.594 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.594 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.594 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:33:58.594 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837860+0000", 2024-04-24T15:33:58.594 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3002000000000003e-05 2024-04-24T15:33:58.594 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:33:58.594 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:33:58.595 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:33:58.595 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837931+0000", 2024-04-24T15:33:58.595 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1222000000000004e-05 2024-04-24T15:33:58.595 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:58.595 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:58.595 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:58.595 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:33:58.595 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:33:58.595 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:33:58.597 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-04-24T15:33:58.597 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-04-24T15:33:58.597 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-04-24T15:33:58.597 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-04-24T15:33:58.597 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:33:58.597+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:33:58.597 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-24T15:33:58.614 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-04-24T15:33:58.632 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-04-24T15:33:58.655 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-04-24T15:33:58.676 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-04-24T15:33:58.677 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T15:33:58.677 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T15:33:58.677 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T15:33:58.677 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T15:33:58.678 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T15:33:58.678 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T15:33:58.678 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T15:33:58.678 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T15:33:58.697 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:33:58.699+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:33:58.713 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-24T15:33:58.732 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:58.732 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:58.743 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-24T15:33:58.798 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:58.799+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:58.812 INFO:teuthology.orchestra.run.smithi096.stdout:moved 0 / 90 (0%) 2024-04-24T15:33:58.812 INFO:teuthology.orchestra.run.smithi096.stdout:avg 15 2024-04-24T15:33:58.812 INFO:teuthology.orchestra.run.smithi096.stdout:stddev 2.38048 -> 2.38048 (expected baseline 3.53553) 2024-04-24T15:33:58.812 INFO:teuthology.orchestra.run.smithi096.stdout:min osd.2 with 10 -> 10 pgs (0.666667 -> 0.666667 * mean) 2024-04-24T15:33:58.812 INFO:teuthology.orchestra.run.smithi096.stdout:max osd.3 with 17 -> 17 pgs (1.13333 -> 1.13333 * mean) 2024-04-24T15:33:58.812 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:33:58.812 INFO:teuthology.orchestra.run.smithi096.stdout:oload 1000 2024-04-24T15:33:58.812 INFO:teuthology.orchestra.run.smithi096.stdout:max_change 3 2024-04-24T15:33:58.812 INFO:teuthology.orchestra.run.smithi096.stdout:max_change_osds 4 2024-04-24T15:33:58.813 INFO:teuthology.orchestra.run.smithi096.stdout:average_utilization 0.0032 2024-04-24T15:33:58.813 INFO:teuthology.orchestra.run.smithi096.stdout:overload_utilization 0.0325 2024-04-24T15:33:58.815 INFO:teuthology.orchestra.run.smithi096.stderr:no change 2024-04-24T15:33:58.827 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-04-24T15:33:58.827 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 3.0 2024-04-24T15:33:58.907 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:58.908 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:58.910 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:33:58.912+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:33:58.917 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-24T15:33:58.999 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:33:59.001+0000 7f0f45efb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 59992) UID: 0 2024-04-24T15:33:59.042 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:59.042 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:59.052 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-24T15:33:59.100 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:33:59.101+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:33:59.201 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:33:59.202+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:33:59.217 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:59.217 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:59.222 INFO:teuthology.orchestra.run.smithi096.stdout:moved 0 / 90 (0%) 2024-04-24T15:33:59.222 INFO:teuthology.orchestra.run.smithi096.stdout:avg 15 2024-04-24T15:33:59.222 INFO:teuthology.orchestra.run.smithi096.stdout:stddev 2.38048 -> 2.38048 (expected baseline 3.53553) 2024-04-24T15:33:59.222 INFO:teuthology.orchestra.run.smithi096.stdout:min osd.2 with 10 -> 10 pgs (0.666667 -> 0.666667 * mean) 2024-04-24T15:33:59.222 INFO:teuthology.orchestra.run.smithi096.stdout:max osd.3 with 17 -> 17 pgs (1.13333 -> 1.13333 * mean) 2024-04-24T15:33:59.222 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:33:59.222 INFO:teuthology.orchestra.run.smithi096.stdout:oload 110 2024-04-24T15:33:59.222 INFO:teuthology.orchestra.run.smithi096.stdout:max_change 3 2024-04-24T15:33:59.222 INFO:teuthology.orchestra.run.smithi096.stdout:max_change_osds 4 2024-04-24T15:33:59.222 INFO:teuthology.orchestra.run.smithi096.stdout:average_utilization 0.0032 2024-04-24T15:33:59.222 INFO:teuthology.orchestra.run.smithi096.stdout:overload_utilization 0.0036 2024-04-24T15:33:59.225 INFO:teuthology.orchestra.run.smithi096.stderr:no change 2024-04-24T15:33:59.227 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-24T15:33:59.235 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '1000', 'type': 'reweight-by-utilization'} 2024-04-24T15:33:59.235 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 3.0 2024-04-24T15:33:59.319 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:33:59.321+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:33:59.349 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:59.349 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:59.359 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-24T15:33:59.402 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:33:59.404+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:33:59.483 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:59.483 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:59.492 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-24T15:33:59.535 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:33:59.537+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:33:59.593 INFO:teuthology.orchestra.run.smithi096.stdout:moved 0 / 90 (0%) 2024-04-24T15:33:59.593 INFO:teuthology.orchestra.run.smithi096.stdout:avg 15 2024-04-24T15:33:59.593 INFO:teuthology.orchestra.run.smithi096.stdout:stddev 2.38048 -> 2.38048 (expected baseline 3.53553) 2024-04-24T15:33:59.593 INFO:teuthology.orchestra.run.smithi096.stdout:min osd.2 with 10 -> 10 pgs (0.666667 -> 0.666667 * mean) 2024-04-24T15:33:59.593 INFO:teuthology.orchestra.run.smithi096.stdout:max osd.3 with 17 -> 17 pgs (1.13333 -> 1.13333 * mean) 2024-04-24T15:33:59.593 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:33:59.593 INFO:teuthology.orchestra.run.smithi096.stdout:oload 1000 2024-04-24T15:33:59.593 INFO:teuthology.orchestra.run.smithi096.stdout:max_change 3 2024-04-24T15:33:59.593 INFO:teuthology.orchestra.run.smithi096.stdout:max_change_osds 4 2024-04-24T15:33:59.593 INFO:teuthology.orchestra.run.smithi096.stdout:average_utilization 0.0032 2024-04-24T15:33:59.594 INFO:teuthology.orchestra.run.smithi096.stdout:overload_utilization 0.0324 2024-04-24T15:33:59.596 INFO:teuthology.orchestra.run.smithi096.stderr:no change 2024-04-24T15:33:59.604 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:59.606+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:59.619 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:59.619 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:59.629 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-24T15:33:59.705 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:59.707+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:59.721 INFO:teuthology.orchestra.run.smithi096.stderr:noscrub is unset 2024-04-24T15:33:59.806 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:33:59.808+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:33:59.813 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:59.814 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:59.824 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-24T15:33:59.908 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:33:59.909+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:33:59.988 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:33:59.988 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:33:59.998 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-24T15:34:00.009 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:34:00.010+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:34:00.109 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:34:00.111+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:34:00.121 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:34:00.121 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:34:00.131 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-24T15:34:00.210 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:34:00.211+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:34:00.253 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:34:00.253 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:34:00.263 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-24T15:34:00.311 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:34:00.312+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:34:00.387 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:34:00.387 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:34:00.397 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-24T15:34:00.438 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:34:00.439+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:34:00.512 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:34:00.514+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:34:00.560 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:34:00.560 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:34:00.570 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-24T15:34:00.613 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:34:00.615+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:34:00.690 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-24T15:34:00.692 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:34:00.692 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:34:00.703 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-24T15:34:00.744 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:34:00.745+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:34:00.815 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:34:00.817+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:34:00.845 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-04-24T15:34:00.865 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-04-24T15:34:00.884 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-04-24T15:34:00.888 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:34:00.888 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-04-24T15:34:00.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-04-24T15:34:00.888 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-04-24T15:34:00.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.889 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:2522 4.6 4:60ceedf0:::rbd_data.114a1f60cb66.000000000000011d:head [set-alloc-hint object_size 262144 write_size 262144,write 110592~2048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-24T15:34:00.889 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:23.088240+0000", 2024-04-24T15:34:00.889 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 97.800446105999995, 2024-04-24T15:34:00.889 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.988232797, 2024-04-24T15:34:00.889 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:34:00.889 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:34:00.889 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:34:00.889 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:34:00.889 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:00.889 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2522 2024-04-24T15:34:00.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.889 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:34:00.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.890 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:34:00.890 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088240+0000", 2024-04-24T15:34:00.890 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:00.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.890 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.890 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:34:00.890 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088240+0000", 2024-04-24T15:34:00.890 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:00.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.890 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.890 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:34:00.890 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088238+0000", 2024-04-24T15:34:00.890 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999981 2024-04-24T15:34:00.891 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.891 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.891 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:34:00.891 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088254+0000", 2024-04-24T15:34:00.891 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5562000000000001e-05 2024-04-24T15:34:00.891 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.891 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.891 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:34:00.891 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088275+0000", 2024-04-24T15:34:00.891 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1256000000000001e-05 2024-04-24T15:34:00.891 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.891 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.891 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:34:00.891 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088283+0000", 2024-04-24T15:34:00.892 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3219999999999993e-06 2024-04-24T15:34:00.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.892 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.892 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:34:00.892 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088351+0000", 2024-04-24T15:34:00.892 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7595000000000006e-05 2024-04-24T15:34:00.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.892 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.892 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-04-24T15:34:00.892 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:23.088362+0000", 2024-04-24T15:34:00.892 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1764e-05 2024-04-24T15:34:00.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.892 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.892 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:34:00.893 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.073748+0000", 2024-04-24T15:34:00.893 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98538541899999998 2024-04-24T15:34:00.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.893 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.893 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:34:00.893 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.073925+0000", 2024-04-24T15:34:00.893 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017747600000000001 2024-04-24T15:34:00.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.893 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.893 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-24T15:34:00.893 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.074092+0000", 2024-04-24T15:34:00.893 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000166622 2024-04-24T15:34:00.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.893 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.893 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-24T15:34:00.894 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.074774+0000", 2024-04-24T15:34:00.894 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00068225000000000002 2024-04-24T15:34:00.894 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.894 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.894 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:34:00.894 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.076412+0000", 2024-04-24T15:34:00.894 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001638189 2024-04-24T15:34:00.894 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.894 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.894 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-24T15:34:00.894 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.076446+0000", 2024-04-24T15:34:00.894 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3587000000000002e-05 2024-04-24T15:34:00.894 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.894 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.895 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:34:00.895 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:24.076473+0000", 2024-04-24T15:34:00.895 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6886999999999999e-05 2024-04-24T15:34:00.895 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:00.895 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:34:00.895 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:00.895 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.895 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.895 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43121 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 5865472~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:34:00.895 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094178+0000", 2024-04-24T15:34:00.895 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 53.794508423000003, 2024-04-24T15:34:00.895 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032860598999999997, 2024-04-24T15:34:00.895 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:34:00.895 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:34:00.895 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:34:00.896 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:34:00.896 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:00.896 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43121 2024-04-24T15:34:00.896 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.896 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:34:00.896 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.896 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:34:00.896 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094178+0000", 2024-04-24T15:34:00.896 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:00.896 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.896 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.896 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:34:00.896 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094178+0000", 2024-04-24T15:34:00.896 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:00.896 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.897 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.897 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:34:00.897 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094178+0000", 2024-04-24T15:34:00.897 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:00.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.897 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.897 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:34:00.897 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094180+0000", 2024-04-24T15:34:00.897 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9989999999999998e-06 2024-04-24T15:34:00.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.897 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.897 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:34:00.897 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094185+0000", 2024-04-24T15:34:00.897 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.541e-06 2024-04-24T15:34:00.898 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.898 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.898 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:34:00.898 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094189+0000", 2024-04-24T15:34:00.898 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0509999999999998e-06 2024-04-24T15:34:00.898 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.898 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.898 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:34:00.898 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.126877+0000", 2024-04-24T15:34:00.898 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032687473000000002 2024-04-24T15:34:00.898 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.898 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.898 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:34:00.898 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.126952+0000", 2024-04-24T15:34:00.899 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5924e-05 2024-04-24T15:34:00.899 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.899 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.899 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:34:00.899 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127038+0000", 2024-04-24T15:34:00.899 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5908999999999994e-05 2024-04-24T15:34:00.899 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:00.899 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:34:00.899 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:00.899 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.899 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.899 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43122 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 6291456~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:34:00.899 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094193+0000", 2024-04-24T15:34:00.899 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 53.794493179, 2024-04-24T15:34:00.899 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032978383999999999, 2024-04-24T15:34:00.899 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:34:00.900 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:34:00.900 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:34:00.900 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:34:00.900 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:00.900 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43122 2024-04-24T15:34:00.900 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.900 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:34:00.900 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.900 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:34:00.900 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094193+0000", 2024-04-24T15:34:00.900 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:00.900 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.900 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.900 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:34:00.900 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094193+0000", 2024-04-24T15:34:00.900 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:00.901 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.901 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.901 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:34:00.901 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094193+0000", 2024-04-24T15:34:00.901 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:00.901 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.901 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.901 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:34:00.901 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094195+0000", 2024-04-24T15:34:00.901 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3269999999999999e-06 2024-04-24T15:34:00.901 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.901 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.901 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:34:00.901 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094201+0000", 2024-04-24T15:34:00.901 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9850000000000003e-06 2024-04-24T15:34:00.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.902 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.902 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:34:00.902 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094205+0000", 2024-04-24T15:34:00.902 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8430000000000003e-06 2024-04-24T15:34:00.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.902 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.902 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:34:00.902 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127068+0000", 2024-04-24T15:34:00.902 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032863305000000002 2024-04-24T15:34:00.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.902 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.902 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:34:00.903 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127113+0000", 2024-04-24T15:34:00.903 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4412000000000001e-05 2024-04-24T15:34:00.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.903 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:34:00.903 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127171+0000", 2024-04-24T15:34:00.903 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8807999999999997e-05 2024-04-24T15:34:00.903 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:00.903 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:34:00.903 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:00.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.903 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43123 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 6717440~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:34:00.903 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094209+0000", 2024-04-24T15:34:00.903 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 53.794477012999998, 2024-04-24T15:34:00.903 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033144583999999998, 2024-04-24T15:34:00.904 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:34:00.904 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:34:00.904 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:34:00.904 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:34:00.904 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:00.904 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43123 2024-04-24T15:34:00.904 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.904 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:34:00.904 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.904 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:34:00.904 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094209+0000", 2024-04-24T15:34:00.904 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:00.904 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.905 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.905 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:34:00.905 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094209+0000", 2024-04-24T15:34:00.905 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:00.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.905 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.905 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:34:00.905 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094209+0000", 2024-04-24T15:34:00.905 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:00.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.905 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.905 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:34:00.905 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094211+0000", 2024-04-24T15:34:00.905 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2089999999999999e-06 2024-04-24T15:34:00.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.905 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.906 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:34:00.906 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094217+0000", 2024-04-24T15:34:00.906 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6799999999999998e-06 2024-04-24T15:34:00.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.906 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.906 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:34:00.906 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094220+0000", 2024-04-24T15:34:00.906 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.444e-06 2024-04-24T15:34:00.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.906 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.906 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:34:00.906 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127190+0000", 2024-04-24T15:34:00.906 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032969471 2024-04-24T15:34:00.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.907 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:34:00.907 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127269+0000", 2024-04-24T15:34:00.907 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9246999999999995e-05 2024-04-24T15:34:00.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.907 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:34:00.907 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127354+0000", 2024-04-24T15:34:00.907 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4853999999999998e-05 2024-04-24T15:34:00.907 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:00.907 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:34:00.907 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:00.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.908 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43124 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 7143424~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:34:00.908 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094224+0000", 2024-04-24T15:34:00.908 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 53.794462039999999, 2024-04-24T15:34:00.908 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033271955999999998, 2024-04-24T15:34:00.908 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:34:00.908 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:34:00.908 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:34:00.908 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:34:00.908 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:00.908 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43124 2024-04-24T15:34:00.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.908 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:34:00.908 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.908 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:34:00.909 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094224+0000", 2024-04-24T15:34:00.909 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:00.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.909 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.909 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:34:00.909 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094224+0000", 2024-04-24T15:34:00.909 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:00.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.909 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.909 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:34:00.909 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094224+0000", 2024-04-24T15:34:00.909 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:00.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.909 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.909 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:34:00.909 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094226+0000", 2024-04-24T15:34:00.910 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9039999999999999e-06 2024-04-24T15:34:00.910 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.910 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.910 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:34:00.910 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094233+0000", 2024-04-24T15:34:00.910 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7970000000000003e-06 2024-04-24T15:34:00.910 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.910 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.910 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:34:00.910 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094236+0000", 2024-04-24T15:34:00.910 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6339999999999999e-06 2024-04-24T15:34:00.910 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.910 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.910 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:34:00.911 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127390+0000", 2024-04-24T15:34:00.911 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.03315353 2024-04-24T15:34:00.911 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.911 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.911 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:34:00.911 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127439+0000", 2024-04-24T15:34:00.911 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8890000000000001e-05 2024-04-24T15:34:00.911 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.911 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.911 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:34:00.911 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127496+0000", 2024-04-24T15:34:00.911 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7476999999999999e-05 2024-04-24T15:34:00.911 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:00.911 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:34:00.911 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:00.911 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.912 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.912 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43125 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 7569408~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:34:00.912 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094240+0000", 2024-04-24T15:34:00.912 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 53.794445953, 2024-04-24T15:34:00.912 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033446278000000003, 2024-04-24T15:34:00.912 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:34:00.912 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:34:00.912 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:34:00.912 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:34:00.912 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:00.912 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43125 2024-04-24T15:34:00.912 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.912 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:34:00.912 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.912 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:34:00.913 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094240+0000", 2024-04-24T15:34:00.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:00.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.913 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.913 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:34:00.913 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094240+0000", 2024-04-24T15:34:00.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:00.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.913 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.913 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:34:00.913 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094240+0000", 2024-04-24T15:34:00.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:00.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.913 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.913 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:34:00.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094247+0000", 2024-04-24T15:34:00.914 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6209999999999998e-06 2024-04-24T15:34:00.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.914 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.914 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:34:00.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094253+0000", 2024-04-24T15:34:00.914 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0739999999999998e-06 2024-04-24T15:34:00.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.914 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.914 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:34:00.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094256+0000", 2024-04-24T15:34:00.914 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7699999999999999e-06 2024-04-24T15:34:00.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.914 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.915 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:34:00.915 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127526+0000", 2024-04-24T15:34:00.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033269703999999997 2024-04-24T15:34:00.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.915 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:34:00.915 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127603+0000", 2024-04-24T15:34:00.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6964000000000004e-05 2024-04-24T15:34:00.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.915 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:34:00.915 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127687+0000", 2024-04-24T15:34:00.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3431999999999997e-05 2024-04-24T15:34:00.915 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:00.915 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:34:00.916 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:00.916 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.916 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.916 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:43126 4.4 4:242f03d0:::rbd_data.114af25c4d4b.0000000000000003:15 [read 7995392~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T15:34:00.916 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:07.094261+0000", 2024-04-24T15:34:00.916 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 53.794425449999999, 2024-04-24T15:34:00.916 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033555201999999999, 2024-04-24T15:34:00.916 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:34:00.916 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:34:00.916 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:34:00.916 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:34:00.916 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:00.916 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43126 2024-04-24T15:34:00.916 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.916 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:34:00.916 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.917 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:34:00.917 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094261+0000", 2024-04-24T15:34:00.917 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:00.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.917 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.917 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:34:00.917 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094261+0000", 2024-04-24T15:34:00.917 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:00.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.917 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.917 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:34:00.917 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094260+0000", 2024-04-24T15:34:00.917 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:00.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.917 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.918 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:34:00.918 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094263+0000", 2024-04-24T15:34:00.918 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2579999999999998e-06 2024-04-24T15:34:00.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.918 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.918 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:34:00.918 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094269+0000", 2024-04-24T15:34:00.918 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.823e-06 2024-04-24T15:34:00.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.918 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.918 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:34:00.918 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.094272+0000", 2024-04-24T15:34:00.918 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8999999999999999e-06 2024-04-24T15:34:00.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.919 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:34:00.919 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127720+0000", 2024-04-24T15:34:00.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033447521000000001 2024-04-24T15:34:00.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.919 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:34:00.919 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127765+0000", 2024-04-24T15:34:00.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5352000000000002e-05 2024-04-24T15:34:00.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.919 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:34:00.919 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:07.127816+0000", 2024-04-24T15:34:00.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0689999999999997e-05 2024-04-24T15:34:00.919 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:00.919 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:34:00.920 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:00.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.920 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.920 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63800 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1032192~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:34:00.920 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555407+0000", 2024-04-24T15:34:00.920 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 41.333279464, 2024-04-24T15:34:00.920 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032809030000000003, 2024-04-24T15:34:00.920 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:34:00.920 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:34:00.920 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:34:00.920 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:34:00.920 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:00.920 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63800 2024-04-24T15:34:00.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.920 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:34:00.920 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.921 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:34:00.921 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555407+0000", 2024-04-24T15:34:00.921 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:00.921 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.921 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.921 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:34:00.921 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555407+0000", 2024-04-24T15:34:00.921 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:00.921 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.921 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.921 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:34:00.921 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555406+0000", 2024-04-24T15:34:00.921 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:00.921 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.921 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.922 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:34:00.922 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555409+0000", 2024-04-24T15:34:00.922 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.706e-06 2024-04-24T15:34:00.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.922 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.922 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:34:00.922 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555416+0000", 2024-04-24T15:34:00.922 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1609999999999997e-06 2024-04-24T15:34:00.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.922 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.922 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:34:00.922 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555420+0000", 2024-04-24T15:34:00.922 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0369999999999996e-06 2024-04-24T15:34:00.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:34:00.923 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588088+0000", 2024-04-24T15:34:00.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032668228000000001 2024-04-24T15:34:00.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:34:00.923 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588143+0000", 2024-04-24T15:34:00.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4852999999999998e-05 2024-04-24T15:34:00.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:34:00.923 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588216+0000", 2024-04-24T15:34:00.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2467999999999998e-05 2024-04-24T15:34:00.923 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:00.924 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:34:00.924 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:00.924 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.924 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.924 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63801 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1114112~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:34:00.924 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555425+0000", 2024-04-24T15:34:00.924 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 41.333261557, 2024-04-24T15:34:00.924 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032924691999999998, 2024-04-24T15:34:00.924 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:34:00.924 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:34:00.924 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:34:00.924 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:34:00.924 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:00.924 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63801 2024-04-24T15:34:00.924 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.925 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:34:00.925 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.925 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:34:00.925 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555425+0000", 2024-04-24T15:34:00.925 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:00.925 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.925 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.925 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:34:00.925 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555425+0000", 2024-04-24T15:34:00.925 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:00.925 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.925 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.925 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:34:00.925 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555424+0000", 2024-04-24T15:34:00.925 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:00.925 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.926 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:34:00.926 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555427+0000", 2024-04-24T15:34:00.926 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2350000000000002e-06 2024-04-24T15:34:00.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.926 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:34:00.926 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555432+0000", 2024-04-24T15:34:00.926 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.434e-06 2024-04-24T15:34:00.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.926 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:34:00.926 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555435+0000", 2024-04-24T15:34:00.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2009999999999999e-06 2024-04-24T15:34:00.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.927 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:34:00.927 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588237+0000", 2024-04-24T15:34:00.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032801508 2024-04-24T15:34:00.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.927 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:34:00.927 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588281+0000", 2024-04-24T15:34:00.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4100999999999997e-05 2024-04-24T15:34:00.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.927 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:34:00.927 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588349+0000", 2024-04-24T15:34:00.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8551000000000001e-05 2024-04-24T15:34:00.928 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:00.928 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:34:00.928 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:00.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.928 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63802 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1196032~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:34:00.928 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555439+0000", 2024-04-24T15:34:00.928 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 41.333247036000003, 2024-04-24T15:34:00.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033057744999999999, 2024-04-24T15:34:00.928 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:34:00.928 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:34:00.928 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:34:00.928 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:34:00.928 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:00.929 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63802 2024-04-24T15:34:00.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.929 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:34:00.929 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.929 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:34:00.929 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555439+0000", 2024-04-24T15:34:00.929 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:00.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.929 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.929 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:34:00.929 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555439+0000", 2024-04-24T15:34:00.929 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:00.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.929 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.929 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:34:00.930 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555439+0000", 2024-04-24T15:34:00.930 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:00.930 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.930 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.930 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:34:00.930 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555445+0000", 2024-04-24T15:34:00.930 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4019999999999999e-06 2024-04-24T15:34:00.930 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.930 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.930 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:34:00.930 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555452+0000", 2024-04-24T15:34:00.930 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8600000000000004e-06 2024-04-24T15:34:00.930 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.930 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.931 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:34:00.931 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555461+0000", 2024-04-24T15:34:00.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0499999999999997e-06 2024-04-24T15:34:00.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.931 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:34:00.931 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588378+0000", 2024-04-24T15:34:00.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032916527000000001 2024-04-24T15:34:00.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.931 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:34:00.931 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588423+0000", 2024-04-24T15:34:00.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5092999999999997e-05 2024-04-24T15:34:00.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.931 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:34:00.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588497+0000", 2024-04-24T15:34:00.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4140999999999994e-05 2024-04-24T15:34:00.932 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:00.932 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:34:00.932 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:00.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.932 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.932 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63803 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1277952~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:34:00.932 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555466+0000", 2024-04-24T15:34:00.932 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 41.333220091000001, 2024-04-24T15:34:00.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033162297, 2024-04-24T15:34:00.932 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:34:00.932 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:34:00.932 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:34:00.932 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:34:00.933 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:00.933 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63803 2024-04-24T15:34:00.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.933 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:34:00.933 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.933 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:34:00.933 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555466+0000", 2024-04-24T15:34:00.933 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:00.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.933 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.933 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:34:00.933 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555466+0000", 2024-04-24T15:34:00.933 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:00.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.933 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.933 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:34:00.934 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555466+0000", 2024-04-24T15:34:00.934 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:00.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.934 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.934 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:34:00.934 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555468+0000", 2024-04-24T15:34:00.934 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4040000000000002e-06 2024-04-24T15:34:00.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.934 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.934 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:34:00.934 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555475+0000", 2024-04-24T15:34:00.934 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9480000000000002e-06 2024-04-24T15:34:00.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.934 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.934 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:34:00.935 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555479+0000", 2024-04-24T15:34:00.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0289999999999999e-06 2024-04-24T15:34:00.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.935 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:34:00.935 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588516+0000", 2024-04-24T15:34:00.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033037094000000003 2024-04-24T15:34:00.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.935 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:34:00.935 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588562+0000", 2024-04-24T15:34:00.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6215999999999999e-05 2024-04-24T15:34:00.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.936 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:34:00.936 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588628+0000", 2024-04-24T15:34:00.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5971999999999998e-05 2024-04-24T15:34:00.936 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:00.936 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:34:00.936 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:00.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.936 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63804 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1359872~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:34:00.936 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555483+0000", 2024-04-24T15:34:00.936 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 41.333202923999998, 2024-04-24T15:34:00.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033287068000000003, 2024-04-24T15:34:00.936 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:34:00.937 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:34:00.937 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:34:00.937 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:34:00.937 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:00.937 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63804 2024-04-24T15:34:00.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.937 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:34:00.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:34:00.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555483+0000", 2024-04-24T15:34:00.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:00.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:34:00.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555483+0000", 2024-04-24T15:34:00.938 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:00.938 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.938 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.938 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:34:00.938 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555483+0000", 2024-04-24T15:34:00.938 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:00.938 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.938 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.938 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:34:00.938 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555485+0000", 2024-04-24T15:34:00.938 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0710000000000002e-06 2024-04-24T15:34:00.938 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.938 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.938 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:34:00.938 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555498+0000", 2024-04-24T15:34:00.939 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2748999999999999e-05 2024-04-24T15:34:00.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.939 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.939 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:34:00.939 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555502+0000", 2024-04-24T15:34:00.939 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0219999999999998e-06 2024-04-24T15:34:00.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.939 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.939 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:34:00.939 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588657+0000", 2024-04-24T15:34:00.939 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033154683999999997 2024-04-24T15:34:00.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.939 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.939 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:34:00.939 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588702+0000", 2024-04-24T15:34:00.940 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5809000000000001e-05 2024-04-24T15:34:00.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.940 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:34:00.940 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588770+0000", 2024-04-24T15:34:00.940 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8056e-05 2024-04-24T15:34:00.940 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:00.940 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:34:00.940 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:00.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.940 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63805 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1441792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:34:00.940 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555508+0000", 2024-04-24T15:34:00.940 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 41.333178074999999, 2024-04-24T15:34:00.940 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033375952, 2024-04-24T15:34:00.941 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:34:00.941 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:34:00.941 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:34:00.941 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:34:00.941 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:00.941 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63805 2024-04-24T15:34:00.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.941 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:34:00.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.941 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:34:00.941 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555508+0000", 2024-04-24T15:34:00.941 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:00.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.941 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:34:00.941 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555508+0000", 2024-04-24T15:34:00.942 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:00.942 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.942 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.942 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:34:00.942 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555508+0000", 2024-04-24T15:34:00.942 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:00.942 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.942 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.942 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:34:00.942 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555510+0000", 2024-04-24T15:34:00.942 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.424e-06 2024-04-24T15:34:00.942 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.942 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.942 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:34:00.943 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555516+0000", 2024-04-24T15:34:00.943 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3730000000000001e-06 2024-04-24T15:34:00.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.943 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.943 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:34:00.943 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555519+0000", 2024-04-24T15:34:00.943 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.039e-06 2024-04-24T15:34:00.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.943 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.943 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:34:00.943 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588789+0000", 2024-04-24T15:34:00.943 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0332702 2024-04-24T15:34:00.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.943 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.943 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:34:00.943 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588833+0000", 2024-04-24T15:34:00.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4660999999999998e-05 2024-04-24T15:34:00.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.944 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:34:00.944 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588884+0000", 2024-04-24T15:34:00.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0661999999999997e-05 2024-04-24T15:34:00.944 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:00.944 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:34:00.944 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:00.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.944 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63806 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1523712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:34:00.944 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555523+0000", 2024-04-24T15:34:00.944 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 41.333163493999997, 2024-04-24T15:34:00.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033479653999999998, 2024-04-24T15:34:00.944 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:34:00.945 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:34:00.945 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:34:00.945 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:34:00.945 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:00.945 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63806 2024-04-24T15:34:00.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.945 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:34:00.945 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.945 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:34:00.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555523+0000", 2024-04-24T15:34:00.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:00.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.945 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.945 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:34:00.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555523+0000", 2024-04-24T15:34:00.946 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:00.946 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.946 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.946 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:34:00.946 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555522+0000", 2024-04-24T15:34:00.946 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:00.946 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.946 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.946 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:34:00.946 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555524+0000", 2024-04-24T15:34:00.946 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0700000000000001e-06 2024-04-24T15:34:00.946 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.946 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.946 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:34:00.946 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555530+0000", 2024-04-24T15:34:00.946 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5450000000000003e-06 2024-04-24T15:34:00.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.947 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.947 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:34:00.947 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555533+0000", 2024-04-24T15:34:00.947 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2200000000000001e-06 2024-04-24T15:34:00.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.947 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.947 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:34:00.947 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588908+0000", 2024-04-24T15:34:00.947 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033374556 2024-04-24T15:34:00.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.947 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.947 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:34:00.947 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.588950+0000", 2024-04-24T15:34:00.947 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2361999999999999e-05 2024-04-24T15:34:00.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.948 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:34:00.948 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589002+0000", 2024-04-24T15:34:00.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2224e-05 2024-04-24T15:34:00.948 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:00.948 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:34:00.948 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:00.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.948 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63807 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1605632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:34:00.948 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555537+0000", 2024-04-24T15:34:00.948 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 41.333148979000001, 2024-04-24T15:34:00.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033577227000000001, 2024-04-24T15:34:00.948 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:34:00.948 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:34:00.949 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:34:00.949 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:34:00.949 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:00.949 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63807 2024-04-24T15:34:00.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.949 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:34:00.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:34:00.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555537+0000", 2024-04-24T15:34:00.949 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:00.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:34:00.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555537+0000", 2024-04-24T15:34:00.949 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:00.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.950 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.950 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:34:00.950 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555537+0000", 2024-04-24T15:34:00.950 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:00.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.950 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.950 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:34:00.950 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555539+0000", 2024-04-24T15:34:00.950 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9769999999999999e-06 2024-04-24T15:34:00.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.950 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.950 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:34:00.950 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555544+0000", 2024-04-24T15:34:00.951 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.468e-06 2024-04-24T15:34:00.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.951 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.951 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:34:00.951 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555548+0000", 2024-04-24T15:34:00.951 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4319999999999999e-06 2024-04-24T15:34:00.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.951 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.951 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:34:00.951 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589021+0000", 2024-04-24T15:34:00.951 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033473655999999997 2024-04-24T15:34:00.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.951 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.951 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:34:00.951 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589064+0000", 2024-04-24T15:34:00.952 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2397000000000001e-05 2024-04-24T15:34:00.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.952 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:34:00.952 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589114+0000", 2024-04-24T15:34:00.952 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0611e-05 2024-04-24T15:34:00.952 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:00.952 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:34:00.952 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:00.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.952 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63808 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1687552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:34:00.952 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555552+0000", 2024-04-24T15:34:00.952 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 41.333134287, 2024-04-24T15:34:00.952 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033686019999999997, 2024-04-24T15:34:00.952 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:34:00.953 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:34:00.953 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:34:00.953 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:34:00.953 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:00.953 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63808 2024-04-24T15:34:00.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.953 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:34:00.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.953 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:34:00.953 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555552+0000", 2024-04-24T15:34:00.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:00.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.953 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:34:00.953 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555552+0000", 2024-04-24T15:34:00.954 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:00.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.954 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.954 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:34:00.954 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555552+0000", 2024-04-24T15:34:00.954 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:00.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.954 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.954 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:34:00.954 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555554+0000", 2024-04-24T15:34:00.954 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2170000000000001e-06 2024-04-24T15:34:00.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.954 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.955 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:34:00.955 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555562+0000", 2024-04-24T15:34:00.955 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8779999999999998e-06 2024-04-24T15:34:00.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.955 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.955 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:34:00.955 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555565+0000", 2024-04-24T15:34:00.955 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4230000000000001e-06 2024-04-24T15:34:00.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.955 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.955 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:34:00.955 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589143+0000", 2024-04-24T15:34:00.955 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033577382000000003 2024-04-24T15:34:00.956 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.956 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.956 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:34:00.956 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589185+0000", 2024-04-24T15:34:00.956 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2206999999999998e-05 2024-04-24T15:34:00.956 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.956 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.956 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:34:00.956 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589238+0000", 2024-04-24T15:34:00.956 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3211000000000003e-05 2024-04-24T15:34:00.956 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:00.956 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:34:00.956 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:00.956 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.957 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.957 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63809 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1769472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:34:00.957 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555569+0000", 2024-04-24T15:34:00.957 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 41.333116746000002, 2024-04-24T15:34:00.957 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.03378068, 2024-04-24T15:34:00.957 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:34:00.957 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:34:00.957 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:34:00.957 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:34:00.957 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:00.957 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63809 2024-04-24T15:34:00.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.957 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:34:00.957 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.957 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:34:00.958 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555569+0000", 2024-04-24T15:34:00.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:00.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.958 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.958 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:34:00.958 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555569+0000", 2024-04-24T15:34:00.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:00.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.958 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.958 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:34:00.958 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555569+0000", 2024-04-24T15:34:00.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:00.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.959 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.959 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:34:00.959 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555571+0000", 2024-04-24T15:34:00.959 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0889999999999998e-06 2024-04-24T15:34:00.959 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.959 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.959 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:34:00.959 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555577+0000", 2024-04-24T15:34:00.959 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4149999999999998e-06 2024-04-24T15:34:00.959 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.959 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.959 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:34:00.959 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555580+0000", 2024-04-24T15:34:00.959 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2399999999999999e-06 2024-04-24T15:34:00.959 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.960 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.960 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:34:00.960 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589258+0000", 2024-04-24T15:34:00.960 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033677817999999998 2024-04-24T15:34:00.960 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.960 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.960 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:34:00.960 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589300+0000", 2024-04-24T15:34:00.960 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2317999999999997e-05 2024-04-24T15:34:00.960 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.960 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.960 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:34:00.960 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589350+0000", 2024-04-24T15:34:00.960 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0111000000000002e-05 2024-04-24T15:34:00.960 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:00.960 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:34:00.961 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:00.961 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.961 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.961 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63810 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1851392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:34:00.961 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555584+0000", 2024-04-24T15:34:00.961 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 41.333102275999998, 2024-04-24T15:34:00.961 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033886251999999999, 2024-04-24T15:34:00.961 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:34:00.961 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:34:00.961 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:34:00.961 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:34:00.961 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:00.961 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63810 2024-04-24T15:34:00.961 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.961 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:34:00.961 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.962 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:34:00.962 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555584+0000", 2024-04-24T15:34:00.962 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:00.962 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.962 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.962 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:34:00.962 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555584+0000", 2024-04-24T15:34:00.962 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:00.962 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.962 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.962 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:34:00.962 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555584+0000", 2024-04-24T15:34:00.962 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:00.962 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.963 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.963 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:34:00.963 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555586+0000", 2024-04-24T15:34:00.963 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0899999999999999e-06 2024-04-24T15:34:00.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.963 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.963 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:34:00.963 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555591+0000", 2024-04-24T15:34:00.963 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5260000000000001e-06 2024-04-24T15:34:00.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.963 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.963 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:34:00.963 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555595+0000", 2024-04-24T15:34:00.963 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.41e-06 2024-04-24T15:34:00.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.963 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.964 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:34:00.964 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589377+0000", 2024-04-24T15:34:00.964 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033781868 2024-04-24T15:34:00.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.964 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.964 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:34:00.964 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589418+0000", 2024-04-24T15:34:00.964 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1461000000000002e-05 2024-04-24T15:34:00.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.964 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.964 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:34:00.964 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589470+0000", 2024-04-24T15:34:00.964 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2194999999999997e-05 2024-04-24T15:34:00.964 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:00.964 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:34:00.965 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:00.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.965 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.965 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63811 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 1933312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:34:00.965 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555599+0000", 2024-04-24T15:34:00.965 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 41.333087466999999, 2024-04-24T15:34:00.965 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033987493000000001, 2024-04-24T15:34:00.965 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:34:00.965 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:34:00.965 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:34:00.965 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:34:00.965 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:00.965 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63811 2024-04-24T15:34:00.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.965 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:34:00.965 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.966 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:34:00.966 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555599+0000", 2024-04-24T15:34:00.966 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:00.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.966 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.966 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:34:00.966 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555599+0000", 2024-04-24T15:34:00.966 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:00.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.966 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.966 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:34:00.966 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555598+0000", 2024-04-24T15:34:00.966 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:00.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.967 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.967 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:34:00.967 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555600+0000", 2024-04-24T15:34:00.967 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.04e-06 2024-04-24T15:34:00.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.967 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.967 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:34:00.967 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555606+0000", 2024-04-24T15:34:00.967 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4199999999999998e-06 2024-04-24T15:34:00.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.967 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.967 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:34:00.967 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555616+0000", 2024-04-24T15:34:00.967 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0054000000000001e-05 2024-04-24T15:34:00.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.967 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.968 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:34:00.968 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589493+0000", 2024-04-24T15:34:00.968 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033877009999999999 2024-04-24T15:34:00.968 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.968 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.968 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:34:00.968 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589536+0000", 2024-04-24T15:34:00.968 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2855999999999998e-05 2024-04-24T15:34:00.968 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.968 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.968 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:34:00.968 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589586+0000", 2024-04-24T15:34:00.968 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0420999999999997e-05 2024-04-24T15:34:00.968 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:00.968 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:34:00.968 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:00.969 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.969 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.969 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:63812 4.12 4:49b869c8:::rbd_data.114a9052fca3.000000000000001a:1e [read 2015232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-24T15:34:00.969 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:33:19.555620+0000", 2024-04-24T15:34:00.969 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 41.333065828999999, 2024-04-24T15:34:00.969 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034081696000000002, 2024-04-24T15:34:00.969 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:34:00.969 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:34:00.969 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:34:00.969 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:34:00.969 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:00.969 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63812 2024-04-24T15:34:00.969 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.969 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:34:00.969 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.970 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:34:00.970 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555620+0000", 2024-04-24T15:34:00.970 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:00.970 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.970 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.970 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:34:00.970 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555620+0000", 2024-04-24T15:34:00.970 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:00.970 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.970 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.970 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:34:00.970 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555620+0000", 2024-04-24T15:34:00.970 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:00.970 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.971 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.971 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:34:00.971 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555625+0000", 2024-04-24T15:34:00.971 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2859999999999999e-06 2024-04-24T15:34:00.971 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.971 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.971 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:34:00.971 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555633+0000", 2024-04-24T15:34:00.971 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8879999999999997e-06 2024-04-24T15:34:00.971 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.971 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.971 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:34:00.971 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.555637+0000", 2024-04-24T15:34:00.971 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9500000000000003e-06 2024-04-24T15:34:00.971 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.972 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:34:00.972 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589604+0000", 2024-04-24T15:34:00.972 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033966478000000001 2024-04-24T15:34:00.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.972 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:34:00.972 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589650+0000", 2024-04-24T15:34:00.972 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6721000000000002e-05 2024-04-24T15:34:00.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:00.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:00.972 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:34:00.972 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:33:19.589702+0000", 2024-04-24T15:34:00.972 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1709000000000002e-05 2024-04-24T15:34:00.972 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:00.973 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:34:00.973 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:00.973 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:00.973 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:34:00.973 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:34:00.974 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-24T15:34:00.975 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-04-24T15:34:00.975 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-04-24T15:34:00.975 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-04-24T15:34:00.975 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-04-24T15:34:00.976 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:34:00.975+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:34:00.976 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-04-24T15:34:00.978 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T15:34:00.978 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T15:34:00.978 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T15:34:00.978 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T15:34:00.978 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T15:34:00.978 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T15:34:00.979 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T15:34:00.979 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T15:34:01.075 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:34:01.077+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:34:01.099 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:34:01.099 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:34:01.099 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-04-24T15:34:01.099 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:34:01.109 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-24T15:34:01.176 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:34:01.178+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:34:01.231 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:34:01.232 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-24T15:34:01.232 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-04-24T15:34:01.232 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-04-24T15:34:01.232 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:34:01.241 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-24T15:34:01.277 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:34:01.278+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:34:01.366 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:34:01.366 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-04-24T15:34:01.366 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-04-24T15:34:01.366 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-04-24T15:34:01.366 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.366 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14043 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2383872~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:01.366 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892347+0000", 2024-04-24T15:34:01.366 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 87.473667339000002, 2024-04-24T15:34:01.367 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038537279000000001, 2024-04-24T15:34:01.367 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:34:01.367 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:34:01.367 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:34:01.367 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:34:01.367 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:01.367 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14043 2024-04-24T15:34:01.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.367 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:34:01.367 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.367 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:34:01.367 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892347+0000", 2024-04-24T15:34:01.367 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:01.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.368 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.368 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:34:01.368 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892347+0000", 2024-04-24T15:34:01.368 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:01.368 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.368 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.368 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:34:01.368 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892347+0000", 2024-04-24T15:34:01.368 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:01.368 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.368 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.368 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:34:01.368 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892349+0000", 2024-04-24T15:34:01.368 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3400000000000001e-06 2024-04-24T15:34:01.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.369 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.369 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:34:01.369 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892357+0000", 2024-04-24T15:34:01.369 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1540000000000002e-06 2024-04-24T15:34:01.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.369 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.369 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:34:01.369 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892360+0000", 2024-04-24T15:34:01.369 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5829999999999998e-06 2024-04-24T15:34:01.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.369 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.369 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:34:01.369 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930795+0000", 2024-04-24T15:34:01.370 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038434590999999997 2024-04-24T15:34:01.370 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.370 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.370 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:34:01.370 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930836+0000", 2024-04-24T15:34:01.370 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0827000000000001e-05 2024-04-24T15:34:01.370 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.370 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.370 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:34:01.370 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930885+0000", 2024-04-24T15:34:01.370 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9029000000000001e-05 2024-04-24T15:34:01.370 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:01.370 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:34:01.370 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:01.371 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.371 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.371 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14044 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2424832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:01.371 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892372+0000", 2024-04-24T15:34:01.371 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 87.473642694999995, 2024-04-24T15:34:01.371 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038624890000000002, 2024-04-24T15:34:01.371 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:34:01.371 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:34:01.371 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:34:01.371 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:34:01.371 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:01.371 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14044 2024-04-24T15:34:01.371 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.371 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:34:01.371 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.372 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:34:01.372 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892372+0000", 2024-04-24T15:34:01.372 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:01.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.372 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.372 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:34:01.372 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892372+0000", 2024-04-24T15:34:01.372 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:01.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.372 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.372 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:34:01.372 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892372+0000", 2024-04-24T15:34:01.372 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:01.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.372 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.373 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:34:01.373 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892374+0000", 2024-04-24T15:34:01.373 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4899999999999999e-06 2024-04-24T15:34:01.373 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.373 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.373 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:34:01.373 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892380+0000", 2024-04-24T15:34:01.373 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.631e-06 2024-04-24T15:34:01.373 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.373 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.373 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:34:01.373 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892383+0000", 2024-04-24T15:34:01.373 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7400000000000002e-06 2024-04-24T15:34:01.373 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.373 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.374 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:34:01.374 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930905+0000", 2024-04-24T15:34:01.374 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038521396999999999 2024-04-24T15:34:01.374 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.374 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.374 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:34:01.374 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930949+0000", 2024-04-24T15:34:01.374 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4153999999999998e-05 2024-04-24T15:34:01.374 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.374 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.374 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:34:01.377 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930997+0000", 2024-04-24T15:34:01.377 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7998999999999999e-05 2024-04-24T15:34:01.378 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:01.378 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:34:01.378 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:01.378 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.378 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.378 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14045 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2465792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:01.378 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892387+0000", 2024-04-24T15:34:01.378 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 87.473627428, 2024-04-24T15:34:01.378 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038720933999999999, 2024-04-24T15:34:01.378 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:34:01.378 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:34:01.378 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:34:01.378 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:34:01.378 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:01.379 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14045 2024-04-24T15:34:01.379 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.379 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:34:01.379 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.379 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:34:01.379 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892387+0000", 2024-04-24T15:34:01.379 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:01.379 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.379 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.379 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:34:01.379 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892387+0000", 2024-04-24T15:34:01.379 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:01.379 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.379 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.380 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:34:01.380 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892387+0000", 2024-04-24T15:34:01.380 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:01.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.380 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.380 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:34:01.380 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892392+0000", 2024-04-24T15:34:01.380 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0039999999999999e-06 2024-04-24T15:34:01.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.380 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.380 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:34:01.380 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892398+0000", 2024-04-24T15:34:01.380 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0900000000000001e-06 2024-04-24T15:34:01.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.380 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.381 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:34:01.381 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892402+0000", 2024-04-24T15:34:01.381 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.399e-06 2024-04-24T15:34:01.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.381 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:34:01.381 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931018+0000", 2024-04-24T15:34:01.381 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038616551999999998 2024-04-24T15:34:01.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.381 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:34:01.381 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931059+0000", 2024-04-24T15:34:01.381 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1205000000000003e-05 2024-04-24T15:34:01.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.382 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:34:01.382 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931108+0000", 2024-04-24T15:34:01.382 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8996000000000003e-05 2024-04-24T15:34:01.382 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:01.382 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:34:01.382 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:01.382 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.382 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.382 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14046 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2506752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:01.382 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892406+0000", 2024-04-24T15:34:01.382 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 87.473608838000004, 2024-04-24T15:34:01.382 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038814507999999998, 2024-04-24T15:34:01.382 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:34:01.383 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:34:01.383 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:34:01.383 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:34:01.383 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:01.383 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14046 2024-04-24T15:34:01.383 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.383 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:34:01.383 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.383 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:34:01.383 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892406+0000", 2024-04-24T15:34:01.383 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:01.383 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.383 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.383 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:34:01.384 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892406+0000", 2024-04-24T15:34:01.384 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:01.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.384 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.384 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:34:01.384 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892406+0000", 2024-04-24T15:34:01.384 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:01.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.384 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.384 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:34:01.384 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892413+0000", 2024-04-24T15:34:01.384 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7449999999999995e-06 2024-04-24T15:34:01.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.384 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.384 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:34:01.385 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892419+0000", 2024-04-24T15:34:01.385 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5790000000000003e-06 2024-04-24T15:34:01.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.385 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.385 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:34:01.385 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892423+0000", 2024-04-24T15:34:01.385 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8199999999999998e-06 2024-04-24T15:34:01.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.385 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.385 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:34:01.385 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931128+0000", 2024-04-24T15:34:01.385 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038705502000000003 2024-04-24T15:34:01.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.385 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.385 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:34:01.386 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931172+0000", 2024-04-24T15:34:01.386 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4066999999999997e-05 2024-04-24T15:34:01.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.386 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.386 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:34:01.386 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931220+0000", 2024-04-24T15:34:01.386 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8167999999999997e-05 2024-04-24T15:34:01.386 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:01.386 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:34:01.386 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:01.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.386 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.386 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14047 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2547712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:01.387 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892427+0000", 2024-04-24T15:34:01.387 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 87.473587609000006, 2024-04-24T15:34:01.387 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038904506999999998, 2024-04-24T15:34:01.387 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:34:01.387 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:34:01.387 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:34:01.387 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:34:01.387 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:01.387 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14047 2024-04-24T15:34:01.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.387 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:34:01.387 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.387 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:34:01.387 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892427+0000", 2024-04-24T15:34:01.388 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:01.388 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.388 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.388 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:34:01.388 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892427+0000", 2024-04-24T15:34:01.388 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:01.388 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.388 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.388 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:34:01.388 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892427+0000", 2024-04-24T15:34:01.388 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:01.388 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.388 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.388 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:34:01.389 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892429+0000", 2024-04-24T15:34:01.389 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3099999999999999e-06 2024-04-24T15:34:01.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.389 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.389 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:34:01.389 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892435+0000", 2024-04-24T15:34:01.389 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5520000000000004e-06 2024-04-24T15:34:01.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.389 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.389 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:34:01.389 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892438+0000", 2024-04-24T15:34:01.389 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4240000000000002e-06 2024-04-24T15:34:01.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.389 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.389 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:34:01.390 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931242+0000", 2024-04-24T15:34:01.390 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038804162000000003 2024-04-24T15:34:01.390 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.390 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.390 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:34:01.390 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931284+0000", 2024-04-24T15:34:01.390 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1414e-05 2024-04-24T15:34:01.390 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.390 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.390 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:34:01.390 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931332+0000", 2024-04-24T15:34:01.390 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8024e-05 2024-04-24T15:34:01.390 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:01.391 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:34:01.391 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:01.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.391 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.391 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14048 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2588672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:01.391 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892442+0000", 2024-04-24T15:34:01.391 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 87.473572645999994, 2024-04-24T15:34:01.391 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039003132000000003, 2024-04-24T15:34:01.391 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:34:01.391 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:34:01.391 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:34:01.391 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:34:01.391 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:01.391 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14048 2024-04-24T15:34:01.392 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.392 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:34:01.392 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.392 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:34:01.392 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892442+0000", 2024-04-24T15:34:01.392 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:01.392 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.392 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.392 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:34:01.392 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892442+0000", 2024-04-24T15:34:01.392 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:01.392 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.392 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.392 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:34:01.392 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892442+0000", 2024-04-24T15:34:01.392 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:01.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.393 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.393 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:34:01.393 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892444+0000", 2024-04-24T15:34:01.393 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1169999999999998e-06 2024-04-24T15:34:01.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.393 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.393 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:34:01.393 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892449+0000", 2024-04-24T15:34:01.393 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1599999999999997e-06 2024-04-24T15:34:01.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.393 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.393 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:34:01.393 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892453+0000", 2024-04-24T15:34:01.393 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5930000000000002e-06 2024-04-24T15:34:01.394 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.394 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.394 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:34:01.394 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931352+0000", 2024-04-24T15:34:01.394 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038899361 2024-04-24T15:34:01.394 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.394 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.394 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:34:01.394 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931398+0000", 2024-04-24T15:34:01.394 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6315e-05 2024-04-24T15:34:01.394 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.394 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.394 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:34:01.394 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931445+0000", 2024-04-24T15:34:01.395 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6950000000000003e-05 2024-04-24T15:34:01.395 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:01.395 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:34:01.395 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:01.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.395 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.395 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14049 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2629632~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:01.395 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892458+0000", 2024-04-24T15:34:01.395 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 87.473557225999997, 2024-04-24T15:34:01.395 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039103322000000003, 2024-04-24T15:34:01.395 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:34:01.395 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:34:01.395 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:34:01.395 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:34:01.396 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:01.396 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14049 2024-04-24T15:34:01.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.396 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:34:01.396 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.396 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:34:01.396 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892458+0000", 2024-04-24T15:34:01.396 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:01.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.396 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.396 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:34:01.396 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892458+0000", 2024-04-24T15:34:01.396 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:01.397 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.397 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.397 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:34:01.397 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892457+0000", 2024-04-24T15:34:01.397 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:01.397 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.397 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.397 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:34:01.397 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892459+0000", 2024-04-24T15:34:01.397 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.305e-06 2024-04-24T15:34:01.397 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.397 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.397 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:34:01.397 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892465+0000", 2024-04-24T15:34:01.398 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4360000000000001e-06 2024-04-24T15:34:01.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.398 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.398 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:34:01.398 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892474+0000", 2024-04-24T15:34:01.398 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7639999999999994e-06 2024-04-24T15:34:01.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.398 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.398 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:34:01.398 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931466+0000", 2024-04-24T15:34:01.398 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038992644 2024-04-24T15:34:01.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.398 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.398 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:34:01.399 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931512+0000", 2024-04-24T15:34:01.399 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5200000000000001e-05 2024-04-24T15:34:01.399 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.399 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.399 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:34:01.399 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931561+0000", 2024-04-24T15:34:01.399 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9357999999999997e-05 2024-04-24T15:34:01.399 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:01.399 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:34:01.399 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:01.399 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.399 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.399 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14050 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2670592~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:01.399 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892478+0000", 2024-04-24T15:34:01.400 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 87.473536639000002, 2024-04-24T15:34:01.400 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039195102000000002, 2024-04-24T15:34:01.400 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:34:01.400 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:34:01.400 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:34:01.400 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:34:01.400 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:01.400 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14050 2024-04-24T15:34:01.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.400 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:34:01.400 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.400 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:34:01.400 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892478+0000", 2024-04-24T15:34:01.400 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:01.401 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.401 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.401 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:34:01.401 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892478+0000", 2024-04-24T15:34:01.401 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:01.401 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.401 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.401 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:34:01.401 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892478+0000", 2024-04-24T15:34:01.401 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:01.401 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.401 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.401 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:34:01.401 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892480+0000", 2024-04-24T15:34:01.401 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3300000000000001e-06 2024-04-24T15:34:01.402 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.402 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.402 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:34:01.402 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892486+0000", 2024-04-24T15:34:01.402 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9200000000000001e-06 2024-04-24T15:34:01.402 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.402 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.402 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:34:01.402 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892494+0000", 2024-04-24T15:34:01.402 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5159999999999997e-06 2024-04-24T15:34:01.402 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.402 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.402 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:34:01.403 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931581+0000", 2024-04-24T15:34:01.403 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039087894999999998 2024-04-24T15:34:01.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.403 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.403 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:34:01.403 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931626+0000", 2024-04-24T15:34:01.403 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4943e-05 2024-04-24T15:34:01.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.403 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.403 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:34:01.403 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931673+0000", 2024-04-24T15:34:01.403 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6906000000000001e-05 2024-04-24T15:34:01.403 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:01.403 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:34:01.403 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:01.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.404 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.404 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14051 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2711552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:01.404 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892507+0000", 2024-04-24T15:34:01.404 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 87.473507940000005, 2024-04-24T15:34:01.404 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039274393999999997, 2024-04-24T15:34:01.404 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:34:01.404 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:34:01.404 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:34:01.404 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:34:01.404 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:01.404 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14051 2024-04-24T15:34:01.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.404 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:34:01.404 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.404 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:34:01.405 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892507+0000", 2024-04-24T15:34:01.405 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:01.405 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.405 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.405 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:34:01.405 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892507+0000", 2024-04-24T15:34:01.405 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:01.405 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.405 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.405 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:34:01.405 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892506+0000", 2024-04-24T15:34:01.405 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:01.405 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.405 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.406 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:34:01.406 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892509+0000", 2024-04-24T15:34:01.406 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8760000000000001e-06 2024-04-24T15:34:01.406 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.406 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.406 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:34:01.406 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892515+0000", 2024-04-24T15:34:01.406 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1840000000000004e-06 2024-04-24T15:34:01.406 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.406 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.406 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:34:01.406 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892519+0000", 2024-04-24T15:34:01.406 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7570000000000002e-06 2024-04-24T15:34:01.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.407 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.407 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:34:01.407 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931694+0000", 2024-04-24T15:34:01.407 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039174960000000002 2024-04-24T15:34:01.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.407 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.407 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:34:01.407 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931734+0000", 2024-04-24T15:34:01.407 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0284000000000003e-05 2024-04-24T15:34:01.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.407 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.407 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:34:01.407 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931781+0000", 2024-04-24T15:34:01.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6813000000000001e-05 2024-04-24T15:34:01.408 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:01.408 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:34:01.408 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:01.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.408 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.408 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14052 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2752512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:01.408 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892524+0000", 2024-04-24T15:34:01.408 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 87.473490792999996, 2024-04-24T15:34:01.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039371122000000001, 2024-04-24T15:34:01.408 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:34:01.408 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:34:01.408 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:34:01.408 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:34:01.409 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:01.409 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14052 2024-04-24T15:34:01.409 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.409 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:34:01.409 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.409 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:34:01.409 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892524+0000", 2024-04-24T15:34:01.409 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:01.409 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.409 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.409 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:34:01.409 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892524+0000", 2024-04-24T15:34:01.409 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:01.409 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.410 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.410 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:34:01.410 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892524+0000", 2024-04-24T15:34:01.410 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:01.410 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.410 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.410 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:34:01.410 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892526+0000", 2024-04-24T15:34:01.410 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3630000000000001e-06 2024-04-24T15:34:01.410 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.410 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.410 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:34:01.410 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892538+0000", 2024-04-24T15:34:01.411 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1677e-05 2024-04-24T15:34:01.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:34:01.411 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892541+0000", 2024-04-24T15:34:01.411 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.822e-06 2024-04-24T15:34:01.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:34:01.411 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931802+0000", 2024-04-24T15:34:01.411 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039260911000000003 2024-04-24T15:34:01.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:34:01.412 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931847+0000", 2024-04-24T15:34:01.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4450999999999998e-05 2024-04-24T15:34:01.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:34:01.412 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931895+0000", 2024-04-24T15:34:01.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8257999999999998e-05 2024-04-24T15:34:01.412 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:01.412 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:34:01.412 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:01.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.412 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14053 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2793472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:01.412 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892546+0000", 2024-04-24T15:34:01.413 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 87.473468815000004, 2024-04-24T15:34:01.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039457712999999998, 2024-04-24T15:34:01.413 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:34:01.413 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:34:01.413 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:34:01.413 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:34:01.413 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:01.413 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14053 2024-04-24T15:34:01.413 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.413 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:34:01.413 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.413 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:34:01.413 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892546+0000", 2024-04-24T15:34:01.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:01.414 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.414 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.414 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:34:01.414 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892546+0000", 2024-04-24T15:34:01.414 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:01.414 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.414 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.414 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:34:01.414 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892546+0000", 2024-04-24T15:34:01.414 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:01.414 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.414 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.414 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:34:01.414 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892548+0000", 2024-04-24T15:34:01.415 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3120000000000001e-06 2024-04-24T15:34:01.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.415 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.415 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:34:01.415 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892553+0000", 2024-04-24T15:34:01.415 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2320000000000001e-06 2024-04-24T15:34:01.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.415 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.415 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:34:01.415 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892559+0000", 2024-04-24T15:34:01.415 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5480000000000001e-06 2024-04-24T15:34:01.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.415 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.415 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:34:01.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931916+0000", 2024-04-24T15:34:01.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039357180999999998 2024-04-24T15:34:01.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.416 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.416 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:34:01.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931956+0000", 2024-04-24T15:34:01.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0565000000000002e-05 2024-04-24T15:34:01.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.416 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.416 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:34:01.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932004+0000", 2024-04-24T15:34:01.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7224000000000001e-05 2024-04-24T15:34:01.416 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:01.416 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:34:01.416 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:01.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.417 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.417 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14054 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2834432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:01.417 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892563+0000", 2024-04-24T15:34:01.417 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 87.473452163000005, 2024-04-24T15:34:01.417 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039552406999999998, 2024-04-24T15:34:01.417 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:34:01.417 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:34:01.417 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:34:01.417 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:34:01.417 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:01.417 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14054 2024-04-24T15:34:01.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.418 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:34:01.418 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.418 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:34:01.418 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892563+0000", 2024-04-24T15:34:01.418 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:01.418 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.418 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.418 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:34:01.418 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892563+0000", 2024-04-24T15:34:01.418 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:01.418 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.418 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.418 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:34:01.418 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892562+0000", 2024-04-24T15:34:01.419 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:01.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.419 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.419 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:34:01.419 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892564+0000", 2024-04-24T15:34:01.419 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2029999999999999e-06 2024-04-24T15:34:01.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.419 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.419 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:34:01.419 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892570+0000", 2024-04-24T15:34:01.419 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1460000000000004e-06 2024-04-24T15:34:01.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.419 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.419 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:34:01.420 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892573+0000", 2024-04-24T15:34:01.420 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2030000000000001e-06 2024-04-24T15:34:01.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.420 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:34:01.420 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932024+0000", 2024-04-24T15:34:01.420 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039451696000000001 2024-04-24T15:34:01.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.420 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:34:01.420 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932068+0000", 2024-04-24T15:34:01.420 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3970000000000001e-05 2024-04-24T15:34:01.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.421 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.421 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:34:01.421 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932115+0000", 2024-04-24T15:34:01.421 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6569000000000001e-05 2024-04-24T15:34:01.421 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:01.421 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:34:01.421 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:01.421 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.421 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.421 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14055 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2875392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:01.421 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892591+0000", 2024-04-24T15:34:01.421 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 87.473424190000003, 2024-04-24T15:34:01.421 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039634309, 2024-04-24T15:34:01.421 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:34:01.421 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:34:01.422 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:34:01.422 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:34:01.422 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:01.422 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14055 2024-04-24T15:34:01.422 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.422 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:34:01.422 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.422 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:34:01.422 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892591+0000", 2024-04-24T15:34:01.422 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:01.422 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.422 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.422 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:34:01.423 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892591+0000", 2024-04-24T15:34:01.423 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:01.423 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.423 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.423 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:34:01.423 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892590+0000", 2024-04-24T15:34:01.423 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:01.423 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.423 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.423 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:34:01.423 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892592+0000", 2024-04-24T15:34:01.423 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7519999999999999e-06 2024-04-24T15:34:01.423 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.423 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.424 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:34:01.424 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892596+0000", 2024-04-24T15:34:01.424 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.608e-06 2024-04-24T15:34:01.424 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.424 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.424 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:34:01.424 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892598+0000", 2024-04-24T15:34:01.424 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.531e-06 2024-04-24T15:34:01.424 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.424 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.424 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:34:01.424 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932136+0000", 2024-04-24T15:34:01.424 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039537668999999998 2024-04-24T15:34:01.424 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.425 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.425 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:34:01.425 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932177+0000", 2024-04-24T15:34:01.425 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0880000000000002e-05 2024-04-24T15:34:01.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.425 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.425 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:34:01.425 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932225+0000", 2024-04-24T15:34:01.425 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8223999999999998e-05 2024-04-24T15:34:01.425 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:01.425 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:34:01.425 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:01.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.425 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.426 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14056 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2916352~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:01.426 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892600+0000", 2024-04-24T15:34:01.426 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 87.473414356000006, 2024-04-24T15:34:01.426 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039736447000000001, 2024-04-24T15:34:01.426 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:34:01.426 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:34:01.426 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:34:01.426 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:34:01.426 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:01.426 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14056 2024-04-24T15:34:01.426 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.426 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:34:01.426 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.427 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:34:01.427 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892600+0000", 2024-04-24T15:34:01.427 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:01.427 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.427 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.427 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:34:01.427 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892600+0000", 2024-04-24T15:34:01.427 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:01.427 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.427 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.427 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:34:01.427 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892600+0000", 2024-04-24T15:34:01.427 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967296 2024-04-24T15:34:01.427 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.427 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.428 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:34:01.428 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892602+0000", 2024-04-24T15:34:01.428 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.576e-06 2024-04-24T15:34:01.428 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.428 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.428 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:34:01.428 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892605+0000", 2024-04-24T15:34:01.428 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.574e-06 2024-04-24T15:34:01.428 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.428 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.428 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:34:01.428 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892608+0000", 2024-04-24T15:34:01.428 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3800000000000001e-06 2024-04-24T15:34:01.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.429 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:34:01.429 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932245+0000", 2024-04-24T15:34:01.429 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039637378000000001 2024-04-24T15:34:01.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.429 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:34:01.429 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932290+0000", 2024-04-24T15:34:01.429 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4734999999999998e-05 2024-04-24T15:34:01.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.429 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:34:01.429 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932337+0000", 2024-04-24T15:34:01.429 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7033000000000002e-05 2024-04-24T15:34:01.430 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:01.430 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:34:01.430 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:01.430 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.430 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.430 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14063 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 3203072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:01.430 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.894564+0000", 2024-04-24T15:34:01.430 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 87.471450765, 2024-04-24T15:34:01.430 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038554995000000002, 2024-04-24T15:34:01.430 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:34:01.430 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:34:01.430 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:34:01.430 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:34:01.431 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:01.431 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14063 2024-04-24T15:34:01.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.431 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:34:01.431 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.431 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:34:01.431 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894564+0000", 2024-04-24T15:34:01.431 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:01.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.431 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.431 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:34:01.431 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894564+0000", 2024-04-24T15:34:01.431 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:01.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.431 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.432 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:34:01.432 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894564+0000", 2024-04-24T15:34:01.432 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967296 2024-04-24T15:34:01.432 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.432 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.432 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:34:01.432 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894565+0000", 2024-04-24T15:34:01.432 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2640000000000001e-06 2024-04-24T15:34:01.432 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.432 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.432 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:34:01.432 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894569+0000", 2024-04-24T15:34:01.432 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1710000000000004e-06 2024-04-24T15:34:01.432 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.433 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.433 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:34:01.433 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894573+0000", 2024-04-24T15:34:01.433 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4410000000000002e-06 2024-04-24T15:34:01.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.433 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.433 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:34:01.433 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933030+0000", 2024-04-24T15:34:01.433 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038457709999999999 2024-04-24T15:34:01.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.433 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.433 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:34:01.433 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933072+0000", 2024-04-24T15:34:01.433 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.193e-05 2024-04-24T15:34:01.434 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.434 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.434 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:34:01.434 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933119+0000", 2024-04-24T15:34:01.434 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6715000000000003e-05 2024-04-24T15:34:01.434 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:01.434 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:34:01.434 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:01.434 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.434 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.434 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14064 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 3244032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:01.434 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.894591+0000", 2024-04-24T15:34:01.434 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 87.471424123000006, 2024-04-24T15:34:01.434 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038639806999999998, 2024-04-24T15:34:01.435 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:34:01.435 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:34:01.435 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:34:01.435 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:34:01.435 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:01.435 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14064 2024-04-24T15:34:01.435 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.435 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:34:01.435 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.435 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:34:01.435 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894591+0000", 2024-04-24T15:34:01.435 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:01.435 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.435 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.436 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:34:01.436 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894591+0000", 2024-04-24T15:34:01.436 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:01.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.436 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:34:01.436 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894590+0000", 2024-04-24T15:34:01.436 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:01.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.436 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:34:01.436 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894593+0000", 2024-04-24T15:34:01.436 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4719999999999998e-06 2024-04-24T15:34:01.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.437 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.437 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:34:01.437 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894598+0000", 2024-04-24T15:34:01.437 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.57e-06 2024-04-24T15:34:01.437 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.437 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.437 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:34:01.437 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894602+0000", 2024-04-24T15:34:01.437 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.614e-06 2024-04-24T15:34:01.437 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.437 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.437 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:34:01.437 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933139+0000", 2024-04-24T15:34:01.437 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038536929999999997 2024-04-24T15:34:01.438 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.438 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.438 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:34:01.438 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933183+0000", 2024-04-24T15:34:01.438 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4217e-05 2024-04-24T15:34:01.438 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.438 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.438 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:34:01.438 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933230+0000", 2024-04-24T15:34:01.438 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7434e-05 2024-04-24T15:34:01.438 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:01.438 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:34:01.438 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:01.438 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.439 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14065 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 3284992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:01.439 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.894606+0000", 2024-04-24T15:34:01.439 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 87.471408741000005, 2024-04-24T15:34:01.439 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038734126000000001, 2024-04-24T15:34:01.439 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:34:01.439 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:34:01.439 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:34:01.439 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:34:01.439 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:01.439 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14065 2024-04-24T15:34:01.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.439 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:34:01.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.440 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:34:01.440 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894606+0000", 2024-04-24T15:34:01.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:01.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.440 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.440 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:34:01.440 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894606+0000", 2024-04-24T15:34:01.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:01.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.440 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.440 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:34:01.440 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894606+0000", 2024-04-24T15:34:01.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:01.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.441 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:34:01.441 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894608+0000", 2024-04-24T15:34:01.441 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0880000000000002e-06 2024-04-24T15:34:01.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.441 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:34:01.441 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894614+0000", 2024-04-24T15:34:01.441 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7830000000000004e-06 2024-04-24T15:34:01.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.441 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:34:01.441 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894621+0000", 2024-04-24T15:34:01.442 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5809999999999999e-06 2024-04-24T15:34:01.442 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.442 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.442 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:34:01.442 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933251+0000", 2024-04-24T15:34:01.442 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038629995 2024-04-24T15:34:01.442 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.442 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.442 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:34:01.442 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933292+0000", 2024-04-24T15:34:01.442 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0380000000000003e-05 2024-04-24T15:34:01.442 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.442 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.443 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:34:01.443 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933340+0000", 2024-04-24T15:34:01.443 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8662999999999998e-05 2024-04-24T15:34:01.443 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:01.443 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:34:01.443 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:01.443 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.443 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.443 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14066 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 3325952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:01.443 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.894625+0000", 2024-04-24T15:34:01.443 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 87.471389302000006, 2024-04-24T15:34:01.443 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038825539999999999, 2024-04-24T15:34:01.443 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:34:01.443 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:34:01.444 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:34:01.444 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:34:01.444 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:01.444 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14066 2024-04-24T15:34:01.444 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.444 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:34:01.444 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.444 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:34:01.444 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894625+0000", 2024-04-24T15:34:01.444 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:01.444 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.444 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.444 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:34:01.444 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894625+0000", 2024-04-24T15:34:01.445 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:01.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.445 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.445 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:34:01.445 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894625+0000", 2024-04-24T15:34:01.445 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:01.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.445 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.445 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:34:01.445 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894627+0000", 2024-04-24T15:34:01.445 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.092e-06 2024-04-24T15:34:01.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.445 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.445 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:34:01.446 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894635+0000", 2024-04-24T15:34:01.446 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6660000000000008e-06 2024-04-24T15:34:01.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.446 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.446 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:34:01.446 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894639+0000", 2024-04-24T15:34:01.446 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.799e-06 2024-04-24T15:34:01.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.446 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.446 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:34:01.447 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933360+0000", 2024-04-24T15:34:01.447 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038721569999999997 2024-04-24T15:34:01.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.447 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:34:01.447 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933403+0000", 2024-04-24T15:34:01.447 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2605999999999998e-05 2024-04-24T15:34:01.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.447 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:34:01.447 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933451+0000", 2024-04-24T15:34:01.447 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8149999999999998e-05 2024-04-24T15:34:01.447 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:01.447 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:34:01.448 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:01.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.448 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.448 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14067 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 3366912~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:01.448 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.894644+0000", 2024-04-24T15:34:01.448 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 87.471371243999997, 2024-04-24T15:34:01.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038926068000000001, 2024-04-24T15:34:01.448 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:34:01.448 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:34:01.448 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:34:01.448 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:34:01.448 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:01.448 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14067 2024-04-24T15:34:01.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.449 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:34:01.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.449 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:34:01.449 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894644+0000", 2024-04-24T15:34:01.449 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:01.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.449 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:34:01.449 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894644+0000", 2024-04-24T15:34:01.449 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:01.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.449 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:34:01.449 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894643+0000", 2024-04-24T15:34:01.449 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:01.450 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.450 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.450 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:34:01.450 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894646+0000", 2024-04-24T15:34:01.450 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3379999999999999e-06 2024-04-24T15:34:01.450 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.450 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.450 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:34:01.450 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894651+0000", 2024-04-24T15:34:01.450 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.485e-06 2024-04-24T15:34:01.450 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.450 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.450 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:34:01.451 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894655+0000", 2024-04-24T15:34:01.451 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.709e-06 2024-04-24T15:34:01.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.451 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.451 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:34:01.451 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933472+0000", 2024-04-24T15:34:01.451 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038816992000000002 2024-04-24T15:34:01.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.451 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.451 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:34:01.451 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933521+0000", 2024-04-24T15:34:01.451 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8841999999999998e-05 2024-04-24T15:34:01.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.451 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.452 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:34:01.452 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933570+0000", 2024-04-24T15:34:01.452 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9063000000000001e-05 2024-04-24T15:34:01.452 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:01.452 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:34:01.452 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:01.452 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.452 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.452 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:30240 4.0 4:059b194c:::rbd_object_map.114a2e5eebc:head [read 0~43] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-04-24T15:34:01.452 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:47.244473+0000", 2024-04-24T15:34:01.452 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 74.121542058000003, 2024-04-24T15:34:01.452 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4146627299999999, 2024-04-24T15:34:01.452 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:34:01.452 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:34:01.453 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:34:01.453 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:34:01.453 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:01.453 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30240 2024-04-24T15:34:01.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.453 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:34:01.453 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.453 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:34:01.453 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.244473+0000", 2024-04-24T15:34:01.453 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:01.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.453 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.453 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:34:01.453 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.244473+0000", 2024-04-24T15:34:01.454 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:01.454 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.454 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.454 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:34:01.454 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.244471+0000", 2024-04-24T15:34:01.454 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999986 2024-04-24T15:34:01.454 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.454 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.454 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:34:01.454 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.244477+0000", 2024-04-24T15:34:01.454 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7819999999999999e-06 2024-04-24T15:34:01.454 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.454 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.454 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:34:01.455 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.244487+0000", 2024-04-24T15:34:01.455 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0247e-05 2024-04-24T15:34:01.455 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.455 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.455 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:34:01.455 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.244499+0000", 2024-04-24T15:34:01.455 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1324e-05 2024-04-24T15:34:01.455 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.455 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.455 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:34:01.455 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.244545+0000", 2024-04-24T15:34:01.455 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6186000000000001e-05 2024-04-24T15:34:01.455 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.455 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.455 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-04-24T15:34:01.456 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.244553+0000", 2024-04-24T15:34:01.456 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5380000000000002e-06 2024-04-24T15:34:01.456 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.456 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.456 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:34:01.456 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.631596+0000", 2024-04-24T15:34:01.456 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.38704289600000003 2024-04-24T15:34:01.456 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.456 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.456 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-04-24T15:34:01.456 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.631606+0000", 2024-04-24T15:34:01.456 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6390000000000004e-06 2024-04-24T15:34:01.456 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.457 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:34:01.457 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:48.658831+0000", 2024-04-24T15:34:01.457 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0272250540000001 2024-04-24T15:34:01.457 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.457 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:34:01.457 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:48.658991+0000", 2024-04-24T15:34:01.457 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015994800000000001 2024-04-24T15:34:01.457 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:01.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:01.457 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:34:01.457 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:48.659135+0000", 2024-04-24T15:34:01.457 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000144456 2024-04-24T15:34:01.457 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:01.457 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:34:01.458 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:01.458 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:01.458 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:34:01.458 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:34:01.459 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-24T15:34:01.460 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:34:01.461+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:34:01.560 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:34:01.562+0000 7f0f45efb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 59992) UID: 0 2024-04-24T15:34:01.585 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:34:01.586 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:34:01.586 INFO:teuthology.orchestra.run.smithi151.stdout: "num_ops": 0 2024-04-24T15:34:01.586 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:34:01.595 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-24T15:34:01.661 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:34:01.662+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:34:01.731 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-24T15:34:01.766 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:34:01.766 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:34:01.766 INFO:teuthology.orchestra.run.smithi151.stdout: "complaint_time": 30, 2024-04-24T15:34:01.766 INFO:teuthology.orchestra.run.smithi151.stdout: "num_blocked_ops": 0 2024-04-24T15:34:01.766 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:34:01.775 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:34:01.777+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:34:01.776 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-24T15:34:01.864 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:34:01.865+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:34:01.906 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:34:01.906 INFO:teuthology.orchestra.run.smithi151.stdout: "size": 20, 2024-04-24T15:34:01.907 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 600, 2024-04-24T15:34:01.907 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [ 2024-04-24T15:34:01.907 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.907 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:39041 4.1 4:82c1a902:::rbd_data.114abe589119.000000000000032c:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e68)", 2024-04-24T15:34:01.907 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:01.760552+0000", 2024-04-24T15:34:01.907 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 60.145815378999998, 2024-04-24T15:34:01.907 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.87570748300000001, 2024-04-24T15:34:01.907 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:01.907 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "reached pg", 2024-04-24T15:34:01.907 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:01.907 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:01.907 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:01.908 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 39041 2024-04-24T15:34:01.908 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.908 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:01.908 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.908 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:01.908 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760552+0000", 2024-04-24T15:34:01.908 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:01.908 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.908 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.908 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:01.908 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760552+0000", 2024-04-24T15:34:01.908 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:01.908 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.909 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.909 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:01.909 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760550+0000", 2024-04-24T15:34:01.909 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999986 2024-04-24T15:34:01.909 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.909 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.909 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:01.909 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760557+0000", 2024-04-24T15:34:01.909 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1420000000000004e-06 2024-04-24T15:34:01.909 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.909 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.909 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:01.909 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760572+0000", 2024-04-24T15:34:01.909 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.483e-05 2024-04-24T15:34:01.909 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.910 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.910 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:01.910 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760578+0000", 2024-04-24T15:34:01.910 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.2820000000000002e-06 2024-04-24T15:34:01.910 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.910 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.910 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:01.910 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760628+0000", 2024-04-24T15:34:01.910 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.0040000000000002e-05 2024-04-24T15:34:01.910 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.910 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.910 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op must wait for map", 2024-04-24T15:34:01.910 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760637+0000", 2024-04-24T15:34:01.911 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.6899999999999998e-06 2024-04-24T15:34:01.911 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.911 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.911 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:01.911 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.770562+0000", 2024-04-24T15:34:01.911 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0099252739999999996 2024-04-24T15:34:01.911 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.911 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.911 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for peered", 2024-04-24T15:34:01.911 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.770572+0000", 2024-04-24T15:34:01.911 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0105999999999999e-05 2024-04-24T15:34:01.911 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.911 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.912 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:01.912 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:02.636170+0000", 2024-04-24T15:34:01.912 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.86559722299999997 2024-04-24T15:34:01.912 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.912 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.912 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:01.912 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:02.636259+0000", 2024-04-24T15:34:01.912 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.9446999999999999e-05 2024-04-24T15:34:01.912 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:01.912 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:01.912 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:01.912 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.912 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.912 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48675 4.d 4:b465d49b:::rbd_data.114a858e6641.0000000000000002:17 [read 2768896~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:34:01.912 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.926403+0000", 2024-04-24T15:34:01.913 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 50.979963808999997, 2024-04-24T15:34:01.913 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035742807000000001, 2024-04-24T15:34:01.913 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:01.913 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:01.913 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:01.913 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:01.913 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:01.913 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48675 2024-04-24T15:34:01.913 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.913 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:01.913 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.913 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:01.913 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926403+0000", 2024-04-24T15:34:01.913 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:01.914 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.914 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.914 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:01.914 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926403+0000", 2024-04-24T15:34:01.914 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:01.914 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.914 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.914 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:01.914 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926403+0000", 2024-04-24T15:34:01.914 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:01.914 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.914 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.915 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:01.915 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926404+0000", 2024-04-24T15:34:01.915 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.424e-06 2024-04-24T15:34:01.915 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.915 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.915 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:01.915 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926413+0000", 2024-04-24T15:34:01.915 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.6850000000000007e-06 2024-04-24T15:34:01.915 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.915 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.915 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:01.915 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926417+0000", 2024-04-24T15:34:01.915 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.7859999999999998e-06 2024-04-24T15:34:01.915 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.916 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.916 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:01.916 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.961986+0000", 2024-04-24T15:34:01.916 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035568862 2024-04-24T15:34:01.916 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.916 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.916 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:01.916 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.962063+0000", 2024-04-24T15:34:01.916 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7257999999999998e-05 2024-04-24T15:34:01.916 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.916 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.916 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:01.916 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.962146+0000", 2024-04-24T15:34:01.916 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3094000000000001e-05 2024-04-24T15:34:01.916 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:01.917 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:01.917 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:01.917 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.917 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.917 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48676 4.d 4:b465d49b:::rbd_data.114a858e6641.0000000000000002:17 [read 2834432~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:34:01.917 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.926421+0000", 2024-04-24T15:34:01.917 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 50.979946265999999, 2024-04-24T15:34:01.917 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035829002999999998, 2024-04-24T15:34:01.917 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:01.917 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:01.917 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:01.917 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:01.917 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:01.917 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48676 2024-04-24T15:34:01.917 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.918 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:01.918 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.918 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:01.918 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926421+0000", 2024-04-24T15:34:01.918 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:01.918 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.918 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.918 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:01.918 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926421+0000", 2024-04-24T15:34:01.918 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:01.918 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.918 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.918 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:01.919 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926420+0000", 2024-04-24T15:34:01.919 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:01.919 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.919 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.919 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:01.919 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926423+0000", 2024-04-24T15:34:01.919 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.356e-06 2024-04-24T15:34:01.919 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.919 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.919 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:01.919 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926428+0000", 2024-04-24T15:34:01.919 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.3340000000000001e-06 2024-04-24T15:34:01.919 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.919 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.920 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:01.920 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926431+0000", 2024-04-24T15:34:01.920 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.2899999999999998e-06 2024-04-24T15:34:01.920 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.920 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.920 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:01.920 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.962156+0000", 2024-04-24T15:34:01.920 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035725159999999999 2024-04-24T15:34:01.920 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.920 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.920 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:01.920 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.962197+0000", 2024-04-24T15:34:01.920 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0457999999999998e-05 2024-04-24T15:34:01.921 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.921 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.921 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:01.921 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.962250+0000", 2024-04-24T15:34:01.921 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.2812000000000002e-05 2024-04-24T15:34:01.921 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:01.921 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:01.921 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:01.921 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.921 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.921 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48704 4.d 4:b59d5735:::rbd_data.114a858e6641.0000000000000004:17 [read 2703360~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:34:01.921 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928541+0000", 2024-04-24T15:34:01.921 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 50.977826296000003, 2024-04-24T15:34:01.921 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035763016000000002, 2024-04-24T15:34:01.922 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:01.922 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:01.922 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:01.922 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:01.922 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:01.922 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48704 2024-04-24T15:34:01.922 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.922 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:01.922 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.922 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:01.922 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928541+0000", 2024-04-24T15:34:01.922 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:01.922 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.923 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.923 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:01.923 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928541+0000", 2024-04-24T15:34:01.923 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:01.923 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.923 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.923 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:01.923 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928540+0000", 2024-04-24T15:34:01.923 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:34:01.923 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.923 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.923 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:01.923 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928542+0000", 2024-04-24T15:34:01.923 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.37e-06 2024-04-24T15:34:01.924 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.924 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.924 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:01.924 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928545+0000", 2024-04-24T15:34:01.924 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8229999999999999e-06 2024-04-24T15:34:01.924 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.924 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.924 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:01.924 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928547+0000", 2024-04-24T15:34:01.924 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.9360000000000002e-06 2024-04-24T15:34:01.924 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.924 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.924 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:01.925 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964219+0000", 2024-04-24T15:34:01.925 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035672813999999997 2024-04-24T15:34:01.925 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.925 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.925 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:01.925 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964259+0000", 2024-04-24T15:34:01.925 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0070000000000001e-05 2024-04-24T15:34:01.925 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.925 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.925 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:01.925 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964304+0000", 2024-04-24T15:34:01.925 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4229e-05 2024-04-24T15:34:01.925 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:01.925 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:01.925 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:01.926 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.926 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.926 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48719 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 1851392~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:34:01.926 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928549+0000", 2024-04-24T15:34:01.926 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 50.977818149000001, 2024-04-24T15:34:01.926 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035890645999999998, 2024-04-24T15:34:01.926 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:01.926 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:01.926 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:01.926 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:01.926 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:01.926 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48719 2024-04-24T15:34:01.926 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.927 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:01.927 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.927 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:01.927 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928549+0000", 2024-04-24T15:34:01.927 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:01.927 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.927 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.927 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:01.927 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928549+0000", 2024-04-24T15:34:01.927 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:01.927 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.927 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.927 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:01.927 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928549+0000", 2024-04-24T15:34:01.928 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:01.928 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.928 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.928 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:01.928 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928550+0000", 2024-04-24T15:34:01.928 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3829999999999999e-06 2024-04-24T15:34:01.928 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.928 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.928 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:01.928 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928553+0000", 2024-04-24T15:34:01.928 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7369999999999998e-06 2024-04-24T15:34:01.928 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.928 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.928 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:01.929 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928555+0000", 2024-04-24T15:34:01.929 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.8920000000000001e-06 2024-04-24T15:34:01.929 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.929 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.929 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:01.929 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964240+0000", 2024-04-24T15:34:01.929 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035685537000000003 2024-04-24T15:34:01.929 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.929 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.929 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:01.929 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964318+0000", 2024-04-24T15:34:01.929 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7838999999999998e-05 2024-04-24T15:34:01.929 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.929 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.930 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:01.930 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964439+0000", 2024-04-24T15:34:01.930 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000121564 2024-04-24T15:34:01.930 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:01.930 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:01.930 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:01.930 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.930 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.930 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48720 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 1916928~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:34:01.930 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928557+0000", 2024-04-24T15:34:01.930 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 50.977810304000002, 2024-04-24T15:34:01.930 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036005219999999998, 2024-04-24T15:34:01.930 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:01.930 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:01.931 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:01.931 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:01.931 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:01.931 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48720 2024-04-24T15:34:01.931 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.931 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:01.931 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.931 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:01.931 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928557+0000", 2024-04-24T15:34:01.931 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:01.931 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.931 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.931 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:01.931 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928557+0000", 2024-04-24T15:34:01.932 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:01.932 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.932 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.932 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:01.932 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928556+0000", 2024-04-24T15:34:01.932 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:34:01.932 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.932 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.932 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:01.932 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928558+0000", 2024-04-24T15:34:01.932 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.13e-06 2024-04-24T15:34:01.932 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.932 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.932 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:01.932 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928561+0000", 2024-04-24T15:34:01.933 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5530000000000001e-06 2024-04-24T15:34:01.933 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.933 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.933 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:01.933 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928563+0000", 2024-04-24T15:34:01.933 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.8810000000000001e-06 2024-04-24T15:34:01.933 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.933 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.933 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:01.933 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964450+0000", 2024-04-24T15:34:01.933 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035887040000000002 2024-04-24T15:34:01.933 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.933 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.933 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:01.934 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964494+0000", 2024-04-24T15:34:01.934 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3699000000000003e-05 2024-04-24T15:34:01.934 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.934 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.934 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:01.934 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964562+0000", 2024-04-24T15:34:01.934 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.8149000000000001e-05 2024-04-24T15:34:01.934 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:01.934 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:01.934 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:01.934 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.934 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.934 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48721 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 1982464~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:34:01.934 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928565+0000", 2024-04-24T15:34:01.935 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 50.97780178, 2024-04-24T15:34:01.935 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036233466999999998, 2024-04-24T15:34:01.935 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:01.935 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:01.935 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:01.935 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:01.935 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:01.935 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48721 2024-04-24T15:34:01.935 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.935 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:01.935 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.935 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:01.935 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928565+0000", 2024-04-24T15:34:01.935 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:01.935 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.936 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.936 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:01.936 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928565+0000", 2024-04-24T15:34:01.936 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:01.936 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.936 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.936 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:01.936 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928565+0000", 2024-04-24T15:34:01.936 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:01.936 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.936 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.936 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:01.936 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928566+0000", 2024-04-24T15:34:01.936 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.336e-06 2024-04-24T15:34:01.937 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.937 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.937 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:01.937 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928569+0000", 2024-04-24T15:34:01.937 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.9189999999999999e-06 2024-04-24T15:34:01.937 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.937 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.937 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:01.937 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928571+0000", 2024-04-24T15:34:01.937 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.1909999999999999e-06 2024-04-24T15:34:01.937 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.937 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.937 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:01.937 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964592+0000", 2024-04-24T15:34:01.937 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036020120000000003 2024-04-24T15:34:01.937 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.938 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.938 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:01.938 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964673+0000", 2024-04-24T15:34:01.938 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.1254000000000005e-05 2024-04-24T15:34:01.938 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.938 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.938 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:01.938 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964799+0000", 2024-04-24T15:34:01.938 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00012588700000000001 2024-04-24T15:34:01.938 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:01.938 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:01.938 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:01.938 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.938 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.939 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48722 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2048000~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:34:01.939 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928586+0000", 2024-04-24T15:34:01.939 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 50.977780654999997, 2024-04-24T15:34:01.939 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036331347999999999, 2024-04-24T15:34:01.939 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:01.939 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:01.939 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:01.939 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:01.939 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:01.939 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48722 2024-04-24T15:34:01.939 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.939 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:01.939 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.939 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:01.940 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928586+0000", 2024-04-24T15:34:01.940 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:01.940 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.940 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.940 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:01.940 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928586+0000", 2024-04-24T15:34:01.940 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:01.940 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.940 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.940 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:01.940 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928586+0000", 2024-04-24T15:34:01.940 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:01.940 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.940 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.940 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:01.941 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928587+0000", 2024-04-24T15:34:01.941 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3519999999999999e-06 2024-04-24T15:34:01.941 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.941 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.941 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:01.941 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928590+0000", 2024-04-24T15:34:01.941 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.9699999999999999e-06 2024-04-24T15:34:01.941 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.941 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.941 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:01.941 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928592+0000", 2024-04-24T15:34:01.941 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.097e-06 2024-04-24T15:34:01.941 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.941 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.942 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:01.942 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964810+0000", 2024-04-24T15:34:01.942 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036217249 2024-04-24T15:34:01.942 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.942 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.942 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:01.942 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964851+0000", 2024-04-24T15:34:01.942 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.1151e-05 2024-04-24T15:34:01.942 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.942 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.942 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:01.942 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964918+0000", 2024-04-24T15:34:01.942 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.6898999999999998e-05 2024-04-24T15:34:01.942 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:01.942 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:01.943 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:01.943 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.943 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.943 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48723 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2113536~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:34:01.943 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928598+0000", 2024-04-24T15:34:01.943 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 50.977769266000003, 2024-04-24T15:34:01.943 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036552125999999997, 2024-04-24T15:34:01.943 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:01.943 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:01.943 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:01.943 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:01.943 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:01.943 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48723 2024-04-24T15:34:01.943 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.944 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:01.944 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.944 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:01.944 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928598+0000", 2024-04-24T15:34:01.944 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:01.944 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.944 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.944 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:01.944 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928598+0000", 2024-04-24T15:34:01.944 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:01.944 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.944 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.944 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:01.944 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928597+0000", 2024-04-24T15:34:01.944 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:01.945 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.945 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.945 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:01.945 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928599+0000", 2024-04-24T15:34:01.945 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.621e-06 2024-04-24T15:34:01.945 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.945 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.945 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:01.945 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928602+0000", 2024-04-24T15:34:01.945 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.9629999999999998e-06 2024-04-24T15:34:01.945 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.945 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.945 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:01.945 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928607+0000", 2024-04-24T15:34:01.945 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8940000000000001e-06 2024-04-24T15:34:01.946 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.946 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.946 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:01.946 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964947+0000", 2024-04-24T15:34:01.946 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036340439000000002 2024-04-24T15:34:01.946 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.946 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.946 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:01.946 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965028+0000", 2024-04-24T15:34:01.946 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.1054e-05 2024-04-24T15:34:01.946 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.946 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.946 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:01.946 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965150+0000", 2024-04-24T15:34:01.947 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00012154000000000001 2024-04-24T15:34:01.947 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:01.947 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:01.947 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:01.947 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.947 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.947 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48724 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2179072~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:34:01.947 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928609+0000", 2024-04-24T15:34:01.947 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 50.977757891000003, 2024-04-24T15:34:01.947 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036679614999999999, 2024-04-24T15:34:01.947 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:01.947 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:01.947 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:01.947 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:01.947 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:01.948 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48724 2024-04-24T15:34:01.948 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.948 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:01.948 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.948 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:01.948 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928609+0000", 2024-04-24T15:34:01.948 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:01.948 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.948 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.948 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:01.948 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928609+0000", 2024-04-24T15:34:01.948 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:01.948 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.948 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.949 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:01.949 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928609+0000", 2024-04-24T15:34:01.949 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:34:01.949 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.949 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.949 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:01.949 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928610+0000", 2024-04-24T15:34:01.949 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3030000000000001e-06 2024-04-24T15:34:01.949 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.949 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.949 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:01.949 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928613+0000", 2024-04-24T15:34:01.949 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.762e-06 2024-04-24T15:34:01.949 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.950 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.950 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:01.950 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928615+0000", 2024-04-24T15:34:01.950 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.832e-06 2024-04-24T15:34:01.950 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.950 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.950 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:01.950 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965174+0000", 2024-04-24T15:34:01.950 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036559633000000001 2024-04-24T15:34:01.950 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.950 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.950 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:01.950 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965221+0000", 2024-04-24T15:34:01.950 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6239999999999998e-05 2024-04-24T15:34:01.951 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.951 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.951 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:01.951 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965289+0000", 2024-04-24T15:34:01.951 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.8053999999999996e-05 2024-04-24T15:34:01.951 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:01.951 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:01.951 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:01.951 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.951 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.951 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48725 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2244608~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:34:01.951 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928617+0000", 2024-04-24T15:34:01.951 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 50.977750121, 2024-04-24T15:34:01.951 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036902659999999997, 2024-04-24T15:34:01.951 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:01.952 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:01.952 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:01.952 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:01.952 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:01.952 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48725 2024-04-24T15:34:01.952 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.952 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:01.952 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.952 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:01.952 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928617+0000", 2024-04-24T15:34:01.952 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:01.952 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.952 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.952 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:01.952 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928617+0000", 2024-04-24T15:34:01.953 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:01.953 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.953 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.953 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:01.953 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928617+0000", 2024-04-24T15:34:01.953 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:34:01.953 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.953 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.953 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:01.953 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928618+0000", 2024-04-24T15:34:01.953 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.206e-06 2024-04-24T15:34:01.953 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.953 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.953 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:01.953 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928621+0000", 2024-04-24T15:34:01.954 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5130000000000001e-06 2024-04-24T15:34:01.954 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.954 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.954 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:01.954 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928623+0000", 2024-04-24T15:34:01.954 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.9640000000000002e-06 2024-04-24T15:34:01.954 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.954 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.954 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:01.954 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965318+0000", 2024-04-24T15:34:01.954 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036694953000000002 2024-04-24T15:34:01.954 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.955 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.955 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:01.955 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965398+0000", 2024-04-24T15:34:01.955 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.9986000000000003e-05 2024-04-24T15:34:01.955 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.955 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.955 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:01.955 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965520+0000", 2024-04-24T15:34:01.955 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000121247 2024-04-24T15:34:01.955 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:01.955 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:01.955 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:01.955 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.955 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.956 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48726 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2310144~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:34:01.956 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928625+0000", 2024-04-24T15:34:01.956 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 50.977741574, 2024-04-24T15:34:01.956 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037015521000000003, 2024-04-24T15:34:01.956 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:01.956 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:01.956 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:01.956 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:01.956 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:01.956 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48726 2024-04-24T15:34:01.956 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.956 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:01.956 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.956 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:01.957 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928625+0000", 2024-04-24T15:34:01.957 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:01.957 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.957 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.957 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:01.957 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928625+0000", 2024-04-24T15:34:01.957 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:01.957 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.957 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.957 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:01.957 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928625+0000", 2024-04-24T15:34:01.957 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:34:01.957 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.957 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.957 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:01.958 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928627+0000", 2024-04-24T15:34:01.958 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3179999999999999e-06 2024-04-24T15:34:01.958 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.958 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.958 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:01.958 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928629+0000", 2024-04-24T15:34:01.958 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.819e-06 2024-04-24T15:34:01.958 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.958 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.958 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:01.958 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928634+0000", 2024-04-24T15:34:01.958 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6999999999999999e-06 2024-04-24T15:34:01.959 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.959 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.959 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:01.959 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965531+0000", 2024-04-24T15:34:01.959 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036897040999999998 2024-04-24T15:34:01.959 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.959 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.959 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:01.959 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965573+0000", 2024-04-24T15:34:01.959 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.189e-05 2024-04-24T15:34:01.959 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.959 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.959 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:01.959 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965641+0000", 2024-04-24T15:34:01.960 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.7959000000000004e-05 2024-04-24T15:34:01.960 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:01.960 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:01.960 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:01.960 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.960 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.960 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48727 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2375680~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:34:01.960 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928637+0000", 2024-04-24T15:34:01.960 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 50.977730213000001, 2024-04-24T15:34:01.960 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037235129999999998, 2024-04-24T15:34:01.960 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:01.960 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:01.960 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:01.960 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:01.961 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:01.961 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48727 2024-04-24T15:34:01.961 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.961 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:01.961 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.961 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:01.961 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928637+0000", 2024-04-24T15:34:01.961 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:01.961 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.961 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.961 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:01.961 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928637+0000", 2024-04-24T15:34:01.961 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:01.961 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.962 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.962 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:01.962 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928637+0000", 2024-04-24T15:34:01.962 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:34:01.962 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.962 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.962 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:01.962 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928638+0000", 2024-04-24T15:34:01.962 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.4279999999999999e-06 2024-04-24T15:34:01.962 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.962 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.962 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:01.962 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928641+0000", 2024-04-24T15:34:01.962 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.909e-06 2024-04-24T15:34:01.963 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.963 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.963 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:01.963 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928643+0000", 2024-04-24T15:34:01.963 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.9269999999999999e-06 2024-04-24T15:34:01.963 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.963 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.963 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:01.963 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965671+0000", 2024-04-24T15:34:01.963 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037028393 2024-04-24T15:34:01.963 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.963 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.963 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:01.963 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965752+0000", 2024-04-24T15:34:01.964 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0921999999999995e-05 2024-04-24T15:34:01.964 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.964 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.964 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:01.964 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965872+0000", 2024-04-24T15:34:01.964 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000119759 2024-04-24T15:34:01.964 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:01.964 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:01.964 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:01.964 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.964 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.964 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48728 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2441216~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:34:01.964 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928645+0000", 2024-04-24T15:34:01.964 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 50.977722030000002, 2024-04-24T15:34:01.965 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037346852999999999, 2024-04-24T15:34:01.965 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:01.965 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:01.965 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:01.965 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:01.965 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:01.965 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48728 2024-04-24T15:34:01.965 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.965 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:01.965 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.965 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:01.965 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928645+0000", 2024-04-24T15:34:01.965 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:01.965 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.966 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.966 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:01.966 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928645+0000", 2024-04-24T15:34:01.966 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:01.966 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.966 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.966 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:01.966 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928645+0000", 2024-04-24T15:34:01.966 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:34:01.966 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.966 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.966 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:01.966 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928646+0000", 2024-04-24T15:34:01.967 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.291e-06 2024-04-24T15:34:01.967 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.967 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.967 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:01.967 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928649+0000", 2024-04-24T15:34:01.967 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8509999999999999e-06 2024-04-24T15:34:01.967 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.967 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.967 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:01.967 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928651+0000", 2024-04-24T15:34:01.967 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.809e-06 2024-04-24T15:34:01.967 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.967 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.967 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:01.967 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965882+0000", 2024-04-24T15:34:01.968 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037231715999999998 2024-04-24T15:34:01.968 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.968 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.968 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:01.968 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965925+0000", 2024-04-24T15:34:01.968 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2493999999999997e-05 2024-04-24T15:34:01.968 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.968 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.968 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:01.968 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965992+0000", 2024-04-24T15:34:01.968 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.6907000000000002e-05 2024-04-24T15:34:01.968 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:01.968 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:01.968 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:01.969 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.969 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.969 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48729 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2506752~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:34:01.969 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928653+0000", 2024-04-24T15:34:01.969 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 50.977714155000001, 2024-04-24T15:34:01.969 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037576904000000001, 2024-04-24T15:34:01.969 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:01.969 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:01.969 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:01.969 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:01.969 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:01.969 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48729 2024-04-24T15:34:01.969 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.969 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:01.970 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.970 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:01.970 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928653+0000", 2024-04-24T15:34:01.970 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:01.970 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.970 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.970 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:01.970 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928653+0000", 2024-04-24T15:34:01.970 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:01.970 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.970 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.970 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:01.970 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928653+0000", 2024-04-24T15:34:01.970 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:34:01.971 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.971 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.971 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:01.971 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928654+0000", 2024-04-24T15:34:01.971 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3209999999999999e-06 2024-04-24T15:34:01.971 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.971 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.971 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:01.971 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928657+0000", 2024-04-24T15:34:01.971 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8559999999999998e-06 2024-04-24T15:34:01.971 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.971 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.971 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:01.971 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928659+0000", 2024-04-24T15:34:01.972 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7080000000000002e-06 2024-04-24T15:34:01.972 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.972 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.972 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:01.972 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966021+0000", 2024-04-24T15:34:01.972 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037361296000000002 2024-04-24T15:34:01.972 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.972 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.972 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:01.972 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966102+0000", 2024-04-24T15:34:01.972 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0939999999999994e-05 2024-04-24T15:34:01.972 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.972 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.972 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:01.972 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966230+0000", 2024-04-24T15:34:01.973 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00012799999999999999 2024-04-24T15:34:01.973 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:01.973 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:01.973 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:01.973 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.973 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.973 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48730 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2572288~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:34:01.973 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928662+0000", 2024-04-24T15:34:01.973 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 50.977705067000002, 2024-04-24T15:34:01.973 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037687246000000001, 2024-04-24T15:34:01.973 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:01.973 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:01.973 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:01.973 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:01.973 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:01.973 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48730 2024-04-24T15:34:01.974 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.974 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:01.974 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.974 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:01.974 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928662+0000", 2024-04-24T15:34:01.974 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:01.974 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.974 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.974 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:01.974 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928662+0000", 2024-04-24T15:34:01.974 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:01.974 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.974 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.974 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:01.975 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928662+0000", 2024-04-24T15:34:01.975 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:01.975 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.975 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.975 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:01.975 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928663+0000", 2024-04-24T15:34:01.975 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.381e-06 2024-04-24T15:34:01.975 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.975 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.975 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:01.975 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928666+0000", 2024-04-24T15:34:01.975 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.796e-06 2024-04-24T15:34:01.975 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.975 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.975 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:01.976 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928668+0000", 2024-04-24T15:34:01.976 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.8920000000000001e-06 2024-04-24T15:34:01.976 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.976 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.976 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:01.976 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966240+0000", 2024-04-24T15:34:01.976 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037572886999999999 2024-04-24T15:34:01.976 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.976 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.976 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:01.976 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966281+0000", 2024-04-24T15:34:01.976 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0450000000000001e-05 2024-04-24T15:34:01.976 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.976 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.976 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:01.977 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966349+0000", 2024-04-24T15:34:01.977 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.8215999999999999e-05 2024-04-24T15:34:01.977 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:01.977 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:01.977 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:01.977 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.977 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.977 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48731 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2637824~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:34:01.977 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928747+0000", 2024-04-24T15:34:01.977 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 50.977620062, 2024-04-24T15:34:01.977 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037833602000000001, 2024-04-24T15:34:01.977 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:01.977 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:01.977 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:01.978 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:01.978 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:01.978 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48731 2024-04-24T15:34:01.978 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.978 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:01.978 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.978 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:01.978 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928747+0000", 2024-04-24T15:34:01.978 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:01.978 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.978 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.978 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:01.978 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928747+0000", 2024-04-24T15:34:01.978 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:01.979 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.979 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.979 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:01.979 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928670+0000", 2024-04-24T15:34:01.979 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999228 2024-04-24T15:34:01.979 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.979 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.979 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:01.979 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928750+0000", 2024-04-24T15:34:01.979 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0527999999999999e-05 2024-04-24T15:34:01.979 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.979 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.979 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:01.979 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928757+0000", 2024-04-24T15:34:01.980 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.9419999999999998e-06 2024-04-24T15:34:01.980 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.980 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.980 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:01.980 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928762+0000", 2024-04-24T15:34:01.980 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4240000000000003e-06 2024-04-24T15:34:01.980 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.980 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.980 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:01.980 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966379+0000", 2024-04-24T15:34:01.980 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037617267000000003 2024-04-24T15:34:01.980 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.980 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.980 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:01.980 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966461+0000", 2024-04-24T15:34:01.981 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.2720999999999996e-05 2024-04-24T15:34:01.981 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.981 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.981 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:01.981 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966581+0000", 2024-04-24T15:34:01.981 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000119014 2024-04-24T15:34:01.981 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:01.981 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:01.981 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:01.981 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.981 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.981 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48732 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2703360~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:34:01.981 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928766+0000", 2024-04-24T15:34:01.981 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 50.977600578000001, 2024-04-24T15:34:01.981 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037930924999999997, 2024-04-24T15:34:01.982 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:01.982 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:01.982 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:01.982 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:01.982 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:01.982 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48732 2024-04-24T15:34:01.982 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.982 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:01.982 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.982 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:01.982 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928766+0000", 2024-04-24T15:34:01.982 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:01.982 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.982 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.983 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:01.983 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928766+0000", 2024-04-24T15:34:01.983 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:01.983 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.983 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.983 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:01.983 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928766+0000", 2024-04-24T15:34:01.983 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:01.983 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.983 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.983 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:01.983 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928768+0000", 2024-04-24T15:34:01.983 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.3420000000000002e-06 2024-04-24T15:34:01.983 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.983 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.984 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:01.984 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928783+0000", 2024-04-24T15:34:01.984 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.4813e-05 2024-04-24T15:34:01.984 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.984 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.984 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:01.984 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928787+0000", 2024-04-24T15:34:01.984 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.1999999999999996e-06 2024-04-24T15:34:01.984 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.984 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.984 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:01.984 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966592+0000", 2024-04-24T15:34:01.984 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037804484999999999 2024-04-24T15:34:01.984 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.984 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.985 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:01.985 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966632+0000", 2024-04-24T15:34:01.985 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0014e-05 2024-04-24T15:34:01.985 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.985 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.985 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:01.985 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966697+0000", 2024-04-24T15:34:01.985 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.5489999999999998e-05 2024-04-24T15:34:01.985 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:01.985 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:01.985 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:01.985 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.985 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.985 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48813 4.d 4:b39400aa:::rbd_data.114a8b543e91.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:34:01.986 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.936015+0000", 2024-04-24T15:34:01.986 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 50.970352243999997, 2024-04-24T15:34:01.986 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.039312294999999997, 2024-04-24T15:34:01.986 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:01.986 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:34:01.986 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:01.986 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:01.986 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:01.986 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48813 2024-04-24T15:34:01.986 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.986 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:01.986 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.986 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:01.986 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.936015+0000", 2024-04-24T15:34:01.987 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:01.987 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.987 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.987 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:01.987 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.936015+0000", 2024-04-24T15:34:01.987 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:01.987 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.987 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.987 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:01.987 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.936014+0000", 2024-04-24T15:34:01.987 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:01.987 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.987 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.988 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:01.988 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.942618+0000", 2024-04-24T15:34:01.988 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0066038499999999997 2024-04-24T15:34:01.988 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.988 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.988 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:01.988 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.942629+0000", 2024-04-24T15:34:01.988 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0606e-05 2024-04-24T15:34:01.988 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.988 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.988 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:01.988 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.942634+0000", 2024-04-24T15:34:01.988 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.2000000000000002e-06 2024-04-24T15:34:01.988 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.988 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.988 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:01.989 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.969006+0000", 2024-04-24T15:34:01.989 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.026371564 2024-04-24T15:34:01.989 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.989 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.989 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:01.989 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.969126+0000", 2024-04-24T15:34:01.989 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000120085 2024-04-24T15:34:01.989 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.989 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.989 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 2", 2024-04-24T15:34:01.989 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.969469+0000", 2024-04-24T15:34:01.989 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00034295400000000002 2024-04-24T15:34:01.989 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.989 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.989 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:34:01.990 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.973118+0000", 2024-04-24T15:34:01.990 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0036489700000000001 2024-04-24T15:34:01.990 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.990 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.990 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:34:01.990 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.975250+0000", 2024-04-24T15:34:01.990 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0021327030000000001 2024-04-24T15:34:01.990 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.990 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.990 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:34:01.990 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.975293+0000", 2024-04-24T15:34:01.990 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2951999999999998e-05 2024-04-24T15:34:01.990 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.990 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.990 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:01.991 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.975327+0000", 2024-04-24T15:34:01.991 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.3840000000000001e-05 2024-04-24T15:34:01.991 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:01.991 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:01.991 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:01.991 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.991 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.991 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:75520 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:head [delete] snapc 17=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e105)", 2024-04-24T15:34:01.991 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:42.638168+0000", 2024-04-24T15:34:01.991 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 19.268198622, 2024-04-24T15:34:01.991 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0190266560000001, 2024-04-24T15:34:01.991 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:01.991 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:34:01.991 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:01.991 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:01.992 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:01.992 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 75520 2024-04-24T15:34:01.992 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.992 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:01.992 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.992 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:01.992 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.638168+0000", 2024-04-24T15:34:01.992 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:01.992 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.992 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.992 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:01.992 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.638168+0000", 2024-04-24T15:34:01.992 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:01.992 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.993 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.993 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:01.993 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.638167+0000", 2024-04-24T15:34:01.993 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999986 2024-04-24T15:34:01.993 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.993 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.993 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:01.993 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.638174+0000", 2024-04-24T15:34:01.993 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.861e-06 2024-04-24T15:34:01.993 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.993 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.993 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:01.993 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.638186+0000", 2024-04-24T15:34:01.993 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.2109e-05 2024-04-24T15:34:01.993 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.994 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.994 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:01.994 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.638194+0000", 2024-04-24T15:34:01.994 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.8040000000000002e-06 2024-04-24T15:34:01.994 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.994 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.994 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:01.994 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.638682+0000", 2024-04-24T15:34:01.994 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00048809599999999998 2024-04-24T15:34:01.994 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.994 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.994 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for peered", 2024-04-24T15:34:01.994 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.638692+0000", 2024-04-24T15:34:01.994 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.8970000000000003e-06 2024-04-24T15:34:01.995 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.995 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.995 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:01.995 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:43.649868+0000", 2024-04-24T15:34:01.995 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.011176641 2024-04-24T15:34:01.995 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.995 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.995 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:01.995 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:43.649960+0000", 2024-04-24T15:34:01.995 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.1632999999999995e-05 2024-04-24T15:34:01.995 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.995 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.995 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 6", 2024-04-24T15:34:01.995 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:43.650060+0000", 2024-04-24T15:34:01.995 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.9562999999999995e-05 2024-04-24T15:34:01.995 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.996 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.996 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:34:01.996 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:43.654465+0000", 2024-04-24T15:34:01.996 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0044056620000000003 2024-04-24T15:34:01.996 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.996 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.996 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:34:01.996 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:43.656963+0000", 2024-04-24T15:34:01.996 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.002497829 2024-04-24T15:34:01.996 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.996 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.996 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:34:01.996 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:43.657083+0000", 2024-04-24T15:34:01.997 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000120338 2024-04-24T15:34:01.997 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:01.997 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:01.997 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:01.997 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:43.657195+0000", 2024-04-24T15:34:01.997 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000111624 2024-04-24T15:34:01.997 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:01.997 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:01.997 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:01.997 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:01.997 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:01.997 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:34:01.999 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-24T15:34:01.999 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:34:02.000+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:34:02.099 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:34:02.101+0000 7f0f45efb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 59992) UID: 0 2024-04-24T15:34:02.124 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:34:02.124 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:34:02.124 INFO:teuthology.orchestra.run.smithi151.stdout: "num_ops": 0 2024-04-24T15:34:02.124 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:34:02.134 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-24T15:34:02.200 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:34:02.201+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:34:02.297 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:34:02.297 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:34:02.297 INFO:teuthology.orchestra.run.smithi151.stdout: "complaint_time": 30, 2024-04-24T15:34:02.297 INFO:teuthology.orchestra.run.smithi151.stdout: "num_blocked_ops": 0 2024-04-24T15:34:02.297 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:34:02.301 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:34:02.302+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:34:02.307 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-24T15:34:02.401 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:34:02.403+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:34:02.431 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:34:02.431 INFO:teuthology.orchestra.run.smithi151.stdout: "size": 20, 2024-04-24T15:34:02.431 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 600, 2024-04-24T15:34:02.431 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [ 2024-04-24T15:34:02.431 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.431 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23613 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 14688256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:02.431 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.792522+0000", 2024-04-24T15:34:02.432 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 87.638858830000004, 2024-04-24T15:34:02.432 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040507356000000001, 2024-04-24T15:34:02.432 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:02.432 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:02.432 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:02.432 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:02.432 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:02.432 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23613 2024-04-24T15:34:02.432 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.432 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:02.432 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.432 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:02.432 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792522+0000", 2024-04-24T15:34:02.433 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:02.433 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.433 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.433 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:02.433 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792522+0000", 2024-04-24T15:34:02.433 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:02.433 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.433 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.433 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:02.433 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792522+0000", 2024-04-24T15:34:02.433 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:02.433 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.433 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.433 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:02.434 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792523+0000", 2024-04-24T15:34:02.434 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.576e-06 2024-04-24T15:34:02.434 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.434 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.434 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:02.434 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792526+0000", 2024-04-24T15:34:02.434 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7439999999999999e-06 2024-04-24T15:34:02.434 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.434 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.434 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:02.434 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792528+0000", 2024-04-24T15:34:02.434 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.0600000000000002e-06 2024-04-24T15:34:02.434 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.435 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.435 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:02.435 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.832864+0000", 2024-04-24T15:34:02.435 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040335742000000001 2024-04-24T15:34:02.435 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.435 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.435 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:02.435 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.832953+0000", 2024-04-24T15:34:02.435 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.9281000000000001e-05 2024-04-24T15:34:02.435 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.435 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.435 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:02.435 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.833029+0000", 2024-04-24T15:34:02.435 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.6259000000000003e-05 2024-04-24T15:34:02.435 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:02.436 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:02.436 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:02.436 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.436 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.436 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23614 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 14729216~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:02.436 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.792530+0000", 2024-04-24T15:34:02.436 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 87.638850566000002, 2024-04-24T15:34:02.436 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040633957999999998, 2024-04-24T15:34:02.436 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:02.436 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:02.436 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:02.436 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:02.436 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:02.436 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23614 2024-04-24T15:34:02.437 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.437 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:02.437 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.437 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:02.437 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792530+0000", 2024-04-24T15:34:02.437 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:02.437 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.437 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.437 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:02.437 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792530+0000", 2024-04-24T15:34:02.437 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:02.437 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.437 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.437 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:02.437 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792530+0000", 2024-04-24T15:34:02.438 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:34:02.438 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.438 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.438 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:02.438 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792531+0000", 2024-04-24T15:34:02.438 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.46e-06 2024-04-24T15:34:02.438 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.438 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.438 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:02.438 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792534+0000", 2024-04-24T15:34:02.438 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8959999999999999e-06 2024-04-24T15:34:02.438 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.438 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.438 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:02.439 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792536+0000", 2024-04-24T15:34:02.439 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.841e-06 2024-04-24T15:34:02.439 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.439 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.439 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:02.439 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.833060+0000", 2024-04-24T15:34:02.439 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040524125000000001 2024-04-24T15:34:02.439 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.439 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.439 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:02.439 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.833108+0000", 2024-04-24T15:34:02.439 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.7547999999999999e-05 2024-04-24T15:34:02.439 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.439 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.440 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:02.440 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.833164+0000", 2024-04-24T15:34:02.440 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6297e-05 2024-04-24T15:34:02.440 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:02.440 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:02.440 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:02.440 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.440 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.440 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23623 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15097856~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:02.440 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.793887+0000", 2024-04-24T15:34:02.440 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 87.637493605000003, 2024-04-24T15:34:02.440 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040432784999999999, 2024-04-24T15:34:02.440 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:02.440 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:02.440 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:02.441 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:02.441 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:02.441 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23623 2024-04-24T15:34:02.441 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.441 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:02.441 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.441 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:02.441 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793887+0000", 2024-04-24T15:34:02.441 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:02.441 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.441 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.441 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:02.441 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793887+0000", 2024-04-24T15:34:02.442 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:02.442 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.442 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.442 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:02.442 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793887+0000", 2024-04-24T15:34:02.442 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:34:02.442 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.442 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.442 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:02.442 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793888+0000", 2024-04-24T15:34:02.442 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3659999999999999e-06 2024-04-24T15:34:02.442 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.442 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.442 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:02.443 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793891+0000", 2024-04-24T15:34:02.443 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8389999999999998e-06 2024-04-24T15:34:02.443 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.443 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.443 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:02.443 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793893+0000", 2024-04-24T15:34:02.443 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.9549999999999999e-06 2024-04-24T15:34:02.443 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.443 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.443 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:02.443 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834239+0000", 2024-04-24T15:34:02.443 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040345487999999999 2024-04-24T15:34:02.443 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.443 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.444 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:02.444 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834280+0000", 2024-04-24T15:34:02.444 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.1878000000000001e-05 2024-04-24T15:34:02.444 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.444 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.444 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:02.444 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834320+0000", 2024-04-24T15:34:02.444 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9480000000000001e-05 2024-04-24T15:34:02.444 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:02.444 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:02.444 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:02.444 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.444 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.445 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23624 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15138816~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:02.445 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.794053+0000", 2024-04-24T15:34:02.445 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 87.637327657, 2024-04-24T15:34:02.445 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040370801999999997, 2024-04-24T15:34:02.445 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:02.445 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:02.445 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:02.445 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:02.445 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:02.445 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23624 2024-04-24T15:34:02.445 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.445 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:02.445 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.445 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:02.446 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794053+0000", 2024-04-24T15:34:02.446 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:02.446 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.446 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.446 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:02.446 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794053+0000", 2024-04-24T15:34:02.446 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:02.446 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.446 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.446 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:02.446 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793895+0000", 2024-04-24T15:34:02.446 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998422 2024-04-24T15:34:02.446 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.447 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.447 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:02.447 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794057+0000", 2024-04-24T15:34:02.447 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00016149400000000001 2024-04-24T15:34:02.447 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.447 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.447 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:02.447 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794065+0000", 2024-04-24T15:34:02.447 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.5259999999999993e-06 2024-04-24T15:34:02.447 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.447 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.447 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:02.447 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794071+0000", 2024-04-24T15:34:02.447 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6149999999999996e-06 2024-04-24T15:34:02.448 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.448 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.448 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:02.448 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834341+0000", 2024-04-24T15:34:02.448 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040270366000000002 2024-04-24T15:34:02.448 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.448 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.448 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:02.448 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834381+0000", 2024-04-24T15:34:02.448 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0102999999999999e-05 2024-04-24T15:34:02.448 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.448 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.448 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:02.448 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834424+0000", 2024-04-24T15:34:02.449 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2713000000000003e-05 2024-04-24T15:34:02.449 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:02.449 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:02.449 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:02.449 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.449 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.449 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23625 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15179776~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:02.449 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.794160+0000", 2024-04-24T15:34:02.449 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 87.637220775000003, 2024-04-24T15:34:02.449 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040413004000000002, 2024-04-24T15:34:02.449 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:02.449 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:02.450 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:02.450 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:02.450 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:02.450 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23625 2024-04-24T15:34:02.453 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.453 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:02.453 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.454 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:02.454 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794160+0000", 2024-04-24T15:34:02.454 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:02.454 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.454 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.454 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:02.454 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794160+0000", 2024-04-24T15:34:02.454 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:02.454 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.454 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.454 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:02.454 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794086+0000", 2024-04-24T15:34:02.454 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999256 2024-04-24T15:34:02.455 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.455 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.455 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:02.455 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794163+0000", 2024-04-24T15:34:02.455 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7896000000000001e-05 2024-04-24T15:34:02.455 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.455 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.455 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:02.455 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794171+0000", 2024-04-24T15:34:02.455 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.9500000000000001e-06 2024-04-24T15:34:02.455 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.455 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.455 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:02.455 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794179+0000", 2024-04-24T15:34:02.456 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7970000000000001e-06 2024-04-24T15:34:02.456 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.456 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.456 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:02.456 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834451+0000", 2024-04-24T15:34:02.456 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040271492999999998 2024-04-24T15:34:02.456 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.456 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.457 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:02.457 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834519+0000", 2024-04-24T15:34:02.457 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.7984000000000005e-05 2024-04-24T15:34:02.457 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.457 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.458 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:02.458 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834573+0000", 2024-04-24T15:34:02.458 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.4339999999999998e-05 2024-04-24T15:34:02.458 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:02.458 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:02.458 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:02.458 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.458 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.458 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23629 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15343616~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:02.458 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.794748+0000", 2024-04-24T15:34:02.458 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 87.636632973000005, 2024-04-24T15:34:02.459 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040419718, 2024-04-24T15:34:02.459 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:02.459 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:02.459 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:02.459 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:02.459 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:02.459 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23629 2024-04-24T15:34:02.459 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.459 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:02.459 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.459 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:02.459 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794748+0000", 2024-04-24T15:34:02.460 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:02.460 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.460 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.460 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:02.460 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794748+0000", 2024-04-24T15:34:02.460 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:02.460 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.460 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.460 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:02.460 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794673+0000", 2024-04-24T15:34:02.460 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999251 2024-04-24T15:34:02.461 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.461 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.461 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:02.461 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794751+0000", 2024-04-24T15:34:02.461 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.8480999999999996e-05 2024-04-24T15:34:02.461 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.461 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.461 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:02.461 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794760+0000", 2024-04-24T15:34:02.461 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3429999999999996e-06 2024-04-24T15:34:02.461 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.461 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.461 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:02.462 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794764+0000", 2024-04-24T15:34:02.462 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5600000000000004e-06 2024-04-24T15:34:02.462 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.462 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.462 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:02.462 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835040+0000", 2024-04-24T15:34:02.462 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040275188000000003 2024-04-24T15:34:02.462 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.462 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.462 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:02.462 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835105+0000", 2024-04-24T15:34:02.462 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.5992000000000001e-05 2024-04-24T15:34:02.463 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.463 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.463 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:02.463 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835167+0000", 2024-04-24T15:34:02.463 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.1981000000000002e-05 2024-04-24T15:34:02.463 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:02.463 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:02.463 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:02.463 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.463 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.463 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23630 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15384576~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:02.463 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.794839+0000", 2024-04-24T15:34:02.463 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 87.636541903999998, 2024-04-24T15:34:02.464 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040523894999999997, 2024-04-24T15:34:02.464 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:02.464 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:02.464 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:02.464 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:02.464 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:02.464 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23630 2024-04-24T15:34:02.464 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.464 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:02.464 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.464 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:02.464 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794839+0000", 2024-04-24T15:34:02.464 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:02.465 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.465 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.465 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:02.465 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794839+0000", 2024-04-24T15:34:02.465 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:02.465 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.465 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.465 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:02.465 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794768+0000", 2024-04-24T15:34:02.465 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999294 2024-04-24T15:34:02.465 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.465 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.465 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:02.466 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794841+0000", 2024-04-24T15:34:02.466 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2793000000000006e-05 2024-04-24T15:34:02.466 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.466 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.466 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:02.466 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794846+0000", 2024-04-24T15:34:02.466 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.7230000000000004e-06 2024-04-24T15:34:02.466 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.466 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.466 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:02.466 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794848+0000", 2024-04-24T15:34:02.467 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7659999999999999e-06 2024-04-24T15:34:02.467 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.467 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.467 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:02.468 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835223+0000", 2024-04-24T15:34:02.468 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040374275000000001 2024-04-24T15:34:02.468 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.468 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.468 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:02.468 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835295+0000", 2024-04-24T15:34:02.468 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2633999999999996e-05 2024-04-24T15:34:02.468 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.468 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.468 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:02.468 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835363+0000", 2024-04-24T15:34:02.468 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.7355999999999997e-05 2024-04-24T15:34:02.468 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:02.468 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:02.469 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:02.469 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.469 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.469 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23631 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15425536~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:02.469 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.794956+0000", 2024-04-24T15:34:02.469 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 87.636425063000004, 2024-04-24T15:34:02.469 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040562383, 2024-04-24T15:34:02.469 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:02.469 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:02.469 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:02.469 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:02.470 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:02.470 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23631 2024-04-24T15:34:02.470 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.470 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:02.470 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.470 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:02.470 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794956+0000", 2024-04-24T15:34:02.470 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:02.470 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.470 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.470 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:02.470 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794956+0000", 2024-04-24T15:34:02.470 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:02.470 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.471 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.471 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:02.471 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794859+0000", 2024-04-24T15:34:02.471 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999032 2024-04-24T15:34:02.471 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.471 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.471 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:02.471 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794959+0000", 2024-04-24T15:34:02.471 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000100207 2024-04-24T15:34:02.471 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.471 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.471 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:02.471 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794967+0000", 2024-04-24T15:34:02.472 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7640000000000005e-06 2024-04-24T15:34:02.472 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.472 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.472 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:02.472 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794971+0000", 2024-04-24T15:34:02.472 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.335e-06 2024-04-24T15:34:02.472 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.472 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.472 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:02.472 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835388+0000", 2024-04-24T15:34:02.472 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040417340000000003 2024-04-24T15:34:02.472 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.472 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.472 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:02.473 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835459+0000", 2024-04-24T15:34:02.473 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.0118999999999997e-05 2024-04-24T15:34:02.473 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.473 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.473 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:02.473 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835518+0000", 2024-04-24T15:34:02.473 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.9438999999999998e-05 2024-04-24T15:34:02.473 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:02.473 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:02.473 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:02.473 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.473 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.473 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23632 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15466496~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:02.474 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795060+0000", 2024-04-24T15:34:02.474 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 87.636320353000002, 2024-04-24T15:34:02.474 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040667993, 2024-04-24T15:34:02.474 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:02.474 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:02.474 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:02.474 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:02.474 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:02.474 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23632 2024-04-24T15:34:02.474 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.474 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:02.474 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.474 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:02.475 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795060+0000", 2024-04-24T15:34:02.475 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:02.475 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.475 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.475 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:02.475 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795060+0000", 2024-04-24T15:34:02.475 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:02.475 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.475 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.475 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:02.475 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794976+0000", 2024-04-24T15:34:02.475 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999156 2024-04-24T15:34:02.475 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.475 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.475 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:02.476 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795064+0000", 2024-04-24T15:34:02.476 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.7709999999999999e-05 2024-04-24T15:34:02.476 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.476 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.476 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:02.476 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795072+0000", 2024-04-24T15:34:02.476 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3769999999999996e-06 2024-04-24T15:34:02.476 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.476 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.476 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:02.476 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795078+0000", 2024-04-24T15:34:02.476 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.5079999999999996e-06 2024-04-24T15:34:02.476 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.476 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.477 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:02.477 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835541+0000", 2024-04-24T15:34:02.477 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040463440000000003 2024-04-24T15:34:02.477 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.477 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.477 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:02.477 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835657+0000", 2024-04-24T15:34:02.477 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00011572499999999999 2024-04-24T15:34:02.477 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.477 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.477 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:02.477 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835728+0000", 2024-04-24T15:34:02.477 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1402000000000005e-05 2024-04-24T15:34:02.478 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:02.478 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:02.478 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:02.478 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.478 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.478 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23633 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15507456~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:02.478 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795134+0000", 2024-04-24T15:34:02.478 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 87.636246697999994, 2024-04-24T15:34:02.478 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040748591000000001, 2024-04-24T15:34:02.478 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:02.478 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:02.478 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:02.478 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:02.478 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:02.479 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23633 2024-04-24T15:34:02.479 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.479 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:02.479 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.479 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:02.479 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795134+0000", 2024-04-24T15:34:02.479 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:02.479 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.479 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.479 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:02.479 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795134+0000", 2024-04-24T15:34:02.479 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:02.479 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.479 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.480 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:02.480 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795083+0000", 2024-04-24T15:34:02.480 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999949 2024-04-24T15:34:02.480 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.480 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.480 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:02.480 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795138+0000", 2024-04-24T15:34:02.480 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.4472000000000003e-05 2024-04-24T15:34:02.480 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.480 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.480 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:02.480 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795145+0000", 2024-04-24T15:34:02.480 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2779999999999996e-06 2024-04-24T15:34:02.481 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.481 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.481 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:02.481 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795149+0000", 2024-04-24T15:34:02.481 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.568e-06 2024-04-24T15:34:02.481 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.481 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.481 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:02.481 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835757+0000", 2024-04-24T15:34:02.481 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040608894999999999 2024-04-24T15:34:02.481 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.481 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.481 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:02.481 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835829+0000", 2024-04-24T15:34:02.481 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1879999999999996e-05 2024-04-24T15:34:02.482 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.482 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.482 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:02.482 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835883+0000", 2024-04-24T15:34:02.482 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.3346000000000001e-05 2024-04-24T15:34:02.482 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:02.482 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:02.482 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:02.482 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.482 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.482 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23634 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15548416~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:02.482 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795271+0000", 2024-04-24T15:34:02.482 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 87.636109664000003, 2024-04-24T15:34:02.483 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040755845999999998, 2024-04-24T15:34:02.483 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:02.483 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:02.483 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:02.483 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:02.483 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:02.483 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23634 2024-04-24T15:34:02.483 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.483 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:02.483 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.483 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:02.483 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795271+0000", 2024-04-24T15:34:02.483 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:02.483 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.484 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.484 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:02.484 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795271+0000", 2024-04-24T15:34:02.484 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:02.484 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.484 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.484 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:02.484 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795152+0000", 2024-04-24T15:34:02.484 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998813 2024-04-24T15:34:02.484 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.484 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.484 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:02.484 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795276+0000", 2024-04-24T15:34:02.484 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00012401799999999999 2024-04-24T15:34:02.484 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.485 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.485 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:02.485 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795281+0000", 2024-04-24T15:34:02.485 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8860000000000003e-06 2024-04-24T15:34:02.485 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.485 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.485 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:02.485 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795284+0000", 2024-04-24T15:34:02.485 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.0450000000000001e-06 2024-04-24T15:34:02.485 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.485 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.485 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:02.485 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835905+0000", 2024-04-24T15:34:02.485 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040621035999999999 2024-04-24T15:34:02.485 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.485 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.486 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:02.486 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835971+0000", 2024-04-24T15:34:02.486 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.5314000000000005e-05 2024-04-24T15:34:02.486 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.486 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.486 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:02.486 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836027+0000", 2024-04-24T15:34:02.486 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6305999999999999e-05 2024-04-24T15:34:02.486 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:02.486 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:02.486 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:02.486 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.486 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.486 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23635 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15589376~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:02.486 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795382+0000", 2024-04-24T15:34:02.487 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 87.635998891, 2024-04-24T15:34:02.487 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040805363999999997, 2024-04-24T15:34:02.487 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:02.487 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:02.487 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:02.487 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:02.487 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:02.487 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23635 2024-04-24T15:34:02.487 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.487 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:02.487 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.487 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:02.487 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795382+0000", 2024-04-24T15:34:02.487 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:02.487 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.487 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.488 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:02.488 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795382+0000", 2024-04-24T15:34:02.488 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:02.488 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.488 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.488 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:02.488 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795288+0000", 2024-04-24T15:34:02.488 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999056 2024-04-24T15:34:02.488 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.488 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.488 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:02.488 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795385+0000", 2024-04-24T15:34:02.488 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.7288999999999996e-05 2024-04-24T15:34:02.488 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.489 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.489 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:02.489 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795392+0000", 2024-04-24T15:34:02.489 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.452e-06 2024-04-24T15:34:02.489 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.489 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.489 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:02.489 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795397+0000", 2024-04-24T15:34:02.489 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2860000000000001e-06 2024-04-24T15:34:02.489 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.489 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.489 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:02.489 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836054+0000", 2024-04-24T15:34:02.489 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040657581999999998 2024-04-24T15:34:02.489 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.490 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.490 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:02.490 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836124+0000", 2024-04-24T15:34:02.490 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.0020999999999999e-05 2024-04-24T15:34:02.490 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.490 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.490 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:02.490 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836187+0000", 2024-04-24T15:34:02.490 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.3009999999999995e-05 2024-04-24T15:34:02.490 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:02.490 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:02.490 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:02.490 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.490 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.491 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23636 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15630336~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:02.491 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795610+0000", 2024-04-24T15:34:02.491 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 87.635770532999999, 2024-04-24T15:34:02.491 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040730942999999999, 2024-04-24T15:34:02.491 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:02.491 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:02.491 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:02.491 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:02.491 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:02.491 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23636 2024-04-24T15:34:02.491 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.491 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:02.491 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.491 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:02.491 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795610+0000", 2024-04-24T15:34:02.492 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:02.492 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.492 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.492 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:02.492 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795610+0000", 2024-04-24T15:34:02.492 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:02.492 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.492 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.492 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:02.492 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795401+0000", 2024-04-24T15:34:02.492 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9997911 2024-04-24T15:34:02.492 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.492 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.492 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:02.492 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795614+0000", 2024-04-24T15:34:02.493 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00021228600000000001 2024-04-24T15:34:02.493 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.493 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.493 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:02.493 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795622+0000", 2024-04-24T15:34:02.493 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3920000000000003e-06 2024-04-24T15:34:02.493 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.493 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.493 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:02.493 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795627+0000", 2024-04-24T15:34:02.493 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8199999999999996e-06 2024-04-24T15:34:02.493 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.493 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.493 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:02.493 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836211+0000", 2024-04-24T15:34:02.494 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040583803000000002 2024-04-24T15:34:02.494 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.494 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.494 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:02.494 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836284+0000", 2024-04-24T15:34:02.494 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.3617999999999999e-05 2024-04-24T15:34:02.494 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.494 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.494 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:02.494 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836341+0000", 2024-04-24T15:34:02.494 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6959999999999997e-05 2024-04-24T15:34:02.494 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:02.494 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:02.494 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:02.495 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.495 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.495 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23637 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15671296~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:02.495 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795699+0000", 2024-04-24T15:34:02.495 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 87.635681366, 2024-04-24T15:34:02.495 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040775556999999997, 2024-04-24T15:34:02.495 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:02.495 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:02.495 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:02.495 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:02.495 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:02.495 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23637 2024-04-24T15:34:02.495 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.495 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:02.496 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.496 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:02.496 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795699+0000", 2024-04-24T15:34:02.496 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:02.496 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.496 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.496 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:02.496 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795699+0000", 2024-04-24T15:34:02.496 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:02.496 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.496 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.496 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:02.496 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795632+0000", 2024-04-24T15:34:02.496 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999323 2024-04-24T15:34:02.496 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.497 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.497 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:02.497 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795704+0000", 2024-04-24T15:34:02.497 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2428000000000005e-05 2024-04-24T15:34:02.497 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.497 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.497 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:02.497 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795715+0000", 2024-04-24T15:34:02.497 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.1532000000000001e-05 2024-04-24T15:34:02.497 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.497 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.497 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:02.497 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795721+0000", 2024-04-24T15:34:02.497 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.8200000000000002e-06 2024-04-24T15:34:02.497 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.498 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.498 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:02.498 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836364+0000", 2024-04-24T15:34:02.498 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040642997 2024-04-24T15:34:02.498 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.498 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.498 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:02.498 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836435+0000", 2024-04-24T15:34:02.498 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.0821000000000005e-05 2024-04-24T15:34:02.498 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.498 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.498 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:02.498 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836475+0000", 2024-04-24T15:34:02.499 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9796999999999999e-05 2024-04-24T15:34:02.499 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:02.499 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:02.499 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:02.499 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.499 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.499 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23638 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15712256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:02.499 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795932+0000", 2024-04-24T15:34:02.499 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 87.635448335999996, 2024-04-24T15:34:02.499 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040668682999999997, 2024-04-24T15:34:02.499 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:02.499 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:02.499 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:02.499 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:02.499 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:02.500 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23638 2024-04-24T15:34:02.500 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.500 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:02.500 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.500 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:02.500 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795932+0000", 2024-04-24T15:34:02.500 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:02.500 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.500 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.500 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:02.500 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795932+0000", 2024-04-24T15:34:02.500 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:02.500 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.501 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.501 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:02.501 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795728+0000", 2024-04-24T15:34:02.501 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9997959 2024-04-24T15:34:02.501 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.501 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.501 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:02.501 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795936+0000", 2024-04-24T15:34:02.501 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000208128 2024-04-24T15:34:02.501 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.501 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.501 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:02.501 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795945+0000", 2024-04-24T15:34:02.501 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.7110000000000001e-06 2024-04-24T15:34:02.502 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.502 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.502 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:02.502 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795957+0000", 2024-04-24T15:34:02.502 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.1800999999999999e-05 2024-04-24T15:34:02.502 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.502 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.502 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:02.502 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836505+0000", 2024-04-24T15:34:02.502 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040547622999999998 2024-04-24T15:34:02.502 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.502 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.502 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:02.502 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836561+0000", 2024-04-24T15:34:02.502 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6598999999999998e-05 2024-04-24T15:34:02.503 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.503 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.503 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:02.503 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836601+0000", 2024-04-24T15:34:02.503 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9907000000000003e-05 2024-04-24T15:34:02.503 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:02.503 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:02.503 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:02.503 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.503 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.503 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23639 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15753216~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:02.503 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.796071+0000", 2024-04-24T15:34:02.503 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 87.635310075000007, 2024-04-24T15:34:02.503 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040648199000000003, 2024-04-24T15:34:02.503 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:02.504 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:02.504 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:02.504 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:02.504 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:02.504 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23639 2024-04-24T15:34:02.504 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.504 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:02.504 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.504 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:02.504 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796071+0000", 2024-04-24T15:34:02.504 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:02.504 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.504 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.504 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:02.504 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796071+0000", 2024-04-24T15:34:02.505 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:02.505 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.505 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.505 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:02.505 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795967+0000", 2024-04-24T15:34:02.505 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998965 2024-04-24T15:34:02.505 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.505 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.505 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:02.505 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796074+0000", 2024-04-24T15:34:02.505 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000107013 2024-04-24T15:34:02.505 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.505 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.505 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:02.505 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796081+0000", 2024-04-24T15:34:02.505 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2590000000000003e-06 2024-04-24T15:34:02.506 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.506 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.506 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:02.506 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796085+0000", 2024-04-24T15:34:02.506 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5269999999999999e-06 2024-04-24T15:34:02.506 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.506 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.506 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:02.506 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836624+0000", 2024-04-24T15:34:02.506 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040539324000000002 2024-04-24T15:34:02.506 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.506 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.506 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:02.506 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836667+0000", 2024-04-24T15:34:02.507 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2562999999999999e-05 2024-04-24T15:34:02.507 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.507 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.507 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:02.507 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836719+0000", 2024-04-24T15:34:02.507 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.2015999999999998e-05 2024-04-24T15:34:02.507 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:02.507 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:02.507 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:02.507 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.507 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.507 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23640 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15794176~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:02.507 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.796492+0000", 2024-04-24T15:34:02.507 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 87.634888317000005, 2024-04-24T15:34:02.508 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040420000999999997, 2024-04-24T15:34:02.508 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:02.508 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:02.508 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:02.508 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:02.508 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:02.508 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23640 2024-04-24T15:34:02.508 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.508 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:02.508 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.508 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:02.508 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796492+0000", 2024-04-24T15:34:02.508 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:02.508 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.509 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.509 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:02.509 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796492+0000", 2024-04-24T15:34:02.509 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:02.509 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.509 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.509 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:02.509 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796088+0000", 2024-04-24T15:34:02.509 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9995961 2024-04-24T15:34:02.509 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.509 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.509 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:02.509 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796496+0000", 2024-04-24T15:34:02.509 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00040785700000000002 2024-04-24T15:34:02.510 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.510 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.510 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:02.510 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796503+0000", 2024-04-24T15:34:02.510 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1049999999999997e-06 2024-04-24T15:34:02.510 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.510 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.510 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:02.510 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796508+0000", 2024-04-24T15:34:02.510 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.155e-06 2024-04-24T15:34:02.510 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.510 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.510 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:02.511 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836756+0000", 2024-04-24T15:34:02.511 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040248850000000003 2024-04-24T15:34:02.511 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.511 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.511 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:02.511 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836841+0000", 2024-04-24T15:34:02.511 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.4735999999999996e-05 2024-04-24T15:34:02.511 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.511 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.511 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:02.511 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836912+0000", 2024-04-24T15:34:02.511 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1229999999999994e-05 2024-04-24T15:34:02.511 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:02.511 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:02.512 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:02.512 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.512 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.512 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23641 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15835136~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:02.512 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.796658+0000", 2024-04-24T15:34:02.512 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 87.634722241000006, 2024-04-24T15:34:02.512 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040420706000000001, 2024-04-24T15:34:02.512 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:02.512 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:02.512 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:02.512 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:02.512 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:02.512 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23641 2024-04-24T15:34:02.513 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.513 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:02.513 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.513 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:02.513 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796658+0000", 2024-04-24T15:34:02.513 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:02.513 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.513 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.513 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:02.513 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796658+0000", 2024-04-24T15:34:02.513 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:02.513 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.513 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.514 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:02.514 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796512+0000", 2024-04-24T15:34:02.514 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998531 2024-04-24T15:34:02.514 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.514 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.514 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:02.514 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796669+0000", 2024-04-24T15:34:02.514 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000157577 2024-04-24T15:34:02.514 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.514 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.514 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:02.514 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796677+0000", 2024-04-24T15:34:02.514 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7800000000000001e-06 2024-04-24T15:34:02.515 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.515 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.515 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:02.515 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796681+0000", 2024-04-24T15:34:02.515 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.721e-06 2024-04-24T15:34:02.515 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.515 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.515 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:02.515 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836941+0000", 2024-04-24T15:34:02.515 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040259907999999997 2024-04-24T15:34:02.515 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.515 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.515 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:02.516 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837017+0000", 2024-04-24T15:34:02.516 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.6071999999999999e-05 2024-04-24T15:34:02.516 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.516 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.516 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:02.516 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837079+0000", 2024-04-24T15:34:02.516 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.2358000000000002e-05 2024-04-24T15:34:02.516 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:02.516 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:02.516 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:02.516 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.516 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.516 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23645 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15998976~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:02.516 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.797360+0000", 2024-04-24T15:34:02.517 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 87.634020742999994, 2024-04-24T15:34:02.517 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040387645999999999, 2024-04-24T15:34:02.517 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:02.517 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:02.517 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:02.517 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:02.517 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:02.517 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23645 2024-04-24T15:34:02.517 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.517 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:02.517 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.517 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:02.517 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797360+0000", 2024-04-24T15:34:02.517 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:02.518 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.518 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.518 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:02.518 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797360+0000", 2024-04-24T15:34:02.518 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:02.518 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.518 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.518 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:02.518 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797288+0000", 2024-04-24T15:34:02.518 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999928 2024-04-24T15:34:02.518 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.518 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.518 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:02.519 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797364+0000", 2024-04-24T15:34:02.519 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.5347999999999998e-05 2024-04-24T15:34:02.519 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.519 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.519 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:02.519 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797372+0000", 2024-04-24T15:34:02.519 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.2409999999999995e-06 2024-04-24T15:34:02.519 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.519 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.519 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:02.519 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797376+0000", 2024-04-24T15:34:02.519 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.515e-06 2024-04-24T15:34:02.519 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.519 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.520 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:02.520 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837586+0000", 2024-04-24T15:34:02.520 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040210047999999998 2024-04-24T15:34:02.520 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.520 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.520 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:02.520 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837672+0000", 2024-04-24T15:34:02.520 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.5997999999999999e-05 2024-04-24T15:34:02.520 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.520 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.520 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:02.520 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837748+0000", 2024-04-24T15:34:02.520 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.5328999999999997e-05 2024-04-24T15:34:02.520 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:02.520 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:02.521 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:02.521 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.521 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.521 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23646 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 16039936~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:02.521 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.797498+0000", 2024-04-24T15:34:02.521 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 87.633882642000003, 2024-04-24T15:34:02.521 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040433139, 2024-04-24T15:34:02.521 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:02.521 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:02.521 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:02.521 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:02.521 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:02.521 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23646 2024-04-24T15:34:02.521 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.521 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:02.521 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.522 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:02.522 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797498+0000", 2024-04-24T15:34:02.522 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:02.522 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.522 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.522 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:02.522 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797498+0000", 2024-04-24T15:34:02.522 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:02.522 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.522 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.522 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:02.522 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797381+0000", 2024-04-24T15:34:02.522 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998832 2024-04-24T15:34:02.523 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.523 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.523 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:02.523 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797501+0000", 2024-04-24T15:34:02.523 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000119759 2024-04-24T15:34:02.523 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.523 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.523 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:02.523 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797510+0000", 2024-04-24T15:34:02.523 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.4770000000000003e-06 2024-04-24T15:34:02.523 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.523 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.523 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:02.523 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797515+0000", 2024-04-24T15:34:02.523 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8330000000000001e-06 2024-04-24T15:34:02.524 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.524 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.524 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:02.524 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837777+0000", 2024-04-24T15:34:02.524 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040262502999999998 2024-04-24T15:34:02.524 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.524 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.524 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:02.524 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837860+0000", 2024-04-24T15:34:02.524 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3002000000000003e-05 2024-04-24T15:34:02.524 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.524 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.524 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:02.524 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837931+0000", 2024-04-24T15:34:02.524 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1222000000000004e-05 2024-04-24T15:34:02.524 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:02.524 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:02.525 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:02.525 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:02.525 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:02.525 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:34:02.526 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-24T15:34:02.527 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:34:02.527+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:34:02.627 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:34:02.628+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:34:02.643 INFO:teuthology.orchestra.run.smithi096.stderr:nodeep-scrub is unset 2024-04-24T15:34:02.727 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:34:02.729+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:34:02.756 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:34:02.757 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [ 2024-04-24T15:34:02.757 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.757 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:77690 4.8 4.80eafec8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-04-24T15:34:02.757 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:34:02.758014+0000", 2024-04-24T15:34:02.757 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 3.7808999999999997e-05, 2024-04-24T15:34:02.757 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.7402999999999999e-05, 2024-04-24T15:34:02.758 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:02.758 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "reached pg", 2024-04-24T15:34:02.758 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:02.758 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:02.758 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:02.758 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 77690 2024-04-24T15:34:02.759 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.759 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:02.759 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.759 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:02.759 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:34:02.758014+0000", 2024-04-24T15:34:02.759 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:02.759 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.760 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.760 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:02.760 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:34:02.758014+0000", 2024-04-24T15:34:02.760 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:02.760 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.760 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.760 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:02.760 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:34:02.758013+0000", 2024-04-24T15:34:02.760 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:02.760 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.760 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.760 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:02.761 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:34:02.758018+0000", 2024-04-24T15:34:02.761 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.9710000000000003e-06 2024-04-24T15:34:02.761 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.761 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.761 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:02.761 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:34:02.758026+0000", 2024-04-24T15:34:02.761 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.5990000000000002e-06 2024-04-24T15:34:02.761 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.761 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.761 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:02.761 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:34:02.758031+0000", 2024-04-24T15:34:02.761 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.1729999999999997e-06 2024-04-24T15:34:02.761 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:02.761 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:02.762 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:02.762 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:34:02.758062+0000", 2024-04-24T15:34:02.762 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.1887999999999997e-05 2024-04-24T15:34:02.762 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:02.762 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:02.762 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:02.762 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:02.762 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-24T15:34:02.762 INFO:teuthology.orchestra.run.smithi151.stdout: "num_ops": 1 2024-04-24T15:34:02.762 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:34:02.767 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-24T15:34:02.828 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:34:02.830+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:34:02.929 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:34:02.931+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:34:02.930 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:34:02.930 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:34:02.930 INFO:teuthology.orchestra.run.smithi151.stdout: "complaint_time": 30, 2024-04-24T15:34:02.930 INFO:teuthology.orchestra.run.smithi151.stdout: "num_blocked_ops": 0 2024-04-24T15:34:02.930 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:34:02.940 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-24T15:34:02.990 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-24T15:34:03.034 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:34:03.036+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:34:03.064 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:34:03.064 INFO:teuthology.orchestra.run.smithi151.stdout: "size": 20, 2024-04-24T15:34:03.064 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 600, 2024-04-24T15:34:03.064 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [ 2024-04-24T15:34:03.064 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.065 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18007 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12460032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:03.065 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.207961+0000", 2024-04-24T15:34:03.065 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 88.856436841000004, 2024-04-24T15:34:03.065 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042965470999999998, 2024-04-24T15:34:03.065 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:03.065 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:03.065 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:03.065 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:03.065 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:03.065 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18007 2024-04-24T15:34:03.065 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.065 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:03.065 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.065 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:03.065 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207961+0000", 2024-04-24T15:34:03.066 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:03.066 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.066 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.066 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:03.066 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207961+0000", 2024-04-24T15:34:03.066 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:03.066 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.066 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.066 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:03.066 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207960+0000", 2024-04-24T15:34:03.066 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:03.066 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.066 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.067 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:03.067 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207963+0000", 2024-04-24T15:34:03.067 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.3759999999999999e-06 2024-04-24T15:34:03.067 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.067 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.067 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:03.067 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207970+0000", 2024-04-24T15:34:03.067 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.0489999999999998e-06 2024-04-24T15:34:03.069 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.070 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.070 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:03.070 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207975+0000", 2024-04-24T15:34:03.070 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.1640000000000003e-06 2024-04-24T15:34:03.070 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.070 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.070 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:03.070 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250834+0000", 2024-04-24T15:34:03.070 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042859060999999997 2024-04-24T15:34:03.070 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.070 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.070 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:03.071 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250874+0000", 2024-04-24T15:34:03.071 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0674999999999999e-05 2024-04-24T15:34:03.071 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.071 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.071 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:03.071 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250926+0000", 2024-04-24T15:34:03.071 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.1693000000000001e-05 2024-04-24T15:34:03.071 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:03.071 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:03.071 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:03.071 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.071 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.071 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18008 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12500992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:03.071 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208030+0000", 2024-04-24T15:34:03.072 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 88.856367747999997, 2024-04-24T15:34:03.072 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043003337000000003, 2024-04-24T15:34:03.072 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:03.072 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:03.072 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:03.072 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:03.072 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:03.072 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18008 2024-04-24T15:34:03.072 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.072 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:03.072 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.072 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:03.072 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208030+0000", 2024-04-24T15:34:03.072 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:03.073 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.073 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.073 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:03.073 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208030+0000", 2024-04-24T15:34:03.073 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:03.073 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.073 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.073 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:03.073 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208029+0000", 2024-04-24T15:34:03.073 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:03.073 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.073 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.073 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:03.073 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208032+0000", 2024-04-24T15:34:03.073 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.117e-06 2024-04-24T15:34:03.074 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.074 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.074 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:03.074 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208039+0000", 2024-04-24T15:34:03.074 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.0129999999999996e-06 2024-04-24T15:34:03.074 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.074 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.074 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:03.074 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208043+0000", 2024-04-24T15:34:03.074 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0600000000000001e-06 2024-04-24T15:34:03.074 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.074 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.074 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:03.074 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250948+0000", 2024-04-24T15:34:03.075 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042904577999999999 2024-04-24T15:34:03.075 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.075 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.075 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:03.075 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250989+0000", 2024-04-24T15:34:03.075 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0978e-05 2024-04-24T15:34:03.075 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.075 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.075 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:03.075 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251033+0000", 2024-04-24T15:34:03.075 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4161e-05 2024-04-24T15:34:03.075 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:03.075 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:03.075 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:03.076 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.076 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.076 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18009 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12541952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:03.076 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208093+0000", 2024-04-24T15:34:03.076 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 88.856304002000002, 2024-04-24T15:34:03.076 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043043451000000003, 2024-04-24T15:34:03.076 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:03.076 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:03.076 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:03.076 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:03.076 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:03.076 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18009 2024-04-24T15:34:03.076 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.076 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:03.076 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.077 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:03.077 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208093+0000", 2024-04-24T15:34:03.077 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:03.077 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.077 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.077 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:03.077 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208093+0000", 2024-04-24T15:34:03.077 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:03.077 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.077 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.077 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:03.077 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208093+0000", 2024-04-24T15:34:03.077 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:03.078 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.078 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.078 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:03.078 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208096+0000", 2024-04-24T15:34:03.078 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.439e-06 2024-04-24T15:34:03.078 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.078 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.078 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:03.078 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208103+0000", 2024-04-24T15:34:03.078 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1019999999999999e-06 2024-04-24T15:34:03.078 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.078 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.078 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:03.078 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208107+0000", 2024-04-24T15:34:03.079 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0910000000000003e-06 2024-04-24T15:34:03.079 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.079 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.079 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:03.079 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251054+0000", 2024-04-24T15:34:03.079 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042946826 2024-04-24T15:34:03.079 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.079 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.079 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:03.079 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251092+0000", 2024-04-24T15:34:03.079 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.7941999999999997e-05 2024-04-24T15:34:03.079 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.079 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.079 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:03.080 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251137+0000", 2024-04-24T15:34:03.080 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4638000000000002e-05 2024-04-24T15:34:03.080 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:03.080 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:03.080 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:03.080 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.080 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.080 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18010 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12615680~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:03.080 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208202+0000", 2024-04-24T15:34:03.080 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 88.856195170999996, 2024-04-24T15:34:03.080 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043046549000000003, 2024-04-24T15:34:03.080 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:03.080 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:03.080 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:03.080 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:03.081 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:03.081 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18010 2024-04-24T15:34:03.081 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.081 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:03.081 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.081 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:03.081 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208202+0000", 2024-04-24T15:34:03.081 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:03.081 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.081 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.081 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:03.081 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208202+0000", 2024-04-24T15:34:03.081 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:03.081 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.082 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.082 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:03.082 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208202+0000", 2024-04-24T15:34:03.082 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:03.082 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.082 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.082 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:03.082 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208205+0000", 2024-04-24T15:34:03.082 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.2490000000000001e-06 2024-04-24T15:34:03.082 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.082 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.082 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:03.082 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208215+0000", 2024-04-24T15:34:03.083 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.9070000000000002e-06 2024-04-24T15:34:03.083 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.083 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.083 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:03.083 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208220+0000", 2024-04-24T15:34:03.083 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8849999999999998e-06 2024-04-24T15:34:03.083 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.083 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.083 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:03.083 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251158+0000", 2024-04-24T15:34:03.083 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042938416 2024-04-24T15:34:03.083 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.083 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.083 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:03.084 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251197+0000", 2024-04-24T15:34:03.084 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9341000000000002e-05 2024-04-24T15:34:03.084 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.084 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.084 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:03.084 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251249+0000", 2024-04-24T15:34:03.084 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.1391000000000003e-05 2024-04-24T15:34:03.084 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:03.084 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:03.084 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:03.084 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.084 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.084 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18011 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12656640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:03.085 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208258+0000", 2024-04-24T15:34:03.085 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 88.856139325000001, 2024-04-24T15:34:03.085 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043102708000000003, 2024-04-24T15:34:03.085 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:03.085 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:03.085 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:03.085 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:03.085 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:03.085 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18011 2024-04-24T15:34:03.085 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.085 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:03.085 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.085 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:03.085 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208258+0000", 2024-04-24T15:34:03.085 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:03.086 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.086 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.086 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:03.086 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208258+0000", 2024-04-24T15:34:03.086 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:03.086 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.086 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.086 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:03.086 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208258+0000", 2024-04-24T15:34:03.086 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:03.086 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.086 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.086 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:03.086 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208260+0000", 2024-04-24T15:34:03.087 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7810000000000001e-06 2024-04-24T15:34:03.087 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.087 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.087 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:03.087 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208267+0000", 2024-04-24T15:34:03.087 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.7610000000000002e-06 2024-04-24T15:34:03.087 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.087 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.087 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:03.087 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208271+0000", 2024-04-24T15:34:03.087 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0890000000000002e-06 2024-04-24T15:34:03.087 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.087 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.087 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:03.088 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251271+0000", 2024-04-24T15:34:03.088 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043000159000000003 2024-04-24T15:34:03.088 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.088 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.088 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:03.088 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251315+0000", 2024-04-24T15:34:03.088 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3446999999999999e-05 2024-04-24T15:34:03.088 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.088 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.088 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:03.088 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251361+0000", 2024-04-24T15:34:03.088 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5952000000000003e-05 2024-04-24T15:34:03.088 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:03.088 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:03.089 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:03.089 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.089 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.089 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18012 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12697600~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:03.089 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208289+0000", 2024-04-24T15:34:03.089 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 88.856108281000004, 2024-04-24T15:34:03.089 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043174774999999999, 2024-04-24T15:34:03.089 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:03.089 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:03.089 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:03.089 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:03.089 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:03.089 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18012 2024-04-24T15:34:03.089 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.089 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:03.090 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.090 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:03.090 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208289+0000", 2024-04-24T15:34:03.090 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:03.090 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.090 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.090 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:03.090 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208289+0000", 2024-04-24T15:34:03.090 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:03.090 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.090 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.090 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:03.090 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208289+0000", 2024-04-24T15:34:03.091 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:03.091 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.091 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.091 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:03.091 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208291+0000", 2024-04-24T15:34:03.091 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.542e-06 2024-04-24T15:34:03.091 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.091 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.091 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:03.091 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208298+0000", 2024-04-24T15:34:03.091 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.3899999999999998e-06 2024-04-24T15:34:03.091 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.091 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.091 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:03.092 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208302+0000", 2024-04-24T15:34:03.092 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.019e-06 2024-04-24T15:34:03.092 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.092 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.092 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:03.092 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251382+0000", 2024-04-24T15:34:03.092 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043080790000000001 2024-04-24T15:34:03.092 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.092 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.092 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:03.092 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251421+0000", 2024-04-24T15:34:03.092 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8473000000000002e-05 2024-04-24T15:34:03.092 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.092 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.092 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:03.093 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251464+0000", 2024-04-24T15:34:03.093 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3133999999999997e-05 2024-04-24T15:34:03.093 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:03.093 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:03.093 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:03.093 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.093 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.093 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18013 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12738560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:03.093 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208423+0000", 2024-04-24T15:34:03.093 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 88.855974173000007, 2024-04-24T15:34:03.093 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043145554000000003, 2024-04-24T15:34:03.093 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:03.093 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:03.093 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:03.094 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:03.094 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:03.094 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18013 2024-04-24T15:34:03.094 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.094 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:03.094 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.094 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:03.094 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208423+0000", 2024-04-24T15:34:03.094 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:03.094 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.094 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.094 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:03.094 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208423+0000", 2024-04-24T15:34:03.094 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:03.095 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.095 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.095 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:03.095 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208423+0000", 2024-04-24T15:34:03.095 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:03.095 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.095 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.095 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:03.095 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208426+0000", 2024-04-24T15:34:03.095 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.0539999999999999e-06 2024-04-24T15:34:03.095 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.095 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.095 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:03.095 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208433+0000", 2024-04-24T15:34:03.096 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.43e-06 2024-04-24T15:34:03.096 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.096 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.096 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:03.096 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208437+0000", 2024-04-24T15:34:03.096 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5590000000000001e-06 2024-04-24T15:34:03.096 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.096 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.096 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:03.096 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251485+0000", 2024-04-24T15:34:03.096 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043048259999999998 2024-04-24T15:34:03.096 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.096 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.096 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:03.097 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251525+0000", 2024-04-24T15:34:03.097 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9916999999999997e-05 2024-04-24T15:34:03.097 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.097 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.097 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:03.097 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251569+0000", 2024-04-24T15:34:03.097 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3939000000000001e-05 2024-04-24T15:34:03.097 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:03.097 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:03.097 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:03.097 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.097 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.097 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18015 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12779520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:03.097 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208630+0000", 2024-04-24T15:34:03.098 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 88.855767647999997, 2024-04-24T15:34:03.098 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043047328000000003, 2024-04-24T15:34:03.098 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:03.098 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:03.098 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:03.098 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:03.098 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:03.098 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18015 2024-04-24T15:34:03.098 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.098 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:03.098 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.098 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:03.098 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208630+0000", 2024-04-24T15:34:03.098 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:03.099 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.099 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.099 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:03.099 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208630+0000", 2024-04-24T15:34:03.099 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:03.099 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.099 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.099 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:03.099 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208629+0000", 2024-04-24T15:34:03.099 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999999 2024-04-24T15:34:03.099 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.099 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.099 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:03.099 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208634+0000", 2024-04-24T15:34:03.100 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8960000000000002e-06 2024-04-24T15:34:03.100 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.100 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.100 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:03.100 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208645+0000", 2024-04-24T15:34:03.100 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0939e-05 2024-04-24T15:34:03.100 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.100 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.100 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:03.100 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208651+0000", 2024-04-24T15:34:03.100 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.896e-06 2024-04-24T15:34:03.100 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.100 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.100 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:03.101 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251591+0000", 2024-04-24T15:34:03.101 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042939862000000002 2024-04-24T15:34:03.101 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.101 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.101 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:03.101 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251631+0000", 2024-04-24T15:34:03.101 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0216000000000003e-05 2024-04-24T15:34:03.101 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.101 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.101 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:03.101 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251677+0000", 2024-04-24T15:34:03.101 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6390999999999997e-05 2024-04-24T15:34:03.101 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:03.102 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:03.102 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:03.102 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.102 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.102 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18016 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12820480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:03.102 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208660+0000", 2024-04-24T15:34:03.102 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 88.855737859000001, 2024-04-24T15:34:03.102 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043127510000000001, 2024-04-24T15:34:03.102 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:03.102 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:03.102 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:03.102 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:03.102 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:03.103 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18016 2024-04-24T15:34:03.103 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.103 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:03.103 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.103 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:03.103 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208660+0000", 2024-04-24T15:34:03.103 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:03.103 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.103 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.103 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:03.103 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208660+0000", 2024-04-24T15:34:03.103 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:03.103 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.103 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.103 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:03.104 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208659+0000", 2024-04-24T15:34:03.104 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:03.104 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.104 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.104 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:03.104 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208662+0000", 2024-04-24T15:34:03.104 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.0979999999999998e-06 2024-04-24T15:34:03.104 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.104 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.104 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:03.104 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208669+0000", 2024-04-24T15:34:03.104 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.4470000000000003e-06 2024-04-24T15:34:03.104 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.104 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.104 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:03.104 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208673+0000", 2024-04-24T15:34:03.104 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2810000000000002e-06 2024-04-24T15:34:03.105 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.105 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.105 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:03.105 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251699+0000", 2024-04-24T15:34:03.105 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043025728999999999 2024-04-24T15:34:03.105 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.105 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.105 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:03.105 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251743+0000", 2024-04-24T15:34:03.105 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4469999999999999e-05 2024-04-24T15:34:03.105 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.105 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.105 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:03.106 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251787+0000", 2024-04-24T15:34:03.106 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3949999999999998e-05 2024-04-24T15:34:03.106 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:03.106 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:03.106 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:03.106 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.106 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.106 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18017 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12861440~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:03.106 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208721+0000", 2024-04-24T15:34:03.106 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 88.855676411000005, 2024-04-24T15:34:03.106 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043167749999999998, 2024-04-24T15:34:03.106 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:03.106 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:03.106 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:03.107 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:03.107 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:03.107 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18017 2024-04-24T15:34:03.107 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.107 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:03.107 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.107 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:03.107 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208721+0000", 2024-04-24T15:34:03.107 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:03.107 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.107 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.107 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:03.108 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208721+0000", 2024-04-24T15:34:03.108 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:03.108 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.108 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.108 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:03.108 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208720+0000", 2024-04-24T15:34:03.108 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:03.108 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.108 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.108 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:03.108 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208724+0000", 2024-04-24T15:34:03.108 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.3919999999999999e-06 2024-04-24T15:34:03.108 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.108 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.108 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:03.109 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208729+0000", 2024-04-24T15:34:03.109 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.3480000000000003e-06 2024-04-24T15:34:03.109 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.109 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.109 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:03.109 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208732+0000", 2024-04-24T15:34:03.109 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.9529999999999999e-06 2024-04-24T15:34:03.109 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.109 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.109 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:03.109 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251807+0000", 2024-04-24T15:34:03.109 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043074492999999998 2024-04-24T15:34:03.109 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.109 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.109 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:03.109 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251846+0000", 2024-04-24T15:34:03.110 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8958999999999997e-05 2024-04-24T15:34:03.110 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.110 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.110 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:03.110 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251889+0000", 2024-04-24T15:34:03.110 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3253000000000001e-05 2024-04-24T15:34:03.110 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:03.110 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:03.110 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:03.110 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.110 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.110 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18018 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12902400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:03.110 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208752+0000", 2024-04-24T15:34:03.110 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 88.855645797999998, 2024-04-24T15:34:03.111 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043241012000000002, 2024-04-24T15:34:03.111 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:03.111 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:03.111 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:03.111 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:03.111 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:03.111 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18018 2024-04-24T15:34:03.111 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.111 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:03.111 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.111 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:03.111 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208752+0000", 2024-04-24T15:34:03.111 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:03.111 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.111 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.112 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:03.112 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208752+0000", 2024-04-24T15:34:03.112 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:03.112 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.112 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.112 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:03.112 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208751+0000", 2024-04-24T15:34:03.112 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:03.112 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.112 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.112 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:03.112 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208753+0000", 2024-04-24T15:34:03.112 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.894e-06 2024-04-24T15:34:03.112 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.112 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.112 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:03.113 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208757+0000", 2024-04-24T15:34:03.113 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.856e-06 2024-04-24T15:34:03.113 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.113 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.113 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:03.113 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208760+0000", 2024-04-24T15:34:03.113 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.6749999999999998e-06 2024-04-24T15:34:03.113 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.113 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.113 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:03.113 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251909+0000", 2024-04-24T15:34:03.113 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043149145999999999 2024-04-24T15:34:03.113 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.113 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.113 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:03.114 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251948+0000", 2024-04-24T15:34:03.114 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9159000000000002e-05 2024-04-24T15:34:03.114 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.114 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.114 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:03.114 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251993+0000", 2024-04-24T15:34:03.114 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4638999999999997e-05 2024-04-24T15:34:03.114 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:03.114 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:03.114 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:03.114 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.114 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.114 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18019 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12943360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:03.114 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208938+0000", 2024-04-24T15:34:03.115 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 88.855459507000006, 2024-04-24T15:34:03.115 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043162272000000002, 2024-04-24T15:34:03.115 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:03.115 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:03.115 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:03.115 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:03.115 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:03.115 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18019 2024-04-24T15:34:03.115 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.115 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:03.115 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.115 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:03.115 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208938+0000", 2024-04-24T15:34:03.115 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:03.115 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.116 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.116 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:03.116 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208938+0000", 2024-04-24T15:34:03.116 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:03.116 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.116 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.116 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:03.116 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208762+0000", 2024-04-24T15:34:03.116 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999824 2024-04-24T15:34:03.116 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.116 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.116 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:03.116 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208942+0000", 2024-04-24T15:34:03.116 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000180008 2024-04-24T15:34:03.116 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.117 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.117 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:03.117 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208950+0000", 2024-04-24T15:34:03.117 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.1550000000000007e-06 2024-04-24T15:34:03.117 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.117 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.117 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:03.117 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208956+0000", 2024-04-24T15:34:03.117 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.7910000000000001e-06 2024-04-24T15:34:03.117 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.117 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.117 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:03.117 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252014+0000", 2024-04-24T15:34:03.117 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043058427000000003 2024-04-24T15:34:03.117 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.118 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.118 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:03.118 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252054+0000", 2024-04-24T15:34:03.118 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0071999999999999e-05 2024-04-24T15:34:03.118 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.118 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.118 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:03.118 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252100+0000", 2024-04-24T15:34:03.118 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5917000000000001e-05 2024-04-24T15:34:03.118 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:03.118 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:03.118 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:03.118 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.118 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.118 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18020 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12984320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:03.119 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.209092+0000", 2024-04-24T15:34:03.119 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 88.855305509000004, 2024-04-24T15:34:03.119 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043111295000000001, 2024-04-24T15:34:03.119 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:03.119 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:03.119 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:03.119 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:03.119 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:03.119 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18020 2024-04-24T15:34:03.119 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.119 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:03.119 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.119 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:03.119 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209092+0000", 2024-04-24T15:34:03.119 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:03.120 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.120 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.120 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:03.120 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209092+0000", 2024-04-24T15:34:03.120 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:03.120 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.120 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.120 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:03.120 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208964+0000", 2024-04-24T15:34:03.120 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998722 2024-04-24T15:34:03.120 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.120 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.120 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:03.120 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209095+0000", 2024-04-24T15:34:03.120 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000130648 2024-04-24T15:34:03.121 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.121 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.121 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:03.121 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209103+0000", 2024-04-24T15:34:03.121 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0320000000000003e-06 2024-04-24T15:34:03.121 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.121 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.121 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:03.121 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209108+0000", 2024-04-24T15:34:03.121 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8690000000000003e-06 2024-04-24T15:34:03.121 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.121 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.121 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:03.121 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252121+0000", 2024-04-24T15:34:03.122 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043013323999999999 2024-04-24T15:34:03.122 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.122 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.122 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:03.122 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252160+0000", 2024-04-24T15:34:03.122 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8376e-05 2024-04-24T15:34:03.122 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.122 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.122 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:03.122 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252203+0000", 2024-04-24T15:34:03.122 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.371e-05 2024-04-24T15:34:03.122 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:03.122 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:03.123 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:03.123 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.123 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.123 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18021 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 13025280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:03.123 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.209132+0000", 2024-04-24T15:34:03.123 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 88.855265150999998, 2024-04-24T15:34:03.123 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043183903000000003, 2024-04-24T15:34:03.123 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:03.123 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:03.123 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:03.123 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:03.123 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:03.123 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18021 2024-04-24T15:34:03.123 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.124 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:03.124 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.124 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:03.124 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209132+0000", 2024-04-24T15:34:03.124 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:03.124 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.124 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.124 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:03.124 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209132+0000", 2024-04-24T15:34:03.124 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:03.124 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.124 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.124 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:03.124 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209132+0000", 2024-04-24T15:34:03.125 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:03.125 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.125 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.125 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:03.125 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209134+0000", 2024-04-24T15:34:03.125 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.891e-06 2024-04-24T15:34:03.125 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.125 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.125 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:03.125 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209138+0000", 2024-04-24T15:34:03.125 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0659999999999997e-06 2024-04-24T15:34:03.125 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.125 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.126 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:03.126 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209140+0000", 2024-04-24T15:34:03.126 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.5339999999999998e-06 2024-04-24T15:34:03.126 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.126 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.126 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:03.126 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252232+0000", 2024-04-24T15:34:03.126 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043091142999999998 2024-04-24T15:34:03.126 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.126 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.126 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:03.126 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252270+0000", 2024-04-24T15:34:03.126 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8872000000000003e-05 2024-04-24T15:34:03.127 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.127 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.127 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:03.127 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252316+0000", 2024-04-24T15:34:03.127 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5788000000000003e-05 2024-04-24T15:34:03.127 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:03.127 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:03.127 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:03.127 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.127 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.127 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18022 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 13066240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:03.127 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.209411+0000", 2024-04-24T15:34:03.127 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 88.854986603, 2024-04-24T15:34:03.127 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043016616000000001, 2024-04-24T15:34:03.127 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:03.128 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:03.128 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:03.128 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:03.128 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:03.128 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18022 2024-04-24T15:34:03.128 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.128 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:03.128 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.128 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:03.128 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209411+0000", 2024-04-24T15:34:03.128 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:03.128 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.128 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.129 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:03.129 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209411+0000", 2024-04-24T15:34:03.129 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:03.129 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.129 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.129 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:03.129 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209143+0000", 2024-04-24T15:34:03.129 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999732 2024-04-24T15:34:03.129 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.129 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.129 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:03.129 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209415+0000", 2024-04-24T15:34:03.129 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000271873 2024-04-24T15:34:03.129 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.130 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.130 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:03.130 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209423+0000", 2024-04-24T15:34:03.130 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3080000000000008e-06 2024-04-24T15:34:03.130 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.130 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.130 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:03.130 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209428+0000", 2024-04-24T15:34:03.130 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.5040000000000002e-06 2024-04-24T15:34:03.130 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.130 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.130 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:03.130 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252339+0000", 2024-04-24T15:34:03.130 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042910419999999998 2024-04-24T15:34:03.131 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.131 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.131 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:03.131 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252380+0000", 2024-04-24T15:34:03.131 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0991000000000001e-05 2024-04-24T15:34:03.131 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.131 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.131 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:03.131 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252427+0000", 2024-04-24T15:34:03.131 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.7647e-05 2024-04-24T15:34:03.131 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:03.131 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:03.131 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:03.131 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.131 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.132 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18023 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 13107200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:03.132 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.209594+0000", 2024-04-24T15:34:03.132 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 88.854803090000004, 2024-04-24T15:34:03.132 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042938543000000003, 2024-04-24T15:34:03.132 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:03.132 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:03.132 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:03.132 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:03.132 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:03.132 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18023 2024-04-24T15:34:03.132 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.132 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:03.132 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.132 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:03.133 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209594+0000", 2024-04-24T15:34:03.133 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:03.133 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.133 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.133 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:03.133 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209594+0000", 2024-04-24T15:34:03.133 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:03.133 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.133 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.133 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:03.133 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209440+0000", 2024-04-24T15:34:03.133 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999846 2024-04-24T15:34:03.133 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.133 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.133 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:03.134 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209598+0000", 2024-04-24T15:34:03.134 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00015731100000000001 2024-04-24T15:34:03.134 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.134 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.134 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:03.134 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209604+0000", 2024-04-24T15:34:03.134 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.2910000000000004e-06 2024-04-24T15:34:03.134 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.134 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.134 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:03.134 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209607+0000", 2024-04-24T15:34:03.134 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5389999999999999e-06 2024-04-24T15:34:03.134 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.134 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.134 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:03.135 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252449+0000", 2024-04-24T15:34:03.135 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042841833000000003 2024-04-24T15:34:03.135 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.135 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.135 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:03.135 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252489+0000", 2024-04-24T15:34:03.135 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9898000000000003e-05 2024-04-24T15:34:03.135 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.135 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.135 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:03.135 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252533+0000", 2024-04-24T15:34:03.135 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3702999999999998e-05 2024-04-24T15:34:03.135 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:03.135 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:03.135 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:03.136 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.136 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.136 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18025 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 13189120~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:03.136 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.209832+0000", 2024-04-24T15:34:03.136 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 88.854565589000003, 2024-04-24T15:34:03.136 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042913556999999998, 2024-04-24T15:34:03.136 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:03.136 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:03.136 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:03.136 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:03.136 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:03.136 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18025 2024-04-24T15:34:03.136 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.136 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:03.136 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.137 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:03.137 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209832+0000", 2024-04-24T15:34:03.137 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:03.137 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.137 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.137 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:03.137 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209832+0000", 2024-04-24T15:34:03.137 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:03.137 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.137 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.137 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:03.137 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209774+0000", 2024-04-24T15:34:03.137 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999418 2024-04-24T15:34:03.137 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.137 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.138 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:03.138 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209835+0000", 2024-04-24T15:34:03.138 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.1377000000000005e-05 2024-04-24T15:34:03.138 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.138 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.138 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:03.138 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209843+0000", 2024-04-24T15:34:03.138 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.2330000000000006e-06 2024-04-24T15:34:03.138 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.138 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.138 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:03.138 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209848+0000", 2024-04-24T15:34:03.138 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6149999999999999e-06 2024-04-24T15:34:03.138 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.139 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.139 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:03.139 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252658+0000", 2024-04-24T15:34:03.139 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042810002 2024-04-24T15:34:03.139 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.139 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.139 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:03.139 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252697+0000", 2024-04-24T15:34:03.139 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8757999999999997e-05 2024-04-24T15:34:03.139 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.139 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.139 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:03.139 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252745+0000", 2024-04-24T15:34:03.139 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.863e-05 2024-04-24T15:34:03.139 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:03.139 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:03.140 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:03.140 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.140 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.140 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:24749 4.8 4:137f5701:::rbd_header.114ae15b9a20:head [watch reconnect cookie 140672595753696 gen 11] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-04-24T15:34:03.140 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:37.628703+0000", 2024-04-24T15:34:03.140 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 85.435693970000003, 2024-04-24T15:34:03.140 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0102918700000001, 2024-04-24T15:34:03.140 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:03.140 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:03.140 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:03.140 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:03.140 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:03.140 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 24749 2024-04-24T15:34:03.140 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.140 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:03.141 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.141 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:03.141 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628703+0000", 2024-04-24T15:34:03.141 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:03.141 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.141 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.141 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:03.141 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628703+0000", 2024-04-24T15:34:03.141 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:03.141 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.141 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.141 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:03.141 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628703+0000", 2024-04-24T15:34:03.141 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999999 2024-04-24T15:34:03.141 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.142 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.142 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:03.142 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628708+0000", 2024-04-24T15:34:03.142 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.4589999999999997e-06 2024-04-24T15:34:03.142 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.142 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.142 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:03.142 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628721+0000", 2024-04-24T15:34:03.142 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.2601e-05 2024-04-24T15:34:03.142 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.142 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.142 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:03.142 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628728+0000", 2024-04-24T15:34:03.142 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.9059999999999996e-06 2024-04-24T15:34:03.143 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.143 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.143 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:03.143 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628789+0000", 2024-04-24T15:34:03.143 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.1672000000000002e-05 2024-04-24T15:34:03.143 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.143 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.143 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting_for_map not empty", 2024-04-24T15:34:03.143 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628797+0000", 2024-04-24T15:34:03.143 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.8189999999999992e-06 2024-04-24T15:34:03.143 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.143 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.143 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:03.143 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.631255+0000", 2024-04-24T15:34:03.144 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0024577610000000001 2024-04-24T15:34:03.144 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.144 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.144 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for peered", 2024-04-24T15:34:03.144 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.631263+0000", 2024-04-24T15:34:03.144 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.4109999999999996e-06 2024-04-24T15:34:03.144 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.144 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.144 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:03.144 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:38.637560+0000", 2024-04-24T15:34:03.144 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0062970339999999 2024-04-24T15:34:03.144 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.144 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.144 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:03.144 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:38.637747+0000", 2024-04-24T15:34:03.145 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00018693400000000001 2024-04-24T15:34:03.145 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.145 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.145 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:03.145 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:38.638995+0000", 2024-04-24T15:34:03.145 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0012481300000000001 2024-04-24T15:34:03.145 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:03.145 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:03.145 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:03.145 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.145 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.145 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:24750 4.8 4:137f5701:::rbd_header.114ae15b9a20:head [notify cookie 140672594434416] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-04-24T15:34:03.145 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:37.628736+0000", 2024-04-24T15:34:03.145 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 85.435661710000005, 2024-04-24T15:34:03.145 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0096094309999999, 2024-04-24T15:34:03.145 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:03.146 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:03.146 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:03.146 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:03.146 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:03.146 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 24750 2024-04-24T15:34:03.146 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.146 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:03.146 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.146 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:03.146 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628736+0000", 2024-04-24T15:34:03.146 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:03.146 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.146 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.146 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:03.147 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628736+0000", 2024-04-24T15:34:03.147 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:03.147 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.147 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.147 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:03.147 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628735+0000", 2024-04-24T15:34:03.147 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:03.147 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.147 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.147 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:03.147 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628739+0000", 2024-04-24T15:34:03.147 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5109999999999999e-06 2024-04-24T15:34:03.147 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.147 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.147 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:03.147 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628745+0000", 2024-04-24T15:34:03.148 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.8300000000000001e-06 2024-04-24T15:34:03.148 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.148 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.148 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:03.148 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628749+0000", 2024-04-24T15:34:03.148 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.7110000000000003e-06 2024-04-24T15:34:03.148 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.148 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.148 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:03.148 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628826+0000", 2024-04-24T15:34:03.148 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.6678e-05 2024-04-24T15:34:03.148 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.148 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.148 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting_for_map not empty", 2024-04-24T15:34:03.148 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628834+0000", 2024-04-24T15:34:03.149 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.6190000000000002e-06 2024-04-24T15:34:03.149 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.149 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.149 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:03.149 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.631305+0000", 2024-04-24T15:34:03.149 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0024714400000000001 2024-04-24T15:34:03.149 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.149 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.149 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for peered", 2024-04-24T15:34:03.149 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.631313+0000", 2024-04-24T15:34:03.149 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.6529999999999994e-06 2024-04-24T15:34:03.149 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.149 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.149 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:03.149 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:38.638068+0000", 2024-04-24T15:34:03.149 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0067548369999999 2024-04-24T15:34:03.150 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.150 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.150 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:03.150 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:38.638158+0000", 2024-04-24T15:34:03.150 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.0079000000000003e-05 2024-04-24T15:34:03.150 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.150 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.150 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:03.150 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:38.638345+0000", 2024-04-24T15:34:03.150 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00018757100000000001 2024-04-24T15:34:03.150 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:03.150 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:03.150 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:03.150 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.150 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.151 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:76870 4.0 4:0746368d:::rbd_data.114abb9508e9.0000000000000001:head [delete] snapc f=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-04-24T15:34:03.151 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:56.733703+0000", 2024-04-24T15:34:03.151 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 6.3306942819999996, 2024-04-24T15:34:03.151 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.91258213700000002, 2024-04-24T15:34:03.151 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:03.151 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:34:03.151 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:03.151 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:03.151 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:03.151 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 76870 2024-04-24T15:34:03.151 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.151 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:03.151 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.151 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:03.151 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:56.733703+0000", 2024-04-24T15:34:03.152 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:03.152 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.152 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.152 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:03.152 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:56.733703+0000", 2024-04-24T15:34:03.152 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:03.152 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.152 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.152 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:03.152 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:56.733702+0000", 2024-04-24T15:34:03.152 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999999 2024-04-24T15:34:03.152 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.152 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.152 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:03.152 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:56.733708+0000", 2024-04-24T15:34:03.153 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6060000000000002e-06 2024-04-24T15:34:03.153 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.153 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.153 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:03.153 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:56.733720+0000", 2024-04-24T15:34:03.153 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.1727e-05 2024-04-24T15:34:03.153 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.153 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.153 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:03.153 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:56.733725+0000", 2024-04-24T15:34:03.153 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.1800000000000004e-06 2024-04-24T15:34:03.153 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.153 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.153 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:03.153 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:56.733775+0000", 2024-04-24T15:34:03.154 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.0692000000000002e-05 2024-04-24T15:34:03.154 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.154 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.154 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for peered", 2024-04-24T15:34:03.154 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:56.733787+0000", 2024-04-24T15:34:03.154 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.1341e-05 2024-04-24T15:34:03.154 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.154 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.154 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:03.154 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:57.642929+0000", 2024-04-24T15:34:03.154 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.90914176199999996 2024-04-24T15:34:03.154 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.154 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.154 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:03.154 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:57.643088+0000", 2024-04-24T15:34:03.155 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00015902599999999999 2024-04-24T15:34:03.155 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.155 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.155 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 2", 2024-04-24T15:34:03.155 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:57.643227+0000", 2024-04-24T15:34:03.155 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00013894499999999999 2024-04-24T15:34:03.155 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.155 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.155 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:34:03.155 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:57.644936+0000", 2024-04-24T15:34:03.155 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0017091490000000001 2024-04-24T15:34:03.155 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.155 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.155 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:34:03.155 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:57.646233+0000", 2024-04-24T15:34:03.155 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0012974880000000001 2024-04-24T15:34:03.156 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.156 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.156 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:34:03.156 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:57.646263+0000", 2024-04-24T15:34:03.156 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.9609000000000001e-05 2024-04-24T15:34:03.156 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:03.156 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:03.156 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:03.156 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:57.646285+0000", 2024-04-24T15:34:03.156 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.2509000000000001e-05 2024-04-24T15:34:03.156 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:03.156 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:03.156 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:03.156 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:03.156 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:03.156 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:34:03.158 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-04-24T15:34:03.158 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-04-24T15:34:03.158 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:34:03.159+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:34:03.159 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-24T15:34:03.169 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-04-24T15:34:03.187 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-04-24T15:34:03.205 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-04-24T15:34:03.224 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-04-24T15:34:03.242 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-04-24T15:34:03.258 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:34:03.260+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:34:03.260 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-04-24T15:34:03.262 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T15:34:03.262 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T15:34:03.262 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T15:34:03.262 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T15:34:03.262 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T15:34:03.262 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T15:34:03.262 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T15:34:03.262 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T15:34:03.284 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:34:03.284 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:34:03.294 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-24T15:34:03.378 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:34:03.380+0000 7f0f45efb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 59992) UID: 0 2024-04-24T15:34:03.458 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:34:03.458 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:34:03.461 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:34:03.462+0000 7f0f45efb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 59992) UID: 0 2024-04-24T15:34:03.468 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-24T15:34:03.562 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:34:03.563+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:34:03.630 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:34:03.631 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:34:03.640 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-24T15:34:03.683 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:34:03.685+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:34:03.764 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:34:03.765+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:34:03.774 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:34:03.774 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:34:03.786 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-24T15:34:03.865 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:34:03.867+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:34:03.950 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:34:03.950 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:34:03.960 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-24T15:34:04.004 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:34:04.005+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:34:04.067 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:34:04.069+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:34:04.082 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:34:04.083 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:34:04.092 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-24T15:34:04.177 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:34:04.178+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:34:04.255 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:34:04.255 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:34:04.265 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-24T15:34:04.269 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:34:04.271+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:34:04.370 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:34:04.372+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:34:04.427 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:34:04.428 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:34:04.437 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-24T15:34:04.479 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:34:04.480+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:34:04.572 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:34:04.573+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:34:04.601 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:34:04.601 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:34:04.607 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 6, 4, 7] out_osds: [1, 5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 7, 6] 2024-04-24T15:34:04.608 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-24T15:34:04.608 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-04-24T15:34:04.608 INFO:tasks.thrashosds.thrasher:Setting osd 6 primary_affinity to 0.827554 2024-04-24T15:34:04.608 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 0.8275539996520849 2024-04-24T15:34:04.611 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-24T15:34:04.660 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-24T15:34:04.672 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:34:04.674+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:34:04.773 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:34:04.774+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:34:04.785 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:34:04.785 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:34:04.796 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-24T15:34:04.882 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:34:04.884+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:34:04.966 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:34:04.967 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:34:04.975 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:34:04.976+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:34:04.976 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-24T15:34:05.076 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:34:05.077+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:34:05.138 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:34:05.138 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:34:05.148 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-24T15:34:05.192 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:34:05.193+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:34:05.270 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:34:05.271 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:34:05.273 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-24T15:34:05.278 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:34:05.279+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:34:05.280 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-24T15:34:05.379 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:34:05.380+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:34:05.417 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-04-24T15:34:05.435 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-04-24T15:34:05.450 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:34:05.450 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:34:05.453 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-04-24T15:34:05.461 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-24T15:34:05.471 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-04-24T15:34:05.490 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-04-24T15:34:05.490 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:34:05.492+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:34:05.509 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-04-24T15:34:05.528 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-04-24T15:34:05.546 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-04-24T15:34:05.548 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T15:34:05.548 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T15:34:05.548 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T15:34:05.548 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T15:34:05.548 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T15:34:05.548 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T15:34:05.548 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T15:34:05.548 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T15:34:05.598 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:34:05.598 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-04-24T15:34:05.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-04-24T15:34:05.598 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-04-24T15:34:05.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.598 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14043 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2383872~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:05.598 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892347+0000", 2024-04-24T15:34:05.598 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 91.705881911999995, 2024-04-24T15:34:05.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038537279000000001, 2024-04-24T15:34:05.599 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:34:05.599 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:34:05.599 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:34:05.599 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:34:05.599 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:05.599 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14043 2024-04-24T15:34:05.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.599 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:34:05.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:34:05.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892347+0000", 2024-04-24T15:34:05.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:05.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:34:05.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892347+0000", 2024-04-24T15:34:05.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:05.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:34:05.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892347+0000", 2024-04-24T15:34:05.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:05.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:34:05.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892349+0000", 2024-04-24T15:34:05.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3400000000000001e-06 2024-04-24T15:34:05.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:34:05.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892357+0000", 2024-04-24T15:34:05.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1540000000000002e-06 2024-04-24T15:34:05.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:34:05.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892360+0000", 2024-04-24T15:34:05.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5829999999999998e-06 2024-04-24T15:34:05.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:34:05.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930795+0000", 2024-04-24T15:34:05.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038434590999999997 2024-04-24T15:34:05.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:34:05.602 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930836+0000", 2024-04-24T15:34:05.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0827000000000001e-05 2024-04-24T15:34:05.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:34:05.602 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930885+0000", 2024-04-24T15:34:05.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9029000000000001e-05 2024-04-24T15:34:05.602 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:05.602 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:34:05.602 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:05.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.602 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14044 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2424832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:05.602 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892372+0000", 2024-04-24T15:34:05.603 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 91.705857268000003, 2024-04-24T15:34:05.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038624890000000002, 2024-04-24T15:34:05.603 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:34:05.603 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:34:05.603 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:34:05.603 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:34:05.603 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:05.603 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14044 2024-04-24T15:34:05.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.603 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:34:05.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:34:05.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892372+0000", 2024-04-24T15:34:05.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:05.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:34:05.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892372+0000", 2024-04-24T15:34:05.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:05.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:34:05.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892372+0000", 2024-04-24T15:34:05.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:05.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:34:05.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892374+0000", 2024-04-24T15:34:05.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4899999999999999e-06 2024-04-24T15:34:05.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:34:05.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892380+0000", 2024-04-24T15:34:05.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.631e-06 2024-04-24T15:34:05.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:34:05.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892383+0000", 2024-04-24T15:34:05.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7400000000000002e-06 2024-04-24T15:34:05.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:34:05.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930905+0000", 2024-04-24T15:34:05.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038521396999999999 2024-04-24T15:34:05.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:34:05.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930949+0000", 2024-04-24T15:34:05.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4153999999999998e-05 2024-04-24T15:34:05.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:34:05.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.930997+0000", 2024-04-24T15:34:05.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7998999999999999e-05 2024-04-24T15:34:05.606 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:05.606 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:34:05.606 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:05.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.606 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14045 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2465792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:05.606 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892387+0000", 2024-04-24T15:34:05.606 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 91.705842000999994, 2024-04-24T15:34:05.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038720933999999999, 2024-04-24T15:34:05.607 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:34:05.607 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:34:05.607 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:34:05.607 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:34:05.607 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:05.607 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14045 2024-04-24T15:34:05.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.607 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:34:05.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:34:05.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892387+0000", 2024-04-24T15:34:05.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:05.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:34:05.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892387+0000", 2024-04-24T15:34:05.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:05.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:34:05.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892387+0000", 2024-04-24T15:34:05.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:05.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:34:05.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892392+0000", 2024-04-24T15:34:05.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0039999999999999e-06 2024-04-24T15:34:05.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:34:05.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892398+0000", 2024-04-24T15:34:05.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0900000000000001e-06 2024-04-24T15:34:05.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:34:05.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892402+0000", 2024-04-24T15:34:05.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.399e-06 2024-04-24T15:34:05.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:34:05.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931018+0000", 2024-04-24T15:34:05.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038616551999999998 2024-04-24T15:34:05.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:34:05.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931059+0000", 2024-04-24T15:34:05.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1205000000000003e-05 2024-04-24T15:34:05.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:34:05.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931108+0000", 2024-04-24T15:34:05.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8996000000000003e-05 2024-04-24T15:34:05.610 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:05.610 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:34:05.610 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:05.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.610 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14046 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2506752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:05.610 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892406+0000", 2024-04-24T15:34:05.610 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 91.705823410999997, 2024-04-24T15:34:05.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038814507999999998, 2024-04-24T15:34:05.611 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:34:05.611 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:34:05.611 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:34:05.611 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:34:05.611 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:05.611 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14046 2024-04-24T15:34:05.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.611 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:34:05.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:34:05.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892406+0000", 2024-04-24T15:34:05.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:05.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:34:05.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892406+0000", 2024-04-24T15:34:05.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:05.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:34:05.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892406+0000", 2024-04-24T15:34:05.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:05.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:34:05.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892413+0000", 2024-04-24T15:34:05.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7449999999999995e-06 2024-04-24T15:34:05.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:34:05.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892419+0000", 2024-04-24T15:34:05.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5790000000000003e-06 2024-04-24T15:34:05.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:34:05.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892423+0000", 2024-04-24T15:34:05.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8199999999999998e-06 2024-04-24T15:34:05.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:34:05.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931128+0000", 2024-04-24T15:34:05.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038705502000000003 2024-04-24T15:34:05.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:34:05.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931172+0000", 2024-04-24T15:34:05.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4066999999999997e-05 2024-04-24T15:34:05.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.614 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:34:05.614 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931220+0000", 2024-04-24T15:34:05.614 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8167999999999997e-05 2024-04-24T15:34:05.614 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:05.614 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:34:05.614 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:05.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.614 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14047 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2547712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:05.614 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892427+0000", 2024-04-24T15:34:05.614 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 91.705802181999999, 2024-04-24T15:34:05.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038904506999999998, 2024-04-24T15:34:05.615 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:34:05.615 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:34:05.615 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:34:05.615 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:34:05.615 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:05.615 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14047 2024-04-24T15:34:05.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.615 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:34:05.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:34:05.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892427+0000", 2024-04-24T15:34:05.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:05.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:34:05.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892427+0000", 2024-04-24T15:34:05.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:05.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:34:05.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892427+0000", 2024-04-24T15:34:05.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:05.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:34:05.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892429+0000", 2024-04-24T15:34:05.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3099999999999999e-06 2024-04-24T15:34:05.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:34:05.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892435+0000", 2024-04-24T15:34:05.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5520000000000004e-06 2024-04-24T15:34:05.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:34:05.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892438+0000", 2024-04-24T15:34:05.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4240000000000002e-06 2024-04-24T15:34:05.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:34:05.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931242+0000", 2024-04-24T15:34:05.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038804162000000003 2024-04-24T15:34:05.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:34:05.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931284+0000", 2024-04-24T15:34:05.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1414e-05 2024-04-24T15:34:05.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.618 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:34:05.618 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931332+0000", 2024-04-24T15:34:05.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8024e-05 2024-04-24T15:34:05.618 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:05.618 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:34:05.618 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:05.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.618 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14048 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2588672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:05.618 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892442+0000", 2024-04-24T15:34:05.618 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 91.705787219000001, 2024-04-24T15:34:05.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039003132000000003, 2024-04-24T15:34:05.619 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:34:05.619 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:34:05.619 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:34:05.619 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:34:05.619 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:05.619 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14048 2024-04-24T15:34:05.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.619 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:34:05.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:34:05.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892442+0000", 2024-04-24T15:34:05.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:05.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:34:05.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892442+0000", 2024-04-24T15:34:05.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:05.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:34:05.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892442+0000", 2024-04-24T15:34:05.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:05.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:34:05.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892444+0000", 2024-04-24T15:34:05.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1169999999999998e-06 2024-04-24T15:34:05.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:34:05.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892449+0000", 2024-04-24T15:34:05.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1599999999999997e-06 2024-04-24T15:34:05.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.621 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:34:05.621 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892453+0000", 2024-04-24T15:34:05.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5930000000000002e-06 2024-04-24T15:34:05.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.621 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:34:05.621 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931352+0000", 2024-04-24T15:34:05.622 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038899361 2024-04-24T15:34:05.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.622 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.622 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:34:05.622 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931398+0000", 2024-04-24T15:34:05.622 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6315e-05 2024-04-24T15:34:05.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.622 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.622 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:34:05.623 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931445+0000", 2024-04-24T15:34:05.623 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6950000000000003e-05 2024-04-24T15:34:05.623 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:05.623 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:34:05.623 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:05.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.623 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14049 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2629632~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:05.623 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892458+0000", 2024-04-24T15:34:05.623 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 91.705771799000004, 2024-04-24T15:34:05.623 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039103322000000003, 2024-04-24T15:34:05.623 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:34:05.623 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:34:05.623 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:34:05.623 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:34:05.624 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:05.624 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14049 2024-04-24T15:34:05.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.624 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:34:05.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.624 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:34:05.624 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892458+0000", 2024-04-24T15:34:05.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:05.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.624 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:34:05.624 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892458+0000", 2024-04-24T15:34:05.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:05.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.625 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:34:05.625 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892457+0000", 2024-04-24T15:34:05.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:05.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.625 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:34:05.625 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892459+0000", 2024-04-24T15:34:05.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.305e-06 2024-04-24T15:34:05.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.625 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:34:05.625 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892465+0000", 2024-04-24T15:34:05.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4360000000000001e-06 2024-04-24T15:34:05.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.626 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:34:05.626 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892474+0000", 2024-04-24T15:34:05.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7639999999999994e-06 2024-04-24T15:34:05.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.626 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:34:05.626 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931466+0000", 2024-04-24T15:34:05.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038992644 2024-04-24T15:34:05.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.626 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:34:05.626 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931512+0000", 2024-04-24T15:34:05.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5200000000000001e-05 2024-04-24T15:34:05.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:34:05.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931561+0000", 2024-04-24T15:34:05.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9357999999999997e-05 2024-04-24T15:34:05.627 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:05.627 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:34:05.627 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:05.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.627 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14050 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2670592~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:05.627 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892478+0000", 2024-04-24T15:34:05.627 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 91.705751211999996, 2024-04-24T15:34:05.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039195102000000002, 2024-04-24T15:34:05.627 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:34:05.627 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:34:05.627 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:34:05.628 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:34:05.628 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:05.628 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14050 2024-04-24T15:34:05.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.628 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:34:05.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:34:05.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892478+0000", 2024-04-24T15:34:05.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:05.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:34:05.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892478+0000", 2024-04-24T15:34:05.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:05.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:34:05.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892478+0000", 2024-04-24T15:34:05.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:05.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:34:05.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892480+0000", 2024-04-24T15:34:05.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3300000000000001e-06 2024-04-24T15:34:05.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:34:05.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892486+0000", 2024-04-24T15:34:05.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9200000000000001e-06 2024-04-24T15:34:05.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:34:05.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892494+0000", 2024-04-24T15:34:05.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5159999999999997e-06 2024-04-24T15:34:05.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:34:05.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931581+0000", 2024-04-24T15:34:05.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039087894999999998 2024-04-24T15:34:05.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:34:05.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931626+0000", 2024-04-24T15:34:05.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4943e-05 2024-04-24T15:34:05.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:34:05.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931673+0000", 2024-04-24T15:34:05.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6906000000000001e-05 2024-04-24T15:34:05.631 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:05.631 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:34:05.631 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:05.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.631 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14051 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2711552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:05.631 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892507+0000", 2024-04-24T15:34:05.631 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 91.705722512999998, 2024-04-24T15:34:05.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039274393999999997, 2024-04-24T15:34:05.631 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:34:05.632 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:34:05.632 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:34:05.632 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:34:05.632 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:05.632 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14051 2024-04-24T15:34:05.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.632 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:34:05.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:34:05.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892507+0000", 2024-04-24T15:34:05.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:05.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:34:05.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892507+0000", 2024-04-24T15:34:05.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:05.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.633 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:34:05.633 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892506+0000", 2024-04-24T15:34:05.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:05.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.633 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:34:05.633 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892509+0000", 2024-04-24T15:34:05.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8760000000000001e-06 2024-04-24T15:34:05.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.633 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:34:05.633 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892515+0000", 2024-04-24T15:34:05.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1840000000000004e-06 2024-04-24T15:34:05.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:34:05.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892519+0000", 2024-04-24T15:34:05.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7570000000000002e-06 2024-04-24T15:34:05.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:34:05.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931694+0000", 2024-04-24T15:34:05.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039174960000000002 2024-04-24T15:34:05.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:34:05.635 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931734+0000", 2024-04-24T15:34:05.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0284000000000003e-05 2024-04-24T15:34:05.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:34:05.635 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931781+0000", 2024-04-24T15:34:05.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6813000000000001e-05 2024-04-24T15:34:05.635 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:05.635 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:34:05.635 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:05.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.635 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14052 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2752512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:05.635 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892524+0000", 2024-04-24T15:34:05.635 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 91.705705366000004, 2024-04-24T15:34:05.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039371122000000001, 2024-04-24T15:34:05.636 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:34:05.636 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:34:05.636 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:34:05.636 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:34:05.636 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:05.636 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14052 2024-04-24T15:34:05.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.636 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:34:05.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:34:05.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892524+0000", 2024-04-24T15:34:05.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:05.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:34:05.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892524+0000", 2024-04-24T15:34:05.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:05.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.637 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:34:05.637 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892524+0000", 2024-04-24T15:34:05.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:05.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.637 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:34:05.637 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892526+0000", 2024-04-24T15:34:05.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3630000000000001e-06 2024-04-24T15:34:05.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.637 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:34:05.637 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892538+0000", 2024-04-24T15:34:05.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1677e-05 2024-04-24T15:34:05.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.638 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:34:05.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892541+0000", 2024-04-24T15:34:05.638 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.822e-06 2024-04-24T15:34:05.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.638 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:34:05.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931802+0000", 2024-04-24T15:34:05.638 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039260911000000003 2024-04-24T15:34:05.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.638 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:34:05.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931847+0000", 2024-04-24T15:34:05.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4450999999999998e-05 2024-04-24T15:34:05.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.639 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:34:05.639 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931895+0000", 2024-04-24T15:34:05.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8257999999999998e-05 2024-04-24T15:34:05.639 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:05.639 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:34:05.639 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:05.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.639 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14053 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2793472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:05.639 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892546+0000", 2024-04-24T15:34:05.640 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 91.705683387999997, 2024-04-24T15:34:05.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039457712999999998, 2024-04-24T15:34:05.640 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:34:05.640 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:34:05.640 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:34:05.640 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:34:05.640 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:05.640 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14053 2024-04-24T15:34:05.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.640 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:34:05.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:34:05.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892546+0000", 2024-04-24T15:34:05.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:05.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.641 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:34:05.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892546+0000", 2024-04-24T15:34:05.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:05.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.641 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:34:05.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892546+0000", 2024-04-24T15:34:05.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:05.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.641 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:34:05.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892548+0000", 2024-04-24T15:34:05.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3120000000000001e-06 2024-04-24T15:34:05.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.642 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.642 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:34:05.642 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892553+0000", 2024-04-24T15:34:05.642 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2320000000000001e-06 2024-04-24T15:34:05.642 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.642 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.642 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:34:05.642 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892559+0000", 2024-04-24T15:34:05.642 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5480000000000001e-06 2024-04-24T15:34:05.642 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.642 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.642 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:34:05.642 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931916+0000", 2024-04-24T15:34:05.642 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039357180999999998 2024-04-24T15:34:05.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.643 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:34:05.643 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.931956+0000", 2024-04-24T15:34:05.643 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0565000000000002e-05 2024-04-24T15:34:05.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.643 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:34:05.643 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932004+0000", 2024-04-24T15:34:05.643 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7224000000000001e-05 2024-04-24T15:34:05.643 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:05.643 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:34:05.643 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:05.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.643 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14054 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2834432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:05.644 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892563+0000", 2024-04-24T15:34:05.644 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 91.705666735999998, 2024-04-24T15:34:05.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039552406999999998, 2024-04-24T15:34:05.644 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:34:05.644 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:34:05.644 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:34:05.644 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:34:05.644 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:05.644 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14054 2024-04-24T15:34:05.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.644 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:34:05.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:34:05.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892563+0000", 2024-04-24T15:34:05.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:05.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:34:05.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892563+0000", 2024-04-24T15:34:05.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:05.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:34:05.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892562+0000", 2024-04-24T15:34:05.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:05.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:34:05.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892564+0000", 2024-04-24T15:34:05.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2029999999999999e-06 2024-04-24T15:34:05.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:34:05.646 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892570+0000", 2024-04-24T15:34:05.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1460000000000004e-06 2024-04-24T15:34:05.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.646 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:34:05.646 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892573+0000", 2024-04-24T15:34:05.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2030000000000001e-06 2024-04-24T15:34:05.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.646 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:34:05.646 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932024+0000", 2024-04-24T15:34:05.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039451696000000001 2024-04-24T15:34:05.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.647 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:34:05.647 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932068+0000", 2024-04-24T15:34:05.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3970000000000001e-05 2024-04-24T15:34:05.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.647 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:34:05.647 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932115+0000", 2024-04-24T15:34:05.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6569000000000001e-05 2024-04-24T15:34:05.647 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:05.647 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:34:05.647 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:05.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.647 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14055 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2875392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:05.647 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892591+0000", 2024-04-24T15:34:05.648 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 91.705638762999996, 2024-04-24T15:34:05.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039634309, 2024-04-24T15:34:05.648 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:34:05.648 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:34:05.648 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:34:05.648 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:34:05.648 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:05.648 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14055 2024-04-24T15:34:05.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.648 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:34:05.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.648 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:34:05.648 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892591+0000", 2024-04-24T15:34:05.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:05.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:34:05.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892591+0000", 2024-04-24T15:34:05.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:05.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:34:05.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892590+0000", 2024-04-24T15:34:05.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:05.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:34:05.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892592+0000", 2024-04-24T15:34:05.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7519999999999999e-06 2024-04-24T15:34:05.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:34:05.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892596+0000", 2024-04-24T15:34:05.650 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.608e-06 2024-04-24T15:34:05.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.650 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:34:05.650 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892598+0000", 2024-04-24T15:34:05.650 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.531e-06 2024-04-24T15:34:05.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.650 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:34:05.650 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932136+0000", 2024-04-24T15:34:05.650 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039537668999999998 2024-04-24T15:34:05.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.651 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:34:05.651 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932177+0000", 2024-04-24T15:34:05.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0880000000000002e-05 2024-04-24T15:34:05.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.651 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:34:05.651 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932225+0000", 2024-04-24T15:34:05.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8223999999999998e-05 2024-04-24T15:34:05.651 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:05.651 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:34:05.651 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:05.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.651 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14056 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 2916352~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:05.652 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.892600+0000", 2024-04-24T15:34:05.652 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 91.705628929, 2024-04-24T15:34:05.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039736447000000001, 2024-04-24T15:34:05.652 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:34:05.652 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:34:05.652 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:34:05.652 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:34:05.652 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:05.652 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14056 2024-04-24T15:34:05.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.652 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:34:05.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:34:05.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892600+0000", 2024-04-24T15:34:05.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:05.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:34:05.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892600+0000", 2024-04-24T15:34:05.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:05.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:34:05.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892600+0000", 2024-04-24T15:34:05.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967296 2024-04-24T15:34:05.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.654 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:34:05.654 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892602+0000", 2024-04-24T15:34:05.654 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.576e-06 2024-04-24T15:34:05.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.654 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:34:05.654 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892605+0000", 2024-04-24T15:34:05.654 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.574e-06 2024-04-24T15:34:05.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.654 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:34:05.654 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.892608+0000", 2024-04-24T15:34:05.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3800000000000001e-06 2024-04-24T15:34:05.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.655 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:34:05.655 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932245+0000", 2024-04-24T15:34:05.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039637378000000001 2024-04-24T15:34:05.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.655 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:34:05.655 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932290+0000", 2024-04-24T15:34:05.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4734999999999998e-05 2024-04-24T15:34:05.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.655 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:34:05.655 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.932337+0000", 2024-04-24T15:34:05.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7033000000000002e-05 2024-04-24T15:34:05.656 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:05.656 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:34:05.656 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:05.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.656 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14063 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 3203072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:05.656 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.894564+0000", 2024-04-24T15:34:05.656 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 91.703665337999993, 2024-04-24T15:34:05.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038554995000000002, 2024-04-24T15:34:05.656 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:34:05.656 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:34:05.656 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:34:05.656 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:34:05.656 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:05.656 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14063 2024-04-24T15:34:05.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.657 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:34:05.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:34:05.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894564+0000", 2024-04-24T15:34:05.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:05.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:34:05.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894564+0000", 2024-04-24T15:34:05.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:05.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:34:05.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894564+0000", 2024-04-24T15:34:05.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967296 2024-04-24T15:34:05.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:34:05.658 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894565+0000", 2024-04-24T15:34:05.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2640000000000001e-06 2024-04-24T15:34:05.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:34:05.658 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894569+0000", 2024-04-24T15:34:05.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1710000000000004e-06 2024-04-24T15:34:05.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.659 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:34:05.659 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894573+0000", 2024-04-24T15:34:05.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4410000000000002e-06 2024-04-24T15:34:05.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.659 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:34:05.659 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933030+0000", 2024-04-24T15:34:05.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038457709999999999 2024-04-24T15:34:05.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.660 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:34:05.660 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933072+0000", 2024-04-24T15:34:05.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.193e-05 2024-04-24T15:34:05.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.660 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:34:05.660 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933119+0000", 2024-04-24T15:34:05.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6715000000000003e-05 2024-04-24T15:34:05.660 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:05.660 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:34:05.661 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:05.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.661 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14064 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 3244032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:05.661 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.894591+0000", 2024-04-24T15:34:05.661 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 91.703638695999999, 2024-04-24T15:34:05.661 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038639806999999998, 2024-04-24T15:34:05.661 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:34:05.661 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:34:05.661 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:34:05.661 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:34:05.661 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:05.662 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14064 2024-04-24T15:34:05.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.662 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:34:05.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.662 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:34:05.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894591+0000", 2024-04-24T15:34:05.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:05.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.662 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:34:05.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894591+0000", 2024-04-24T15:34:05.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:05.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.663 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:34:05.663 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894590+0000", 2024-04-24T15:34:05.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:05.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.663 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:34:05.663 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894593+0000", 2024-04-24T15:34:05.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4719999999999998e-06 2024-04-24T15:34:05.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.663 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:34:05.663 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894598+0000", 2024-04-24T15:34:05.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.57e-06 2024-04-24T15:34:05.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.664 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:34:05.664 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894602+0000", 2024-04-24T15:34:05.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.614e-06 2024-04-24T15:34:05.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.664 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:34:05.664 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933139+0000", 2024-04-24T15:34:05.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038536929999999997 2024-04-24T15:34:05.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.664 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:34:05.664 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933183+0000", 2024-04-24T15:34:05.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4217e-05 2024-04-24T15:34:05.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:34:05.665 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933230+0000", 2024-04-24T15:34:05.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7434e-05 2024-04-24T15:34:05.665 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:05.665 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:34:05.665 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:05.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.665 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14065 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 3284992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:05.665 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.894606+0000", 2024-04-24T15:34:05.665 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 91.703623313999998, 2024-04-24T15:34:05.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038734126000000001, 2024-04-24T15:34:05.666 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:34:05.666 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:34:05.666 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:34:05.666 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:34:05.666 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:05.666 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14065 2024-04-24T15:34:05.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.666 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:34:05.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.666 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:34:05.666 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894606+0000", 2024-04-24T15:34:05.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:05.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.666 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:34:05.667 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894606+0000", 2024-04-24T15:34:05.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:05.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.667 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:34:05.667 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894606+0000", 2024-04-24T15:34:05.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:05.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.667 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:34:05.667 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894608+0000", 2024-04-24T15:34:05.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0880000000000002e-06 2024-04-24T15:34:05.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.667 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:34:05.668 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894614+0000", 2024-04-24T15:34:05.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7830000000000004e-06 2024-04-24T15:34:05.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.668 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:34:05.668 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894621+0000", 2024-04-24T15:34:05.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5809999999999999e-06 2024-04-24T15:34:05.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.668 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:34:05.668 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933251+0000", 2024-04-24T15:34:05.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038629995 2024-04-24T15:34:05.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.668 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:34:05.668 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933292+0000", 2024-04-24T15:34:05.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0380000000000003e-05 2024-04-24T15:34:05.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:34:05.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933340+0000", 2024-04-24T15:34:05.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8662999999999998e-05 2024-04-24T15:34:05.669 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:05.669 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:34:05.669 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:05.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.669 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14066 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 3325952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:05.669 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.894625+0000", 2024-04-24T15:34:05.669 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 91.703603874999999, 2024-04-24T15:34:05.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038825539999999999, 2024-04-24T15:34:05.669 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:34:05.670 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:34:05.670 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:34:05.670 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:34:05.670 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:05.670 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14066 2024-04-24T15:34:05.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.670 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:34:05.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:34:05.670 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894625+0000", 2024-04-24T15:34:05.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:05.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:34:05.670 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894625+0000", 2024-04-24T15:34:05.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:05.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.671 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:34:05.671 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894625+0000", 2024-04-24T15:34:05.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:05.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.671 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:34:05.671 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894627+0000", 2024-04-24T15:34:05.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.092e-06 2024-04-24T15:34:05.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.671 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:34:05.671 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894635+0000", 2024-04-24T15:34:05.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6660000000000008e-06 2024-04-24T15:34:05.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.672 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:34:05.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894639+0000", 2024-04-24T15:34:05.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.799e-06 2024-04-24T15:34:05.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.672 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:34:05.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933360+0000", 2024-04-24T15:34:05.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038721569999999997 2024-04-24T15:34:05.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.672 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:34:05.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933403+0000", 2024-04-24T15:34:05.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2605999999999998e-05 2024-04-24T15:34:05.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.673 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:34:05.673 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933451+0000", 2024-04-24T15:34:05.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8149999999999998e-05 2024-04-24T15:34:05.673 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:05.673 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:34:05.673 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:05.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.673 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:14067 4.1a 4:58d36a0e:::rbd_data.114a38cedbad.0000000000000002:7 [read 3366912~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:05.673 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:33.894644+0000", 2024-04-24T15:34:05.673 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 91.703585817000004, 2024-04-24T15:34:05.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038926068000000001, 2024-04-24T15:34:05.673 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:34:05.674 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:34:05.674 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:34:05.674 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:34:05.674 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:05.674 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 14067 2024-04-24T15:34:05.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.674 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:34:05.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.674 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:34:05.674 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894644+0000", 2024-04-24T15:34:05.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:05.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.674 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:34:05.674 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894644+0000", 2024-04-24T15:34:05.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:05.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:34:05.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894643+0000", 2024-04-24T15:34:05.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:05.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:34:05.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894646+0000", 2024-04-24T15:34:05.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3379999999999999e-06 2024-04-24T15:34:05.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:34:05.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894651+0000", 2024-04-24T15:34:05.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.485e-06 2024-04-24T15:34:05.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:34:05.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.894655+0000", 2024-04-24T15:34:05.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.709e-06 2024-04-24T15:34:05.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:34:05.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933472+0000", 2024-04-24T15:34:05.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038816992000000002 2024-04-24T15:34:05.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:34:05.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933521+0000", 2024-04-24T15:34:05.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8841999999999998e-05 2024-04-24T15:34:05.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:34:05.677 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:33.933570+0000", 2024-04-24T15:34:05.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9063000000000001e-05 2024-04-24T15:34:05.677 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:05.677 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:34:05.677 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:05.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.677 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4426.0:30240 4.0 4:059b194c:::rbd_object_map.114a2e5eebc:head [read 0~43] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-04-24T15:34:05.677 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-24T15:32:47.244473+0000", 2024-04-24T15:34:05.677 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 78.353756630999996, 2024-04-24T15:34:05.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4146627299999999, 2024-04-24T15:34:05.677 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-24T15:34:05.677 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-24T15:34:05.677 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-24T15:34:05.677 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4426", 2024-04-24T15:34:05.677 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:05.678 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30240 2024-04-24T15:34:05.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.678 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-24T15:34:05.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-24T15:34:05.678 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.244473+0000", 2024-04-24T15:34:05.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:05.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-24T15:34:05.678 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.244473+0000", 2024-04-24T15:34:05.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-24T15:34:05.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-24T15:34:05.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.244471+0000", 2024-04-24T15:34:05.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4294967295.9999986 2024-04-24T15:34:05.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.679 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-24T15:34:05.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.244477+0000", 2024-04-24T15:34:05.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7819999999999999e-06 2024-04-24T15:34:05.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.679 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-24T15:34:05.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.244487+0000", 2024-04-24T15:34:05.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0247e-05 2024-04-24T15:34:05.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.679 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-24T15:34:05.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.244499+0000", 2024-04-24T15:34:05.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1324e-05 2024-04-24T15:34:05.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.680 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:34:05.680 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.244545+0000", 2024-04-24T15:34:05.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6186000000000001e-05 2024-04-24T15:34:05.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.680 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-04-24T15:34:05.680 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.244553+0000", 2024-04-24T15:34:05.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5380000000000002e-06 2024-04-24T15:34:05.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.680 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:34:05.680 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.631596+0000", 2024-04-24T15:34:05.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.38704289600000003 2024-04-24T15:34:05.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-04-24T15:34:05.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:47.631606+0000", 2024-04-24T15:34:05.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6390000000000004e-06 2024-04-24T15:34:05.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-24T15:34:05.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:48.658831+0000", 2024-04-24T15:34:05.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0272250540000001 2024-04-24T15:34:05.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-24T15:34:05.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:48.658991+0000", 2024-04-24T15:34:05.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015994800000000001 2024-04-24T15:34:05.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-24T15:34:05.682 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-24T15:34:05.682 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-24T15:34:05.682 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-24T15:32:48.659135+0000", 2024-04-24T15:34:05.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000144456 2024-04-24T15:34:05.682 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:05.682 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:34:05.682 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:05.682 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-24T15:34:05.682 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-24T15:34:05.682 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:34:05.683 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:34:05.599+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:34:05.683 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-24T15:34:05.725 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:34:05.726+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:34:05.744 INFO:teuthology.orchestra.run.smithi096.stderr:noscrub is set 2024-04-24T15:34:05.744 INFO:teuthology.orchestra.run.smithi096.stderr:set osd.6 primary-affinity to 0.827554 (8542342) 2024-04-24T15:34:05.786 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:34:05.787+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:34:05.828 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:34:05.828 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:34:05.829 INFO:teuthology.orchestra.run.smithi151.stdout: "num_ops": 0 2024-04-24T15:34:05.829 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:34:05.839 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-24T15:34:05.887 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-04-24T15:34:05.888+0000 7faf0d5bb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 44083) UID: 0 2024-04-24T15:34:05.987 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:34:05.989+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:34:06.002 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:34:06.002 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:34:06.002 INFO:teuthology.orchestra.run.smithi151.stdout: "complaint_time": 30, 2024-04-24T15:34:06.003 INFO:teuthology.orchestra.run.smithi151.stdout: "num_blocked_ops": 0 2024-04-24T15:34:06.003 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:34:06.012 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-24T15:34:06.089 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:34:06.090+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:34:06.176 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:34:06.176 INFO:teuthology.orchestra.run.smithi151.stdout: "size": 20, 2024-04-24T15:34:06.176 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 600, 2024-04-24T15:34:06.177 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [ 2024-04-24T15:34:06.177 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.177 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:39041 4.1 4:82c1a902:::rbd_data.114abe589119.000000000000032c:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e68)", 2024-04-24T15:34:06.177 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:01.760552+0000", 2024-04-24T15:34:06.177 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 64.415822786999996, 2024-04-24T15:34:06.177 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.87570748300000001, 2024-04-24T15:34:06.177 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:06.177 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "reached pg", 2024-04-24T15:34:06.177 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:06.177 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:06.177 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:06.177 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 39041 2024-04-24T15:34:06.177 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.177 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:06.177 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.178 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:06.178 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760552+0000", 2024-04-24T15:34:06.178 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.178 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.178 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.178 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:06.178 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760552+0000", 2024-04-24T15:34:06.178 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.178 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.178 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.178 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:06.178 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760550+0000", 2024-04-24T15:34:06.178 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999986 2024-04-24T15:34:06.178 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.179 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.179 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:06.179 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760557+0000", 2024-04-24T15:34:06.179 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1420000000000004e-06 2024-04-24T15:34:06.179 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.179 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.179 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:06.179 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760572+0000", 2024-04-24T15:34:06.179 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.483e-05 2024-04-24T15:34:06.179 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.179 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.179 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:06.179 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760578+0000", 2024-04-24T15:34:06.179 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.2820000000000002e-06 2024-04-24T15:34:06.179 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.179 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.180 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:06.180 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760628+0000", 2024-04-24T15:34:06.180 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.0040000000000002e-05 2024-04-24T15:34:06.180 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.180 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.180 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op must wait for map", 2024-04-24T15:34:06.180 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.760637+0000", 2024-04-24T15:34:06.180 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.6899999999999998e-06 2024-04-24T15:34:06.180 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.180 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.180 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:06.180 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.770562+0000", 2024-04-24T15:34:06.180 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0099252739999999996 2024-04-24T15:34:06.180 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.180 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.180 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for peered", 2024-04-24T15:34:06.181 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:01.770572+0000", 2024-04-24T15:34:06.181 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0105999999999999e-05 2024-04-24T15:34:06.181 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.181 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.181 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:06.181 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:02.636170+0000", 2024-04-24T15:34:06.181 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.86559722299999997 2024-04-24T15:34:06.181 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.181 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.181 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:06.181 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:02.636259+0000", 2024-04-24T15:34:06.181 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.9446999999999999e-05 2024-04-24T15:34:06.181 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.181 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:06.181 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.181 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.182 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.182 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48675 4.d 4:b465d49b:::rbd_data.114a858e6641.0000000000000002:17 [read 2768896~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:34:06.182 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.926403+0000", 2024-04-24T15:34:06.182 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 55.249971217000002, 2024-04-24T15:34:06.182 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035742807000000001, 2024-04-24T15:34:06.182 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:06.182 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:06.182 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:06.182 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:06.182 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:06.182 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48675 2024-04-24T15:34:06.182 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.182 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:06.182 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.183 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:06.183 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926403+0000", 2024-04-24T15:34:06.183 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.183 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.183 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.183 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:06.183 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926403+0000", 2024-04-24T15:34:06.183 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.183 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.183 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.183 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:06.183 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926403+0000", 2024-04-24T15:34:06.183 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:06.183 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.183 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.183 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:06.184 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926404+0000", 2024-04-24T15:34:06.184 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.424e-06 2024-04-24T15:34:06.184 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.184 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.184 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:06.184 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926413+0000", 2024-04-24T15:34:06.184 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.6850000000000007e-06 2024-04-24T15:34:06.184 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.184 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.184 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:06.184 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926417+0000", 2024-04-24T15:34:06.184 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.7859999999999998e-06 2024-04-24T15:34:06.184 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.184 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.184 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:06.185 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.961986+0000", 2024-04-24T15:34:06.185 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035568862 2024-04-24T15:34:06.185 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.185 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.185 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:06.185 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.962063+0000", 2024-04-24T15:34:06.185 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7257999999999998e-05 2024-04-24T15:34:06.185 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.185 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.185 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:06.185 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.962146+0000", 2024-04-24T15:34:06.185 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3094000000000001e-05 2024-04-24T15:34:06.185 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.185 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:06.186 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.186 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.186 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.186 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48676 4.d 4:b465d49b:::rbd_data.114a858e6641.0000000000000002:17 [read 2834432~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:34:06.186 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.926421+0000", 2024-04-24T15:34:06.186 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 55.249953673999997, 2024-04-24T15:34:06.186 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035829002999999998, 2024-04-24T15:34:06.186 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:06.186 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:06.186 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:06.186 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:06.186 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:06.186 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48676 2024-04-24T15:34:06.187 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.187 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:06.187 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.187 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:06.187 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926421+0000", 2024-04-24T15:34:06.187 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.187 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.187 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.187 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:06.187 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926421+0000", 2024-04-24T15:34:06.187 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.187 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.187 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.187 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:06.187 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926420+0000", 2024-04-24T15:34:06.187 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:06.188 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.188 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.188 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:06.188 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926423+0000", 2024-04-24T15:34:06.188 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.356e-06 2024-04-24T15:34:06.188 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.188 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.188 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:06.188 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926428+0000", 2024-04-24T15:34:06.188 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.3340000000000001e-06 2024-04-24T15:34:06.188 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.188 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.188 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:06.188 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.926431+0000", 2024-04-24T15:34:06.188 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.2899999999999998e-06 2024-04-24T15:34:06.189 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.189 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.189 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:06.189 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.962156+0000", 2024-04-24T15:34:06.189 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035725159999999999 2024-04-24T15:34:06.189 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.189 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.189 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:06.189 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.962197+0000", 2024-04-24T15:34:06.189 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0457999999999998e-05 2024-04-24T15:34:06.189 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.189 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.189 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:06.189 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.962250+0000", 2024-04-24T15:34:06.190 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.2812000000000002e-05 2024-04-24T15:34:06.190 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.190 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:06.190 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.190 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.190 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.190 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48704 4.d 4:b59d5735:::rbd_data.114a858e6641.0000000000000004:17 [read 2703360~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:34:06.190 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928541+0000", 2024-04-24T15:34:06.190 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 55.247833704000001, 2024-04-24T15:34:06.190 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035763016000000002, 2024-04-24T15:34:06.190 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:06.190 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:06.190 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:06.190 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:06.191 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:06.191 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48704 2024-04-24T15:34:06.191 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.191 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:06.191 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.191 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:06.191 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928541+0000", 2024-04-24T15:34:06.191 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.191 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.191 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.191 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:06.191 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928541+0000", 2024-04-24T15:34:06.191 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.191 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.191 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.192 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:06.192 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928540+0000", 2024-04-24T15:34:06.192 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:34:06.192 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.192 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.192 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:06.192 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928542+0000", 2024-04-24T15:34:06.192 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.37e-06 2024-04-24T15:34:06.192 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.192 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.192 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:06.192 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928545+0000", 2024-04-24T15:34:06.192 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8229999999999999e-06 2024-04-24T15:34:06.192 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.192 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.193 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:06.193 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928547+0000", 2024-04-24T15:34:06.193 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.9360000000000002e-06 2024-04-24T15:34:06.193 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.193 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.193 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:06.193 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964219+0000", 2024-04-24T15:34:06.193 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035672813999999997 2024-04-24T15:34:06.193 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.193 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.193 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:06.193 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964259+0000", 2024-04-24T15:34:06.193 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0070000000000001e-05 2024-04-24T15:34:06.193 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.193 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.194 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:06.194 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964304+0000", 2024-04-24T15:34:06.194 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4229e-05 2024-04-24T15:34:06.194 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.194 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:06.194 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.194 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.194 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.194 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48719 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 1851392~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:34:06.194 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928549+0000", 2024-04-24T15:34:06.194 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 55.247825556999999, 2024-04-24T15:34:06.194 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035890645999999998, 2024-04-24T15:34:06.194 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:06.194 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:06.195 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:06.195 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:06.195 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:06.195 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48719 2024-04-24T15:34:06.195 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.195 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:06.195 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.195 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:06.195 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928549+0000", 2024-04-24T15:34:06.195 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.195 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.195 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.195 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:06.195 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928549+0000", 2024-04-24T15:34:06.195 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.196 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.196 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.196 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:06.196 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928549+0000", 2024-04-24T15:34:06.196 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:06.196 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.196 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.196 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:06.196 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928550+0000", 2024-04-24T15:34:06.196 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3829999999999999e-06 2024-04-24T15:34:06.196 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.196 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.196 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:06.196 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928553+0000", 2024-04-24T15:34:06.196 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7369999999999998e-06 2024-04-24T15:34:06.197 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.197 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.197 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:06.197 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928555+0000", 2024-04-24T15:34:06.197 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.8920000000000001e-06 2024-04-24T15:34:06.197 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.197 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.197 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:06.197 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964240+0000", 2024-04-24T15:34:06.197 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035685537000000003 2024-04-24T15:34:06.197 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.197 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.197 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:06.197 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964318+0000", 2024-04-24T15:34:06.198 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7838999999999998e-05 2024-04-24T15:34:06.198 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.198 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.198 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:06.198 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964439+0000", 2024-04-24T15:34:06.198 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000121564 2024-04-24T15:34:06.198 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.198 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:06.198 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.198 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.198 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.198 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48720 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 1916928~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:34:06.198 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928557+0000", 2024-04-24T15:34:06.199 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 55.247817712, 2024-04-24T15:34:06.199 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036005219999999998, 2024-04-24T15:34:06.199 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:06.199 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:06.199 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:06.199 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:06.199 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:06.199 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48720 2024-04-24T15:34:06.199 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.199 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:06.199 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.199 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:06.199 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928557+0000", 2024-04-24T15:34:06.199 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.200 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.200 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.200 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:06.200 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928557+0000", 2024-04-24T15:34:06.200 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.200 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.200 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.200 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:06.200 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928556+0000", 2024-04-24T15:34:06.200 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:34:06.200 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.200 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.200 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:06.200 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928558+0000", 2024-04-24T15:34:06.200 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.13e-06 2024-04-24T15:34:06.200 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.201 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.201 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:06.201 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928561+0000", 2024-04-24T15:34:06.201 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5530000000000001e-06 2024-04-24T15:34:06.201 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.201 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.201 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:06.201 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928563+0000", 2024-04-24T15:34:06.201 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.8810000000000001e-06 2024-04-24T15:34:06.201 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.201 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.201 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:06.201 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964450+0000", 2024-04-24T15:34:06.201 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.035887040000000002 2024-04-24T15:34:06.201 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.202 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.202 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:06.202 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964494+0000", 2024-04-24T15:34:06.202 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3699000000000003e-05 2024-04-24T15:34:06.202 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.202 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.202 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:06.202 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964562+0000", 2024-04-24T15:34:06.202 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.8149000000000001e-05 2024-04-24T15:34:06.202 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.202 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:06.202 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.202 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.202 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.203 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48721 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 1982464~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:34:06.203 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928565+0000", 2024-04-24T15:34:06.203 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 55.247809187999998, 2024-04-24T15:34:06.203 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036233466999999998, 2024-04-24T15:34:06.203 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:06.203 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:06.203 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:06.203 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:06.203 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:06.203 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48721 2024-04-24T15:34:06.203 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.203 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:06.203 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.203 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:06.203 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928565+0000", 2024-04-24T15:34:06.204 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.204 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.204 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.204 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:06.204 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928565+0000", 2024-04-24T15:34:06.204 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.204 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.204 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.204 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:06.204 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928565+0000", 2024-04-24T15:34:06.204 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:06.204 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.204 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.204 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:06.204 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928566+0000", 2024-04-24T15:34:06.204 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.336e-06 2024-04-24T15:34:06.205 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.205 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.205 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:06.205 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928569+0000", 2024-04-24T15:34:06.205 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.9189999999999999e-06 2024-04-24T15:34:06.205 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.205 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.205 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:06.205 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928571+0000", 2024-04-24T15:34:06.205 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.1909999999999999e-06 2024-04-24T15:34:06.205 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.205 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.205 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:06.205 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964592+0000", 2024-04-24T15:34:06.205 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036020120000000003 2024-04-24T15:34:06.206 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.206 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.206 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:06.206 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964673+0000", 2024-04-24T15:34:06.206 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.1254000000000005e-05 2024-04-24T15:34:06.206 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.206 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.206 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:06.206 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964799+0000", 2024-04-24T15:34:06.206 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00012588700000000001 2024-04-24T15:34:06.206 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.206 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:06.206 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.206 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.207 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.207 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48722 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2048000~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:34:06.207 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928586+0000", 2024-04-24T15:34:06.207 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 55.247788063000002, 2024-04-24T15:34:06.207 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036331347999999999, 2024-04-24T15:34:06.207 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:06.207 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:06.207 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:06.207 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:06.207 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:06.207 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48722 2024-04-24T15:34:06.207 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.207 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:06.207 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.207 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:06.207 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928586+0000", 2024-04-24T15:34:06.208 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.208 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.208 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.208 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:06.208 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928586+0000", 2024-04-24T15:34:06.208 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.208 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.208 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.208 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:06.208 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928586+0000", 2024-04-24T15:34:06.208 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:06.208 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.208 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.208 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:06.208 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928587+0000", 2024-04-24T15:34:06.209 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3519999999999999e-06 2024-04-24T15:34:06.209 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.209 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.209 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:06.209 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928590+0000", 2024-04-24T15:34:06.209 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.9699999999999999e-06 2024-04-24T15:34:06.209 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.209 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.209 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:06.209 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928592+0000", 2024-04-24T15:34:06.209 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.097e-06 2024-04-24T15:34:06.209 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.209 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.209 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:06.209 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964810+0000", 2024-04-24T15:34:06.210 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036217249 2024-04-24T15:34:06.210 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.210 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.210 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:06.210 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964851+0000", 2024-04-24T15:34:06.210 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.1151e-05 2024-04-24T15:34:06.210 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.210 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.210 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:06.210 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964918+0000", 2024-04-24T15:34:06.210 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.6898999999999998e-05 2024-04-24T15:34:06.210 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.210 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:06.210 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.210 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.211 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.211 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48723 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2113536~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:34:06.211 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928598+0000", 2024-04-24T15:34:06.211 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 55.247776674000001, 2024-04-24T15:34:06.211 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036552125999999997, 2024-04-24T15:34:06.211 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:06.211 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:06.211 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:06.211 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:06.211 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:06.211 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48723 2024-04-24T15:34:06.211 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.211 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:06.211 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.211 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:06.212 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928598+0000", 2024-04-24T15:34:06.212 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.212 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.212 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.212 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:06.212 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928598+0000", 2024-04-24T15:34:06.212 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.212 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.212 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.212 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:06.212 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928597+0000", 2024-04-24T15:34:06.212 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:06.212 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.212 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.212 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:06.213 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928599+0000", 2024-04-24T15:34:06.213 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.621e-06 2024-04-24T15:34:06.213 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.213 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.213 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:06.213 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928602+0000", 2024-04-24T15:34:06.213 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.9629999999999998e-06 2024-04-24T15:34:06.213 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.213 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.213 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:06.213 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928607+0000", 2024-04-24T15:34:06.213 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8940000000000001e-06 2024-04-24T15:34:06.213 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.213 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.213 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:06.214 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.964947+0000", 2024-04-24T15:34:06.214 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036340439000000002 2024-04-24T15:34:06.214 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.214 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.214 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:06.214 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965028+0000", 2024-04-24T15:34:06.214 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.1054e-05 2024-04-24T15:34:06.214 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.214 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.214 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:06.214 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965150+0000", 2024-04-24T15:34:06.214 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00012154000000000001 2024-04-24T15:34:06.214 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.214 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:06.215 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.215 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.215 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.215 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48724 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2179072~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:34:06.215 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928609+0000", 2024-04-24T15:34:06.215 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 55.247765299000001, 2024-04-24T15:34:06.215 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036679614999999999, 2024-04-24T15:34:06.215 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:06.215 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:06.215 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:06.215 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:06.215 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:06.215 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48724 2024-04-24T15:34:06.216 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.216 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:06.216 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.216 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:06.216 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928609+0000", 2024-04-24T15:34:06.216 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.216 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.216 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.216 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:06.216 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928609+0000", 2024-04-24T15:34:06.216 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.216 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.216 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.216 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:06.217 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928609+0000", 2024-04-24T15:34:06.217 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:34:06.217 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.217 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.217 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:06.217 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928610+0000", 2024-04-24T15:34:06.217 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3030000000000001e-06 2024-04-24T15:34:06.217 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.217 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.217 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:06.217 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928613+0000", 2024-04-24T15:34:06.217 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.762e-06 2024-04-24T15:34:06.217 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.217 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.218 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:06.218 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928615+0000", 2024-04-24T15:34:06.218 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.832e-06 2024-04-24T15:34:06.218 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.218 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.218 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:06.218 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965174+0000", 2024-04-24T15:34:06.218 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036559633000000001 2024-04-24T15:34:06.218 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.218 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.218 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:06.218 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965221+0000", 2024-04-24T15:34:06.218 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6239999999999998e-05 2024-04-24T15:34:06.219 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.219 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.219 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:06.219 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965289+0000", 2024-04-24T15:34:06.219 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.8053999999999996e-05 2024-04-24T15:34:06.219 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.219 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:06.219 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.219 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.219 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.219 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48725 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2244608~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:34:06.219 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928617+0000", 2024-04-24T15:34:06.219 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 55.247757528999998, 2024-04-24T15:34:06.219 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036902659999999997, 2024-04-24T15:34:06.220 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:06.220 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:06.220 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:06.220 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:06.220 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:06.220 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48725 2024-04-24T15:34:06.220 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.220 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:06.220 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.220 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:06.220 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928617+0000", 2024-04-24T15:34:06.220 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.220 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.220 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.221 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:06.221 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928617+0000", 2024-04-24T15:34:06.221 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.221 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.221 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.221 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:06.221 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928617+0000", 2024-04-24T15:34:06.221 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:34:06.221 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.221 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.221 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:06.221 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928618+0000", 2024-04-24T15:34:06.221 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.206e-06 2024-04-24T15:34:06.221 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.222 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.222 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:06.222 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928621+0000", 2024-04-24T15:34:06.222 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5130000000000001e-06 2024-04-24T15:34:06.222 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.222 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.222 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:06.222 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928623+0000", 2024-04-24T15:34:06.222 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.9640000000000002e-06 2024-04-24T15:34:06.222 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.222 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.222 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:06.222 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965318+0000", 2024-04-24T15:34:06.223 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036694953000000002 2024-04-24T15:34:06.223 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.223 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.223 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:06.223 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965398+0000", 2024-04-24T15:34:06.223 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.9986000000000003e-05 2024-04-24T15:34:06.223 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.223 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.223 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:06.223 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965520+0000", 2024-04-24T15:34:06.223 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000121247 2024-04-24T15:34:06.223 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.223 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:06.223 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.223 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.223 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.224 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48726 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2310144~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:34:06.224 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928625+0000", 2024-04-24T15:34:06.224 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 55.247748981999997, 2024-04-24T15:34:06.224 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037015521000000003, 2024-04-24T15:34:06.224 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:06.224 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:06.224 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:06.224 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:06.224 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:06.224 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48726 2024-04-24T15:34:06.224 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.224 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:06.224 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.224 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:06.225 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928625+0000", 2024-04-24T15:34:06.225 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.225 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.225 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.225 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:06.225 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928625+0000", 2024-04-24T15:34:06.225 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.225 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.225 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.225 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:06.225 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928625+0000", 2024-04-24T15:34:06.225 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:34:06.225 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.225 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.225 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:06.226 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928627+0000", 2024-04-24T15:34:06.226 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3179999999999999e-06 2024-04-24T15:34:06.226 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.226 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.226 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:06.226 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928629+0000", 2024-04-24T15:34:06.226 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.819e-06 2024-04-24T15:34:06.226 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.226 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.226 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:06.226 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928634+0000", 2024-04-24T15:34:06.226 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6999999999999999e-06 2024-04-24T15:34:06.226 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.226 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.227 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:06.227 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965531+0000", 2024-04-24T15:34:06.227 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.036897040999999998 2024-04-24T15:34:06.227 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.227 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.227 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:06.227 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965573+0000", 2024-04-24T15:34:06.227 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.189e-05 2024-04-24T15:34:06.227 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.227 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.227 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:06.227 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965641+0000", 2024-04-24T15:34:06.227 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.7959000000000004e-05 2024-04-24T15:34:06.227 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.227 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:06.228 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.228 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.228 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.228 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48727 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2375680~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:34:06.228 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928637+0000", 2024-04-24T15:34:06.228 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 55.247737620999999, 2024-04-24T15:34:06.228 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037235129999999998, 2024-04-24T15:34:06.228 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:06.228 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:06.228 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:06.228 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:06.228 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:06.228 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48727 2024-04-24T15:34:06.228 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.228 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:06.229 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.229 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:06.229 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928637+0000", 2024-04-24T15:34:06.229 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.229 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.229 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.229 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:06.229 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928637+0000", 2024-04-24T15:34:06.229 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.229 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.229 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.229 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:06.229 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928637+0000", 2024-04-24T15:34:06.229 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:34:06.229 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.230 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.230 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:06.230 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928638+0000", 2024-04-24T15:34:06.230 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.4279999999999999e-06 2024-04-24T15:34:06.230 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.230 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.230 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:06.230 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928641+0000", 2024-04-24T15:34:06.230 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.909e-06 2024-04-24T15:34:06.230 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.230 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.230 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:06.230 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928643+0000", 2024-04-24T15:34:06.231 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.9269999999999999e-06 2024-04-24T15:34:06.231 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.231 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.231 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:06.231 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965671+0000", 2024-04-24T15:34:06.231 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037028393 2024-04-24T15:34:06.231 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.231 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.231 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:06.231 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965752+0000", 2024-04-24T15:34:06.231 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0921999999999995e-05 2024-04-24T15:34:06.231 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.231 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.231 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:06.231 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965872+0000", 2024-04-24T15:34:06.232 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000119759 2024-04-24T15:34:06.232 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.232 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:06.232 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.232 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.232 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.232 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48728 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2441216~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:34:06.232 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928645+0000", 2024-04-24T15:34:06.232 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 55.247729438, 2024-04-24T15:34:06.232 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037346852999999999, 2024-04-24T15:34:06.232 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:06.232 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:06.232 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:06.232 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:06.233 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:06.233 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48728 2024-04-24T15:34:06.233 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.233 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:06.233 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.233 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:06.233 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928645+0000", 2024-04-24T15:34:06.233 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.233 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.233 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.233 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:06.233 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928645+0000", 2024-04-24T15:34:06.233 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.233 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.233 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.234 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:06.234 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928645+0000", 2024-04-24T15:34:06.234 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:34:06.234 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.234 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.234 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:06.234 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928646+0000", 2024-04-24T15:34:06.234 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.291e-06 2024-04-24T15:34:06.234 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.234 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.234 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:06.234 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928649+0000", 2024-04-24T15:34:06.234 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8509999999999999e-06 2024-04-24T15:34:06.234 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.235 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.235 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:06.235 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928651+0000", 2024-04-24T15:34:06.235 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.809e-06 2024-04-24T15:34:06.235 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.235 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.235 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:06.235 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965882+0000", 2024-04-24T15:34:06.235 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037231715999999998 2024-04-24T15:34:06.235 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.235 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.235 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:06.235 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965925+0000", 2024-04-24T15:34:06.235 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2493999999999997e-05 2024-04-24T15:34:06.236 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.236 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.236 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:06.236 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.965992+0000", 2024-04-24T15:34:06.236 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.6907000000000002e-05 2024-04-24T15:34:06.236 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.236 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:06.236 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.236 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.236 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.236 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48729 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2506752~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:34:06.236 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928653+0000", 2024-04-24T15:34:06.236 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 55.247721562999999, 2024-04-24T15:34:06.236 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037576904000000001, 2024-04-24T15:34:06.236 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:06.237 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:06.237 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:06.237 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:06.237 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:06.237 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48729 2024-04-24T15:34:06.237 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.237 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:06.237 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.237 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:06.237 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928653+0000", 2024-04-24T15:34:06.237 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.237 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.237 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.237 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:06.238 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928653+0000", 2024-04-24T15:34:06.238 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.238 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.238 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.238 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:06.238 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928653+0000", 2024-04-24T15:34:06.238 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:34:06.238 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.238 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.238 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:06.238 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928654+0000", 2024-04-24T15:34:06.238 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3209999999999999e-06 2024-04-24T15:34:06.238 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.238 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.239 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:06.239 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928657+0000", 2024-04-24T15:34:06.239 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8559999999999998e-06 2024-04-24T15:34:06.239 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.239 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.239 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:06.239 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928659+0000", 2024-04-24T15:34:06.239 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7080000000000002e-06 2024-04-24T15:34:06.239 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.239 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.239 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:06.239 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966021+0000", 2024-04-24T15:34:06.239 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037361296000000002 2024-04-24T15:34:06.239 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.240 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.240 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:06.240 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966102+0000", 2024-04-24T15:34:06.240 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0939999999999994e-05 2024-04-24T15:34:06.240 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.240 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.240 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:06.240 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966230+0000", 2024-04-24T15:34:06.240 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00012799999999999999 2024-04-24T15:34:06.240 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.240 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:06.240 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.240 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.240 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.240 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48730 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2572288~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:34:06.241 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928662+0000", 2024-04-24T15:34:06.241 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 55.247712475, 2024-04-24T15:34:06.241 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037687246000000001, 2024-04-24T15:34:06.241 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:06.241 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:06.241 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:06.241 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:06.241 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:06.241 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48730 2024-04-24T15:34:06.241 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.241 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:06.241 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.241 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:06.241 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928662+0000", 2024-04-24T15:34:06.242 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.242 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.242 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.242 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:06.242 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928662+0000", 2024-04-24T15:34:06.242 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.242 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.242 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.242 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:06.242 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928662+0000", 2024-04-24T15:34:06.242 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:06.242 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.242 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.242 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:06.243 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928663+0000", 2024-04-24T15:34:06.243 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.381e-06 2024-04-24T15:34:06.243 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.243 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.243 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:06.243 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928666+0000", 2024-04-24T15:34:06.243 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.796e-06 2024-04-24T15:34:06.243 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.243 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.243 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:06.243 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928668+0000", 2024-04-24T15:34:06.243 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.8920000000000001e-06 2024-04-24T15:34:06.243 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.243 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.243 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:06.244 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966240+0000", 2024-04-24T15:34:06.244 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037572886999999999 2024-04-24T15:34:06.244 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.244 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.244 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:06.244 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966281+0000", 2024-04-24T15:34:06.244 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0450000000000001e-05 2024-04-24T15:34:06.244 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.244 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.244 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:06.244 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966349+0000", 2024-04-24T15:34:06.244 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.8215999999999999e-05 2024-04-24T15:34:06.244 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.244 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:06.245 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.245 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.245 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.245 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48731 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2637824~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:34:06.245 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928747+0000", 2024-04-24T15:34:06.245 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 55.247627469999998, 2024-04-24T15:34:06.245 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037833602000000001, 2024-04-24T15:34:06.245 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:06.245 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:06.245 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:06.245 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:06.245 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:06.245 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48731 2024-04-24T15:34:06.245 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.246 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:06.246 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.246 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:06.246 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928747+0000", 2024-04-24T15:34:06.246 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.246 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.246 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.246 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:06.246 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928747+0000", 2024-04-24T15:34:06.246 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.246 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.246 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.246 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:06.247 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928670+0000", 2024-04-24T15:34:06.247 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999228 2024-04-24T15:34:06.247 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.247 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.247 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:06.247 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928750+0000", 2024-04-24T15:34:06.247 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0527999999999999e-05 2024-04-24T15:34:06.247 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.247 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.247 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:06.247 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928757+0000", 2024-04-24T15:34:06.247 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.9419999999999998e-06 2024-04-24T15:34:06.247 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.247 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.247 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:06.248 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928762+0000", 2024-04-24T15:34:06.248 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4240000000000003e-06 2024-04-24T15:34:06.248 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.248 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.248 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:06.248 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966379+0000", 2024-04-24T15:34:06.248 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037617267000000003 2024-04-24T15:34:06.248 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.248 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.248 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:06.248 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966461+0000", 2024-04-24T15:34:06.248 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.2720999999999996e-05 2024-04-24T15:34:06.248 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.248 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.248 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:06.249 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966581+0000", 2024-04-24T15:34:06.249 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000119014 2024-04-24T15:34:06.249 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.249 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:06.249 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.249 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.249 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.249 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48732 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:17 [read 2703360~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:34:06.249 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.928766+0000", 2024-04-24T15:34:06.249 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 55.247607985999998, 2024-04-24T15:34:06.249 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037930924999999997, 2024-04-24T15:34:06.249 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:06.249 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:06.249 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:06.249 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:06.250 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:06.250 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48732 2024-04-24T15:34:06.250 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.250 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:06.250 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.250 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:06.250 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928766+0000", 2024-04-24T15:34:06.250 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.250 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.250 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.250 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:06.250 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928766+0000", 2024-04-24T15:34:06.250 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.251 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.251 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.251 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:06.251 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928766+0000", 2024-04-24T15:34:06.251 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:06.251 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.251 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.251 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:06.251 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928768+0000", 2024-04-24T15:34:06.251 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.3420000000000002e-06 2024-04-24T15:34:06.251 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.251 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.251 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:06.251 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928783+0000", 2024-04-24T15:34:06.251 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.4813e-05 2024-04-24T15:34:06.252 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.252 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.252 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:06.252 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.928787+0000", 2024-04-24T15:34:06.252 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.1999999999999996e-06 2024-04-24T15:34:06.252 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.252 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.252 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:06.252 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966592+0000", 2024-04-24T15:34:06.252 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.037804484999999999 2024-04-24T15:34:06.252 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.252 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.252 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:06.252 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966632+0000", 2024-04-24T15:34:06.252 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0014e-05 2024-04-24T15:34:06.253 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.253 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.253 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:06.253 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.966697+0000", 2024-04-24T15:34:06.253 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.5489999999999998e-05 2024-04-24T15:34:06.253 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.253 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:06.253 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.253 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.253 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.253 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:48813 4.d 4:b39400aa:::rbd_data.114a8b543e91.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-04-24T15:34:06.253 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:10.936015+0000", 2024-04-24T15:34:06.253 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 55.240359652000002, 2024-04-24T15:34:06.253 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.039312294999999997, 2024-04-24T15:34:06.254 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:06.254 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:34:06.254 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:06.254 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:06.254 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:06.254 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 48813 2024-04-24T15:34:06.254 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.254 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:06.254 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.254 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:06.254 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.936015+0000", 2024-04-24T15:34:06.254 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.254 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.254 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.255 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:06.255 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.936015+0000", 2024-04-24T15:34:06.255 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.255 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.255 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.255 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:06.255 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.936014+0000", 2024-04-24T15:34:06.255 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:06.255 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.255 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.255 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:06.255 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.942618+0000", 2024-04-24T15:34:06.255 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0066038499999999997 2024-04-24T15:34:06.255 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.255 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.256 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:06.256 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.942629+0000", 2024-04-24T15:34:06.256 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0606e-05 2024-04-24T15:34:06.256 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.256 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.256 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:06.256 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.942634+0000", 2024-04-24T15:34:06.256 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.2000000000000002e-06 2024-04-24T15:34:06.256 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.256 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.256 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:06.256 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.969006+0000", 2024-04-24T15:34:06.256 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.026371564 2024-04-24T15:34:06.256 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.256 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.257 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:06.257 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.969126+0000", 2024-04-24T15:34:06.257 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000120085 2024-04-24T15:34:06.257 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.257 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.257 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 2", 2024-04-24T15:34:06.257 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.969469+0000", 2024-04-24T15:34:06.257 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00034295400000000002 2024-04-24T15:34:06.257 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.257 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.257 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:34:06.257 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.973118+0000", 2024-04-24T15:34:06.257 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0036489700000000001 2024-04-24T15:34:06.257 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.258 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.258 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:34:06.258 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.975250+0000", 2024-04-24T15:34:06.258 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0021327030000000001 2024-04-24T15:34:06.258 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.258 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.258 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:34:06.258 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.975293+0000", 2024-04-24T15:34:06.258 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2951999999999998e-05 2024-04-24T15:34:06.258 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.258 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.258 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:06.258 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:10.975327+0000", 2024-04-24T15:34:06.258 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.3840000000000001e-05 2024-04-24T15:34:06.259 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.259 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:06.259 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.259 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.259 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.259 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:75520 4.5 4:a5f13e1e:::rbd_data.114a858e6641.0000000000000006:head [delete] snapc 17=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e105)", 2024-04-24T15:34:06.259 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:42.638168+0000", 2024-04-24T15:34:06.259 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 23.538206030000001, 2024-04-24T15:34:06.259 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0190266560000001, 2024-04-24T15:34:06.259 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:06.259 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:34:06.259 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:06.259 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:06.259 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:06.260 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 75520 2024-04-24T15:34:06.260 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.260 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:06.260 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.260 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:06.260 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.638168+0000", 2024-04-24T15:34:06.260 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.260 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.260 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.260 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:06.260 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.638168+0000", 2024-04-24T15:34:06.260 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.260 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.260 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.260 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:06.261 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.638167+0000", 2024-04-24T15:34:06.261 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999986 2024-04-24T15:34:06.261 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.261 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.261 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:06.261 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.638174+0000", 2024-04-24T15:34:06.261 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.861e-06 2024-04-24T15:34:06.261 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.261 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.261 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:06.261 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.638186+0000", 2024-04-24T15:34:06.261 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.2109e-05 2024-04-24T15:34:06.261 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.261 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.262 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:06.262 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.638194+0000", 2024-04-24T15:34:06.262 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.8040000000000002e-06 2024-04-24T15:34:06.262 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.262 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.262 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:06.262 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.638682+0000", 2024-04-24T15:34:06.262 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00048809599999999998 2024-04-24T15:34:06.262 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.262 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.262 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for peered", 2024-04-24T15:34:06.262 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:42.638692+0000", 2024-04-24T15:34:06.262 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.8970000000000003e-06 2024-04-24T15:34:06.262 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.263 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.263 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:06.263 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:43.649868+0000", 2024-04-24T15:34:06.263 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.011176641 2024-04-24T15:34:06.263 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.263 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.263 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:06.263 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:43.649960+0000", 2024-04-24T15:34:06.263 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.1632999999999995e-05 2024-04-24T15:34:06.263 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.263 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.263 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 6", 2024-04-24T15:34:06.263 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:43.650060+0000", 2024-04-24T15:34:06.263 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.9562999999999995e-05 2024-04-24T15:34:06.264 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.264 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.264 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:34:06.264 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:43.654465+0000", 2024-04-24T15:34:06.264 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0044056620000000003 2024-04-24T15:34:06.264 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.264 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.264 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:34:06.264 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:43.656963+0000", 2024-04-24T15:34:06.264 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.002497829 2024-04-24T15:34:06.264 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.264 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.264 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:34:06.264 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:43.657083+0000", 2024-04-24T15:34:06.265 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000120338 2024-04-24T15:34:06.265 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.265 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.265 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:06.265 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:43.657195+0000", 2024-04-24T15:34:06.265 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000111624 2024-04-24T15:34:06.265 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.265 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:06.265 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.265 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.265 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:06.265 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:34:06.267 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-24T15:34:06.267 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:34:06.268+0000 7f0f45efb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 59992) UID: 0 2024-04-24T15:34:06.367 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:34:06.368+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:34:06.431 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:34:06.432 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:34:06.432 INFO:teuthology.orchestra.run.smithi151.stdout: "num_ops": 0 2024-04-24T15:34:06.432 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:34:06.441 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-24T15:34:06.484 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:34:06.486+0000 7f0f45efb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 59992) UID: 0 2024-04-24T15:34:06.563 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:34:06.563 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:34:06.563 INFO:teuthology.orchestra.run.smithi151.stdout: "complaint_time": 30, 2024-04-24T15:34:06.564 INFO:teuthology.orchestra.run.smithi151.stdout: "num_blocked_ops": 0 2024-04-24T15:34:06.564 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:34:06.569 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:34:06.570+0000 7f0f45efb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 59992) UID: 0 2024-04-24T15:34:06.573 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-24T15:34:06.670 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:34:06.671+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:34:06.738 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:34:06.738 INFO:teuthology.orchestra.run.smithi151.stdout: "size": 20, 2024-04-24T15:34:06.738 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 600, 2024-04-24T15:34:06.738 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [ 2024-04-24T15:34:06.738 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.738 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23613 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 14688256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:06.738 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.792522+0000", 2024-04-24T15:34:06.739 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 91.945585176999998, 2024-04-24T15:34:06.739 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040507356000000001, 2024-04-24T15:34:06.739 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:06.739 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:06.739 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:06.739 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:06.739 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:06.739 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23613 2024-04-24T15:34:06.739 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.739 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:06.739 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.739 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:06.739 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792522+0000", 2024-04-24T15:34:06.739 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.739 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.740 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.740 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:06.740 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792522+0000", 2024-04-24T15:34:06.740 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.740 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.740 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.740 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:06.740 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792522+0000", 2024-04-24T15:34:06.740 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:06.740 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.740 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.740 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:06.740 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792523+0000", 2024-04-24T15:34:06.740 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.576e-06 2024-04-24T15:34:06.740 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.741 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.741 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:06.741 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792526+0000", 2024-04-24T15:34:06.741 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7439999999999999e-06 2024-04-24T15:34:06.741 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.741 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.741 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:06.741 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792528+0000", 2024-04-24T15:34:06.741 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.0600000000000002e-06 2024-04-24T15:34:06.741 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.741 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.741 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:06.741 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.832864+0000", 2024-04-24T15:34:06.741 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040335742000000001 2024-04-24T15:34:06.741 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.742 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.742 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:06.742 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.832953+0000", 2024-04-24T15:34:06.742 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.9281000000000001e-05 2024-04-24T15:34:06.742 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.742 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.742 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:06.742 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.833029+0000", 2024-04-24T15:34:06.742 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.6259000000000003e-05 2024-04-24T15:34:06.742 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.742 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:06.742 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.742 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.742 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.742 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23614 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 14729216~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:06.743 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.792530+0000", 2024-04-24T15:34:06.743 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 91.945576912999996, 2024-04-24T15:34:06.743 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040633957999999998, 2024-04-24T15:34:06.743 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:06.743 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:06.743 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:06.743 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:06.743 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:06.743 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23614 2024-04-24T15:34:06.743 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.743 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:06.743 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.743 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:06.743 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792530+0000", 2024-04-24T15:34:06.743 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.743 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.743 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.744 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:06.744 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792530+0000", 2024-04-24T15:34:06.744 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.744 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.744 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.744 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:06.744 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792530+0000", 2024-04-24T15:34:06.744 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:34:06.744 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.744 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.744 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:06.744 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792531+0000", 2024-04-24T15:34:06.744 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.46e-06 2024-04-24T15:34:06.744 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.745 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.745 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:06.745 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792534+0000", 2024-04-24T15:34:06.745 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8959999999999999e-06 2024-04-24T15:34:06.745 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.745 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.745 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:06.745 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.792536+0000", 2024-04-24T15:34:06.745 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.841e-06 2024-04-24T15:34:06.745 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.745 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.745 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:06.745 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.833060+0000", 2024-04-24T15:34:06.745 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040524125000000001 2024-04-24T15:34:06.745 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.746 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.746 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:06.746 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.833108+0000", 2024-04-24T15:34:06.746 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.7547999999999999e-05 2024-04-24T15:34:06.746 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.746 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.746 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:06.746 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.833164+0000", 2024-04-24T15:34:06.746 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6297e-05 2024-04-24T15:34:06.746 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.746 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:06.746 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.746 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.746 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.747 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23623 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15097856~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:06.747 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.793887+0000", 2024-04-24T15:34:06.747 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 91.944219951999997, 2024-04-24T15:34:06.747 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040432784999999999, 2024-04-24T15:34:06.747 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:06.747 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:06.747 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:06.747 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:06.747 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:06.747 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23623 2024-04-24T15:34:06.747 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.747 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:06.747 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.747 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:06.747 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793887+0000", 2024-04-24T15:34:06.747 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.748 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.748 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.748 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:06.748 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793887+0000", 2024-04-24T15:34:06.748 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.748 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.748 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.748 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:06.748 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793887+0000", 2024-04-24T15:34:06.748 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967296 2024-04-24T15:34:06.748 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.748 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.748 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:06.749 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793888+0000", 2024-04-24T15:34:06.749 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.3659999999999999e-06 2024-04-24T15:34:06.749 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.749 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.749 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:06.749 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793891+0000", 2024-04-24T15:34:06.749 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.8389999999999998e-06 2024-04-24T15:34:06.749 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.749 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.749 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:06.749 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793893+0000", 2024-04-24T15:34:06.749 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.9549999999999999e-06 2024-04-24T15:34:06.749 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.749 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.749 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:06.750 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834239+0000", 2024-04-24T15:34:06.750 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040345487999999999 2024-04-24T15:34:06.750 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.750 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.750 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:06.750 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834280+0000", 2024-04-24T15:34:06.750 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.1878000000000001e-05 2024-04-24T15:34:06.750 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.750 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.750 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:06.750 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834320+0000", 2024-04-24T15:34:06.750 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9480000000000001e-05 2024-04-24T15:34:06.750 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.750 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:06.751 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.751 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.751 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.751 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23624 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15138816~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:06.751 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.794053+0000", 2024-04-24T15:34:06.751 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 91.944054003999995, 2024-04-24T15:34:06.751 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040370801999999997, 2024-04-24T15:34:06.751 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:06.751 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:06.751 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:06.751 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:06.751 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:06.751 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23624 2024-04-24T15:34:06.751 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.751 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:06.752 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.752 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:06.752 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794053+0000", 2024-04-24T15:34:06.752 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.752 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.752 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.752 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:06.752 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794053+0000", 2024-04-24T15:34:06.752 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.752 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.752 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.752 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:06.752 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.793895+0000", 2024-04-24T15:34:06.752 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998422 2024-04-24T15:34:06.752 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.753 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.753 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:06.753 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794057+0000", 2024-04-24T15:34:06.753 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00016149400000000001 2024-04-24T15:34:06.753 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.753 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.753 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:06.754 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794065+0000", 2024-04-24T15:34:06.754 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.5259999999999993e-06 2024-04-24T15:34:06.754 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.754 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.754 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:06.754 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794071+0000", 2024-04-24T15:34:06.754 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6149999999999996e-06 2024-04-24T15:34:06.754 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.754 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.754 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:06.754 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834341+0000", 2024-04-24T15:34:06.754 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040270366000000002 2024-04-24T15:34:06.754 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.754 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.755 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:06.755 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834381+0000", 2024-04-24T15:34:06.755 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0102999999999999e-05 2024-04-24T15:34:06.755 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.755 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.755 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:06.755 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834424+0000", 2024-04-24T15:34:06.755 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2713000000000003e-05 2024-04-24T15:34:06.755 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.755 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:06.755 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.755 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.755 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.755 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23625 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15179776~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:06.755 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.794160+0000", 2024-04-24T15:34:06.755 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 91.943947121999997, 2024-04-24T15:34:06.756 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040413004000000002, 2024-04-24T15:34:06.756 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:06.756 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:06.756 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:06.756 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:06.756 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:06.756 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23625 2024-04-24T15:34:06.756 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.756 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:06.756 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.756 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:06.756 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794160+0000", 2024-04-24T15:34:06.756 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.756 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.756 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.756 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:06.757 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794160+0000", 2024-04-24T15:34:06.757 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.757 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.757 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.757 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:06.757 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794086+0000", 2024-04-24T15:34:06.757 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999256 2024-04-24T15:34:06.757 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.757 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.757 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:06.757 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794163+0000", 2024-04-24T15:34:06.757 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7896000000000001e-05 2024-04-24T15:34:06.757 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.758 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.758 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:06.758 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794171+0000", 2024-04-24T15:34:06.758 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.9500000000000001e-06 2024-04-24T15:34:06.758 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.758 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.758 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:06.758 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794179+0000", 2024-04-24T15:34:06.758 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7970000000000001e-06 2024-04-24T15:34:06.758 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.758 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.758 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:06.758 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834451+0000", 2024-04-24T15:34:06.758 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040271492999999998 2024-04-24T15:34:06.759 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.759 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.759 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:06.759 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834519+0000", 2024-04-24T15:34:06.759 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.7984000000000005e-05 2024-04-24T15:34:06.759 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.759 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.759 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:06.759 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.834573+0000", 2024-04-24T15:34:06.759 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.4339999999999998e-05 2024-04-24T15:34:06.759 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.759 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:06.759 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.759 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.759 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.760 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23629 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15343616~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:06.760 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.794748+0000", 2024-04-24T15:34:06.760 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 91.943359319999999, 2024-04-24T15:34:06.760 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040419718, 2024-04-24T15:34:06.760 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:06.760 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:06.760 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:06.760 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:06.760 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:06.760 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23629 2024-04-24T15:34:06.760 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.760 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:06.760 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.760 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:06.760 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794748+0000", 2024-04-24T15:34:06.761 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.761 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.761 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.761 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:06.761 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794748+0000", 2024-04-24T15:34:06.761 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.761 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.761 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.761 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:06.761 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794673+0000", 2024-04-24T15:34:06.761 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999251 2024-04-24T15:34:06.761 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.762 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.762 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:06.762 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794751+0000", 2024-04-24T15:34:06.762 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.8480999999999996e-05 2024-04-24T15:34:06.762 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.762 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.762 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:06.762 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794760+0000", 2024-04-24T15:34:06.762 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3429999999999996e-06 2024-04-24T15:34:06.762 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.762 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.762 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:06.762 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794764+0000", 2024-04-24T15:34:06.762 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5600000000000004e-06 2024-04-24T15:34:06.762 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.763 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.763 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:06.763 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835040+0000", 2024-04-24T15:34:06.763 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040275188000000003 2024-04-24T15:34:06.763 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.763 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.763 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:06.763 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835105+0000", 2024-04-24T15:34:06.763 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.5992000000000001e-05 2024-04-24T15:34:06.763 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.763 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.764 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:06.764 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835167+0000", 2024-04-24T15:34:06.764 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.1981000000000002e-05 2024-04-24T15:34:06.764 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.764 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:06.764 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.764 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.764 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.764 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23630 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15384576~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:06.764 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.794839+0000", 2024-04-24T15:34:06.764 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 91.943268251000006, 2024-04-24T15:34:06.764 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040523894999999997, 2024-04-24T15:34:06.764 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:06.765 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:06.765 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:06.765 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:06.765 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:06.765 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23630 2024-04-24T15:34:06.765 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.765 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:06.765 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.765 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:06.765 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794839+0000", 2024-04-24T15:34:06.765 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.765 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.766 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.766 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:06.766 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794839+0000", 2024-04-24T15:34:06.766 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.766 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.766 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.767 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:06.767 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794768+0000", 2024-04-24T15:34:06.767 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999294 2024-04-24T15:34:06.767 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.767 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.767 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:06.767 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794841+0000", 2024-04-24T15:34:06.767 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2793000000000006e-05 2024-04-24T15:34:06.767 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.767 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.767 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:06.767 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794846+0000", 2024-04-24T15:34:06.767 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.7230000000000004e-06 2024-04-24T15:34:06.767 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.768 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.768 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:06.768 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794848+0000", 2024-04-24T15:34:06.768 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7659999999999999e-06 2024-04-24T15:34:06.768 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.768 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.768 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:06.768 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835223+0000", 2024-04-24T15:34:06.768 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040374275000000001 2024-04-24T15:34:06.768 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.768 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.769 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:06.769 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835295+0000", 2024-04-24T15:34:06.769 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2633999999999996e-05 2024-04-24T15:34:06.769 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.769 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.769 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:06.769 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835363+0000", 2024-04-24T15:34:06.769 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.7355999999999997e-05 2024-04-24T15:34:06.769 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.769 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:06.769 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.769 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.769 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.769 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23631 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15425536~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:06.769 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.794956+0000", 2024-04-24T15:34:06.770 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 91.943151409999999, 2024-04-24T15:34:06.770 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040562383, 2024-04-24T15:34:06.770 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:06.770 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:06.770 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:06.770 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:06.770 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:06.770 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23631 2024-04-24T15:34:06.770 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.770 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:06.770 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.770 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:06.770 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794956+0000", 2024-04-24T15:34:06.770 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.771 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.771 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.771 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:06.771 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794956+0000", 2024-04-24T15:34:06.771 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.771 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.771 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.771 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:06.771 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794859+0000", 2024-04-24T15:34:06.771 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999032 2024-04-24T15:34:06.771 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.771 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.771 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:06.771 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794959+0000", 2024-04-24T15:34:06.772 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000100207 2024-04-24T15:34:06.772 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.772 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.772 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:06.772 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794967+0000", 2024-04-24T15:34:06.772 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7640000000000005e-06 2024-04-24T15:34:06.772 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.772 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.772 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:06.772 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794971+0000", 2024-04-24T15:34:06.772 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.335e-06 2024-04-24T15:34:06.772 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.772 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.772 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:06.772 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835388+0000", 2024-04-24T15:34:06.773 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040417340000000003 2024-04-24T15:34:06.773 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.773 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.773 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:06.773 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835459+0000", 2024-04-24T15:34:06.773 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.0118999999999997e-05 2024-04-24T15:34:06.773 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.773 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.773 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:06.773 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835518+0000", 2024-04-24T15:34:06.773 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.9438999999999998e-05 2024-04-24T15:34:06.773 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.773 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:06.773 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.773 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.773 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.774 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23632 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15466496~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:06.774 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795060+0000", 2024-04-24T15:34:06.774 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 91.943046699999996, 2024-04-24T15:34:06.774 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040667993, 2024-04-24T15:34:06.774 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:06.774 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:06.774 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:06.774 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:06.774 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:06.774 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23632 2024-04-24T15:34:06.774 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.774 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:06.774 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.774 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:06.775 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795060+0000", 2024-04-24T15:34:06.775 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.775 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.775 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.775 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:06.775 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795060+0000", 2024-04-24T15:34:06.775 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.775 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.775 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.775 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:06.775 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.794976+0000", 2024-04-24T15:34:06.775 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999156 2024-04-24T15:34:06.775 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.775 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.776 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:06.776 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795064+0000", 2024-04-24T15:34:06.776 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.7709999999999999e-05 2024-04-24T15:34:06.776 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.776 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.776 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:06.776 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795072+0000", 2024-04-24T15:34:06.776 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3769999999999996e-06 2024-04-24T15:34:06.776 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.776 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.776 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:06.776 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795078+0000", 2024-04-24T15:34:06.776 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.5079999999999996e-06 2024-04-24T15:34:06.776 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.777 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.777 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:06.777 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835541+0000", 2024-04-24T15:34:06.777 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040463440000000003 2024-04-24T15:34:06.777 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.777 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.777 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:06.777 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835657+0000", 2024-04-24T15:34:06.777 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00011572499999999999 2024-04-24T15:34:06.777 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.777 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.777 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:06.777 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835728+0000", 2024-04-24T15:34:06.777 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1402000000000005e-05 2024-04-24T15:34:06.777 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.778 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:06.778 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.778 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.778 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.778 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23633 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15507456~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:06.778 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795134+0000", 2024-04-24T15:34:06.778 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 91.942973045000002, 2024-04-24T15:34:06.778 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040748591000000001, 2024-04-24T15:34:06.778 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:06.778 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:06.778 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:06.778 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:06.778 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:06.778 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23633 2024-04-24T15:34:06.779 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.779 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:06.779 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.779 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:06.779 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795134+0000", 2024-04-24T15:34:06.779 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.779 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.779 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.779 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:06.779 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795134+0000", 2024-04-24T15:34:06.779 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.779 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.779 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.779 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:06.779 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795083+0000", 2024-04-24T15:34:06.780 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999949 2024-04-24T15:34:06.780 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.780 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.780 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:06.780 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795138+0000", 2024-04-24T15:34:06.780 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.4472000000000003e-05 2024-04-24T15:34:06.780 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.780 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.780 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:06.780 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795145+0000", 2024-04-24T15:34:06.780 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2779999999999996e-06 2024-04-24T15:34:06.780 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.780 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.780 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:06.781 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795149+0000", 2024-04-24T15:34:06.781 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.568e-06 2024-04-24T15:34:06.781 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.781 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.781 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:06.781 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835757+0000", 2024-04-24T15:34:06.781 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040608894999999999 2024-04-24T15:34:06.781 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.781 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.781 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:06.781 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835829+0000", 2024-04-24T15:34:06.781 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1879999999999996e-05 2024-04-24T15:34:06.781 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.781 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.781 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:06.782 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835883+0000", 2024-04-24T15:34:06.782 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.3346000000000001e-05 2024-04-24T15:34:06.782 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.782 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:06.782 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.782 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.782 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.782 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23634 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15548416~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:06.782 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795271+0000", 2024-04-24T15:34:06.782 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 91.942836010999997, 2024-04-24T15:34:06.782 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040755845999999998, 2024-04-24T15:34:06.782 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:06.782 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:06.783 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:06.783 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:06.783 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:06.783 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23634 2024-04-24T15:34:06.783 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.783 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:06.783 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.783 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:06.783 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795271+0000", 2024-04-24T15:34:06.783 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.783 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.783 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.783 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:06.783 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795271+0000", 2024-04-24T15:34:06.783 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.784 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.784 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.784 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:06.784 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795152+0000", 2024-04-24T15:34:06.784 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998813 2024-04-24T15:34:06.784 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.784 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.784 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:06.784 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795276+0000", 2024-04-24T15:34:06.784 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00012401799999999999 2024-04-24T15:34:06.784 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.784 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.784 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:06.784 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795281+0000", 2024-04-24T15:34:06.785 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8860000000000003e-06 2024-04-24T15:34:06.785 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.785 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.785 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:06.785 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795284+0000", 2024-04-24T15:34:06.785 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.0450000000000001e-06 2024-04-24T15:34:06.785 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.785 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.785 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:06.785 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835905+0000", 2024-04-24T15:34:06.785 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040621035999999999 2024-04-24T15:34:06.785 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.785 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.785 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:06.785 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.835971+0000", 2024-04-24T15:34:06.786 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.5314000000000005e-05 2024-04-24T15:34:06.786 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.786 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.786 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:06.786 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836027+0000", 2024-04-24T15:34:06.786 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6305999999999999e-05 2024-04-24T15:34:06.786 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.786 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:06.786 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.786 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.786 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.786 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23635 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15589376~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:06.786 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795382+0000", 2024-04-24T15:34:06.786 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 91.942725237999994, 2024-04-24T15:34:06.786 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040805363999999997, 2024-04-24T15:34:06.787 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:06.787 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:06.787 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:06.787 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:06.787 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:06.787 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23635 2024-04-24T15:34:06.787 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.787 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:06.787 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.787 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:06.787 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795382+0000", 2024-04-24T15:34:06.787 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.787 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.787 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.787 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:06.788 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795382+0000", 2024-04-24T15:34:06.788 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.788 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.788 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.788 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:06.788 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795288+0000", 2024-04-24T15:34:06.788 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999056 2024-04-24T15:34:06.788 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.788 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.788 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:06.788 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795385+0000", 2024-04-24T15:34:06.788 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.7288999999999996e-05 2024-04-24T15:34:06.788 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.788 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.789 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:06.789 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795392+0000", 2024-04-24T15:34:06.789 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.452e-06 2024-04-24T15:34:06.789 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.789 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.789 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:06.789 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795397+0000", 2024-04-24T15:34:06.789 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2860000000000001e-06 2024-04-24T15:34:06.789 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.789 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.789 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:06.789 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836054+0000", 2024-04-24T15:34:06.789 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040657581999999998 2024-04-24T15:34:06.789 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.790 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.790 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:06.790 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836124+0000", 2024-04-24T15:34:06.790 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.0020999999999999e-05 2024-04-24T15:34:06.790 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.790 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.790 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:06.790 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836187+0000", 2024-04-24T15:34:06.790 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.3009999999999995e-05 2024-04-24T15:34:06.790 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.790 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:06.790 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.790 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.790 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.790 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23636 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15630336~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:06.791 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795610+0000", 2024-04-24T15:34:06.791 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 91.942496879999993, 2024-04-24T15:34:06.791 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040730942999999999, 2024-04-24T15:34:06.791 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:06.791 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:06.791 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:06.791 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:06.791 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:06.791 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23636 2024-04-24T15:34:06.791 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.791 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:06.791 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.791 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:06.791 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795610+0000", 2024-04-24T15:34:06.792 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.792 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.792 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.792 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:06.792 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795610+0000", 2024-04-24T15:34:06.792 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.792 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.792 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.792 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:06.792 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795401+0000", 2024-04-24T15:34:06.792 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9997911 2024-04-24T15:34:06.792 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.792 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.792 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:06.792 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795614+0000", 2024-04-24T15:34:06.792 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00021228600000000001 2024-04-24T15:34:06.792 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.793 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.793 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:06.793 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795622+0000", 2024-04-24T15:34:06.793 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3920000000000003e-06 2024-04-24T15:34:06.793 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.793 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.793 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:06.793 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795627+0000", 2024-04-24T15:34:06.793 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8199999999999996e-06 2024-04-24T15:34:06.793 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.793 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.793 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:06.793 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836211+0000", 2024-04-24T15:34:06.793 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040583803000000002 2024-04-24T15:34:06.793 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.794 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.794 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:06.794 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836284+0000", 2024-04-24T15:34:06.794 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.3617999999999999e-05 2024-04-24T15:34:06.794 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.794 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.794 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:06.794 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836341+0000", 2024-04-24T15:34:06.794 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6959999999999997e-05 2024-04-24T15:34:06.794 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.794 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:06.794 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.794 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.795 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.795 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23637 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15671296~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:06.795 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795699+0000", 2024-04-24T15:34:06.795 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 91.942407712999994, 2024-04-24T15:34:06.795 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040775556999999997, 2024-04-24T15:34:06.795 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:06.795 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:06.795 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:06.795 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:06.795 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:06.795 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23637 2024-04-24T15:34:06.795 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.795 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:06.795 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.796 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:06.796 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795699+0000", 2024-04-24T15:34:06.796 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.796 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.796 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.796 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:06.796 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795699+0000", 2024-04-24T15:34:06.796 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.796 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.796 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.796 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:06.796 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795632+0000", 2024-04-24T15:34:06.796 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999323 2024-04-24T15:34:06.796 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.796 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.796 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:06.797 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795704+0000", 2024-04-24T15:34:06.797 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2428000000000005e-05 2024-04-24T15:34:06.797 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.797 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.797 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:06.797 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795715+0000", 2024-04-24T15:34:06.797 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.1532000000000001e-05 2024-04-24T15:34:06.797 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.797 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.797 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:06.797 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795721+0000", 2024-04-24T15:34:06.797 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.8200000000000002e-06 2024-04-24T15:34:06.797 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.797 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.797 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:06.798 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836364+0000", 2024-04-24T15:34:06.798 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040642997 2024-04-24T15:34:06.798 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.798 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.798 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:06.798 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836435+0000", 2024-04-24T15:34:06.798 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.0821000000000005e-05 2024-04-24T15:34:06.798 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.798 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.798 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:06.798 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836475+0000", 2024-04-24T15:34:06.798 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9796999999999999e-05 2024-04-24T15:34:06.798 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.798 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:06.799 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.799 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.799 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.799 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23638 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15712256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:06.799 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.795932+0000", 2024-04-24T15:34:06.799 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 91.942174683000005, 2024-04-24T15:34:06.799 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040668682999999997, 2024-04-24T15:34:06.799 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:06.799 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:06.799 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:06.799 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:06.799 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:06.799 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23638 2024-04-24T15:34:06.799 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.799 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:06.799 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.800 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:06.800 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795932+0000", 2024-04-24T15:34:06.800 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.800 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.800 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.800 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:06.800 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795932+0000", 2024-04-24T15:34:06.800 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.800 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.800 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.800 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:06.800 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795728+0000", 2024-04-24T15:34:06.800 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9997959 2024-04-24T15:34:06.800 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.800 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.801 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:06.801 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795936+0000", 2024-04-24T15:34:06.801 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000208128 2024-04-24T15:34:06.801 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.801 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.801 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:06.801 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795945+0000", 2024-04-24T15:34:06.801 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.7110000000000001e-06 2024-04-24T15:34:06.801 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.801 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.801 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:06.801 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795957+0000", 2024-04-24T15:34:06.801 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.1800999999999999e-05 2024-04-24T15:34:06.801 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.802 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.802 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:06.802 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836505+0000", 2024-04-24T15:34:06.802 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040547622999999998 2024-04-24T15:34:06.802 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.802 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.802 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:06.802 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836561+0000", 2024-04-24T15:34:06.802 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6598999999999998e-05 2024-04-24T15:34:06.802 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.802 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.802 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:06.802 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836601+0000", 2024-04-24T15:34:06.802 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9907000000000003e-05 2024-04-24T15:34:06.803 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.803 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:06.803 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.803 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.803 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.803 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23639 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15753216~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:06.803 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.796071+0000", 2024-04-24T15:34:06.803 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 91.942036422000001, 2024-04-24T15:34:06.803 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040648199000000003, 2024-04-24T15:34:06.803 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:06.803 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:06.803 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:06.803 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:06.803 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:06.804 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23639 2024-04-24T15:34:06.804 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.804 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:06.804 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.804 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:06.804 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796071+0000", 2024-04-24T15:34:06.804 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.804 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.804 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.804 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:06.804 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796071+0000", 2024-04-24T15:34:06.804 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.804 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.804 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.805 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:06.805 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.795967+0000", 2024-04-24T15:34:06.805 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998965 2024-04-24T15:34:06.805 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.805 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.805 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:06.805 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796074+0000", 2024-04-24T15:34:06.805 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000107013 2024-04-24T15:34:06.805 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.805 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.805 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:06.805 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796081+0000", 2024-04-24T15:34:06.805 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.2590000000000003e-06 2024-04-24T15:34:06.805 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.806 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.806 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:06.806 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796085+0000", 2024-04-24T15:34:06.806 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5269999999999999e-06 2024-04-24T15:34:06.806 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.806 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.806 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:06.806 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836624+0000", 2024-04-24T15:34:06.806 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040539324000000002 2024-04-24T15:34:06.806 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.806 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.806 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:06.806 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836667+0000", 2024-04-24T15:34:06.807 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2562999999999999e-05 2024-04-24T15:34:06.807 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.807 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.807 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:06.807 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836719+0000", 2024-04-24T15:34:06.807 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.2015999999999998e-05 2024-04-24T15:34:06.807 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.807 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:06.807 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.807 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.807 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.807 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23640 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15794176~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:06.807 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.796492+0000", 2024-04-24T15:34:06.808 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 91.941614663999999, 2024-04-24T15:34:06.808 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040420000999999997, 2024-04-24T15:34:06.808 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:06.808 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:06.808 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:06.808 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:06.808 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:06.808 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23640 2024-04-24T15:34:06.808 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.808 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:06.808 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.808 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:06.808 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796492+0000", 2024-04-24T15:34:06.808 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.809 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.809 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.809 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:06.809 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796492+0000", 2024-04-24T15:34:06.809 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.809 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.809 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.809 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:06.809 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796088+0000", 2024-04-24T15:34:06.809 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9995961 2024-04-24T15:34:06.809 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.809 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.809 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:06.809 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796496+0000", 2024-04-24T15:34:06.810 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00040785700000000002 2024-04-24T15:34:06.810 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.810 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.810 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:06.810 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796503+0000", 2024-04-24T15:34:06.810 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1049999999999997e-06 2024-04-24T15:34:06.810 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.810 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.810 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:06.810 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796508+0000", 2024-04-24T15:34:06.810 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.155e-06 2024-04-24T15:34:06.810 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.810 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.811 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:06.811 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836756+0000", 2024-04-24T15:34:06.811 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040248850000000003 2024-04-24T15:34:06.811 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.811 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.811 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:06.811 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836841+0000", 2024-04-24T15:34:06.811 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.4735999999999996e-05 2024-04-24T15:34:06.811 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.811 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.811 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:06.811 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836912+0000", 2024-04-24T15:34:06.811 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1229999999999994e-05 2024-04-24T15:34:06.811 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.812 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:06.812 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.812 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.812 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.812 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23641 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15835136~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:06.812 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.796658+0000", 2024-04-24T15:34:06.812 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 91.941448588, 2024-04-24T15:34:06.812 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040420706000000001, 2024-04-24T15:34:06.812 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:06.812 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:06.812 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:06.812 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:06.812 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:06.813 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23641 2024-04-24T15:34:06.813 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.813 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:06.813 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.813 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:06.813 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796658+0000", 2024-04-24T15:34:06.813 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.813 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.813 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.813 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:06.813 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796658+0000", 2024-04-24T15:34:06.813 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.813 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.813 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.813 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:06.814 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796512+0000", 2024-04-24T15:34:06.814 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998531 2024-04-24T15:34:06.814 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.814 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.814 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:06.814 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796669+0000", 2024-04-24T15:34:06.814 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000157577 2024-04-24T15:34:06.814 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.814 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.814 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:06.814 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796677+0000", 2024-04-24T15:34:06.814 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.7800000000000001e-06 2024-04-24T15:34:06.814 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.815 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.815 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:06.815 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.796681+0000", 2024-04-24T15:34:06.815 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.721e-06 2024-04-24T15:34:06.815 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.815 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.815 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:06.815 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.836941+0000", 2024-04-24T15:34:06.815 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040259907999999997 2024-04-24T15:34:06.815 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.815 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.815 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:06.815 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837017+0000", 2024-04-24T15:34:06.815 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.6071999999999999e-05 2024-04-24T15:34:06.816 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.816 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.816 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:06.816 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837079+0000", 2024-04-24T15:34:06.816 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.2358000000000002e-05 2024-04-24T15:34:06.816 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.816 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:06.816 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.816 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.816 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.816 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23645 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 15998976~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:06.816 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.797360+0000", 2024-04-24T15:34:06.816 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 91.940747090000002, 2024-04-24T15:34:06.816 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040387645999999999, 2024-04-24T15:34:06.817 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:06.817 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:06.817 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:06.817 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:06.817 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:06.817 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23645 2024-04-24T15:34:06.817 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.817 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:06.817 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.817 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:06.817 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797360+0000", 2024-04-24T15:34:06.817 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.817 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.817 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.817 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:06.818 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797360+0000", 2024-04-24T15:34:06.818 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.818 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.818 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.818 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:06.818 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797288+0000", 2024-04-24T15:34:06.818 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999928 2024-04-24T15:34:06.818 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.818 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.818 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:06.818 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797364+0000", 2024-04-24T15:34:06.818 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.5347999999999998e-05 2024-04-24T15:34:06.818 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.818 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.819 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:06.819 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797372+0000", 2024-04-24T15:34:06.819 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.2409999999999995e-06 2024-04-24T15:34:06.819 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.819 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.819 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:06.819 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797376+0000", 2024-04-24T15:34:06.819 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.515e-06 2024-04-24T15:34:06.819 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.819 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.819 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:06.819 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837586+0000", 2024-04-24T15:34:06.819 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040210047999999998 2024-04-24T15:34:06.819 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.819 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.819 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:06.820 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837672+0000", 2024-04-24T15:34:06.820 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.5997999999999999e-05 2024-04-24T15:34:06.820 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.820 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.820 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:06.820 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837748+0000", 2024-04-24T15:34:06.820 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.5328999999999997e-05 2024-04-24T15:34:06.820 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.820 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:06.820 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.820 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.820 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.820 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:23646 4.3 4:c1a0df07:::rbd_data.114a38cedbad.0000000000000003:7 [read 16039936~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:06.820 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.797498+0000", 2024-04-24T15:34:06.821 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 91.940608988999998, 2024-04-24T15:34:06.821 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040433139, 2024-04-24T15:34:06.821 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:06.821 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:06.821 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:06.821 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:06.821 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:06.821 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 23646 2024-04-24T15:34:06.821 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.821 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:06.821 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.821 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:06.821 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797498+0000", 2024-04-24T15:34:06.821 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.821 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.822 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.822 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:06.822 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797498+0000", 2024-04-24T15:34:06.822 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:06.822 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.822 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.822 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:06.822 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797381+0000", 2024-04-24T15:34:06.822 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998832 2024-04-24T15:34:06.822 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.822 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.822 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:06.822 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797501+0000", 2024-04-24T15:34:06.823 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000119759 2024-04-24T15:34:06.823 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.823 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.823 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:06.823 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797510+0000", 2024-04-24T15:34:06.823 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.4770000000000003e-06 2024-04-24T15:34:06.823 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.823 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.823 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:06.823 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.797515+0000", 2024-04-24T15:34:06.823 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8330000000000001e-06 2024-04-24T15:34:06.823 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.823 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.823 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:06.824 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837777+0000", 2024-04-24T15:34:06.824 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.040262502999999998 2024-04-24T15:34:06.824 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.824 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.824 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:06.824 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837860+0000", 2024-04-24T15:34:06.824 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3002000000000003e-05 2024-04-24T15:34:06.824 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:06.824 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:06.824 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:06.824 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.837931+0000", 2024-04-24T15:34:06.824 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1222000000000004e-05 2024-04-24T15:34:06.824 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.824 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:06.824 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.825 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:06.825 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:06.825 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:34:06.826 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-24T15:34:06.827 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:34:06.827+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:34:06.927 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-04-24T15:34:06.928+0000 7f807b5ae700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 44076) UID: 0 2024-04-24T15:34:06.990 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:34:06.990 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:34:06.990 INFO:teuthology.orchestra.run.smithi151.stdout: "num_ops": 0 2024-04-24T15:34:06.990 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:34:06.999 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-24T15:34:07.043 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:34:07.045+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:34:07.122 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:34:07.122 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:34:07.123 INFO:teuthology.orchestra.run.smithi151.stdout: "complaint_time": 30, 2024-04-24T15:34:07.123 INFO:teuthology.orchestra.run.smithi151.stdout: "num_blocked_ops": 0 2024-04-24T15:34:07.123 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:34:07.129 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:34:07.130+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:34:07.132 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-24T15:34:07.229 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:34:07.231+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:34:07.298 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:34:07.299 INFO:teuthology.orchestra.run.smithi151.stdout: "size": 20, 2024-04-24T15:34:07.299 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 600, 2024-04-24T15:34:07.299 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [ 2024-04-24T15:34:07.299 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.299 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18007 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12460032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:07.299 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.207961+0000", 2024-04-24T15:34:07.299 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 93.090696983000001, 2024-04-24T15:34:07.299 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042965470999999998, 2024-04-24T15:34:07.299 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:07.299 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:07.299 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:07.299 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:07.300 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:07.300 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18007 2024-04-24T15:34:07.300 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.300 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:07.300 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.300 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:07.300 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207961+0000", 2024-04-24T15:34:07.300 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:07.300 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.300 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.300 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:07.300 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207961+0000", 2024-04-24T15:34:07.300 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:07.300 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.301 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.301 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:07.301 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207960+0000", 2024-04-24T15:34:07.301 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:07.301 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.301 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.301 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:07.301 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207963+0000", 2024-04-24T15:34:07.301 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.3759999999999999e-06 2024-04-24T15:34:07.301 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.301 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.301 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:07.301 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207970+0000", 2024-04-24T15:34:07.301 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.0489999999999998e-06 2024-04-24T15:34:07.302 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.302 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.302 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:07.302 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.207975+0000", 2024-04-24T15:34:07.302 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.1640000000000003e-06 2024-04-24T15:34:07.302 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.302 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.302 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:07.302 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250834+0000", 2024-04-24T15:34:07.302 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042859060999999997 2024-04-24T15:34:07.302 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.302 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.302 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:07.302 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250874+0000", 2024-04-24T15:34:07.303 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0674999999999999e-05 2024-04-24T15:34:07.303 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.303 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.303 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:07.303 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250926+0000", 2024-04-24T15:34:07.303 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.1693000000000001e-05 2024-04-24T15:34:07.303 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:07.303 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:07.303 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:07.303 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.303 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.303 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18008 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12500992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:07.303 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208030+0000", 2024-04-24T15:34:07.303 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 93.090627889999993, 2024-04-24T15:34:07.303 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043003337000000003, 2024-04-24T15:34:07.304 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:07.304 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:07.304 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:07.304 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:07.304 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:07.304 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18008 2024-04-24T15:34:07.304 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.304 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:07.304 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.304 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:07.304 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208030+0000", 2024-04-24T15:34:07.304 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:07.304 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.304 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.304 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:07.305 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208030+0000", 2024-04-24T15:34:07.305 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:07.305 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.305 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.305 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:07.305 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208029+0000", 2024-04-24T15:34:07.305 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:07.305 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.305 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.305 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:07.305 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208032+0000", 2024-04-24T15:34:07.305 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.117e-06 2024-04-24T15:34:07.305 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.305 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.305 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:07.306 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208039+0000", 2024-04-24T15:34:07.306 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.0129999999999996e-06 2024-04-24T15:34:07.306 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.306 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.306 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:07.306 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208043+0000", 2024-04-24T15:34:07.306 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0600000000000001e-06 2024-04-24T15:34:07.306 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.306 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.306 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:07.306 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250948+0000", 2024-04-24T15:34:07.306 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042904577999999999 2024-04-24T15:34:07.306 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.306 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.307 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:07.307 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.250989+0000", 2024-04-24T15:34:07.307 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0978e-05 2024-04-24T15:34:07.307 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.307 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.307 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:07.307 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251033+0000", 2024-04-24T15:34:07.307 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4161e-05 2024-04-24T15:34:07.307 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:07.307 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:07.307 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:07.307 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.307 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.307 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18009 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12541952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:07.308 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208093+0000", 2024-04-24T15:34:07.308 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 93.090564143999998, 2024-04-24T15:34:07.308 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043043451000000003, 2024-04-24T15:34:07.308 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:07.308 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:07.308 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:07.308 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:07.308 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:07.308 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18009 2024-04-24T15:34:07.308 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.308 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:07.308 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.308 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:07.308 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208093+0000", 2024-04-24T15:34:07.308 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:07.308 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.309 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.309 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:07.309 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208093+0000", 2024-04-24T15:34:07.309 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:07.309 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.309 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.309 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:07.309 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208093+0000", 2024-04-24T15:34:07.309 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:07.309 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.309 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.309 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:07.309 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208096+0000", 2024-04-24T15:34:07.309 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.439e-06 2024-04-24T15:34:07.309 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.309 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.310 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:07.310 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208103+0000", 2024-04-24T15:34:07.310 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.1019999999999999e-06 2024-04-24T15:34:07.310 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.310 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.310 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:07.310 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208107+0000", 2024-04-24T15:34:07.310 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0910000000000003e-06 2024-04-24T15:34:07.310 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.310 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.310 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:07.310 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251054+0000", 2024-04-24T15:34:07.310 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042946826 2024-04-24T15:34:07.310 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.310 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.311 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:07.311 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251092+0000", 2024-04-24T15:34:07.311 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.7941999999999997e-05 2024-04-24T15:34:07.311 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.311 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.311 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:07.311 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251137+0000", 2024-04-24T15:34:07.311 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4638000000000002e-05 2024-04-24T15:34:07.311 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:07.311 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:07.311 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:07.311 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.311 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.311 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18010 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12615680~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:07.312 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208202+0000", 2024-04-24T15:34:07.312 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 93.090455313000007, 2024-04-24T15:34:07.312 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043046549000000003, 2024-04-24T15:34:07.312 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:07.312 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:07.312 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:07.312 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:07.312 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:07.312 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18010 2024-04-24T15:34:07.312 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.312 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:07.312 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.312 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:07.312 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208202+0000", 2024-04-24T15:34:07.313 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:07.313 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.313 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.313 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:07.313 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208202+0000", 2024-04-24T15:34:07.313 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:07.313 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.313 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.313 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:07.313 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208202+0000", 2024-04-24T15:34:07.313 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:07.313 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.313 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.313 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:07.313 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208205+0000", 2024-04-24T15:34:07.314 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.2490000000000001e-06 2024-04-24T15:34:07.314 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.314 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.314 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:07.314 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208215+0000", 2024-04-24T15:34:07.314 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.9070000000000002e-06 2024-04-24T15:34:07.314 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.314 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.314 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:07.314 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208220+0000", 2024-04-24T15:34:07.314 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8849999999999998e-06 2024-04-24T15:34:07.314 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.314 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.314 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:07.315 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251158+0000", 2024-04-24T15:34:07.315 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042938416 2024-04-24T15:34:07.315 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.315 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.315 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:07.315 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251197+0000", 2024-04-24T15:34:07.315 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9341000000000002e-05 2024-04-24T15:34:07.315 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.315 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.315 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:07.315 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251249+0000", 2024-04-24T15:34:07.315 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.1391000000000003e-05 2024-04-24T15:34:07.315 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:07.315 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:07.315 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:07.316 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.316 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.316 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18011 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12656640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:07.316 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208258+0000", 2024-04-24T15:34:07.316 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 93.090399466999997, 2024-04-24T15:34:07.316 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043102708000000003, 2024-04-24T15:34:07.316 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:07.316 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:07.316 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:07.316 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:07.316 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:07.316 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18011 2024-04-24T15:34:07.316 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.316 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:07.316 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.317 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:07.317 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208258+0000", 2024-04-24T15:34:07.317 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:07.317 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.317 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.317 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:07.317 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208258+0000", 2024-04-24T15:34:07.317 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:07.317 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.317 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.317 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:07.317 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208258+0000", 2024-04-24T15:34:07.317 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:07.317 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.317 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.318 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:07.318 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208260+0000", 2024-04-24T15:34:07.318 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.7810000000000001e-06 2024-04-24T15:34:07.318 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.318 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.318 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:07.318 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208267+0000", 2024-04-24T15:34:07.318 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.7610000000000002e-06 2024-04-24T15:34:07.318 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.318 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.318 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:07.318 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208271+0000", 2024-04-24T15:34:07.318 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0890000000000002e-06 2024-04-24T15:34:07.318 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.319 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.319 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:07.319 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251271+0000", 2024-04-24T15:34:07.319 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043000159000000003 2024-04-24T15:34:07.319 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.319 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.319 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:07.319 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251315+0000", 2024-04-24T15:34:07.319 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3446999999999999e-05 2024-04-24T15:34:07.319 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.319 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.319 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:07.319 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251361+0000", 2024-04-24T15:34:07.319 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5952000000000003e-05 2024-04-24T15:34:07.319 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:07.320 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:07.320 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:07.320 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.320 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.320 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18012 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12697600~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:07.320 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208289+0000", 2024-04-24T15:34:07.320 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 93.090368423000001, 2024-04-24T15:34:07.320 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043174774999999999, 2024-04-24T15:34:07.320 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:07.320 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:07.320 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:07.320 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:07.320 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:07.320 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18012 2024-04-24T15:34:07.320 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.321 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:07.321 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.321 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:07.321 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208289+0000", 2024-04-24T15:34:07.321 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:07.321 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.321 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.321 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:07.321 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208289+0000", 2024-04-24T15:34:07.321 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:07.321 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.321 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.321 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:07.321 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208289+0000", 2024-04-24T15:34:07.321 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:07.321 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.322 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.322 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:07.322 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208291+0000", 2024-04-24T15:34:07.322 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.542e-06 2024-04-24T15:34:07.322 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.322 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.322 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:07.322 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208298+0000", 2024-04-24T15:34:07.322 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.3899999999999998e-06 2024-04-24T15:34:07.322 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.322 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.322 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:07.322 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208302+0000", 2024-04-24T15:34:07.323 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.019e-06 2024-04-24T15:34:07.323 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.323 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.323 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:07.323 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251382+0000", 2024-04-24T15:34:07.323 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043080790000000001 2024-04-24T15:34:07.323 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.323 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.323 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:07.323 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251421+0000", 2024-04-24T15:34:07.323 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8473000000000002e-05 2024-04-24T15:34:07.323 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.323 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.323 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:07.323 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251464+0000", 2024-04-24T15:34:07.324 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3133999999999997e-05 2024-04-24T15:34:07.324 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:07.324 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:07.324 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:07.324 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.324 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.324 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18013 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12738560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:07.324 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208423+0000", 2024-04-24T15:34:07.324 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 93.090234315000004, 2024-04-24T15:34:07.324 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043145554000000003, 2024-04-24T15:34:07.324 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:07.324 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:07.324 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:07.324 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:07.324 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:07.325 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18013 2024-04-24T15:34:07.325 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.325 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:07.325 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.325 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:07.325 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208423+0000", 2024-04-24T15:34:07.325 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:07.325 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.325 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.325 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:07.325 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208423+0000", 2024-04-24T15:34:07.325 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:07.325 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.325 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.326 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:07.326 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208423+0000", 2024-04-24T15:34:07.326 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:07.326 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.326 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.326 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:07.326 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208426+0000", 2024-04-24T15:34:07.326 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.0539999999999999e-06 2024-04-24T15:34:07.326 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.326 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.326 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:07.326 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208433+0000", 2024-04-24T15:34:07.326 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.43e-06 2024-04-24T15:34:07.326 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.327 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.327 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:07.327 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208437+0000", 2024-04-24T15:34:07.327 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5590000000000001e-06 2024-04-24T15:34:07.327 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.327 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.327 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:07.327 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251485+0000", 2024-04-24T15:34:07.327 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043048259999999998 2024-04-24T15:34:07.327 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.327 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.327 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:07.327 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251525+0000", 2024-04-24T15:34:07.327 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9916999999999997e-05 2024-04-24T15:34:07.328 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.328 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.328 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:07.328 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251569+0000", 2024-04-24T15:34:07.328 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3939000000000001e-05 2024-04-24T15:34:07.328 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:07.328 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:07.328 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:07.328 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.328 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.328 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18015 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12779520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:07.328 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208630+0000", 2024-04-24T15:34:07.328 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 93.090027789999994, 2024-04-24T15:34:07.328 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043047328000000003, 2024-04-24T15:34:07.328 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:07.329 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:07.329 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:07.329 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:07.329 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:07.329 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18015 2024-04-24T15:34:07.329 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.329 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:07.329 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.329 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:07.329 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208630+0000", 2024-04-24T15:34:07.329 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:07.329 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.329 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.329 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:07.330 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208630+0000", 2024-04-24T15:34:07.330 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:07.330 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.330 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.330 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:07.330 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208629+0000", 2024-04-24T15:34:07.330 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999999 2024-04-24T15:34:07.330 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.330 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.330 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:07.330 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208634+0000", 2024-04-24T15:34:07.330 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8960000000000002e-06 2024-04-24T15:34:07.330 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.330 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.330 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:07.331 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208645+0000", 2024-04-24T15:34:07.331 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0939e-05 2024-04-24T15:34:07.331 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.331 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.331 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:07.331 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208651+0000", 2024-04-24T15:34:07.331 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.896e-06 2024-04-24T15:34:07.331 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.331 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.331 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:07.331 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251591+0000", 2024-04-24T15:34:07.331 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042939862000000002 2024-04-24T15:34:07.331 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.331 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.332 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:07.332 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251631+0000", 2024-04-24T15:34:07.332 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0216000000000003e-05 2024-04-24T15:34:07.332 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.332 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.332 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:07.332 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251677+0000", 2024-04-24T15:34:07.332 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6390999999999997e-05 2024-04-24T15:34:07.332 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:07.332 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:07.332 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:07.332 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.332 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.332 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18016 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12820480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:07.332 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208660+0000", 2024-04-24T15:34:07.333 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 93.089998000999998, 2024-04-24T15:34:07.333 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043127510000000001, 2024-04-24T15:34:07.333 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:07.333 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:07.333 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:07.333 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:07.333 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:07.333 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18016 2024-04-24T15:34:07.333 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.333 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:07.333 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.333 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:07.333 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208660+0000", 2024-04-24T15:34:07.333 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:07.333 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.333 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.334 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:07.334 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208660+0000", 2024-04-24T15:34:07.334 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:07.334 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.334 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.334 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:07.334 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208659+0000", 2024-04-24T15:34:07.334 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:07.334 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.334 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.334 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:07.334 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208662+0000", 2024-04-24T15:34:07.334 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.0979999999999998e-06 2024-04-24T15:34:07.334 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.334 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.335 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:07.335 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208669+0000", 2024-04-24T15:34:07.335 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.4470000000000003e-06 2024-04-24T15:34:07.335 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.335 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.335 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:07.335 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208673+0000", 2024-04-24T15:34:07.335 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.2810000000000002e-06 2024-04-24T15:34:07.335 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.335 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.335 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:07.335 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251699+0000", 2024-04-24T15:34:07.335 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043025728999999999 2024-04-24T15:34:07.335 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.335 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.336 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:07.336 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251743+0000", 2024-04-24T15:34:07.336 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4469999999999999e-05 2024-04-24T15:34:07.336 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.336 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.336 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:07.336 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251787+0000", 2024-04-24T15:34:07.336 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3949999999999998e-05 2024-04-24T15:34:07.336 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:07.336 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:07.336 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:07.336 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.336 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.336 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18017 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12861440~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:07.336 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208721+0000", 2024-04-24T15:34:07.336 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 93.089936553000001, 2024-04-24T15:34:07.337 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043167749999999998, 2024-04-24T15:34:07.337 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:07.337 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:07.337 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:07.337 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:07.337 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:07.337 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18017 2024-04-24T15:34:07.337 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.337 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:07.337 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.337 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:07.337 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208721+0000", 2024-04-24T15:34:07.337 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:07.337 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.337 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.337 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:07.338 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208721+0000", 2024-04-24T15:34:07.338 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:07.338 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.338 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.338 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:07.338 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208720+0000", 2024-04-24T15:34:07.338 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:07.338 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.338 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.338 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:07.338 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208724+0000", 2024-04-24T15:34:07.338 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.3919999999999999e-06 2024-04-24T15:34:07.338 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.338 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.338 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:07.339 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208729+0000", 2024-04-24T15:34:07.339 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.3480000000000003e-06 2024-04-24T15:34:07.339 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.339 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.339 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:07.339 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208732+0000", 2024-04-24T15:34:07.339 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.9529999999999999e-06 2024-04-24T15:34:07.339 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.339 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.339 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:07.339 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251807+0000", 2024-04-24T15:34:07.339 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043074492999999998 2024-04-24T15:34:07.339 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.339 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.339 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:07.339 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251846+0000", 2024-04-24T15:34:07.339 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8958999999999997e-05 2024-04-24T15:34:07.340 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.340 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.340 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:07.340 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251889+0000", 2024-04-24T15:34:07.340 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3253000000000001e-05 2024-04-24T15:34:07.340 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:07.340 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:07.340 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:07.340 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.340 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.340 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18018 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12902400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:07.340 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208752+0000", 2024-04-24T15:34:07.340 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 93.089905939999994, 2024-04-24T15:34:07.340 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043241012000000002, 2024-04-24T15:34:07.340 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:07.341 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:07.341 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:07.341 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:07.341 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:07.341 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18018 2024-04-24T15:34:07.341 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.341 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:07.341 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.341 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:07.341 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208752+0000", 2024-04-24T15:34:07.341 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:07.341 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.341 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.341 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:07.341 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208752+0000", 2024-04-24T15:34:07.342 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:07.342 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.342 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.342 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:07.342 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208751+0000", 2024-04-24T15:34:07.342 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:07.342 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.342 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.342 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:07.342 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208753+0000", 2024-04-24T15:34:07.342 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.894e-06 2024-04-24T15:34:07.342 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.343 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.343 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:07.343 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208757+0000", 2024-04-24T15:34:07.343 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.856e-06 2024-04-24T15:34:07.343 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.343 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.343 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:07.343 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208760+0000", 2024-04-24T15:34:07.343 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.6749999999999998e-06 2024-04-24T15:34:07.343 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.343 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.343 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:07.343 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251909+0000", 2024-04-24T15:34:07.343 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043149145999999999 2024-04-24T15:34:07.343 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.344 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.344 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:07.344 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251948+0000", 2024-04-24T15:34:07.344 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9159000000000002e-05 2024-04-24T15:34:07.344 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.344 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.344 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:07.344 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.251993+0000", 2024-04-24T15:34:07.344 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.4638999999999997e-05 2024-04-24T15:34:07.344 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:07.344 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:07.344 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:07.344 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.344 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.345 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18019 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12943360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:07.345 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.208938+0000", 2024-04-24T15:34:07.345 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 93.089719649000003, 2024-04-24T15:34:07.345 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043162272000000002, 2024-04-24T15:34:07.345 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:07.345 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:07.345 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:07.345 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:07.345 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:07.345 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18019 2024-04-24T15:34:07.345 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.345 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:07.345 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.345 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:07.345 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208938+0000", 2024-04-24T15:34:07.346 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:07.346 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.346 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.346 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:07.346 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208938+0000", 2024-04-24T15:34:07.346 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:07.346 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.346 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.346 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:07.346 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208762+0000", 2024-04-24T15:34:07.346 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999824 2024-04-24T15:34:07.346 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.346 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.347 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:07.347 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208942+0000", 2024-04-24T15:34:07.347 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000180008 2024-04-24T15:34:07.347 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.347 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.347 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:07.347 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208950+0000", 2024-04-24T15:34:07.347 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.1550000000000007e-06 2024-04-24T15:34:07.347 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.347 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.347 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:07.347 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208956+0000", 2024-04-24T15:34:07.347 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.7910000000000001e-06 2024-04-24T15:34:07.347 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.347 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.348 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:07.348 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252014+0000", 2024-04-24T15:34:07.348 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043058427000000003 2024-04-24T15:34:07.348 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.348 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.348 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:07.348 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252054+0000", 2024-04-24T15:34:07.348 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0071999999999999e-05 2024-04-24T15:34:07.348 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.348 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.348 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:07.348 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252100+0000", 2024-04-24T15:34:07.348 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5917000000000001e-05 2024-04-24T15:34:07.348 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:07.348 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:07.348 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:07.349 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.349 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.349 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18020 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 12984320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:07.349 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.209092+0000", 2024-04-24T15:34:07.349 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 93.089565651000001, 2024-04-24T15:34:07.349 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043111295000000001, 2024-04-24T15:34:07.349 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:07.349 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:07.349 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:07.349 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:07.349 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:07.349 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18020 2024-04-24T15:34:07.349 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.349 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:07.350 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.350 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:07.350 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209092+0000", 2024-04-24T15:34:07.350 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:07.350 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.350 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.350 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:07.350 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209092+0000", 2024-04-24T15:34:07.350 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:07.350 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.350 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.350 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:07.350 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.208964+0000", 2024-04-24T15:34:07.350 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9998722 2024-04-24T15:34:07.350 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.351 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.351 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:07.351 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209095+0000", 2024-04-24T15:34:07.351 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000130648 2024-04-24T15:34:07.351 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.351 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.351 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:07.351 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209103+0000", 2024-04-24T15:34:07.351 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.0320000000000003e-06 2024-04-24T15:34:07.351 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.351 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.351 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:07.351 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209108+0000", 2024-04-24T15:34:07.352 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.8690000000000003e-06 2024-04-24T15:34:07.352 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.352 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.352 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:07.352 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252121+0000", 2024-04-24T15:34:07.352 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043013323999999999 2024-04-24T15:34:07.352 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.352 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.352 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:07.352 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252160+0000", 2024-04-24T15:34:07.352 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8376e-05 2024-04-24T15:34:07.352 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.352 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.352 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:07.352 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252203+0000", 2024-04-24T15:34:07.352 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.371e-05 2024-04-24T15:34:07.353 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:07.353 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:07.353 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:07.353 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.353 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.353 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18021 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 13025280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:07.353 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.209132+0000", 2024-04-24T15:34:07.353 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 93.089525292999994, 2024-04-24T15:34:07.353 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043183903000000003, 2024-04-24T15:34:07.353 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:07.353 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:07.353 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:07.353 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:07.353 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:07.353 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18021 2024-04-24T15:34:07.353 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.353 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:07.354 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.354 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:07.354 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209132+0000", 2024-04-24T15:34:07.354 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:07.354 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.354 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.354 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:07.354 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209132+0000", 2024-04-24T15:34:07.354 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:07.354 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.354 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.354 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:07.354 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209132+0000", 2024-04-24T15:34:07.354 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:07.354 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.355 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.355 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:07.355 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209134+0000", 2024-04-24T15:34:07.355 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.891e-06 2024-04-24T15:34:07.355 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.355 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.355 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:07.355 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209138+0000", 2024-04-24T15:34:07.355 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0659999999999997e-06 2024-04-24T15:34:07.355 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.355 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.355 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:07.355 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209140+0000", 2024-04-24T15:34:07.355 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.5339999999999998e-06 2024-04-24T15:34:07.355 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.356 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.356 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:07.356 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252232+0000", 2024-04-24T15:34:07.356 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043091142999999998 2024-04-24T15:34:07.356 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.356 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.356 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:07.356 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252270+0000", 2024-04-24T15:34:07.356 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8872000000000003e-05 2024-04-24T15:34:07.356 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.356 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.356 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:07.356 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252316+0000", 2024-04-24T15:34:07.356 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.5788000000000003e-05 2024-04-24T15:34:07.356 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:07.357 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:07.357 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:07.357 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.357 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.357 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18022 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 13066240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:07.357 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.209411+0000", 2024-04-24T15:34:07.357 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 93.089246744999997, 2024-04-24T15:34:07.357 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.043016616000000001, 2024-04-24T15:34:07.357 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:07.357 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:07.357 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:07.357 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:07.357 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:07.357 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18022 2024-04-24T15:34:07.357 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.358 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:07.358 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.358 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:07.358 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209411+0000", 2024-04-24T15:34:07.358 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:07.358 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.358 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.358 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:07.358 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209411+0000", 2024-04-24T15:34:07.358 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:07.358 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.358 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.358 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:07.359 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209143+0000", 2024-04-24T15:34:07.359 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999732 2024-04-24T15:34:07.359 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.359 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.359 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:07.359 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209415+0000", 2024-04-24T15:34:07.359 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.000271873 2024-04-24T15:34:07.359 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.359 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.359 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:07.359 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209423+0000", 2024-04-24T15:34:07.359 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.3080000000000008e-06 2024-04-24T15:34:07.359 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.359 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.359 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:07.360 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209428+0000", 2024-04-24T15:34:07.360 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.5040000000000002e-06 2024-04-24T15:34:07.360 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.360 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.360 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:07.360 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252339+0000", 2024-04-24T15:34:07.360 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042910419999999998 2024-04-24T15:34:07.360 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.360 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.360 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:07.360 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252380+0000", 2024-04-24T15:34:07.360 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.0991000000000001e-05 2024-04-24T15:34:07.360 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.360 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.360 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:07.361 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252427+0000", 2024-04-24T15:34:07.361 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.7647e-05 2024-04-24T15:34:07.361 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:07.361 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:07.361 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:07.361 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.361 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.361 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18023 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 13107200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:07.361 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.209594+0000", 2024-04-24T15:34:07.361 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 93.089063232000001, 2024-04-24T15:34:07.361 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042938543000000003, 2024-04-24T15:34:07.361 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:07.361 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:07.361 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:07.361 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:07.362 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:07.362 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18023 2024-04-24T15:34:07.362 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.362 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:07.362 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.362 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:07.362 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209594+0000", 2024-04-24T15:34:07.362 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:07.362 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.362 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.362 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:07.362 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209594+0000", 2024-04-24T15:34:07.362 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:07.363 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.363 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.363 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:07.363 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209440+0000", 2024-04-24T15:34:07.363 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999846 2024-04-24T15:34:07.363 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.363 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.363 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:07.363 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209598+0000", 2024-04-24T15:34:07.363 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00015731100000000001 2024-04-24T15:34:07.363 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.363 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.363 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:07.363 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209604+0000", 2024-04-24T15:34:07.363 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.2910000000000004e-06 2024-04-24T15:34:07.363 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.364 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.364 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:07.364 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209607+0000", 2024-04-24T15:34:07.364 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5389999999999999e-06 2024-04-24T15:34:07.364 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.364 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.364 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:07.364 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252449+0000", 2024-04-24T15:34:07.364 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042841833000000003 2024-04-24T15:34:07.364 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.364 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.364 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:07.365 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252489+0000", 2024-04-24T15:34:07.365 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.9898000000000003e-05 2024-04-24T15:34:07.365 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.365 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.365 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:07.365 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252533+0000", 2024-04-24T15:34:07.365 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.3702999999999998e-05 2024-04-24T15:34:07.365 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:07.365 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:07.365 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:07.365 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.365 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.365 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:18025 4.1e 4:7c76f732:::rbd_data.114a38cedbad.0000000000000001:7 [read 13189120~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T15:34:07.365 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:34.209832+0000", 2024-04-24T15:34:07.365 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 93.088825731, 2024-04-24T15:34:07.366 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042913556999999998, 2024-04-24T15:34:07.366 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:07.366 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:07.366 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:07.366 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:07.366 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:07.366 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 18025 2024-04-24T15:34:07.366 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.366 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:07.366 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.366 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:07.366 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209832+0000", 2024-04-24T15:34:07.366 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:07.366 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.367 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.367 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:07.367 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209832+0000", 2024-04-24T15:34:07.367 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:07.367 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.367 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.367 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:07.367 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209774+0000", 2024-04-24T15:34:07.367 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999418 2024-04-24T15:34:07.367 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.367 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.367 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:07.367 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209835+0000", 2024-04-24T15:34:07.367 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.1377000000000005e-05 2024-04-24T15:34:07.367 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.368 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.368 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:07.368 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209843+0000", 2024-04-24T15:34:07.368 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.2330000000000006e-06 2024-04-24T15:34:07.368 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.368 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.368 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:07.368 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.209848+0000", 2024-04-24T15:34:07.368 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.6149999999999999e-06 2024-04-24T15:34:07.368 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.368 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.368 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:07.368 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252658+0000", 2024-04-24T15:34:07.368 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.042810002 2024-04-24T15:34:07.368 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.369 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.369 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:07.369 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252697+0000", 2024-04-24T15:34:07.369 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.8757999999999997e-05 2024-04-24T15:34:07.369 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.369 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.369 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:07.369 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:34.252745+0000", 2024-04-24T15:34:07.369 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.863e-05 2024-04-24T15:34:07.369 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:07.369 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:07.369 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:07.369 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.369 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.369 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:24749 4.8 4:137f5701:::rbd_header.114ae15b9a20:head [watch reconnect cookie 140672595753696 gen 11] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-04-24T15:34:07.369 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:37.628703+0000", 2024-04-24T15:34:07.370 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 89.669954111999999, 2024-04-24T15:34:07.370 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0102918700000001, 2024-04-24T15:34:07.370 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:07.370 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:07.370 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:07.370 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:07.370 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:07.370 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 24749 2024-04-24T15:34:07.370 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.370 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:07.370 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.370 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:07.370 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628703+0000", 2024-04-24T15:34:07.370 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:07.370 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.371 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.371 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:07.371 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628703+0000", 2024-04-24T15:34:07.371 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:07.371 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.371 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.371 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:07.371 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628703+0000", 2024-04-24T15:34:07.371 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999999 2024-04-24T15:34:07.371 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.371 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.371 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:07.371 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628708+0000", 2024-04-24T15:34:07.371 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.4589999999999997e-06 2024-04-24T15:34:07.372 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.372 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.372 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:07.372 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628721+0000", 2024-04-24T15:34:07.372 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.2601e-05 2024-04-24T15:34:07.372 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.372 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.372 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:07.372 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628728+0000", 2024-04-24T15:34:07.372 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.9059999999999996e-06 2024-04-24T15:34:07.372 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.372 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.372 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:07.372 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628789+0000", 2024-04-24T15:34:07.372 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 6.1672000000000002e-05 2024-04-24T15:34:07.373 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.373 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.373 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting_for_map not empty", 2024-04-24T15:34:07.373 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628797+0000", 2024-04-24T15:34:07.373 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.8189999999999992e-06 2024-04-24T15:34:07.373 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.373 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.373 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:07.373 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.631255+0000", 2024-04-24T15:34:07.373 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0024577610000000001 2024-04-24T15:34:07.373 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.373 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.373 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for peered", 2024-04-24T15:34:07.373 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.631263+0000", 2024-04-24T15:34:07.373 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 8.4109999999999996e-06 2024-04-24T15:34:07.374 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.374 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.374 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:07.374 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:38.637560+0000", 2024-04-24T15:34:07.374 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0062970339999999 2024-04-24T15:34:07.374 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.374 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.374 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:07.374 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:38.637747+0000", 2024-04-24T15:34:07.374 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00018693400000000001 2024-04-24T15:34:07.374 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.374 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.374 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:07.374 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:38.638995+0000", 2024-04-24T15:34:07.375 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0012481300000000001 2024-04-24T15:34:07.375 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:07.375 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:07.375 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:07.375 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.375 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.375 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:24750 4.8 4:137f5701:::rbd_header.114ae15b9a20:head [notify cookie 140672594434416] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-04-24T15:34:07.375 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:32:37.628736+0000", 2024-04-24T15:34:07.375 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 89.669921852000002, 2024-04-24T15:34:07.375 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0096094309999999, 2024-04-24T15:34:07.375 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:07.375 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "started", 2024-04-24T15:34:07.375 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:07.375 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:07.375 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:07.376 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 24750 2024-04-24T15:34:07.376 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.376 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:07.376 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.376 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:07.376 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628736+0000", 2024-04-24T15:34:07.376 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:07.376 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.376 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.376 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:07.376 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628736+0000", 2024-04-24T15:34:07.376 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:07.376 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.376 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.376 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:07.377 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628735+0000", 2024-04-24T15:34:07.377 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.9999995 2024-04-24T15:34:07.377 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.377 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.377 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:07.377 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628739+0000", 2024-04-24T15:34:07.377 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 3.5109999999999999e-06 2024-04-24T15:34:07.377 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.377 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.377 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:07.377 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628745+0000", 2024-04-24T15:34:07.377 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.8300000000000001e-06 2024-04-24T15:34:07.377 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.377 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.377 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:07.378 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628749+0000", 2024-04-24T15:34:07.378 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4.7110000000000003e-06 2024-04-24T15:34:07.378 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.378 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.378 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:07.378 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628826+0000", 2024-04-24T15:34:07.378 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.6678e-05 2024-04-24T15:34:07.378 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.378 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.378 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting_for_map not empty", 2024-04-24T15:34:07.378 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.628834+0000", 2024-04-24T15:34:07.378 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.6190000000000002e-06 2024-04-24T15:34:07.378 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.379 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.379 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:07.379 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.631305+0000", 2024-04-24T15:34:07.379 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0024714400000000001 2024-04-24T15:34:07.379 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.379 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.379 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for peered", 2024-04-24T15:34:07.379 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:37.631313+0000", 2024-04-24T15:34:07.379 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 7.6529999999999994e-06 2024-04-24T15:34:07.379 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.379 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.379 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:07.379 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:38.638068+0000", 2024-04-24T15:34:07.379 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.0067548369999999 2024-04-24T15:34:07.380 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.380 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.380 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:07.380 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:38.638158+0000", 2024-04-24T15:34:07.380 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 9.0079000000000003e-05 2024-04-24T15:34:07.380 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.380 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.380 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:07.380 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:32:38.638345+0000", 2024-04-24T15:34:07.380 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00018757100000000001 2024-04-24T15:34:07.380 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:07.380 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:07.380 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:07.380 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.380 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.381 INFO:teuthology.orchestra.run.smithi151.stdout: "description": "osd_op(client.4426.0:76870 4.0 4:0746368d:::rbd_data.114abb9508e9.0000000000000001:head [delete] snapc f=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-04-24T15:34:07.381 INFO:teuthology.orchestra.run.smithi151.stdout: "initiated_at": "2024-04-24T15:33:56.733703+0000", 2024-04-24T15:34:07.381 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 10.564954424, 2024-04-24T15:34:07.381 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.91258213700000002, 2024-04-24T15:34:07.381 INFO:teuthology.orchestra.run.smithi151.stdout: "type_data": { 2024-04-24T15:34:07.381 INFO:teuthology.orchestra.run.smithi151.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T15:34:07.381 INFO:teuthology.orchestra.run.smithi151.stdout: "client_info": { 2024-04-24T15:34:07.381 INFO:teuthology.orchestra.run.smithi151.stdout: "client": "client.4426", 2024-04-24T15:34:07.381 INFO:teuthology.orchestra.run.smithi151.stdout: "client_addr": "172.21.15.96:0/3163644175", 2024-04-24T15:34:07.381 INFO:teuthology.orchestra.run.smithi151.stdout: "tid": 76870 2024-04-24T15:34:07.381 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.381 INFO:teuthology.orchestra.run.smithi151.stdout: "events": [ 2024-04-24T15:34:07.381 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.381 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "initiated", 2024-04-24T15:34:07.381 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:56.733703+0000", 2024-04-24T15:34:07.382 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:07.382 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.382 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.382 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "throttled", 2024-04-24T15:34:07.382 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:56.733703+0000", 2024-04-24T15:34:07.382 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0 2024-04-24T15:34:07.382 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.382 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.382 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "header_read", 2024-04-24T15:34:07.382 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:56.733702+0000", 2024-04-24T15:34:07.382 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 4294967295.999999 2024-04-24T15:34:07.382 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.382 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.383 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "all_read", 2024-04-24T15:34:07.383 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:56.733708+0000", 2024-04-24T15:34:07.383 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.6060000000000002e-06 2024-04-24T15:34:07.383 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.383 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.383 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "dispatched", 2024-04-24T15:34:07.383 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:56.733720+0000", 2024-04-24T15:34:07.383 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.1727e-05 2024-04-24T15:34:07.383 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.383 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.383 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "queued_for_pg", 2024-04-24T15:34:07.383 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:56.733725+0000", 2024-04-24T15:34:07.383 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.1800000000000004e-06 2024-04-24T15:34:07.383 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.384 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.384 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:07.384 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:56.733775+0000", 2024-04-24T15:34:07.384 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 5.0692000000000002e-05 2024-04-24T15:34:07.384 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.384 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.384 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for peered", 2024-04-24T15:34:07.384 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:56.733787+0000", 2024-04-24T15:34:07.384 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 1.1341e-05 2024-04-24T15:34:07.384 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.384 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.384 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "reached_pg", 2024-04-24T15:34:07.384 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:57.642929+0000", 2024-04-24T15:34:07.384 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.90914176199999996 2024-04-24T15:34:07.385 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.385 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.385 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "started", 2024-04-24T15:34:07.385 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:57.643088+0000", 2024-04-24T15:34:07.385 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00015902599999999999 2024-04-24T15:34:07.385 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.385 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.385 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "waiting for subops from 2", 2024-04-24T15:34:07.385 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:57.643227+0000", 2024-04-24T15:34:07.385 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.00013894499999999999 2024-04-24T15:34:07.385 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.385 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.385 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "op_commit", 2024-04-24T15:34:07.385 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:57.644936+0000", 2024-04-24T15:34:07.385 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0017091490000000001 2024-04-24T15:34:07.386 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.386 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.386 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "sub_op_commit_rec", 2024-04-24T15:34:07.386 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:57.646233+0000", 2024-04-24T15:34:07.386 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 0.0012974880000000001 2024-04-24T15:34:07.386 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.386 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.386 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "commit_sent", 2024-04-24T15:34:07.386 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:57.646263+0000", 2024-04-24T15:34:07.386 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.9609000000000001e-05 2024-04-24T15:34:07.386 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-24T15:34:07.386 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-24T15:34:07.386 INFO:teuthology.orchestra.run.smithi151.stdout: "event": "done", 2024-04-24T15:34:07.387 INFO:teuthology.orchestra.run.smithi151.stdout: "time": "2024-04-24T15:33:57.646285+0000", 2024-04-24T15:34:07.387 INFO:teuthology.orchestra.run.smithi151.stdout: "duration": 2.2509000000000001e-05 2024-04-24T15:34:07.387 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:07.387 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:07.387 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:07.387 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-24T15:34:07.387 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-24T15:34:07.387 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:34:07.388 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-24T15:34:07.389 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:34:07.389+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:34:07.489 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-24T15:34:07.489+0000 7fc8a43a6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44015) UID: 0 2024-04-24T15:34:07.553 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:34:07.553 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:34:07.553 INFO:teuthology.orchestra.run.smithi151.stdout: "num_ops": 0 2024-04-24T15:34:07.553 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:34:07.559 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-24T15:34:07.563 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-24T15:34:07.604 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:34:07.605+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:34:07.691 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:34:07.693+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:34:07.722 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-04-24T15:34:07.733 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-24T15:34:07.733 INFO:teuthology.orchestra.run.smithi151.stdout: "ops": [], 2024-04-24T15:34:07.733 INFO:teuthology.orchestra.run.smithi151.stdout: "complaint_time": 30, 2024-04-24T15:34:07.733 INFO:teuthology.orchestra.run.smithi151.stdout: "num_blocked_ops": 0 2024-04-24T15:34:07.733 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-24T15:34:07.743 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-04-24T15:34:07.744 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-24T15:34:07.759 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-24T15:34:07.763 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-04-24T15:34:07.784 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-04-24T15:34:07.793 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:34:07.793+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:34:07.803 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-04-24T15:34:07.821 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-04-24T15:34:07.839 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-04-24T15:34:07.857 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-04-24T15:34:07.859 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T15:34:07.859 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T15:34:07.859 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T15:34:07.859 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T15:34:07.859 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T15:34:07.859 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T15:34:07.859 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T15:34:07.859 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T15:34:07.870 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:34:07.870 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:34:07.880 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-24T15:34:07.912 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:34:07.913+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:34:07.995 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:34:07.996+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:34:08.037 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:34:08.037 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:34:08.047 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-24T15:34:08.096 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:34:08.097+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:34:08.197 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-04-24T15:34:08.198+0000 7f29647ed700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 44067) UID: 0 2024-04-24T15:34:08.208 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:34:08.208 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:34:08.217 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-24T15:34:08.297 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-24T15:34:08.298+0000 7f7e2efb4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 44019) UID: 0 2024-04-24T15:34:08.381 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:34:08.381 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:34:08.391 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-24T15:34:08.401 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-24T15:34:08.399+0000 7f0f45efb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 59992) UID: 0 2024-04-24T15:34:08.499 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:34:08.500+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:34:08.513 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:34:08.513 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:34:08.523 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-24T15:34:08.602 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:34:08.603+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:34:08.686 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:34:08.686 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:34:08.696 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-24T15:34:08.703 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-24T15:34:08.704+0000 7f27cf94e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 44013) UID: 0 2024-04-24T15:34:08.762 INFO:teuthology.orchestra.run.smithi096.stderr:nodeep-scrub is set 2024-04-24T15:34:08.806 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-24T15:34:08.808+0000 7f4f92ebb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44068) UID: 0 2024-04-24T15:34:08.831 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:34:08.831 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:34:08.838 INFO:teuthology.orchestra.run.smithi096.stdout:All operations completed A-OK! 2024-04-24T15:34:08.843 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-24T15:34:08.847 DEBUG:teuthology.parallel:result is None 2024-04-24T15:34:08.847 DEBUG:teuthology.run_tasks:Unwinding manager rbd_fsx 2024-04-24T15:34:08.857 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-04-24T15:34:08.867 INFO:tasks.thrashosds:joining thrashosds 2024-04-24T15:34:09.009 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:34:09.009 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:34:09.020 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-24T15:34:09.143 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:34:09.143 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:34:09.153 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-24T15:34:09.275 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:34:09.276 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:34:09.285 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-24T15:34:09.408 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:34:09.408 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:34:09.418 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-24T15:34:09.540 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:34:09.540 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:34:09.550 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-24T15:34:09.672 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-24T15:34:09.673 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:34:09.682 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-24T15:34:09.821 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:34:09.821 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:34:09.831 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-24T15:34:09.954 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:34:09.954 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:34:09.963 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-24T15:34:10.086 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:34:10.087 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:34:10.097 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-24T15:34:10.219 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:34:10.219 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:34:10.229 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-24T15:34:10.352 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:34:10.352 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:34:10.362 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-24T15:34:10.485 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:34:10.485 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:34:10.495 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-24T15:34:10.617 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:34:10.618 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:34:10.627 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-24T15:34:10.749 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:34:10.749 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:34:10.759 INFO:tasks.thrashosds.thrasher:inning osd 2024-04-24T15:34:10.759 INFO:tasks.thrashosds.thrasher:Adding osd 1 2024-04-24T15:34:10.759 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 1 2024-04-24T15:34:10.761 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-24T15:34:10.772 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-24T15:34:10.884 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:34:10.884 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:34:10.894 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-24T15:34:11.018 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:34:11.018 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:34:11.027 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-24T15:34:11.126 INFO:tasks.ceph.mon.a.smithi096.stderr:2024-04-24T15:34:11.126+0000 7fa88ddc8700 -1 mon.a@0(leader).osd e131 definitely_dead 0 2024-04-24T15:34:11.150 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:34:11.150 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:34:11.159 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-24T15:34:11.282 INFO:teuthology.orchestra.run.smithi151.stdout:ERROR: (22) Invalid argument 2024-04-24T15:34:11.283 INFO:teuthology.orchestra.run.smithi151.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T15:34:11.778 INFO:teuthology.orchestra.run.smithi096.stderr:noscrub is unset 2024-04-24T15:34:11.780 INFO:teuthology.orchestra.run.smithi096.stderr:marked in osd.1. 2024-04-24T15:34:11.792 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-24T15:34:11.793 INFO:tasks.thrashosds.thrasher:Added osd 1 2024-04-24T15:34:11.793 INFO:tasks.thrashosds.thrasher:inning osd 2024-04-24T15:34:11.793 INFO:tasks.thrashosds.thrasher:Adding osd 5 2024-04-24T15:34:11.793 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 5 2024-04-24T15:34:12.173 INFO:tasks.ceph.mon.a.smithi096.stderr:2024-04-24T15:34:12.173+0000 7fa88ddc8700 -1 mon.a@0(leader).osd e132 definitely_dead 0 2024-04-24T15:34:12.648 INFO:teuthology.orchestra.run.smithi096.stderr:nodeep-scrub is unset 2024-04-24T15:34:12.655 INFO:teuthology.orchestra.run.smithi096.stderr:marked in osd.5. 2024-04-24T15:34:12.669 INFO:tasks.thrashosds.thrasher:Added osd 5 2024-04-24T15:34:12.669 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-04-24T15:34:12.843 INFO:teuthology.orchestra.run.smithi096.stdout:mon.a: {} 2024-04-24T15:34:12.854 INFO:teuthology.orchestra.run.smithi096.stdout:mon.b: {} 2024-04-24T15:34:12.863 INFO:teuthology.orchestra.run.smithi096.stdout:mon.c: {} 2024-04-24T15:34:12.864 INFO:teuthology.orchestra.run.smithi096.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-04-24T15:34:12.864 INFO:teuthology.orchestra.run.smithi096.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-04-24T15:34:12.864 INFO:teuthology.orchestra.run.smithi096.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-04-24T15:34:12.875 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-04-24T15:34:13.660 INFO:teuthology.orchestra.run.smithi096.stderr:reweighted osd.0 to 1 (10000) 2024-04-24T15:34:13.673 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-04-24T15:34:14.658 INFO:teuthology.orchestra.run.smithi096.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-04-24T15:34:14.671 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-04-24T15:34:15.657 INFO:teuthology.orchestra.run.smithi096.stderr:reweighted osd.1 to 1 (10000) 2024-04-24T15:34:15.672 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-04-24T15:34:16.660 INFO:teuthology.orchestra.run.smithi096.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-04-24T15:34:16.673 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-04-24T15:34:17.653 INFO:teuthology.orchestra.run.smithi096.stderr:reweighted osd.2 to 1 (10000) 2024-04-24T15:34:17.667 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-04-24T15:34:18.803 INFO:teuthology.orchestra.run.smithi096.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-04-24T15:34:18.816 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-04-24T15:34:19.807 INFO:teuthology.orchestra.run.smithi096.stderr:reweighted osd.3 to 1 (10000) 2024-04-24T15:34:19.820 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-04-24T15:34:20.807 INFO:teuthology.orchestra.run.smithi096.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-04-24T15:34:20.821 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-04-24T15:34:21.814 INFO:teuthology.orchestra.run.smithi096.stderr:reweighted osd.4 to 1 (10000) 2024-04-24T15:34:21.828 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-04-24T15:34:22.825 INFO:teuthology.orchestra.run.smithi096.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-04-24T15:34:22.838 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-04-24T15:34:23.825 INFO:teuthology.orchestra.run.smithi096.stderr:reweighted osd.5 to 1 (10000) 2024-04-24T15:34:23.838 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-04-24T15:34:24.841 INFO:teuthology.orchestra.run.smithi096.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-04-24T15:34:24.854 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-04-24T15:34:25.849 INFO:teuthology.orchestra.run.smithi096.stderr:reweighted osd.7 to 1 (10000) 2024-04-24T15:34:25.861 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2024-04-24T15:34:26.856 INFO:teuthology.orchestra.run.smithi096.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-04-24T15:34:26.868 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-04-24T15:34:27.865 INFO:teuthology.orchestra.run.smithi096.stderr:reweighted osd.6 to 1 (10000) 2024-04-24T15:34:27.877 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-04-24T15:34:28.866 INFO:teuthology.orchestra.run.smithi096.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-04-24T15:34:28.879 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-04-24T15:34:28.880 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-04-24T15:34:28.880 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-04-24T15:34:28.880 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-04-24T15:34:28.880 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-24T15:34:28.880 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T15:34:29.198 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:34:29.199 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":149,"fsid":"253d426c-62cb-4194-8514-6cd970fd85c7","created":"2024-04-24T15:31:47.870327+0000","modified":"2024-04-24T15:34:28.859605+0000","last_up_change":"2024-04-24T15:33:41.967503+0000","last_in_change":"2024-04-24T15:34:12.639389+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T15:31:59.589762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-24T15:31:59.971601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"21","last_force_op_resend_prenautilus":"21","last_force_op_resend_preluminous":"21","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[3],"tier_of":-1,"read_tier":3,"write_tier":3,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cache","create_time":"2024-04-24T15:32:06.914463+0000","flags":9,"flags_names":"hashpspool,incomplete_clones","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":4,"pg_placement_num":4,"pg_placement_num_target":4,"pg_num_target":4,"pg_num_pending":4,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"21","last_force_op_resend_prenautilus":"21","last_force_op_resend_preluminous":"21","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":2,"read_tier":-1,"write_tier":-1,"cache_mode":"writeback","target_max_bytes":0,"target_max_objects":250,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"bloom","false_positive_probability":0.050000000000000003,"target_size":0,"seed":0},"hit_set_period":60,"hit_set_count":8,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"pool_client.0","create_time":"2024-04-24T15:32:15.929114+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":69,"snap_epoch":131,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"92941fb9-96ee-4b2b-8147-3c1d61192f94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":44106},{"type":"v1","addr":"172.21.15.96:6817","nonce":44106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":44106},{"type":"v1","addr":"172.21.15.96:6819","nonce":44106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":44106},{"type":"v1","addr":"172.21.15.96:6823","nonce":44106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":44106},{"type":"v1","addr":"172.21.15.96:6821","nonce":44106}]},"public_addr":"172.21.15.96:6817/44106","cluster_addr":"172.21.15.96:6819/44106","heartbeat_back_addr":"172.21.15.96:6823/44106","heartbeat_front_addr":"172.21.15.96:6821/44106","state":["exists","up"]},{"osd":1,"uuid":"628e868a-9884-42e6-9331-caef97cb6967","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":44089},{"type":"v1","addr":"172.21.15.96:6801","nonce":44089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":44089},{"type":"v1","addr":"172.21.15.96:6803","nonce":44089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":44089},{"type":"v1","addr":"172.21.15.96:6807","nonce":44089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":44089},{"type":"v1","addr":"172.21.15.96:6805","nonce":44089}]},"public_addr":"172.21.15.96:6801/44089","cluster_addr":"172.21.15.96:6803/44089","heartbeat_back_addr":"172.21.15.96:6807/44089","heartbeat_front_addr":"172.21.15.96:6805/44089","state":["exists","up"]},{"osd":2,"uuid":"46de3f65-87a8-460c-b57a-4468d9c89a57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":44090},{"type":"v1","addr":"172.21.15.96:6809","nonce":44090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":44090},{"type":"v1","addr":"172.21.15.96:6811","nonce":44090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":44090},{"type":"v1","addr":"172.21.15.96:6815","nonce":44090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":44090},{"type":"v1","addr":"172.21.15.96:6813","nonce":44090}]},"public_addr":"172.21.15.96:6809/44090","cluster_addr":"172.21.15.96:6811/44090","heartbeat_back_addr":"172.21.15.96:6815/44090","heartbeat_front_addr":"172.21.15.96:6813/44090","state":["exists","up"]},{"osd":3,"uuid":"5507e063-10bc-4e32-b1ec-00aa3e47d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":44121},{"type":"v1","addr":"172.21.15.96:6825","nonce":44121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":44121},{"type":"v1","addr":"172.21.15.96:6827","nonce":44121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":44121},{"type":"v1","addr":"172.21.15.96:6831","nonce":44121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":44121},{"type":"v1","addr":"172.21.15.96:6829","nonce":44121}]},"public_addr":"172.21.15.96:6825/44121","cluster_addr":"172.21.15.96:6827/44121","heartbeat_back_addr":"172.21.15.96:6831/44121","heartbeat_front_addr":"172.21.15.96:6829/44121","state":["exists","up"]},{"osd":4,"uuid":"6a74d453-1048-44ef-a5b0-4277583d76c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":44025},{"type":"v1","addr":"172.21.15.151:6817","nonce":44025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":44025},{"type":"v1","addr":"172.21.15.151:6819","nonce":44025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":44025},{"type":"v1","addr":"172.21.15.151:6823","nonce":44025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":44025},{"type":"v1","addr":"172.21.15.151:6821","nonce":44025}]},"public_addr":"172.21.15.151:6817/44025","cluster_addr":"172.21.15.151:6819/44025","heartbeat_back_addr":"172.21.15.151:6823/44025","heartbeat_front_addr":"172.21.15.151:6821/44025","state":["exists","up"]},{"osd":5,"uuid":"3645ef18-5574-49f8-ab81-b40651208309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":44023},{"type":"v1","addr":"172.21.15.151:6801","nonce":44023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":44023},{"type":"v1","addr":"172.21.15.151:6803","nonce":44023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":44023},{"type":"v1","addr":"172.21.15.151:6807","nonce":44023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":44023},{"type":"v1","addr":"172.21.15.151:6805","nonce":44023}]},"public_addr":"172.21.15.151:6801/44023","cluster_addr":"172.21.15.151:6803/44023","heartbeat_back_addr":"172.21.15.151:6807/44023","heartbeat_front_addr":"172.21.15.151:6805/44023","state":["exists","up"]},{"osd":6,"uuid":"6b8b0b36-9246-4241-a197-c69fa48a6202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":75,"up_from":104,"up_thru":135,"down_at":76,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":59996},{"type":"v1","addr":"172.21.15.151:6825","nonce":59996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":59996},{"type":"v1","addr":"172.21.15.151:6827","nonce":59996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":59996},{"type":"v1","addr":"172.21.15.151:6831","nonce":59996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":59996},{"type":"v1","addr":"172.21.15.151:6829","nonce":59996}]},"public_addr":"172.21.15.151:6825/59996","cluster_addr":"172.21.15.151:6827/59996","heartbeat_back_addr":"172.21.15.151:6831/59996","heartbeat_front_addr":"172.21.15.151:6829/59996","state":["exists","up"]},{"osd":7,"uuid":"4501cd9f-e437-4ddd-af3d-129fdf1ee61a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":44024},{"type":"v1","addr":"172.21.15.151:6809","nonce":44024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":44024},{"type":"v1","addr":"172.21.15.151:6811","nonce":44024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":44024},{"type":"v1","addr":"172.21.15.151:6815","nonce":44024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":44024},{"type":"v1","addr":"172.21.15.151:6813","nonce":44024}]},"public_addr":"172.21.15.151:6809/44024","cluster_addr":"172.21.15.151:6811/44024","heartbeat_back_addr":"172.21.15.151:6815/44024","heartbeat_front_addr":"172.21.15.151:6813/44024","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T15:31:56.300910+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T15:31:56.202064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T15:31:56.198961+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T15:31:56.346777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T15:31:56.282585+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T15:31:56.257943+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T15:31:56.260479+0000","dead_epoch":76},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T15:31:56.242093+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.0","mappings":[{"from":3,"to":2}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T15:34:29.212 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-24T15:34:29.212 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T15:34:29.530 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:34:29.530 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":149,"fsid":"253d426c-62cb-4194-8514-6cd970fd85c7","created":"2024-04-24T15:31:47.870327+0000","modified":"2024-04-24T15:34:28.859605+0000","last_up_change":"2024-04-24T15:33:41.967503+0000","last_in_change":"2024-04-24T15:34:12.639389+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T15:31:59.589762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-24T15:31:59.971601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"21","last_force_op_resend_prenautilus":"21","last_force_op_resend_preluminous":"21","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[3],"tier_of":-1,"read_tier":3,"write_tier":3,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cache","create_time":"2024-04-24T15:32:06.914463+0000","flags":9,"flags_names":"hashpspool,incomplete_clones","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":4,"pg_placement_num":4,"pg_placement_num_target":4,"pg_num_target":4,"pg_num_pending":4,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"21","last_force_op_resend_prenautilus":"21","last_force_op_resend_preluminous":"21","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":2,"read_tier":-1,"write_tier":-1,"cache_mode":"writeback","target_max_bytes":0,"target_max_objects":250,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"bloom","false_positive_probability":0.050000000000000003,"target_size":0,"seed":0},"hit_set_period":60,"hit_set_count":8,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"pool_client.0","create_time":"2024-04-24T15:32:15.929114+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":69,"snap_epoch":131,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"92941fb9-96ee-4b2b-8147-3c1d61192f94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":44106},{"type":"v1","addr":"172.21.15.96:6817","nonce":44106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":44106},{"type":"v1","addr":"172.21.15.96:6819","nonce":44106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":44106},{"type":"v1","addr":"172.21.15.96:6823","nonce":44106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":44106},{"type":"v1","addr":"172.21.15.96:6821","nonce":44106}]},"public_addr":"172.21.15.96:6817/44106","cluster_addr":"172.21.15.96:6819/44106","heartbeat_back_addr":"172.21.15.96:6823/44106","heartbeat_front_addr":"172.21.15.96:6821/44106","state":["exists","up"]},{"osd":1,"uuid":"628e868a-9884-42e6-9331-caef97cb6967","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":44089},{"type":"v1","addr":"172.21.15.96:6801","nonce":44089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":44089},{"type":"v1","addr":"172.21.15.96:6803","nonce":44089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":44089},{"type":"v1","addr":"172.21.15.96:6807","nonce":44089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":44089},{"type":"v1","addr":"172.21.15.96:6805","nonce":44089}]},"public_addr":"172.21.15.96:6801/44089","cluster_addr":"172.21.15.96:6803/44089","heartbeat_back_addr":"172.21.15.96:6807/44089","heartbeat_front_addr":"172.21.15.96:6805/44089","state":["exists","up"]},{"osd":2,"uuid":"46de3f65-87a8-460c-b57a-4468d9c89a57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":44090},{"type":"v1","addr":"172.21.15.96:6809","nonce":44090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":44090},{"type":"v1","addr":"172.21.15.96:6811","nonce":44090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":44090},{"type":"v1","addr":"172.21.15.96:6815","nonce":44090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":44090},{"type":"v1","addr":"172.21.15.96:6813","nonce":44090}]},"public_addr":"172.21.15.96:6809/44090","cluster_addr":"172.21.15.96:6811/44090","heartbeat_back_addr":"172.21.15.96:6815/44090","heartbeat_front_addr":"172.21.15.96:6813/44090","state":["exists","up"]},{"osd":3,"uuid":"5507e063-10bc-4e32-b1ec-00aa3e47d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":44121},{"type":"v1","addr":"172.21.15.96:6825","nonce":44121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":44121},{"type":"v1","addr":"172.21.15.96:6827","nonce":44121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":44121},{"type":"v1","addr":"172.21.15.96:6831","nonce":44121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":44121},{"type":"v1","addr":"172.21.15.96:6829","nonce":44121}]},"public_addr":"172.21.15.96:6825/44121","cluster_addr":"172.21.15.96:6827/44121","heartbeat_back_addr":"172.21.15.96:6831/44121","heartbeat_front_addr":"172.21.15.96:6829/44121","state":["exists","up"]},{"osd":4,"uuid":"6a74d453-1048-44ef-a5b0-4277583d76c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":44025},{"type":"v1","addr":"172.21.15.151:6817","nonce":44025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":44025},{"type":"v1","addr":"172.21.15.151:6819","nonce":44025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":44025},{"type":"v1","addr":"172.21.15.151:6823","nonce":44025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":44025},{"type":"v1","addr":"172.21.15.151:6821","nonce":44025}]},"public_addr":"172.21.15.151:6817/44025","cluster_addr":"172.21.15.151:6819/44025","heartbeat_back_addr":"172.21.15.151:6823/44025","heartbeat_front_addr":"172.21.15.151:6821/44025","state":["exists","up"]},{"osd":5,"uuid":"3645ef18-5574-49f8-ab81-b40651208309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":44023},{"type":"v1","addr":"172.21.15.151:6801","nonce":44023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":44023},{"type":"v1","addr":"172.21.15.151:6803","nonce":44023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":44023},{"type":"v1","addr":"172.21.15.151:6807","nonce":44023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":44023},{"type":"v1","addr":"172.21.15.151:6805","nonce":44023}]},"public_addr":"172.21.15.151:6801/44023","cluster_addr":"172.21.15.151:6803/44023","heartbeat_back_addr":"172.21.15.151:6807/44023","heartbeat_front_addr":"172.21.15.151:6805/44023","state":["exists","up"]},{"osd":6,"uuid":"6b8b0b36-9246-4241-a197-c69fa48a6202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":75,"up_from":104,"up_thru":135,"down_at":76,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":59996},{"type":"v1","addr":"172.21.15.151:6825","nonce":59996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":59996},{"type":"v1","addr":"172.21.15.151:6827","nonce":59996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":59996},{"type":"v1","addr":"172.21.15.151:6831","nonce":59996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":59996},{"type":"v1","addr":"172.21.15.151:6829","nonce":59996}]},"public_addr":"172.21.15.151:6825/59996","cluster_addr":"172.21.15.151:6827/59996","heartbeat_back_addr":"172.21.15.151:6831/59996","heartbeat_front_addr":"172.21.15.151:6829/59996","state":["exists","up"]},{"osd":7,"uuid":"4501cd9f-e437-4ddd-af3d-129fdf1ee61a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":44024},{"type":"v1","addr":"172.21.15.151:6809","nonce":44024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":44024},{"type":"v1","addr":"172.21.15.151:6811","nonce":44024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":44024},{"type":"v1","addr":"172.21.15.151:6815","nonce":44024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":44024},{"type":"v1","addr":"172.21.15.151:6813","nonce":44024}]},"public_addr":"172.21.15.151:6809/44024","cluster_addr":"172.21.15.151:6811/44024","heartbeat_back_addr":"172.21.15.151:6815/44024","heartbeat_front_addr":"172.21.15.151:6813/44024","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T15:31:56.300910+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T15:31:56.202064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T15:31:56.198961+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T15:31:56.346777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T15:31:56.282585+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T15:31:56.257943+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T15:31:56.260479+0000","dead_epoch":76},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T15:31:56.242093+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.0","mappings":[{"from":3,"to":2}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T15:34:29.544 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-24T15:34:29.544 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-24T15:34:29.545 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-24T15:34:29.545 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-24T15:34:29.545 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-24T15:34:29.546 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-24T15:34:29.546 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-24T15:34:29.547 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-24T15:34:29.819 INFO:teuthology.orchestra.run.smithi096.stdout:55834574882 2024-04-24T15:34:29.819 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-24T15:34:29.821 INFO:teuthology.orchestra.run.smithi096.stdout:55834574882 2024-04-24T15:34:29.821 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-24T15:34:29.825 INFO:teuthology.orchestra.run.smithi096.stdout:55834574882 2024-04-24T15:34:29.825 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-24T15:34:29.837 INFO:teuthology.orchestra.run.smithi096.stdout:55834574882 2024-04-24T15:34:29.838 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-24T15:34:29.841 INFO:teuthology.orchestra.run.smithi096.stdout:446676598795 2024-04-24T15:34:29.841 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-24T15:34:29.843 INFO:teuthology.orchestra.run.smithi096.stdout:55834574883 2024-04-24T15:34:29.843 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-24T15:34:29.846 INFO:teuthology.orchestra.run.smithi096.stdout:55834574881 2024-04-24T15:34:29.846 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-24T15:34:29.848 INFO:teuthology.orchestra.run.smithi096.stdout:55834574882 2024-04-24T15:34:29.848 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-24T15:34:30.431 INFO:teuthology.orchestra.run.smithi096.stdout:55834574881 2024-04-24T15:34:30.449 INFO:teuthology.orchestra.run.smithi096.stdout:55834574881 2024-04-24T15:34:30.451 INFO:teuthology.orchestra.run.smithi096.stdout:55834574881 2024-04-24T15:34:30.452 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574882 got 55834574881 for osd.0 2024-04-24T15:34:30.468 INFO:teuthology.orchestra.run.smithi096.stdout:55834574882 2024-04-24T15:34:30.473 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574882 got 55834574881 for osd.7 2024-04-24T15:34:30.473 INFO:teuthology.orchestra.run.smithi096.stdout:55834574881 2024-04-24T15:34:30.474 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574882 got 55834574881 for osd.5 2024-04-24T15:34:30.474 INFO:teuthology.orchestra.run.smithi096.stdout:446676598794 2024-04-24T15:34:30.487 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574883 got 55834574882 for osd.4 2024-04-24T15:34:30.496 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574882 got 55834574881 for osd.2 2024-04-24T15:34:30.497 INFO:tasks.ceph.ceph_manager.ceph:need seq 446676598795 got 446676598794 for osd.6 2024-04-24T15:34:30.504 INFO:teuthology.orchestra.run.smithi096.stdout:55834574881 2024-04-24T15:34:30.517 INFO:teuthology.orchestra.run.smithi096.stdout:55834574880 2024-04-24T15:34:30.518 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574882 got 55834574881 for osd.3 2024-04-24T15:34:30.531 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574881 got 55834574880 for osd.1 2024-04-24T15:34:31.453 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-24T15:34:31.474 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-24T15:34:31.475 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-24T15:34:31.487 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-24T15:34:31.497 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-24T15:34:31.497 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-24T15:34:31.519 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-24T15:34:31.531 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-24T15:34:32.086 INFO:teuthology.orchestra.run.smithi096.stdout:55834574882 2024-04-24T15:34:32.093 INFO:teuthology.orchestra.run.smithi096.stdout:55834574882 2024-04-24T15:34:32.103 INFO:teuthology.orchestra.run.smithi096.stdout:55834574883 2024-04-24T15:34:32.107 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574882 got 55834574882 for osd.0 2024-04-24T15:34:32.107 DEBUG:teuthology.parallel:result is None 2024-04-24T15:34:32.109 INFO:teuthology.orchestra.run.smithi096.stdout:446676598796 2024-04-24T15:34:32.112 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574882 got 55834574882 for osd.2 2024-04-24T15:34:32.113 DEBUG:teuthology.parallel:result is None 2024-04-24T15:34:32.126 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574883 got 55834574883 for osd.4 2024-04-24T15:34:32.126 DEBUG:teuthology.parallel:result is None 2024-04-24T15:34:32.129 INFO:tasks.ceph.ceph_manager.ceph:need seq 446676598795 got 446676598796 for osd.6 2024-04-24T15:34:32.129 DEBUG:teuthology.parallel:result is None 2024-04-24T15:34:32.148 INFO:teuthology.orchestra.run.smithi096.stdout:55834574882 2024-04-24T15:34:32.152 INFO:teuthology.orchestra.run.smithi096.stdout:55834574882 2024-04-24T15:34:32.163 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574882 got 55834574882 for osd.5 2024-04-24T15:34:32.163 DEBUG:teuthology.parallel:result is None 2024-04-24T15:34:32.169 INFO:teuthology.orchestra.run.smithi096.stdout:55834574881 2024-04-24T15:34:32.171 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574882 got 55834574882 for osd.7 2024-04-24T15:34:32.171 DEBUG:teuthology.parallel:result is None 2024-04-24T15:34:32.177 INFO:teuthology.orchestra.run.smithi096.stdout:55834574882 2024-04-24T15:34:32.184 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574881 got 55834574881 for osd.1 2024-04-24T15:34:32.184 DEBUG:teuthology.parallel:result is None 2024-04-24T15:34:32.191 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574882 got 55834574882 for osd.3 2024-04-24T15:34:32.192 DEBUG:teuthology.parallel:result is None 2024-04-24T15:34:32.192 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-04-24T15:34:32.192 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T15:34:32.546 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:34:32.547 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-04-24T15:34:32.558 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":219,"stamp":"2024-04-24T15:34:31.579540+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":51,"num_read":114923,"num_read_kb":727642,"num_write":20449,"num_write_kb":358744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4420,"num_bytes_recovered":106356266,"num_keys_recovered":235,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17371,"ondisk_log_size":17371,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2440640,"kb_used_data":14024,"kb_used_omap":55,"kb_used_meta":2426440,"kb_avail":752534080,"statfs":{"total":773094113280,"available":770594897920,"internally_reserved":0,"allocated":14360576,"data_stored":8434480,"data_compressed":10519,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":56663,"internal_metadata":2484675241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.736205"},"pg_stats":[{"pgid":"4.1f","version":"131'1485","reported_seq":2550,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698777+0000","last_change":"2024-04-24T15:34:15.652525+0000","last_active":"2024-04-24T15:34:28.698777+0000","last_peered":"2024-04-24T15:34:28.698777+0000","last_clean":"2024-04-24T15:34:28.698777+0000","last_became_active":"2024-04-24T15:34:15.650850+0000","last_became_peered":"2024-04-24T15:34:15.650850+0000","last_unstale":"2024-04-24T15:34:28.698777+0000","last_undegraded":"2024-04-24T15:34:28.698777+0000","last_fullsized":"2024-04-24T15:34:28.698777+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":1485,"log_dups_size":0,"ondisk_log_size":1485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:06:34.577605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001749947,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":8143,"num_read_kb":14722,"num_write":2575,"num_write_kb":16077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":5952278,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"129'507","reported_seq":6418,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698830+0000","last_change":"2024-04-24T15:34:15.669801+0000","last_active":"2024-04-24T15:34:28.698830+0000","last_peered":"2024-04-24T15:34:28.698830+0000","last_clean":"2024-04-24T15:34:28.698830+0000","last_became_active":"2024-04-24T15:34:15.668657+0000","last_became_peered":"2024-04-24T15:34:15.668657+0000","last_unstale":"2024-04-24T15:34:28.698830+0000","last_undegraded":"2024-04-24T15:34:28.698830+0000","last_fullsized":"2024-04-24T15:34:28.698830+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":507,"log_dups_size":0,"ondisk_log_size":507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:32:57.253624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029386599999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":7278,"num_read_kb":39113,"num_write":560,"num_write_kb":6592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":4495932,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"129'529","reported_seq":3745,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861949+0000","last_change":"2024-04-24T15:34:16.660701+0000","last_active":"2024-04-24T15:34:27.861949+0000","last_peered":"2024-04-24T15:34:27.861949+0000","last_clean":"2024-04-24T15:34:27.861949+0000","last_became_active":"2024-04-24T15:34:16.659202+0000","last_became_peered":"2024-04-24T15:34:16.659202+0000","last_unstale":"2024-04-24T15:34:27.861949+0000","last_undegraded":"2024-04-24T15:34:27.861949+0000","last_fullsized":"2024-04-24T15:34:27.861949+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":529,"log_dups_size":0,"ondisk_log_size":529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:31:57.762122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015488839999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":4427,"num_read_kb":51896,"num_write":603,"num_write_kb":18759,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":580671,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"131'526","reported_seq":2700,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.699161+0000","last_change":"2024-04-24T15:34:19.807684+0000","last_active":"2024-04-24T15:34:28.699161+0000","last_peered":"2024-04-24T15:34:28.699161+0000","last_clean":"2024-04-24T15:34:28.699161+0000","last_became_active":"2024-04-24T15:34:19.806783+0000","last_became_peered":"2024-04-24T15:34:19.806783+0000","last_unstale":"2024-04-24T15:34:28.699161+0000","last_undegraded":"2024-04-24T15:34:28.699161+0000","last_fullsized":"2024-04-24T15:34:28.699161+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":526,"log_dups_size":0,"ondisk_log_size":526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:06:01.349645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001413493,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":3716,"num_read_kb":10078,"num_write":547,"num_write_kb":2888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"129'385","reported_seq":2359,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861581+0000","last_change":"2024-04-24T15:34:23.827263+0000","last_active":"2024-04-24T15:34:27.861581+0000","last_peered":"2024-04-24T15:34:27.861581+0000","last_clean":"2024-04-24T15:34:27.861581+0000","last_became_active":"2024-04-24T15:34:23.825472+0000","last_became_peered":"2024-04-24T15:34:23.825472+0000","last_unstale":"2024-04-24T15:34:27.861581+0000","last_undegraded":"2024-04-24T15:34:27.861581+0000","last_fullsized":"2024-04-24T15:34:27.861581+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":385,"log_dups_size":0,"ondisk_log_size":385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:19:37.470876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011107479999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":2335,"num_read_kb":25405,"num_write":329,"num_write_kb":6118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":79,"num_bytes_recovered":2143880,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"129'449","reported_seq":6610,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861569+0000","last_change":"2024-04-24T15:34:23.827554+0000","last_active":"2024-04-24T15:34:27.861569+0000","last_peered":"2024-04-24T15:34:27.861569+0000","last_clean":"2024-04-24T15:34:27.861569+0000","last_became_active":"2024-04-24T15:34:23.824861+0000","last_became_peered":"2024-04-24T15:34:23.824861+0000","last_unstale":"2024-04-24T15:34:27.861569+0000","last_undegraded":"2024-04-24T15:34:27.861569+0000","last_fullsized":"2024-04-24T15:34:27.861569+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:42:20.554334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097376600000000004,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":6368,"num_read_kb":19805,"num_write":475,"num_write_kb":10193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":89,"num_bytes_recovered":1137673,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"129'537","reported_seq":2680,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861805+0000","last_change":"2024-04-24T15:34:08.756894+0000","last_active":"2024-04-24T15:34:27.861805+0000","last_peered":"2024-04-24T15:34:27.861805+0000","last_clean":"2024-04-24T15:34:27.861805+0000","last_became_active":"2024-04-24T15:32:17.047842+0000","last_became_peered":"2024-04-24T15:32:17.047842+0000","last_unstale":"2024-04-24T15:34:27.861805+0000","last_undegraded":"2024-04-24T15:34:27.861805+0000","last_fullsized":"2024-04-24T15:34:27.861805+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":537,"log_dups_size":0,"ondisk_log_size":537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:09:52.065772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081826100000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":4119,"num_read_kb":22694,"num_write":527,"num_write_kb":11193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"130'444","reported_seq":1968,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861537+0000","last_change":"2024-04-24T15:34:08.758067+0000","last_active":"2024-04-24T15:34:27.861537+0000","last_peered":"2024-04-24T15:34:27.861537+0000","last_clean":"2024-04-24T15:34:27.861537+0000","last_became_active":"2024-04-24T15:33:42.641679+0000","last_became_peered":"2024-04-24T15:33:42.641679+0000","last_unstale":"2024-04-24T15:34:27.861537+0000","last_undegraded":"2024-04-24T15:34:27.861537+0000","last_fullsized":"2024-04-24T15:34:27.861537+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":444,"log_dups_size":0,"ondisk_log_size":444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:29:58.781375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077716999999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":1887,"num_read_kb":15406,"num_write":547,"num_write_kb":8155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":4099152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"129'523","reported_seq":2447,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861936+0000","last_change":"2024-04-24T15:34:08.758924+0000","last_active":"2024-04-24T15:34:27.861936+0000","last_peered":"2024-04-24T15:34:27.861936+0000","last_clean":"2024-04-24T15:34:27.861936+0000","last_became_active":"2024-04-24T15:33:56.699629+0000","last_became_peered":"2024-04-24T15:33:56.699629+0000","last_unstale":"2024-04-24T15:34:27.861936+0000","last_undegraded":"2024-04-24T15:34:27.861936+0000","last_fullsized":"2024-04-24T15:34:27.861936+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":523,"log_dups_size":0,"ondisk_log_size":523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:53:30.709042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00212766,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":3353,"num_read_kb":16260,"num_write":594,"num_write_kb":12753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":352256,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"129'316","reported_seq":2577,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861623+0000","last_change":"2024-04-24T15:34:23.829028+0000","last_active":"2024-04-24T15:34:27.861623+0000","last_peered":"2024-04-24T15:34:27.861623+0000","last_clean":"2024-04-24T15:34:27.861623+0000","last_became_active":"2024-04-24T15:34:23.827917+0000","last_became_peered":"2024-04-24T15:34:23.827917+0000","last_unstale":"2024-04-24T15:34:27.861623+0000","last_undegraded":"2024-04-24T15:34:27.861623+0000","last_fullsized":"2024-04-24T15:34:27.861623+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":316,"log_dups_size":0,"ondisk_log_size":316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:11:49.794260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065534800000000004,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":1633,"num_read_kb":26736,"num_write":303,"num_write_kb":13822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":9304722,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":121,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698692+0000","last_change":"2024-04-24T15:33:53.644249+0000","last_active":"2024-04-24T15:34:28.698692+0000","last_peered":"2024-04-24T15:34:28.698692+0000","last_clean":"2024-04-24T15:34:28.698692+0000","last_became_active":"2024-04-24T15:33:53.643941+0000","last_became_peered":"2024-04-24T15:33:53.643941+0000","last_unstale":"2024-04-24T15:34:28.698692+0000","last_undegraded":"2024-04-24T15:34:28.698692+0000","last_fullsized":"2024-04-24T15:34:28.698692+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:42:05.771051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"130'862","reported_seq":3009,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698843+0000","last_change":"2024-04-24T15:34:19.810827+0000","last_active":"2024-04-24T15:34:28.698843+0000","last_peered":"2024-04-24T15:34:28.698843+0000","last_clean":"2024-04-24T15:34:28.698843+0000","last_became_active":"2024-04-24T15:34:19.809619+0000","last_became_peered":"2024-04-24T15:34:19.809619+0000","last_unstale":"2024-04-24T15:34:28.698843+0000","last_undegraded":"2024-04-24T15:34:28.698843+0000","last_fullsized":"2024-04-24T15:34:28.698843+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:42:29.391778+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00033353700000000002,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":4689,"num_read_kb":23870,"num_write":1044,"num_write_kb":13092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":176,"num_bytes_recovered":5601476,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":24,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.699509+0000","last_change":"2024-04-24T15:34:12.652580+0000","last_active":"2024-04-24T15:34:28.699509+0000","last_peered":"2024-04-24T15:34:28.699509+0000","last_clean":"2024-04-24T15:34:28.699509+0000","last_became_active":"2024-04-24T15:34:12.652298+0000","last_became_peered":"2024-04-24T15:34:12.652298+0000","last_unstale":"2024-04-24T15:34:28.699509+0000","last_undegraded":"2024-04-24T15:34:28.699509+0000","last_fullsized":"2024-04-24T15:34:28.699509+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:22:59.160275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":41,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.862035+0000","last_change":"2024-04-24T15:33:53.645875+0000","last_active":"2024-04-24T15:34:27.862035+0000","last_peered":"2024-04-24T15:34:27.862035+0000","last_clean":"2024-04-24T15:34:27.862035+0000","last_became_active":"2024-04-24T15:33:53.645696+0000","last_became_peered":"2024-04-24T15:33:53.645696+0000","last_unstale":"2024-04-24T15:34:27.862035+0000","last_undegraded":"2024-04-24T15:34:27.862035+0000","last_fullsized":"2024-04-24T15:34:27.862035+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:29:35.345558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"131'584","reported_seq":5172,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698983+0000","last_change":"2024-04-24T15:34:17.655614+0000","last_active":"2024-04-24T15:34:28.698983+0000","last_peered":"2024-04-24T15:34:28.698983+0000","last_clean":"2024-04-24T15:34:28.698983+0000","last_became_active":"2024-04-24T15:34:17.654759+0000","last_became_peered":"2024-04-24T15:34:17.654759+0000","last_unstale":"2024-04-24T15:34:28.698983+0000","last_undegraded":"2024-04-24T15:34:28.698983+0000","last_fullsized":"2024-04-24T15:34:28.698983+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":584,"log_dups_size":0,"ondisk_log_size":584,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:42:40.438352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061779100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3592,"num_read_kb":14899,"num_write":598,"num_write_kb":11359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":23598,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"60'1","reported_seq":80,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698947+0000","last_change":"2024-04-24T15:33:47.663123+0000","last_active":"2024-04-24T15:34:28.698947+0000","last_peered":"2024-04-24T15:34:28.698947+0000","last_clean":"2024-04-24T15:34:28.698947+0000","last_became_active":"2024-04-24T15:33:47.662685+0000","last_became_peered":"2024-04-24T15:33:47.662685+0000","last_unstale":"2024-04-24T15:34:28.698947+0000","last_undegraded":"2024-04-24T15:34:28.698947+0000","last_fullsized":"2024-04-24T15:34:28.698947+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_clean_scrub_stamp":"2024-04-24T15:32:06.972928+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:05:35.654980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":50,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861972+0000","last_change":"2024-04-24T15:34:12.653126+0000","last_active":"2024-04-24T15:34:27.861972+0000","last_peered":"2024-04-24T15:34:27.861972+0000","last_clean":"2024-04-24T15:34:27.861972+0000","last_became_active":"2024-04-24T15:34:12.652805+0000","last_became_peered":"2024-04-24T15:34:12.652805+0000","last_unstale":"2024-04-24T15:34:27.861972+0000","last_undegraded":"2024-04-24T15:34:27.861972+0000","last_fullsized":"2024-04-24T15:34:27.861972+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:44:23.434720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"131'615","reported_seq":2369,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698926+0000","last_change":"2024-04-24T15:34:16.760282+0000","last_active":"2024-04-24T15:34:28.698926+0000","last_peered":"2024-04-24T15:34:28.698926+0000","last_clean":"2024-04-24T15:34:28.698926+0000","last_became_active":"2024-04-24T15:34:16.655390+0000","last_became_peered":"2024-04-24T15:34:16.655390+0000","last_unstale":"2024-04-24T15:34:28.698926+0000","last_undegraded":"2024-04-24T15:34:28.698926+0000","last_fullsized":"2024-04-24T15:34:28.698926+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":615,"log_dups_size":0,"ondisk_log_size":615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:01:32.241481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042917399999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":3015,"num_read_kb":19823,"num_write":696,"num_write_kb":5112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"90'2","reported_seq":182,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861981+0000","last_change":"2024-04-24T15:34:13.666629+0000","last_active":"2024-04-24T15:34:27.861981+0000","last_peered":"2024-04-24T15:34:27.861981+0000","last_clean":"2024-04-24T15:34:27.861981+0000","last_became_active":"2024-04-24T15:34:13.662363+0000","last_became_peered":"2024-04-24T15:34:13.662363+0000","last_unstale":"2024-04-24T15:34:27.861981+0000","last_undegraded":"2024-04-24T15:34:27.861981+0000","last_fullsized":"2024-04-24T15:34:27.861981+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_clean_scrub_stamp":"2024-04-24T15:32:06.972928+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:09:38.698619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":251,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.862026+0000","last_change":"2024-04-24T15:33:57.644653+0000","last_active":"2024-04-24T15:34:27.862026+0000","last_peered":"2024-04-24T15:34:27.862026+0000","last_clean":"2024-04-24T15:34:27.862026+0000","last_became_active":"2024-04-24T15:33:57.644312+0000","last_became_peered":"2024-04-24T15:33:57.644312+0000","last_unstale":"2024-04-24T15:34:27.862026+0000","last_undegraded":"2024-04-24T15:34:27.862026+0000","last_fullsized":"2024-04-24T15:34:27.862026+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:31:59.935287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:31:59.935287+0000","last_clean_scrub_stamp":"2024-04-24T15:31:59.935287+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:21:46.534664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698724+0000","last_change":"2024-04-24T15:34:13.650533+0000","last_active":"2024-04-24T15:34:28.698724+0000","last_peered":"2024-04-24T15:34:28.698724+0000","last_clean":"2024-04-24T15:34:28.698724+0000","last_became_active":"2024-04-24T15:34:13.650174+0000","last_became_peered":"2024-04-24T15:34:13.650174+0000","last_unstale":"2024-04-24T15:34:28.698724+0000","last_undegraded":"2024-04-24T15:34:28.698724+0000","last_fullsized":"2024-04-24T15:34:28.698724+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:01:43.442578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"129'591","reported_seq":2456,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698898+0000","last_change":"2024-04-24T15:34:08.805854+0000","last_active":"2024-04-24T15:34:28.698898+0000","last_peered":"2024-04-24T15:34:28.698898+0000","last_clean":"2024-04-24T15:34:28.698898+0000","last_became_active":"2024-04-24T15:33:45.658207+0000","last_became_peered":"2024-04-24T15:33:45.658207+0000","last_unstale":"2024-04-24T15:34:28.698898+0000","last_undegraded":"2024-04-24T15:34:28.698898+0000","last_fullsized":"2024-04-24T15:34:28.698898+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":591,"log_dups_size":0,"ondisk_log_size":591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:59:01.456518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010111199999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":3352,"num_read_kb":26205,"num_write":791,"num_write_kb":11816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":8821091,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":24,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.699087+0000","last_change":"2024-04-24T15:34:12.655959+0000","last_active":"2024-04-24T15:34:28.699087+0000","last_peered":"2024-04-24T15:34:28.699087+0000","last_clean":"2024-04-24T15:34:28.699087+0000","last_became_active":"2024-04-24T15:34:12.655508+0000","last_became_peered":"2024-04-24T15:34:12.655508+0000","last_unstale":"2024-04-24T15:34:28.699087+0000","last_undegraded":"2024-04-24T15:34:28.699087+0000","last_fullsized":"2024-04-24T15:34:28.699087+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_clean_scrub_stamp":"2024-04-24T15:32:06.972928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:13:50.464460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":59,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.699249+0000","last_change":"2024-04-24T15:34:19.838358+0000","last_active":"2024-04-24T15:34:28.699249+0000","last_peered":"2024-04-24T15:34:28.699249+0000","last_clean":"2024-04-24T15:34:28.699249+0000","last_became_active":"2024-04-24T15:34:19.838058+0000","last_became_peered":"2024-04-24T15:34:19.838058+0000","last_unstale":"2024-04-24T15:34:28.699249+0000","last_undegraded":"2024-04-24T15:34:28.699249+0000","last_fullsized":"2024-04-24T15:34:28.699249+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:29:43.725200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"131'430","reported_seq":1231,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.699369+0000","last_change":"2024-04-24T15:34:15.660049+0000","last_active":"2024-04-24T15:34:28.699369+0000","last_peered":"2024-04-24T15:34:28.699369+0000","last_clean":"2024-04-24T15:34:28.699369+0000","last_became_active":"2024-04-24T15:34:15.658368+0000","last_became_peered":"2024-04-24T15:34:15.658368+0000","last_unstale":"2024-04-24T15:34:28.699369+0000","last_undegraded":"2024-04-24T15:34:28.699369+0000","last_fullsized":"2024-04-24T15:34:28.699369+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:29:46.353757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015045289999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":1970,"num_read_kb":12710,"num_write":463,"num_write_kb":8069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":208,"num_bytes_recovered":107133,"num_keys_recovered":27,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":93,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861700+0000","last_change":"2024-04-24T15:34:24.428957+0000","last_active":"2024-04-24T15:34:27.861700+0000","last_peered":"2024-04-24T15:34:27.861700+0000","last_clean":"2024-04-24T15:34:27.861700+0000","last_became_active":"2024-04-24T15:34:24.428510+0000","last_became_peered":"2024-04-24T15:34:24.428510+0000","last_unstale":"2024-04-24T15:34:27.861700+0000","last_undegraded":"2024-04-24T15:34:27.861700+0000","last_fullsized":"2024-04-24T15:34:27.861700+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_clean_scrub_stamp":"2024-04-24T15:32:06.972928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:36:35.138337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698664+0000","last_change":"2024-04-24T15:34:13.689898+0000","last_active":"2024-04-24T15:34:28.698664+0000","last_peered":"2024-04-24T15:34:28.698664+0000","last_clean":"2024-04-24T15:34:28.698664+0000","last_became_active":"2024-04-24T15:34:13.689685+0000","last_became_peered":"2024-04-24T15:34:13.689685+0000","last_unstale":"2024-04-24T15:34:28.698664+0000","last_undegraded":"2024-04-24T15:34:28.698664+0000","last_fullsized":"2024-04-24T15:34:28.698664+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:24:27.251519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"130'457","reported_seq":1995,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.699625+0000","last_change":"2024-04-24T15:34:16.654083+0000","last_active":"2024-04-24T15:34:28.699625+0000","last_peered":"2024-04-24T15:34:28.699625+0000","last_clean":"2024-04-24T15:34:28.699625+0000","last_became_active":"2024-04-24T15:34:16.651906+0000","last_became_peered":"2024-04-24T15:34:16.651906+0000","last_unstale":"2024-04-24T15:34:28.699625+0000","last_undegraded":"2024-04-24T15:34:28.699625+0000","last_fullsized":"2024-04-24T15:34:28.699625+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":457,"log_dups_size":0,"ondisk_log_size":457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:54.315154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010565220000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2138,"num_read_kb":24032,"num_write":423,"num_write_kb":14900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"130'408","reported_seq":1974,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.699200+0000","last_change":"2024-04-24T15:34:16.653875+0000","last_active":"2024-04-24T15:34:28.699200+0000","last_peered":"2024-04-24T15:34:28.699200+0000","last_clean":"2024-04-24T15:34:28.699200+0000","last_became_active":"2024-04-24T15:34:16.651931+0000","last_became_peered":"2024-04-24T15:34:16.651931+0000","last_unstale":"2024-04-24T15:34:28.699200+0000","last_undegraded":"2024-04-24T15:34:28.699200+0000","last_fullsized":"2024-04-24T15:34:28.699200+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:59:24.823763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039757799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2388,"num_read_kb":22956,"num_write":379,"num_write_kb":16060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":578400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.699581+0000","last_change":"2024-04-24T15:34:12.653777+0000","last_active":"2024-04-24T15:34:28.699581+0000","last_peered":"2024-04-24T15:34:28.699581+0000","last_clean":"2024-04-24T15:34:28.699581+0000","last_became_active":"2024-04-24T15:34:12.653529+0000","last_became_peered":"2024-04-24T15:34:12.653529+0000","last_unstale":"2024-04-24T15:34:28.699581+0000","last_undegraded":"2024-04-24T15:34:28.699581+0000","last_fullsized":"2024-04-24T15:34:28.699581+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:21:58.411545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"129'479","reported_seq":1997,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698576+0000","last_change":"2024-04-24T15:34:16.656437+0000","last_active":"2024-04-24T15:34:28.698576+0000","last_peered":"2024-04-24T15:34:28.698576+0000","last_clean":"2024-04-24T15:34:28.698576+0000","last_became_active":"2024-04-24T15:34:16.654252+0000","last_became_peered":"2024-04-24T15:34:16.654252+0000","last_unstale":"2024-04-24T15:34:28.698576+0000","last_undegraded":"2024-04-24T15:34:28.698576+0000","last_fullsized":"2024-04-24T15:34:28.698576+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:53:07.507328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035787500000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":2434,"num_read_kb":14959,"num_write":545,"num_write_kb":12213,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":335,"num_bytes_recovered":6920328,"num_keys_recovered":43,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"130'616","reported_seq":2734,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861566+0000","last_change":"2024-04-24T15:34:23.827845+0000","last_active":"2024-04-24T15:34:27.861566+0000","last_peered":"2024-04-24T15:34:27.861566+0000","last_clean":"2024-04-24T15:34:27.861566+0000","last_became_active":"2024-04-24T15:34:23.825759+0000","last_became_peered":"2024-04-24T15:34:23.825759+0000","last_unstale":"2024-04-24T15:34:27.861566+0000","last_undegraded":"2024-04-24T15:34:27.861566+0000","last_fullsized":"2024-04-24T15:34:27.861566+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:49:46.424217+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0055639879999999997,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":4324,"num_read_kb":21128,"num_write":677,"num_write_kb":10933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":5713612,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"130'485","reported_seq":3231,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698742+0000","last_change":"2024-04-24T15:34:15.659323+0000","last_active":"2024-04-24T15:34:28.698742+0000","last_peered":"2024-04-24T15:34:28.698742+0000","last_clean":"2024-04-24T15:34:28.698742+0000","last_became_active":"2024-04-24T15:34:15.657660+0000","last_became_peered":"2024-04-24T15:34:15.657660+0000","last_unstale":"2024-04-24T15:34:28.698742+0000","last_undegraded":"2024-04-24T15:34:28.698742+0000","last_fullsized":"2024-04-24T15:34:28.698742+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:09:16.071953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066763,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3120,"num_read_kb":27458,"num_write":430,"num_write_kb":12267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":237,"num_bytes_recovered":5299136,"num_keys_recovered":49,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"129'551","reported_seq":1976,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698866+0000","last_change":"2024-04-24T15:34:15.669951+0000","last_active":"2024-04-24T15:34:28.698866+0000","last_peered":"2024-04-24T15:34:28.698866+0000","last_clean":"2024-04-24T15:34:28.698866+0000","last_became_active":"2024-04-24T15:34:15.668717+0000","last_became_peered":"2024-04-24T15:34:15.668717+0000","last_unstale":"2024-04-24T15:34:28.698866+0000","last_undegraded":"2024-04-24T15:34:28.698866+0000","last_fullsized":"2024-04-24T15:34:28.698866+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":551,"log_dups_size":0,"ondisk_log_size":551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:46:28.552856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039775200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2839,"num_read_kb":21248,"num_write":721,"num_write_kb":8473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":141,"num_bytes_recovered":1490474,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"127'1046","reported_seq":2505,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861716+0000","last_change":"2024-04-24T15:34:23.826944+0000","last_active":"2024-04-24T15:34:27.861716+0000","last_peered":"2024-04-24T15:34:27.861716+0000","last_clean":"2024-04-24T15:34:27.861716+0000","last_became_active":"2024-04-24T15:34:23.825039+0000","last_became_peered":"2024-04-24T15:34:23.825039+0000","last_unstale":"2024-04-24T15:34:27.861716+0000","last_undegraded":"2024-04-24T15:34:27.861716+0000","last_fullsized":"2024-04-24T15:34:27.861716+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":1046,"log_dups_size":0,"ondisk_log_size":1046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:15:05.349996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029127150000000002,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":6311,"num_read_kb":16558,"num_write":1677,"num_write_kb":12129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":54,"num_bytes_recovered":1219753,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.d","version":"127'298","reported_seq":2572,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861837+0000","last_change":"2024-04-24T15:34:23.825797+0000","last_active":"2024-04-24T15:34:27.861837+0000","last_peered":"2024-04-24T15:34:27.861837+0000","last_clean":"2024-04-24T15:34:27.861837+0000","last_became_active":"2024-04-24T15:34:23.823763+0000","last_became_peered":"2024-04-24T15:34:23.823763+0000","last_unstale":"2024-04-24T15:34:27.861837+0000","last_undegraded":"2024-04-24T15:34:27.861837+0000","last_fullsized":"2024-04-24T15:34:27.861837+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":298,"log_dups_size":0,"ondisk_log_size":298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:24:00.110009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047837299999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":2121,"num_read_kb":46483,"num_write":303,"num_write_kb":20603,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":4246835,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"129'563","reported_seq":2395,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861756+0000","last_change":"2024-04-24T15:34:23.826137+0000","last_active":"2024-04-24T15:34:27.861756+0000","last_peered":"2024-04-24T15:34:27.861756+0000","last_clean":"2024-04-24T15:34:27.861756+0000","last_became_active":"2024-04-24T15:34:23.824037+0000","last_became_peered":"2024-04-24T15:34:23.824037+0000","last_unstale":"2024-04-24T15:34:27.861756+0000","last_undegraded":"2024-04-24T15:34:27.861756+0000","last_fullsized":"2024-04-24T15:34:27.861756+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":563,"log_dups_size":0,"ondisk_log_size":563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:30:41.572188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001084025,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3022,"num_read_kb":13469,"num_write":696,"num_write_kb":6185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":113,"num_bytes_recovered":1676557,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"130'907","reported_seq":3306,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.699413+0000","last_change":"2024-04-24T15:34:16.657695+0000","last_active":"2024-04-24T15:34:28.699413+0000","last_peered":"2024-04-24T15:34:28.699413+0000","last_clean":"2024-04-24T15:34:28.699413+0000","last_became_active":"2024-04-24T15:34:16.655733+0000","last_became_peered":"2024-04-24T15:34:16.655733+0000","last_unstale":"2024-04-24T15:34:28.699413+0000","last_undegraded":"2024-04-24T15:34:28.699413+0000","last_fullsized":"2024-04-24T15:34:28.699413+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":907,"log_dups_size":0,"ondisk_log_size":907,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:59:41.606707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018480180000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":5733,"num_read_kb":14488,"num_write":1011,"num_write_kb":8995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":223,"num_bytes_recovered":630687,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"130'244","reported_seq":1061,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861808+0000","last_change":"2024-04-24T15:34:15.664032+0000","last_active":"2024-04-24T15:34:27.861808+0000","last_peered":"2024-04-24T15:34:27.861808+0000","last_clean":"2024-04-24T15:34:27.861808+0000","last_became_active":"2024-04-24T15:34:15.663081+0000","last_became_peered":"2024-04-24T15:34:15.663081+0000","last_unstale":"2024-04-24T15:34:27.861808+0000","last_undegraded":"2024-04-24T15:34:27.861808+0000","last_fullsized":"2024-04-24T15:34:27.861808+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":244,"log_dups_size":0,"ondisk_log_size":244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:33:57.743630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010945530000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":384,"num_read_kb":4092,"num_write":235,"num_write_kb":5715,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":1989443,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.11","version":"129'705","reported_seq":2162,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861883+0000","last_change":"2024-04-24T15:34:08.757586+0000","last_active":"2024-04-24T15:34:27.861883+0000","last_peered":"2024-04-24T15:34:27.861883+0000","last_clean":"2024-04-24T15:34:27.861883+0000","last_became_active":"2024-04-24T15:33:57.644309+0000","last_became_peered":"2024-04-24T15:33:57.644309+0000","last_unstale":"2024-04-24T15:34:27.861883+0000","last_undegraded":"2024-04-24T15:34:27.861883+0000","last_fullsized":"2024-04-24T15:34:27.861883+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":705,"log_dups_size":0,"ondisk_log_size":705,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:26:21.749588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012982219999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":2615,"num_read_kb":21488,"num_write":891,"num_write_kb":14014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":5651177,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"129'331","reported_seq":5228,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.699265+0000","last_change":"2024-04-24T15:34:15.655138+0000","last_active":"2024-04-24T15:34:28.699265+0000","last_peered":"2024-04-24T15:34:28.699265+0000","last_clean":"2024-04-24T15:34:28.699265+0000","last_became_active":"2024-04-24T15:34:15.653205+0000","last_became_peered":"2024-04-24T15:34:15.653205+0000","last_unstale":"2024-04-24T15:34:28.699265+0000","last_undegraded":"2024-04-24T15:34:28.699265+0000","last_fullsized":"2024-04-24T15:34:28.699265+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":331,"log_dups_size":0,"ondisk_log_size":331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:28:59.539282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013576079999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":4982,"num_read_kb":42072,"num_write":333,"num_write_kb":14625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":3758016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"129'241","reported_seq":2003,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861679+0000","last_change":"2024-04-24T15:34:23.825462+0000","last_active":"2024-04-24T15:34:27.861679+0000","last_peered":"2024-04-24T15:34:27.861679+0000","last_clean":"2024-04-24T15:34:27.861679+0000","last_became_active":"2024-04-24T15:34:23.823962+0000","last_became_peered":"2024-04-24T15:34:23.823962+0000","last_unstale":"2024-04-24T15:34:27.861679+0000","last_undegraded":"2024-04-24T15:34:27.861679+0000","last_fullsized":"2024-04-24T15:34:27.861679+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":241,"log_dups_size":0,"ondisk_log_size":241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:52:34.277987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000635438,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":1312,"num_read_kb":18878,"num_write":253,"num_write_kb":9844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":108,"num_bytes_recovered":2694373,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"131'450","reported_seq":6595,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.862055+0000","last_change":"2024-04-24T15:34:16.660466+0000","last_active":"2024-04-24T15:34:27.862055+0000","last_peered":"2024-04-24T15:34:27.862055+0000","last_clean":"2024-04-24T15:34:27.862055+0000","last_became_active":"2024-04-24T15:34:16.659071+0000","last_became_peered":"2024-04-24T15:34:16.659071+0000","last_unstale":"2024-04-24T15:34:27.862055+0000","last_undegraded":"2024-04-24T15:34:27.862055+0000","last_fullsized":"2024-04-24T15:34:27.862055+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:10:28.933829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013356920000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":7366,"num_read_kb":44142,"num_write":407,"num_write_kb":9850,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":233,"num_bytes_recovered":4458876,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"130'312","reported_seq":1575,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698823+0000","last_change":"2024-04-24T15:34:16.657117+0000","last_active":"2024-04-24T15:34:28.698823+0000","last_peered":"2024-04-24T15:34:28.698823+0000","last_clean":"2024-04-24T15:34:28.698823+0000","last_became_active":"2024-04-24T15:34:16.655084+0000","last_became_peered":"2024-04-24T15:34:16.655084+0000","last_unstale":"2024-04-24T15:34:28.698823+0000","last_undegraded":"2024-04-24T15:34:28.698823+0000","last_fullsized":"2024-04-24T15:34:28.698823+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:26:41.362344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003384743,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":837,"num_read_kb":16692,"num_write":319,"num_write_kb":15488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":277,"num_bytes_recovered":16228001,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.16","version":"129'375","reported_seq":2454,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.699026+0000","last_change":"2024-04-24T15:34:15.669825+0000","last_active":"2024-04-24T15:34:28.699026+0000","last_peered":"2024-04-24T15:34:28.699026+0000","last_clean":"2024-04-24T15:34:28.699026+0000","last_became_active":"2024-04-24T15:34:15.668678+0000","last_became_peered":"2024-04-24T15:34:15.668678+0000","last_unstale":"2024-04-24T15:34:28.699026+0000","last_undegraded":"2024-04-24T15:34:28.699026+0000","last_fullsized":"2024-04-24T15:34:28.699026+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:43:39.484677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018886770000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":3004,"num_read_kb":17777,"num_write":338,"num_write_kb":8036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":48,"num_read":114807,"num_read_kb":727542,"num_write":20290,"num_write_kb":356328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4413,"num_bytes_recovered":105175530,"num_keys_recovered":235,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15242,"internal_metadata":0},"log_size":17251,"ondisk_log_size":17251,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":8,"acting":8,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":16196,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574882,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299136,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072704,"statfs":{"total":96636764160,"available":96330448896,"internally_reserved":0,"allocated":1818624,"data_stored":1494500,"data_compressed":5677,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.46000000000000002,"15min":0.46200000000000002},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.60099999999999998,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.54200000000000004,"15min":0.55600000000000005},"min":{"1min":0.33900000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.67200000000000004,"5min":2.4710000000000001,"15min":2.4710000000000001},"last":0.73999999999999999}]},{"osd":1,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.57399999999999995,"15min":0.59299999999999997},"min":{"1min":0.28100000000000003,"5min":0.248,"15min":0.248},"max":{"1min":0.63600000000000001,"5min":2.4729999999999999,"15min":2.4729999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.51500000000000001,"15min":0.52400000000000002},"min":{"1min":0.29999999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.56899999999999995,"5min":2.4020000000000001,"15min":2.4020000000000001},"last":0.61099999999999999}]},{"osd":2,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.45700000000000002,"15min":0.45800000000000002},"min":{"1min":0.35499999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.59699999999999998,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.503,"5min":0.54800000000000004,"15min":0.55600000000000005},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.82199999999999995,"5min":2.363,"15min":2.363},"last":0.752}]},{"osd":3,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.52200000000000002,"15min":0.51800000000000002},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.53800000000000003,"15min":0.53200000000000003},"min":{"1min":0.38600000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.63600000000000001}]},{"osd":4,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.41399999999999998,"15min":0.41999999999999998},"min":{"1min":0.26100000000000001,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.66100000000000003,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.42299999999999999,"15min":0.42499999999999999},"min":{"1min":0.30599999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.73499999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.441}]},{"osd":5,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.45100000000000001,"15min":0.45500000000000002},"min":{"1min":0.307,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.51600000000000001,"15min":0.52000000000000002},"min":{"1min":0.36399999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.80500000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.67900000000000005}]},{"osd":6,"last update":"Wed Apr 24 15:32:58 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.65700000000000003}]}]},{"osd":6,"up_from":104,"seq":446676598796,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":305896,"kb_used_data":1748,"kb_used_omap":43,"kb_used_meta":304084,"kb_avail":94065944,"statfs":{"total":96636764160,"available":96323526656,"internally_reserved":0,"allocated":1789952,"data_stored":910351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":44735,"internal_metadata":311382337},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574881,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307280,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":305534,"kb_avail":94064560,"statfs":{"total":96636764160,"available":96322109440,"internally_reserved":0,"allocated":1773568,"data_stored":903176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312867160},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 15:34:01 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.35599999999999998,"15min":0.35799999999999998},"min":{"1min":0.20699999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.51100000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.377,"15min":0.376},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":1.353,"5min":1.375,"15min":1.375},"last":0.376}]},{"osd":2,"last update":"Wed Apr 24 15:34:01 2024","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.33600000000000002,"15min":0.34000000000000002},"min":{"1min":0.224,"5min":0.223,"15min":0.223},"max":{"1min":0.46100000000000002,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.255},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.41499999999999998,"15min":0.42699999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.52800000000000002,"5min":1.494,"15min":1.494},"last":0.30399999999999999}]},{"osd":3,"last update":"Wed Apr 24 15:34:01 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.42799999999999999,"15min":0.433},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.97499999999999998,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.377,"15min":0.372},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.40799999999999997}]},{"osd":4,"last update":"Wed Apr 24 15:34:01 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.53500000000000003,"15min":0.54100000000000004},"min":{"1min":0.34799999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.72199999999999998,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.495},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.57799999999999996,"15min":0.58199999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.52100000000000002}]},{"osd":5,"last update":"Wed Apr 24 15:34:01 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.379,"5min":1.454,"15min":1.454},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.53800000000000003,"15min":0.54200000000000004},"min":{"1min":0.35999999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.67100000000000004,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.56799999999999995}]},{"osd":6,"last update":"Wed Apr 24 15:33:01 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.548,"5min":1.548,"15min":1.548},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.51300000000000001}]},{"osd":7,"last update":"Wed Apr 24 15:34:01 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.56899999999999995,"15min":0.57099999999999995},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.55100000000000005,"15min":0.55700000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.69599999999999995,"5min":0.999,"15min":0.999},"last":0.53600000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574882,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303260,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94068580,"statfs":{"total":96636764160,"available":96326225920,"internally_reserved":0,"allocated":1847296,"data_stored":1500719,"data_compressed":4842,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.30499999999999999,"5min":0.45700000000000002,"15min":0.48299999999999998},"min":{"1min":0.17899999999999999,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.47999999999999998,"5min":3.7890000000000001,"15min":3.7890000000000001},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.46700000000000003,"15min":0.48899999999999999},"min":{"1min":0.224,"5min":0.191,"15min":0.191},"max":{"1min":0.59899999999999998,"5min":2.7890000000000001,"15min":2.7890000000000001},"last":0.35099999999999998}]},{"osd":2,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.53100000000000003,"15min":0.55900000000000005},"min":{"1min":0.22700000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.61399999999999999,"5min":3.9950000000000001,"15min":3.9950000000000001},"last":0.312},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.42299999999999999,"15min":0.436},"min":{"1min":0.26000000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.56799999999999995,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":0.28299999999999997}]},{"osd":3,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.48499999999999999,"15min":0.505},"min":{"1min":0.24099999999999999,"5min":0.223,"15min":0.223},"max":{"1min":0.54000000000000004,"5min":2.097,"15min":2.097},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.622,"15min":0.65700000000000003},"min":{"1min":0.30499999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.79500000000000004,"5min":3.2109999999999999,"15min":3.2109999999999999},"last":0.38800000000000001}]},{"osd":4,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.55700000000000005,"15min":0.56699999999999995},"min":{"1min":0.31,"5min":0.308,"15min":0.308},"max":{"1min":0.69099999999999995,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.65000000000000002,"15min":0.67300000000000004},"min":{"1min":0.34899999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.85599999999999998,"5min":3.282,"15min":3.282},"last":0.5}]},{"osd":5,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.60099999999999998,"15min":0.61299999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.79100000000000004,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.64600000000000002,"15min":0.66200000000000003},"min":{"1min":0.38700000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.78000000000000003,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":0.55400000000000005}]},{"osd":6,"last update":"Wed Apr 24 15:33:00 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":3.2509999999999999,"5min":3.2509999999999999,"15min":3.2509999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":2.1909999999999998,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":0.57999999999999996}]},{"osd":7,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.60299999999999998,"15min":0.60899999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.075,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.63200000000000001,"15min":0.64400000000000002},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.876,"5min":2.206,"15min":2.206},"last":0.52800000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574882,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303200,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94068640,"statfs":{"total":96636764160,"available":96326287360,"internally_reserved":0,"allocated":1785856,"data_stored":909395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 15:34:06 2024","interfaces":[{"interface":"back","average":{"1min":0.29199999999999998,"5min":0.312,"15min":0.315},"min":{"1min":0.16200000000000001,"5min":0.16200000000000001,"15min":0.16200000000000001},"max":{"1min":0.41399999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.30599999999999999,"5min":0.33700000000000002,"15min":0.34200000000000003},"min":{"1min":0.185,"5min":0.185,"15min":0.185},"max":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"last":0.442}]},{"osd":1,"last update":"Wed Apr 24 15:34:06 2024","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.36899999999999999,"15min":0.377},"min":{"1min":0.23300000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.47399999999999998,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.46100000000000002,"15min":0.48199999999999998},"min":{"1min":0.24099999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.441,"5min":3.1110000000000002,"15min":3.1110000000000002},"last":0.48999999999999999}]},{"osd":3,"last update":"Wed Apr 24 15:34:06 2024","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.40500000000000003,"15min":0.41899999999999998},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.41399999999999998,"5min":2.056,"15min":2.056},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.39900000000000002,"15min":0.40500000000000003},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.52900000000000003,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.54100000000000004}]},{"osd":4,"last update":"Wed Apr 24 15:34:06 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.51800000000000002,"15min":0.52400000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.53600000000000003,"15min":0.54500000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.70399999999999996,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.57299999999999995}]},{"osd":5,"last update":"Wed Apr 24 15:34:06 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.54300000000000004,"15min":0.54600000000000004},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.63700000000000001,"15min":0.65400000000000003},"min":{"1min":0.41399999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.80500000000000005,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":0.621}]},{"osd":6,"last update":"Wed Apr 24 15:33:03 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.1000000000000001,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.54900000000000004}]},{"osd":7,"last update":"Wed Apr 24 15:34:06 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.63500000000000001,"15min":0.65700000000000003},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.63800000000000001,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.56999999999999995,"15min":0.57799999999999996},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73999999999999999,"5min":1.502,"15min":1.502},"last":0.67400000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574882,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303204,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94068636,"statfs":{"total":96636764160,"available":96326283264,"internally_reserved":0,"allocated":1794048,"data_stored":910370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 15:34:04 2024","interfaces":[{"interface":"back","average":{"1min":0.29399999999999998,"5min":0.34599999999999997,"15min":0.35499999999999998},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.39900000000000002,"5min":1.224,"15min":1.224},"last":0.27600000000000002},{"interface":"front","average":{"1min":0.30199999999999999,"5min":0.34000000000000002,"15min":0.34599999999999997},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.41999999999999998,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.30099999999999999}]},{"osd":1,"last update":"Wed Apr 24 15:34:04 2024","interfaces":[{"interface":"back","average":{"1min":0.30299999999999999,"5min":0.33400000000000002,"15min":0.33900000000000002},"min":{"1min":0.19600000000000001,"5min":0.187,"15min":0.187},"max":{"1min":0.437,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.42199999999999999,"15min":0.434},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.71299999999999997,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.39400000000000002}]},{"osd":2,"last update":"Wed Apr 24 15:34:04 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.39400000000000002,"15min":0.40300000000000002},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.46500000000000002,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.376,"5min":0.41299999999999998,"15min":0.41899999999999998},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.53000000000000003,"5min":1.252,"15min":1.252},"last":0.44800000000000001}]},{"osd":4,"last update":"Wed Apr 24 15:34:04 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.52200000000000002,"15min":0.53400000000000003},"min":{"1min":0.28599999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.56499999999999995,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.52100000000000002,"15min":0.53300000000000003},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.60699999999999998,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.69999999999999996}]},{"osd":5,"last update":"Wed Apr 24 15:34:04 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.55600000000000005,"15min":0.56999999999999995},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.61199999999999999,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.5,"5min":0.58799999999999997,"15min":0.60199999999999998},"min":{"1min":0.38300000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.81100000000000005,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.621}]},{"osd":6,"last update":"Wed Apr 24 15:33:00 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.60199999999999998}]},{"osd":7,"last update":"Wed Apr 24 15:34:04 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.59099999999999997,"15min":0.60499999999999998},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.66800000000000004,"5min":1.635,"15min":1.635},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.495,"5min":0.55600000000000005,"15min":0.56599999999999995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.59699999999999998,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.64500000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574883,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307284,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":305534,"kb_avail":94064556,"statfs":{"total":96636764160,"available":96322105344,"internally_reserved":0,"allocated":1773568,"data_stored":901837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312867160},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 15:34:00 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.505,"15min":0.50900000000000001},"min":{"1min":0.249,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":1.8620000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.626,"5min":0.56599999999999995,"15min":0.55600000000000005},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":5.6239999999999997,"5min":5.6239999999999997,"15min":5.6239999999999997},"last":0.64100000000000001}]},{"osd":1,"last update":"Wed Apr 24 15:34:00 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.58699999999999997,"15min":0.57299999999999995},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":5.7149999999999999,"5min":5.7149999999999999,"15min":5.7149999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.35499999999999998,"5min":0.31,"15min":0.31},"max":{"1min":1.913,"5min":1.913,"15min":1.913},"last":0.53100000000000003}]},{"osd":2,"last update":"Wed Apr 24 15:34:00 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.54600000000000004,"15min":0.52800000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":5.6660000000000004,"5min":5.6660000000000004,"15min":5.6660000000000004},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.53400000000000003,"15min":0.54400000000000004},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.819,"5min":1.819,"15min":1.819},"last":0.46800000000000003}]},{"osd":3,"last update":"Wed Apr 24 15:34:00 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.53400000000000003,"15min":0.53600000000000003},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.9550000000000001,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":0.502},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.55500000000000005,"15min":0.55000000000000004},"min":{"1min":0.36799999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":2.3149999999999999,"5min":2.3149999999999999,"15min":2.3149999999999999},"last":0.55400000000000005}]},{"osd":5,"last update":"Wed Apr 24 15:34:00 2024","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.58299999999999996,"15min":0.51700000000000002},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":14.667999999999999,"5min":14.667999999999999,"15min":14.667999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.434,"5min":0.497,"15min":0.50700000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.9890000000000001,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":0.76400000000000001}]},{"osd":6,"last update":"Wed Apr 24 15:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.73399999999999999}]},{"osd":7,"last update":"Wed Apr 24 15:34:00 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.54000000000000004,"15min":0.53100000000000003},"min":{"1min":0.28599999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":5.2759999999999998,"5min":5.2759999999999998,"15min":5.2759999999999998},"last":0.71999999999999997},{"interface":"front","average":{"1min":1.0469999999999999,"5min":0.64100000000000001,"15min":0.57299999999999995},"min":{"1min":0.29399999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":14.773,"5min":14.773,"15min":14.773},"last":0.78700000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574882,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311380,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":309630,"kb_avail":94060460,"statfs":{"total":96636764160,"available":96317911040,"internally_reserved":0,"allocated":1777664,"data_stored":904132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317061464},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.47899999999999998,"15min":0.48499999999999999},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.46800000000000003,"15min":0.47599999999999998},"min":{"1min":0.246,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.629,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.55300000000000005}]},{"osd":1,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.46300000000000002,"15min":0.47099999999999997},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.64900000000000002,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.46899999999999997,"15min":0.47099999999999997},"min":{"1min":0.317,"5min":0.313,"15min":0.313},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.67300000000000004}]},{"osd":2,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.47499999999999998,"15min":0.46600000000000003},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":2.4169999999999998,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.48599999999999999,"15min":0.49199999999999999},"min":{"1min":0.30099999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.64200000000000002}]},{"osd":3,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.48899999999999999,"15min":0.495},"min":{"1min":0.308,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.70999999999999996,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":2.3370000000000002,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":0.65900000000000003}]},{"osd":4,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.44800000000000001,"15min":0.45700000000000002},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.624,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.752},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.48399999999999999,"15min":0.48599999999999999},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.59899999999999998}]},{"osd":6,"last update":"Wed Apr 24 15:33:02 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.71599999999999997}]},{"osd":7,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.52600000000000002,"15min":0.53100000000000003},"min":{"1min":0.30299999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.78600000000000003,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.496,"5min":0.53200000000000003,"15min":0.53800000000000003},"min":{"1min":0.34499999999999997,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.77100000000000002,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.76500000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4842,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":5677,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":5677,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15242,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T15:34:32.561 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T15:34:32.874 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:34:32.875 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-04-24T15:34:32.886 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":219,"stamp":"2024-04-24T15:34:31.579540+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":51,"num_read":114923,"num_read_kb":727642,"num_write":20449,"num_write_kb":358744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4420,"num_bytes_recovered":106356266,"num_keys_recovered":235,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17371,"ondisk_log_size":17371,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2440640,"kb_used_data":14024,"kb_used_omap":55,"kb_used_meta":2426440,"kb_avail":752534080,"statfs":{"total":773094113280,"available":770594897920,"internally_reserved":0,"allocated":14360576,"data_stored":8434480,"data_compressed":10519,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":56663,"internal_metadata":2484675241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.736205"},"pg_stats":[{"pgid":"4.1f","version":"131'1485","reported_seq":2550,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698777+0000","last_change":"2024-04-24T15:34:15.652525+0000","last_active":"2024-04-24T15:34:28.698777+0000","last_peered":"2024-04-24T15:34:28.698777+0000","last_clean":"2024-04-24T15:34:28.698777+0000","last_became_active":"2024-04-24T15:34:15.650850+0000","last_became_peered":"2024-04-24T15:34:15.650850+0000","last_unstale":"2024-04-24T15:34:28.698777+0000","last_undegraded":"2024-04-24T15:34:28.698777+0000","last_fullsized":"2024-04-24T15:34:28.698777+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":1485,"log_dups_size":0,"ondisk_log_size":1485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:06:34.577605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001749947,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":8143,"num_read_kb":14722,"num_write":2575,"num_write_kb":16077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":5952278,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"129'507","reported_seq":6418,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698830+0000","last_change":"2024-04-24T15:34:15.669801+0000","last_active":"2024-04-24T15:34:28.698830+0000","last_peered":"2024-04-24T15:34:28.698830+0000","last_clean":"2024-04-24T15:34:28.698830+0000","last_became_active":"2024-04-24T15:34:15.668657+0000","last_became_peered":"2024-04-24T15:34:15.668657+0000","last_unstale":"2024-04-24T15:34:28.698830+0000","last_undegraded":"2024-04-24T15:34:28.698830+0000","last_fullsized":"2024-04-24T15:34:28.698830+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":507,"log_dups_size":0,"ondisk_log_size":507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:32:57.253624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029386599999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":7278,"num_read_kb":39113,"num_write":560,"num_write_kb":6592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":4495932,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"129'529","reported_seq":3745,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861949+0000","last_change":"2024-04-24T15:34:16.660701+0000","last_active":"2024-04-24T15:34:27.861949+0000","last_peered":"2024-04-24T15:34:27.861949+0000","last_clean":"2024-04-24T15:34:27.861949+0000","last_became_active":"2024-04-24T15:34:16.659202+0000","last_became_peered":"2024-04-24T15:34:16.659202+0000","last_unstale":"2024-04-24T15:34:27.861949+0000","last_undegraded":"2024-04-24T15:34:27.861949+0000","last_fullsized":"2024-04-24T15:34:27.861949+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":529,"log_dups_size":0,"ondisk_log_size":529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:31:57.762122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015488839999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":4427,"num_read_kb":51896,"num_write":603,"num_write_kb":18759,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":580671,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"131'526","reported_seq":2700,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.699161+0000","last_change":"2024-04-24T15:34:19.807684+0000","last_active":"2024-04-24T15:34:28.699161+0000","last_peered":"2024-04-24T15:34:28.699161+0000","last_clean":"2024-04-24T15:34:28.699161+0000","last_became_active":"2024-04-24T15:34:19.806783+0000","last_became_peered":"2024-04-24T15:34:19.806783+0000","last_unstale":"2024-04-24T15:34:28.699161+0000","last_undegraded":"2024-04-24T15:34:28.699161+0000","last_fullsized":"2024-04-24T15:34:28.699161+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":526,"log_dups_size":0,"ondisk_log_size":526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:06:01.349645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001413493,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":3716,"num_read_kb":10078,"num_write":547,"num_write_kb":2888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"129'385","reported_seq":2359,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861581+0000","last_change":"2024-04-24T15:34:23.827263+0000","last_active":"2024-04-24T15:34:27.861581+0000","last_peered":"2024-04-24T15:34:27.861581+0000","last_clean":"2024-04-24T15:34:27.861581+0000","last_became_active":"2024-04-24T15:34:23.825472+0000","last_became_peered":"2024-04-24T15:34:23.825472+0000","last_unstale":"2024-04-24T15:34:27.861581+0000","last_undegraded":"2024-04-24T15:34:27.861581+0000","last_fullsized":"2024-04-24T15:34:27.861581+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":385,"log_dups_size":0,"ondisk_log_size":385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:19:37.470876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011107479999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":2335,"num_read_kb":25405,"num_write":329,"num_write_kb":6118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":79,"num_bytes_recovered":2143880,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"129'449","reported_seq":6610,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861569+0000","last_change":"2024-04-24T15:34:23.827554+0000","last_active":"2024-04-24T15:34:27.861569+0000","last_peered":"2024-04-24T15:34:27.861569+0000","last_clean":"2024-04-24T15:34:27.861569+0000","last_became_active":"2024-04-24T15:34:23.824861+0000","last_became_peered":"2024-04-24T15:34:23.824861+0000","last_unstale":"2024-04-24T15:34:27.861569+0000","last_undegraded":"2024-04-24T15:34:27.861569+0000","last_fullsized":"2024-04-24T15:34:27.861569+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:42:20.554334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097376600000000004,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":6368,"num_read_kb":19805,"num_write":475,"num_write_kb":10193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":89,"num_bytes_recovered":1137673,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"129'537","reported_seq":2680,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861805+0000","last_change":"2024-04-24T15:34:08.756894+0000","last_active":"2024-04-24T15:34:27.861805+0000","last_peered":"2024-04-24T15:34:27.861805+0000","last_clean":"2024-04-24T15:34:27.861805+0000","last_became_active":"2024-04-24T15:32:17.047842+0000","last_became_peered":"2024-04-24T15:32:17.047842+0000","last_unstale":"2024-04-24T15:34:27.861805+0000","last_undegraded":"2024-04-24T15:34:27.861805+0000","last_fullsized":"2024-04-24T15:34:27.861805+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":537,"log_dups_size":0,"ondisk_log_size":537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:09:52.065772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081826100000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":4119,"num_read_kb":22694,"num_write":527,"num_write_kb":11193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"130'444","reported_seq":1968,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861537+0000","last_change":"2024-04-24T15:34:08.758067+0000","last_active":"2024-04-24T15:34:27.861537+0000","last_peered":"2024-04-24T15:34:27.861537+0000","last_clean":"2024-04-24T15:34:27.861537+0000","last_became_active":"2024-04-24T15:33:42.641679+0000","last_became_peered":"2024-04-24T15:33:42.641679+0000","last_unstale":"2024-04-24T15:34:27.861537+0000","last_undegraded":"2024-04-24T15:34:27.861537+0000","last_fullsized":"2024-04-24T15:34:27.861537+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":444,"log_dups_size":0,"ondisk_log_size":444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:29:58.781375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077716999999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":1887,"num_read_kb":15406,"num_write":547,"num_write_kb":8155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":4099152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"129'523","reported_seq":2447,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861936+0000","last_change":"2024-04-24T15:34:08.758924+0000","last_active":"2024-04-24T15:34:27.861936+0000","last_peered":"2024-04-24T15:34:27.861936+0000","last_clean":"2024-04-24T15:34:27.861936+0000","last_became_active":"2024-04-24T15:33:56.699629+0000","last_became_peered":"2024-04-24T15:33:56.699629+0000","last_unstale":"2024-04-24T15:34:27.861936+0000","last_undegraded":"2024-04-24T15:34:27.861936+0000","last_fullsized":"2024-04-24T15:34:27.861936+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":523,"log_dups_size":0,"ondisk_log_size":523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:53:30.709042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00212766,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":3353,"num_read_kb":16260,"num_write":594,"num_write_kb":12753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":352256,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"129'316","reported_seq":2577,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861623+0000","last_change":"2024-04-24T15:34:23.829028+0000","last_active":"2024-04-24T15:34:27.861623+0000","last_peered":"2024-04-24T15:34:27.861623+0000","last_clean":"2024-04-24T15:34:27.861623+0000","last_became_active":"2024-04-24T15:34:23.827917+0000","last_became_peered":"2024-04-24T15:34:23.827917+0000","last_unstale":"2024-04-24T15:34:27.861623+0000","last_undegraded":"2024-04-24T15:34:27.861623+0000","last_fullsized":"2024-04-24T15:34:27.861623+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":316,"log_dups_size":0,"ondisk_log_size":316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:11:49.794260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065534800000000004,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":1633,"num_read_kb":26736,"num_write":303,"num_write_kb":13822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":9304722,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":121,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698692+0000","last_change":"2024-04-24T15:33:53.644249+0000","last_active":"2024-04-24T15:34:28.698692+0000","last_peered":"2024-04-24T15:34:28.698692+0000","last_clean":"2024-04-24T15:34:28.698692+0000","last_became_active":"2024-04-24T15:33:53.643941+0000","last_became_peered":"2024-04-24T15:33:53.643941+0000","last_unstale":"2024-04-24T15:34:28.698692+0000","last_undegraded":"2024-04-24T15:34:28.698692+0000","last_fullsized":"2024-04-24T15:34:28.698692+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:42:05.771051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"130'862","reported_seq":3009,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698843+0000","last_change":"2024-04-24T15:34:19.810827+0000","last_active":"2024-04-24T15:34:28.698843+0000","last_peered":"2024-04-24T15:34:28.698843+0000","last_clean":"2024-04-24T15:34:28.698843+0000","last_became_active":"2024-04-24T15:34:19.809619+0000","last_became_peered":"2024-04-24T15:34:19.809619+0000","last_unstale":"2024-04-24T15:34:28.698843+0000","last_undegraded":"2024-04-24T15:34:28.698843+0000","last_fullsized":"2024-04-24T15:34:28.698843+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:42:29.391778+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00033353700000000002,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":4689,"num_read_kb":23870,"num_write":1044,"num_write_kb":13092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":176,"num_bytes_recovered":5601476,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":24,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.699509+0000","last_change":"2024-04-24T15:34:12.652580+0000","last_active":"2024-04-24T15:34:28.699509+0000","last_peered":"2024-04-24T15:34:28.699509+0000","last_clean":"2024-04-24T15:34:28.699509+0000","last_became_active":"2024-04-24T15:34:12.652298+0000","last_became_peered":"2024-04-24T15:34:12.652298+0000","last_unstale":"2024-04-24T15:34:28.699509+0000","last_undegraded":"2024-04-24T15:34:28.699509+0000","last_fullsized":"2024-04-24T15:34:28.699509+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:22:59.160275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":41,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.862035+0000","last_change":"2024-04-24T15:33:53.645875+0000","last_active":"2024-04-24T15:34:27.862035+0000","last_peered":"2024-04-24T15:34:27.862035+0000","last_clean":"2024-04-24T15:34:27.862035+0000","last_became_active":"2024-04-24T15:33:53.645696+0000","last_became_peered":"2024-04-24T15:33:53.645696+0000","last_unstale":"2024-04-24T15:34:27.862035+0000","last_undegraded":"2024-04-24T15:34:27.862035+0000","last_fullsized":"2024-04-24T15:34:27.862035+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:29:35.345558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"131'584","reported_seq":5172,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698983+0000","last_change":"2024-04-24T15:34:17.655614+0000","last_active":"2024-04-24T15:34:28.698983+0000","last_peered":"2024-04-24T15:34:28.698983+0000","last_clean":"2024-04-24T15:34:28.698983+0000","last_became_active":"2024-04-24T15:34:17.654759+0000","last_became_peered":"2024-04-24T15:34:17.654759+0000","last_unstale":"2024-04-24T15:34:28.698983+0000","last_undegraded":"2024-04-24T15:34:28.698983+0000","last_fullsized":"2024-04-24T15:34:28.698983+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":584,"log_dups_size":0,"ondisk_log_size":584,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:42:40.438352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061779100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3592,"num_read_kb":14899,"num_write":598,"num_write_kb":11359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":23598,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"60'1","reported_seq":80,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698947+0000","last_change":"2024-04-24T15:33:47.663123+0000","last_active":"2024-04-24T15:34:28.698947+0000","last_peered":"2024-04-24T15:34:28.698947+0000","last_clean":"2024-04-24T15:34:28.698947+0000","last_became_active":"2024-04-24T15:33:47.662685+0000","last_became_peered":"2024-04-24T15:33:47.662685+0000","last_unstale":"2024-04-24T15:34:28.698947+0000","last_undegraded":"2024-04-24T15:34:28.698947+0000","last_fullsized":"2024-04-24T15:34:28.698947+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_clean_scrub_stamp":"2024-04-24T15:32:06.972928+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:05:35.654980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":50,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861972+0000","last_change":"2024-04-24T15:34:12.653126+0000","last_active":"2024-04-24T15:34:27.861972+0000","last_peered":"2024-04-24T15:34:27.861972+0000","last_clean":"2024-04-24T15:34:27.861972+0000","last_became_active":"2024-04-24T15:34:12.652805+0000","last_became_peered":"2024-04-24T15:34:12.652805+0000","last_unstale":"2024-04-24T15:34:27.861972+0000","last_undegraded":"2024-04-24T15:34:27.861972+0000","last_fullsized":"2024-04-24T15:34:27.861972+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:44:23.434720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"131'615","reported_seq":2369,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698926+0000","last_change":"2024-04-24T15:34:16.760282+0000","last_active":"2024-04-24T15:34:28.698926+0000","last_peered":"2024-04-24T15:34:28.698926+0000","last_clean":"2024-04-24T15:34:28.698926+0000","last_became_active":"2024-04-24T15:34:16.655390+0000","last_became_peered":"2024-04-24T15:34:16.655390+0000","last_unstale":"2024-04-24T15:34:28.698926+0000","last_undegraded":"2024-04-24T15:34:28.698926+0000","last_fullsized":"2024-04-24T15:34:28.698926+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":615,"log_dups_size":0,"ondisk_log_size":615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:01:32.241481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042917399999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":3015,"num_read_kb":19823,"num_write":696,"num_write_kb":5112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"90'2","reported_seq":182,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861981+0000","last_change":"2024-04-24T15:34:13.666629+0000","last_active":"2024-04-24T15:34:27.861981+0000","last_peered":"2024-04-24T15:34:27.861981+0000","last_clean":"2024-04-24T15:34:27.861981+0000","last_became_active":"2024-04-24T15:34:13.662363+0000","last_became_peered":"2024-04-24T15:34:13.662363+0000","last_unstale":"2024-04-24T15:34:27.861981+0000","last_undegraded":"2024-04-24T15:34:27.861981+0000","last_fullsized":"2024-04-24T15:34:27.861981+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_clean_scrub_stamp":"2024-04-24T15:32:06.972928+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:09:38.698619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":251,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.862026+0000","last_change":"2024-04-24T15:33:57.644653+0000","last_active":"2024-04-24T15:34:27.862026+0000","last_peered":"2024-04-24T15:34:27.862026+0000","last_clean":"2024-04-24T15:34:27.862026+0000","last_became_active":"2024-04-24T15:33:57.644312+0000","last_became_peered":"2024-04-24T15:33:57.644312+0000","last_unstale":"2024-04-24T15:34:27.862026+0000","last_undegraded":"2024-04-24T15:34:27.862026+0000","last_fullsized":"2024-04-24T15:34:27.862026+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:31:59.935287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:31:59.935287+0000","last_clean_scrub_stamp":"2024-04-24T15:31:59.935287+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:21:46.534664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698724+0000","last_change":"2024-04-24T15:34:13.650533+0000","last_active":"2024-04-24T15:34:28.698724+0000","last_peered":"2024-04-24T15:34:28.698724+0000","last_clean":"2024-04-24T15:34:28.698724+0000","last_became_active":"2024-04-24T15:34:13.650174+0000","last_became_peered":"2024-04-24T15:34:13.650174+0000","last_unstale":"2024-04-24T15:34:28.698724+0000","last_undegraded":"2024-04-24T15:34:28.698724+0000","last_fullsized":"2024-04-24T15:34:28.698724+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:01:43.442578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"129'591","reported_seq":2456,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698898+0000","last_change":"2024-04-24T15:34:08.805854+0000","last_active":"2024-04-24T15:34:28.698898+0000","last_peered":"2024-04-24T15:34:28.698898+0000","last_clean":"2024-04-24T15:34:28.698898+0000","last_became_active":"2024-04-24T15:33:45.658207+0000","last_became_peered":"2024-04-24T15:33:45.658207+0000","last_unstale":"2024-04-24T15:34:28.698898+0000","last_undegraded":"2024-04-24T15:34:28.698898+0000","last_fullsized":"2024-04-24T15:34:28.698898+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":591,"log_dups_size":0,"ondisk_log_size":591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:59:01.456518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010111199999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":3352,"num_read_kb":26205,"num_write":791,"num_write_kb":11816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":8821091,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":24,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.699087+0000","last_change":"2024-04-24T15:34:12.655959+0000","last_active":"2024-04-24T15:34:28.699087+0000","last_peered":"2024-04-24T15:34:28.699087+0000","last_clean":"2024-04-24T15:34:28.699087+0000","last_became_active":"2024-04-24T15:34:12.655508+0000","last_became_peered":"2024-04-24T15:34:12.655508+0000","last_unstale":"2024-04-24T15:34:28.699087+0000","last_undegraded":"2024-04-24T15:34:28.699087+0000","last_fullsized":"2024-04-24T15:34:28.699087+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_clean_scrub_stamp":"2024-04-24T15:32:06.972928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:13:50.464460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":59,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.699249+0000","last_change":"2024-04-24T15:34:19.838358+0000","last_active":"2024-04-24T15:34:28.699249+0000","last_peered":"2024-04-24T15:34:28.699249+0000","last_clean":"2024-04-24T15:34:28.699249+0000","last_became_active":"2024-04-24T15:34:19.838058+0000","last_became_peered":"2024-04-24T15:34:19.838058+0000","last_unstale":"2024-04-24T15:34:28.699249+0000","last_undegraded":"2024-04-24T15:34:28.699249+0000","last_fullsized":"2024-04-24T15:34:28.699249+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:29:43.725200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"131'430","reported_seq":1231,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.699369+0000","last_change":"2024-04-24T15:34:15.660049+0000","last_active":"2024-04-24T15:34:28.699369+0000","last_peered":"2024-04-24T15:34:28.699369+0000","last_clean":"2024-04-24T15:34:28.699369+0000","last_became_active":"2024-04-24T15:34:15.658368+0000","last_became_peered":"2024-04-24T15:34:15.658368+0000","last_unstale":"2024-04-24T15:34:28.699369+0000","last_undegraded":"2024-04-24T15:34:28.699369+0000","last_fullsized":"2024-04-24T15:34:28.699369+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:29:46.353757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015045289999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":1970,"num_read_kb":12710,"num_write":463,"num_write_kb":8069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":208,"num_bytes_recovered":107133,"num_keys_recovered":27,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":93,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861700+0000","last_change":"2024-04-24T15:34:24.428957+0000","last_active":"2024-04-24T15:34:27.861700+0000","last_peered":"2024-04-24T15:34:27.861700+0000","last_clean":"2024-04-24T15:34:27.861700+0000","last_became_active":"2024-04-24T15:34:24.428510+0000","last_became_peered":"2024-04-24T15:34:24.428510+0000","last_unstale":"2024-04-24T15:34:27.861700+0000","last_undegraded":"2024-04-24T15:34:27.861700+0000","last_fullsized":"2024-04-24T15:34:27.861700+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_clean_scrub_stamp":"2024-04-24T15:32:06.972928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:36:35.138337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698664+0000","last_change":"2024-04-24T15:34:13.689898+0000","last_active":"2024-04-24T15:34:28.698664+0000","last_peered":"2024-04-24T15:34:28.698664+0000","last_clean":"2024-04-24T15:34:28.698664+0000","last_became_active":"2024-04-24T15:34:13.689685+0000","last_became_peered":"2024-04-24T15:34:13.689685+0000","last_unstale":"2024-04-24T15:34:28.698664+0000","last_undegraded":"2024-04-24T15:34:28.698664+0000","last_fullsized":"2024-04-24T15:34:28.698664+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:24:27.251519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"130'457","reported_seq":1995,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.699625+0000","last_change":"2024-04-24T15:34:16.654083+0000","last_active":"2024-04-24T15:34:28.699625+0000","last_peered":"2024-04-24T15:34:28.699625+0000","last_clean":"2024-04-24T15:34:28.699625+0000","last_became_active":"2024-04-24T15:34:16.651906+0000","last_became_peered":"2024-04-24T15:34:16.651906+0000","last_unstale":"2024-04-24T15:34:28.699625+0000","last_undegraded":"2024-04-24T15:34:28.699625+0000","last_fullsized":"2024-04-24T15:34:28.699625+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":457,"log_dups_size":0,"ondisk_log_size":457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:54.315154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010565220000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2138,"num_read_kb":24032,"num_write":423,"num_write_kb":14900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"130'408","reported_seq":1974,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.699200+0000","last_change":"2024-04-24T15:34:16.653875+0000","last_active":"2024-04-24T15:34:28.699200+0000","last_peered":"2024-04-24T15:34:28.699200+0000","last_clean":"2024-04-24T15:34:28.699200+0000","last_became_active":"2024-04-24T15:34:16.651931+0000","last_became_peered":"2024-04-24T15:34:16.651931+0000","last_unstale":"2024-04-24T15:34:28.699200+0000","last_undegraded":"2024-04-24T15:34:28.699200+0000","last_fullsized":"2024-04-24T15:34:28.699200+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:59:24.823763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039757799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2388,"num_read_kb":22956,"num_write":379,"num_write_kb":16060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":578400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.699581+0000","last_change":"2024-04-24T15:34:12.653777+0000","last_active":"2024-04-24T15:34:28.699581+0000","last_peered":"2024-04-24T15:34:28.699581+0000","last_clean":"2024-04-24T15:34:28.699581+0000","last_became_active":"2024-04-24T15:34:12.653529+0000","last_became_peered":"2024-04-24T15:34:12.653529+0000","last_unstale":"2024-04-24T15:34:28.699581+0000","last_undegraded":"2024-04-24T15:34:28.699581+0000","last_fullsized":"2024-04-24T15:34:28.699581+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:21:58.411545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"129'479","reported_seq":1997,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698576+0000","last_change":"2024-04-24T15:34:16.656437+0000","last_active":"2024-04-24T15:34:28.698576+0000","last_peered":"2024-04-24T15:34:28.698576+0000","last_clean":"2024-04-24T15:34:28.698576+0000","last_became_active":"2024-04-24T15:34:16.654252+0000","last_became_peered":"2024-04-24T15:34:16.654252+0000","last_unstale":"2024-04-24T15:34:28.698576+0000","last_undegraded":"2024-04-24T15:34:28.698576+0000","last_fullsized":"2024-04-24T15:34:28.698576+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:53:07.507328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035787500000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":2434,"num_read_kb":14959,"num_write":545,"num_write_kb":12213,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":335,"num_bytes_recovered":6920328,"num_keys_recovered":43,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"130'616","reported_seq":2734,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861566+0000","last_change":"2024-04-24T15:34:23.827845+0000","last_active":"2024-04-24T15:34:27.861566+0000","last_peered":"2024-04-24T15:34:27.861566+0000","last_clean":"2024-04-24T15:34:27.861566+0000","last_became_active":"2024-04-24T15:34:23.825759+0000","last_became_peered":"2024-04-24T15:34:23.825759+0000","last_unstale":"2024-04-24T15:34:27.861566+0000","last_undegraded":"2024-04-24T15:34:27.861566+0000","last_fullsized":"2024-04-24T15:34:27.861566+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:49:46.424217+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0055639879999999997,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":4324,"num_read_kb":21128,"num_write":677,"num_write_kb":10933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":5713612,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"130'485","reported_seq":3231,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698742+0000","last_change":"2024-04-24T15:34:15.659323+0000","last_active":"2024-04-24T15:34:28.698742+0000","last_peered":"2024-04-24T15:34:28.698742+0000","last_clean":"2024-04-24T15:34:28.698742+0000","last_became_active":"2024-04-24T15:34:15.657660+0000","last_became_peered":"2024-04-24T15:34:15.657660+0000","last_unstale":"2024-04-24T15:34:28.698742+0000","last_undegraded":"2024-04-24T15:34:28.698742+0000","last_fullsized":"2024-04-24T15:34:28.698742+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:09:16.071953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066763,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3120,"num_read_kb":27458,"num_write":430,"num_write_kb":12267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":237,"num_bytes_recovered":5299136,"num_keys_recovered":49,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"129'551","reported_seq":1976,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698866+0000","last_change":"2024-04-24T15:34:15.669951+0000","last_active":"2024-04-24T15:34:28.698866+0000","last_peered":"2024-04-24T15:34:28.698866+0000","last_clean":"2024-04-24T15:34:28.698866+0000","last_became_active":"2024-04-24T15:34:15.668717+0000","last_became_peered":"2024-04-24T15:34:15.668717+0000","last_unstale":"2024-04-24T15:34:28.698866+0000","last_undegraded":"2024-04-24T15:34:28.698866+0000","last_fullsized":"2024-04-24T15:34:28.698866+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":551,"log_dups_size":0,"ondisk_log_size":551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:46:28.552856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039775200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2839,"num_read_kb":21248,"num_write":721,"num_write_kb":8473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":141,"num_bytes_recovered":1490474,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"127'1046","reported_seq":2505,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861716+0000","last_change":"2024-04-24T15:34:23.826944+0000","last_active":"2024-04-24T15:34:27.861716+0000","last_peered":"2024-04-24T15:34:27.861716+0000","last_clean":"2024-04-24T15:34:27.861716+0000","last_became_active":"2024-04-24T15:34:23.825039+0000","last_became_peered":"2024-04-24T15:34:23.825039+0000","last_unstale":"2024-04-24T15:34:27.861716+0000","last_undegraded":"2024-04-24T15:34:27.861716+0000","last_fullsized":"2024-04-24T15:34:27.861716+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":1046,"log_dups_size":0,"ondisk_log_size":1046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:15:05.349996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029127150000000002,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":6311,"num_read_kb":16558,"num_write":1677,"num_write_kb":12129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":54,"num_bytes_recovered":1219753,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.d","version":"127'298","reported_seq":2572,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861837+0000","last_change":"2024-04-24T15:34:23.825797+0000","last_active":"2024-04-24T15:34:27.861837+0000","last_peered":"2024-04-24T15:34:27.861837+0000","last_clean":"2024-04-24T15:34:27.861837+0000","last_became_active":"2024-04-24T15:34:23.823763+0000","last_became_peered":"2024-04-24T15:34:23.823763+0000","last_unstale":"2024-04-24T15:34:27.861837+0000","last_undegraded":"2024-04-24T15:34:27.861837+0000","last_fullsized":"2024-04-24T15:34:27.861837+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":298,"log_dups_size":0,"ondisk_log_size":298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:24:00.110009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047837299999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":2121,"num_read_kb":46483,"num_write":303,"num_write_kb":20603,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":4246835,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"129'563","reported_seq":2395,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861756+0000","last_change":"2024-04-24T15:34:23.826137+0000","last_active":"2024-04-24T15:34:27.861756+0000","last_peered":"2024-04-24T15:34:27.861756+0000","last_clean":"2024-04-24T15:34:27.861756+0000","last_became_active":"2024-04-24T15:34:23.824037+0000","last_became_peered":"2024-04-24T15:34:23.824037+0000","last_unstale":"2024-04-24T15:34:27.861756+0000","last_undegraded":"2024-04-24T15:34:27.861756+0000","last_fullsized":"2024-04-24T15:34:27.861756+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":563,"log_dups_size":0,"ondisk_log_size":563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:30:41.572188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001084025,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3022,"num_read_kb":13469,"num_write":696,"num_write_kb":6185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":113,"num_bytes_recovered":1676557,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"130'907","reported_seq":3306,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.699413+0000","last_change":"2024-04-24T15:34:16.657695+0000","last_active":"2024-04-24T15:34:28.699413+0000","last_peered":"2024-04-24T15:34:28.699413+0000","last_clean":"2024-04-24T15:34:28.699413+0000","last_became_active":"2024-04-24T15:34:16.655733+0000","last_became_peered":"2024-04-24T15:34:16.655733+0000","last_unstale":"2024-04-24T15:34:28.699413+0000","last_undegraded":"2024-04-24T15:34:28.699413+0000","last_fullsized":"2024-04-24T15:34:28.699413+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":907,"log_dups_size":0,"ondisk_log_size":907,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:59:41.606707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018480180000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":5733,"num_read_kb":14488,"num_write":1011,"num_write_kb":8995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":223,"num_bytes_recovered":630687,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"130'244","reported_seq":1061,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861808+0000","last_change":"2024-04-24T15:34:15.664032+0000","last_active":"2024-04-24T15:34:27.861808+0000","last_peered":"2024-04-24T15:34:27.861808+0000","last_clean":"2024-04-24T15:34:27.861808+0000","last_became_active":"2024-04-24T15:34:15.663081+0000","last_became_peered":"2024-04-24T15:34:15.663081+0000","last_unstale":"2024-04-24T15:34:27.861808+0000","last_undegraded":"2024-04-24T15:34:27.861808+0000","last_fullsized":"2024-04-24T15:34:27.861808+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":244,"log_dups_size":0,"ondisk_log_size":244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:33:57.743630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010945530000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":384,"num_read_kb":4092,"num_write":235,"num_write_kb":5715,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":1989443,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.11","version":"129'705","reported_seq":2162,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861883+0000","last_change":"2024-04-24T15:34:08.757586+0000","last_active":"2024-04-24T15:34:27.861883+0000","last_peered":"2024-04-24T15:34:27.861883+0000","last_clean":"2024-04-24T15:34:27.861883+0000","last_became_active":"2024-04-24T15:33:57.644309+0000","last_became_peered":"2024-04-24T15:33:57.644309+0000","last_unstale":"2024-04-24T15:34:27.861883+0000","last_undegraded":"2024-04-24T15:34:27.861883+0000","last_fullsized":"2024-04-24T15:34:27.861883+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":705,"log_dups_size":0,"ondisk_log_size":705,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:26:21.749588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012982219999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":2615,"num_read_kb":21488,"num_write":891,"num_write_kb":14014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":5651177,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"129'331","reported_seq":5228,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.699265+0000","last_change":"2024-04-24T15:34:15.655138+0000","last_active":"2024-04-24T15:34:28.699265+0000","last_peered":"2024-04-24T15:34:28.699265+0000","last_clean":"2024-04-24T15:34:28.699265+0000","last_became_active":"2024-04-24T15:34:15.653205+0000","last_became_peered":"2024-04-24T15:34:15.653205+0000","last_unstale":"2024-04-24T15:34:28.699265+0000","last_undegraded":"2024-04-24T15:34:28.699265+0000","last_fullsized":"2024-04-24T15:34:28.699265+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":331,"log_dups_size":0,"ondisk_log_size":331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:28:59.539282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013576079999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":4982,"num_read_kb":42072,"num_write":333,"num_write_kb":14625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":3758016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"129'241","reported_seq":2003,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861679+0000","last_change":"2024-04-24T15:34:23.825462+0000","last_active":"2024-04-24T15:34:27.861679+0000","last_peered":"2024-04-24T15:34:27.861679+0000","last_clean":"2024-04-24T15:34:27.861679+0000","last_became_active":"2024-04-24T15:34:23.823962+0000","last_became_peered":"2024-04-24T15:34:23.823962+0000","last_unstale":"2024-04-24T15:34:27.861679+0000","last_undegraded":"2024-04-24T15:34:27.861679+0000","last_fullsized":"2024-04-24T15:34:27.861679+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":241,"log_dups_size":0,"ondisk_log_size":241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:52:34.277987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000635438,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":1312,"num_read_kb":18878,"num_write":253,"num_write_kb":9844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":108,"num_bytes_recovered":2694373,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"131'450","reported_seq":6595,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.862055+0000","last_change":"2024-04-24T15:34:16.660466+0000","last_active":"2024-04-24T15:34:27.862055+0000","last_peered":"2024-04-24T15:34:27.862055+0000","last_clean":"2024-04-24T15:34:27.862055+0000","last_became_active":"2024-04-24T15:34:16.659071+0000","last_became_peered":"2024-04-24T15:34:16.659071+0000","last_unstale":"2024-04-24T15:34:27.862055+0000","last_undegraded":"2024-04-24T15:34:27.862055+0000","last_fullsized":"2024-04-24T15:34:27.862055+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:10:28.933829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013356920000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":7366,"num_read_kb":44142,"num_write":407,"num_write_kb":9850,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":233,"num_bytes_recovered":4458876,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"130'312","reported_seq":1575,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698823+0000","last_change":"2024-04-24T15:34:16.657117+0000","last_active":"2024-04-24T15:34:28.698823+0000","last_peered":"2024-04-24T15:34:28.698823+0000","last_clean":"2024-04-24T15:34:28.698823+0000","last_became_active":"2024-04-24T15:34:16.655084+0000","last_became_peered":"2024-04-24T15:34:16.655084+0000","last_unstale":"2024-04-24T15:34:28.698823+0000","last_undegraded":"2024-04-24T15:34:28.698823+0000","last_fullsized":"2024-04-24T15:34:28.698823+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:26:41.362344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003384743,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":837,"num_read_kb":16692,"num_write":319,"num_write_kb":15488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":277,"num_bytes_recovered":16228001,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.16","version":"129'375","reported_seq":2454,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.699026+0000","last_change":"2024-04-24T15:34:15.669825+0000","last_active":"2024-04-24T15:34:28.699026+0000","last_peered":"2024-04-24T15:34:28.699026+0000","last_clean":"2024-04-24T15:34:28.699026+0000","last_became_active":"2024-04-24T15:34:15.668678+0000","last_became_peered":"2024-04-24T15:34:15.668678+0000","last_unstale":"2024-04-24T15:34:28.699026+0000","last_undegraded":"2024-04-24T15:34:28.699026+0000","last_fullsized":"2024-04-24T15:34:28.699026+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:43:39.484677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018886770000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":3004,"num_read_kb":17777,"num_write":338,"num_write_kb":8036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":48,"num_read":114807,"num_read_kb":727542,"num_write":20290,"num_write_kb":356328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4413,"num_bytes_recovered":105175530,"num_keys_recovered":235,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15242,"internal_metadata":0},"log_size":17251,"ondisk_log_size":17251,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":8,"acting":8,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":16196,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574882,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299136,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072704,"statfs":{"total":96636764160,"available":96330448896,"internally_reserved":0,"allocated":1818624,"data_stored":1494500,"data_compressed":5677,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.46000000000000002,"15min":0.46200000000000002},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.60099999999999998,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.54200000000000004,"15min":0.55600000000000005},"min":{"1min":0.33900000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.67200000000000004,"5min":2.4710000000000001,"15min":2.4710000000000001},"last":0.73999999999999999}]},{"osd":1,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.57399999999999995,"15min":0.59299999999999997},"min":{"1min":0.28100000000000003,"5min":0.248,"15min":0.248},"max":{"1min":0.63600000000000001,"5min":2.4729999999999999,"15min":2.4729999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.51500000000000001,"15min":0.52400000000000002},"min":{"1min":0.29999999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.56899999999999995,"5min":2.4020000000000001,"15min":2.4020000000000001},"last":0.61099999999999999}]},{"osd":2,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.45700000000000002,"15min":0.45800000000000002},"min":{"1min":0.35499999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.59699999999999998,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.503,"5min":0.54800000000000004,"15min":0.55600000000000005},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.82199999999999995,"5min":2.363,"15min":2.363},"last":0.752}]},{"osd":3,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.52200000000000002,"15min":0.51800000000000002},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.53800000000000003,"15min":0.53200000000000003},"min":{"1min":0.38600000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.63600000000000001}]},{"osd":4,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.41399999999999998,"15min":0.41999999999999998},"min":{"1min":0.26100000000000001,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.66100000000000003,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.42299999999999999,"15min":0.42499999999999999},"min":{"1min":0.30599999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.73499999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.441}]},{"osd":5,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.45100000000000001,"15min":0.45500000000000002},"min":{"1min":0.307,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.51600000000000001,"15min":0.52000000000000002},"min":{"1min":0.36399999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.80500000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.67900000000000005}]},{"osd":6,"last update":"Wed Apr 24 15:32:58 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.65700000000000003}]}]},{"osd":6,"up_from":104,"seq":446676598796,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":305896,"kb_used_data":1748,"kb_used_omap":43,"kb_used_meta":304084,"kb_avail":94065944,"statfs":{"total":96636764160,"available":96323526656,"internally_reserved":0,"allocated":1789952,"data_stored":910351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":44735,"internal_metadata":311382337},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574881,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307280,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":305534,"kb_avail":94064560,"statfs":{"total":96636764160,"available":96322109440,"internally_reserved":0,"allocated":1773568,"data_stored":903176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312867160},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 15:34:01 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.35599999999999998,"15min":0.35799999999999998},"min":{"1min":0.20699999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.51100000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.377,"15min":0.376},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":1.353,"5min":1.375,"15min":1.375},"last":0.376}]},{"osd":2,"last update":"Wed Apr 24 15:34:01 2024","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.33600000000000002,"15min":0.34000000000000002},"min":{"1min":0.224,"5min":0.223,"15min":0.223},"max":{"1min":0.46100000000000002,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.255},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.41499999999999998,"15min":0.42699999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.52800000000000002,"5min":1.494,"15min":1.494},"last":0.30399999999999999}]},{"osd":3,"last update":"Wed Apr 24 15:34:01 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.42799999999999999,"15min":0.433},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.97499999999999998,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.377,"15min":0.372},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.40799999999999997}]},{"osd":4,"last update":"Wed Apr 24 15:34:01 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.53500000000000003,"15min":0.54100000000000004},"min":{"1min":0.34799999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.72199999999999998,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.495},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.57799999999999996,"15min":0.58199999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.52100000000000002}]},{"osd":5,"last update":"Wed Apr 24 15:34:01 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.379,"5min":1.454,"15min":1.454},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.53800000000000003,"15min":0.54200000000000004},"min":{"1min":0.35999999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.67100000000000004,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.56799999999999995}]},{"osd":6,"last update":"Wed Apr 24 15:33:01 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.548,"5min":1.548,"15min":1.548},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.51300000000000001}]},{"osd":7,"last update":"Wed Apr 24 15:34:01 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.56899999999999995,"15min":0.57099999999999995},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.55100000000000005,"15min":0.55700000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.69599999999999995,"5min":0.999,"15min":0.999},"last":0.53600000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574882,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303260,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94068580,"statfs":{"total":96636764160,"available":96326225920,"internally_reserved":0,"allocated":1847296,"data_stored":1500719,"data_compressed":4842,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.30499999999999999,"5min":0.45700000000000002,"15min":0.48299999999999998},"min":{"1min":0.17899999999999999,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.47999999999999998,"5min":3.7890000000000001,"15min":3.7890000000000001},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.46700000000000003,"15min":0.48899999999999999},"min":{"1min":0.224,"5min":0.191,"15min":0.191},"max":{"1min":0.59899999999999998,"5min":2.7890000000000001,"15min":2.7890000000000001},"last":0.35099999999999998}]},{"osd":2,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.53100000000000003,"15min":0.55900000000000005},"min":{"1min":0.22700000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.61399999999999999,"5min":3.9950000000000001,"15min":3.9950000000000001},"last":0.312},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.42299999999999999,"15min":0.436},"min":{"1min":0.26000000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.56799999999999995,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":0.28299999999999997}]},{"osd":3,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.48499999999999999,"15min":0.505},"min":{"1min":0.24099999999999999,"5min":0.223,"15min":0.223},"max":{"1min":0.54000000000000004,"5min":2.097,"15min":2.097},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.622,"15min":0.65700000000000003},"min":{"1min":0.30499999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.79500000000000004,"5min":3.2109999999999999,"15min":3.2109999999999999},"last":0.38800000000000001}]},{"osd":4,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.55700000000000005,"15min":0.56699999999999995},"min":{"1min":0.31,"5min":0.308,"15min":0.308},"max":{"1min":0.69099999999999995,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.65000000000000002,"15min":0.67300000000000004},"min":{"1min":0.34899999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.85599999999999998,"5min":3.282,"15min":3.282},"last":0.5}]},{"osd":5,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.60099999999999998,"15min":0.61299999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.79100000000000004,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.64600000000000002,"15min":0.66200000000000003},"min":{"1min":0.38700000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.78000000000000003,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":0.55400000000000005}]},{"osd":6,"last update":"Wed Apr 24 15:33:00 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":3.2509999999999999,"5min":3.2509999999999999,"15min":3.2509999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":2.1909999999999998,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":0.57999999999999996}]},{"osd":7,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.60299999999999998,"15min":0.60899999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.075,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.63200000000000001,"15min":0.64400000000000002},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.876,"5min":2.206,"15min":2.206},"last":0.52800000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574882,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303200,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94068640,"statfs":{"total":96636764160,"available":96326287360,"internally_reserved":0,"allocated":1785856,"data_stored":909395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 15:34:06 2024","interfaces":[{"interface":"back","average":{"1min":0.29199999999999998,"5min":0.312,"15min":0.315},"min":{"1min":0.16200000000000001,"5min":0.16200000000000001,"15min":0.16200000000000001},"max":{"1min":0.41399999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.30599999999999999,"5min":0.33700000000000002,"15min":0.34200000000000003},"min":{"1min":0.185,"5min":0.185,"15min":0.185},"max":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"last":0.442}]},{"osd":1,"last update":"Wed Apr 24 15:34:06 2024","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.36899999999999999,"15min":0.377},"min":{"1min":0.23300000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.47399999999999998,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.46100000000000002,"15min":0.48199999999999998},"min":{"1min":0.24099999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.441,"5min":3.1110000000000002,"15min":3.1110000000000002},"last":0.48999999999999999}]},{"osd":3,"last update":"Wed Apr 24 15:34:06 2024","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.40500000000000003,"15min":0.41899999999999998},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.41399999999999998,"5min":2.056,"15min":2.056},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.39900000000000002,"15min":0.40500000000000003},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.52900000000000003,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.54100000000000004}]},{"osd":4,"last update":"Wed Apr 24 15:34:06 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.51800000000000002,"15min":0.52400000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.53600000000000003,"15min":0.54500000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.70399999999999996,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.57299999999999995}]},{"osd":5,"last update":"Wed Apr 24 15:34:06 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.54300000000000004,"15min":0.54600000000000004},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.63700000000000001,"15min":0.65400000000000003},"min":{"1min":0.41399999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.80500000000000005,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":0.621}]},{"osd":6,"last update":"Wed Apr 24 15:33:03 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.1000000000000001,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.54900000000000004}]},{"osd":7,"last update":"Wed Apr 24 15:34:06 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.63500000000000001,"15min":0.65700000000000003},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.63800000000000001,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.56999999999999995,"15min":0.57799999999999996},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73999999999999999,"5min":1.502,"15min":1.502},"last":0.67400000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574882,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303204,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94068636,"statfs":{"total":96636764160,"available":96326283264,"internally_reserved":0,"allocated":1794048,"data_stored":910370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 15:34:04 2024","interfaces":[{"interface":"back","average":{"1min":0.29399999999999998,"5min":0.34599999999999997,"15min":0.35499999999999998},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.39900000000000002,"5min":1.224,"15min":1.224},"last":0.27600000000000002},{"interface":"front","average":{"1min":0.30199999999999999,"5min":0.34000000000000002,"15min":0.34599999999999997},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.41999999999999998,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.30099999999999999}]},{"osd":1,"last update":"Wed Apr 24 15:34:04 2024","interfaces":[{"interface":"back","average":{"1min":0.30299999999999999,"5min":0.33400000000000002,"15min":0.33900000000000002},"min":{"1min":0.19600000000000001,"5min":0.187,"15min":0.187},"max":{"1min":0.437,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.42199999999999999,"15min":0.434},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.71299999999999997,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.39400000000000002}]},{"osd":2,"last update":"Wed Apr 24 15:34:04 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.39400000000000002,"15min":0.40300000000000002},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.46500000000000002,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.376,"5min":0.41299999999999998,"15min":0.41899999999999998},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.53000000000000003,"5min":1.252,"15min":1.252},"last":0.44800000000000001}]},{"osd":4,"last update":"Wed Apr 24 15:34:04 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.52200000000000002,"15min":0.53400000000000003},"min":{"1min":0.28599999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.56499999999999995,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.52100000000000002,"15min":0.53300000000000003},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.60699999999999998,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.69999999999999996}]},{"osd":5,"last update":"Wed Apr 24 15:34:04 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.55600000000000005,"15min":0.56999999999999995},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.61199999999999999,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.5,"5min":0.58799999999999997,"15min":0.60199999999999998},"min":{"1min":0.38300000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.81100000000000005,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.621}]},{"osd":6,"last update":"Wed Apr 24 15:33:00 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.60199999999999998}]},{"osd":7,"last update":"Wed Apr 24 15:34:04 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.59099999999999997,"15min":0.60499999999999998},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.66800000000000004,"5min":1.635,"15min":1.635},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.495,"5min":0.55600000000000005,"15min":0.56599999999999995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.59699999999999998,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.64500000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574883,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307284,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":305534,"kb_avail":94064556,"statfs":{"total":96636764160,"available":96322105344,"internally_reserved":0,"allocated":1773568,"data_stored":901837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312867160},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 15:34:00 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.505,"15min":0.50900000000000001},"min":{"1min":0.249,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":1.8620000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.626,"5min":0.56599999999999995,"15min":0.55600000000000005},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":5.6239999999999997,"5min":5.6239999999999997,"15min":5.6239999999999997},"last":0.64100000000000001}]},{"osd":1,"last update":"Wed Apr 24 15:34:00 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.58699999999999997,"15min":0.57299999999999995},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":5.7149999999999999,"5min":5.7149999999999999,"15min":5.7149999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.35499999999999998,"5min":0.31,"15min":0.31},"max":{"1min":1.913,"5min":1.913,"15min":1.913},"last":0.53100000000000003}]},{"osd":2,"last update":"Wed Apr 24 15:34:00 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.54600000000000004,"15min":0.52800000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":5.6660000000000004,"5min":5.6660000000000004,"15min":5.6660000000000004},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.53400000000000003,"15min":0.54400000000000004},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.819,"5min":1.819,"15min":1.819},"last":0.46800000000000003}]},{"osd":3,"last update":"Wed Apr 24 15:34:00 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.53400000000000003,"15min":0.53600000000000003},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.9550000000000001,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":0.502},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.55500000000000005,"15min":0.55000000000000004},"min":{"1min":0.36799999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":2.3149999999999999,"5min":2.3149999999999999,"15min":2.3149999999999999},"last":0.55400000000000005}]},{"osd":5,"last update":"Wed Apr 24 15:34:00 2024","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.58299999999999996,"15min":0.51700000000000002},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":14.667999999999999,"5min":14.667999999999999,"15min":14.667999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.434,"5min":0.497,"15min":0.50700000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.9890000000000001,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":0.76400000000000001}]},{"osd":6,"last update":"Wed Apr 24 15:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.73399999999999999}]},{"osd":7,"last update":"Wed Apr 24 15:34:00 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.54000000000000004,"15min":0.53100000000000003},"min":{"1min":0.28599999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":5.2759999999999998,"5min":5.2759999999999998,"15min":5.2759999999999998},"last":0.71999999999999997},{"interface":"front","average":{"1min":1.0469999999999999,"5min":0.64100000000000001,"15min":0.57299999999999995},"min":{"1min":0.29399999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":14.773,"5min":14.773,"15min":14.773},"last":0.78700000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574882,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311380,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":309630,"kb_avail":94060460,"statfs":{"total":96636764160,"available":96317911040,"internally_reserved":0,"allocated":1777664,"data_stored":904132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317061464},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.47899999999999998,"15min":0.48499999999999999},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.46800000000000003,"15min":0.47599999999999998},"min":{"1min":0.246,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.629,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.55300000000000005}]},{"osd":1,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.46300000000000002,"15min":0.47099999999999997},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.64900000000000002,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.46899999999999997,"15min":0.47099999999999997},"min":{"1min":0.317,"5min":0.313,"15min":0.313},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.67300000000000004}]},{"osd":2,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.47499999999999998,"15min":0.46600000000000003},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":2.4169999999999998,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.48599999999999999,"15min":0.49199999999999999},"min":{"1min":0.30099999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.64200000000000002}]},{"osd":3,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.48899999999999999,"15min":0.495},"min":{"1min":0.308,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.70999999999999996,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":2.3370000000000002,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":0.65900000000000003}]},{"osd":4,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.44800000000000001,"15min":0.45700000000000002},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.624,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.752},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.48399999999999999,"15min":0.48599999999999999},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.59899999999999998}]},{"osd":6,"last update":"Wed Apr 24 15:33:02 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.71599999999999997}]},{"osd":7,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.52600000000000002,"15min":0.53100000000000003},"min":{"1min":0.30299999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.78600000000000003,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.496,"5min":0.53200000000000003,"15min":0.53800000000000003},"min":{"1min":0.34499999999999997,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.77100000000000002,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.76500000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4842,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":5677,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":5677,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15242,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T15:34:32.889 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-04-24T15:34:32.889 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-04-24T15:34:33.219 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-24T15:34:33.228 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2024-04-24T15:34:33.228 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T15:34:33.540 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:34:33.540 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":149,"fsid":"253d426c-62cb-4194-8514-6cd970fd85c7","created":"2024-04-24T15:31:47.870327+0000","modified":"2024-04-24T15:34:28.859605+0000","last_up_change":"2024-04-24T15:33:41.967503+0000","last_in_change":"2024-04-24T15:34:12.639389+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T15:31:59.589762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-24T15:31:59.971601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"21","last_force_op_resend_prenautilus":"21","last_force_op_resend_preluminous":"21","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[3],"tier_of":-1,"read_tier":3,"write_tier":3,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cache","create_time":"2024-04-24T15:32:06.914463+0000","flags":9,"flags_names":"hashpspool,incomplete_clones","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":4,"pg_placement_num":4,"pg_placement_num_target":4,"pg_num_target":4,"pg_num_pending":4,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"21","last_force_op_resend_prenautilus":"21","last_force_op_resend_preluminous":"21","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":2,"read_tier":-1,"write_tier":-1,"cache_mode":"writeback","target_max_bytes":0,"target_max_objects":250,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"bloom","false_positive_probability":0.050000000000000003,"target_size":0,"seed":0},"hit_set_period":60,"hit_set_count":8,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"pool_client.0","create_time":"2024-04-24T15:32:15.929114+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":69,"snap_epoch":131,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"92941fb9-96ee-4b2b-8147-3c1d61192f94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":44106},{"type":"v1","addr":"172.21.15.96:6817","nonce":44106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":44106},{"type":"v1","addr":"172.21.15.96:6819","nonce":44106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":44106},{"type":"v1","addr":"172.21.15.96:6823","nonce":44106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":44106},{"type":"v1","addr":"172.21.15.96:6821","nonce":44106}]},"public_addr":"172.21.15.96:6817/44106","cluster_addr":"172.21.15.96:6819/44106","heartbeat_back_addr":"172.21.15.96:6823/44106","heartbeat_front_addr":"172.21.15.96:6821/44106","state":["exists","up"]},{"osd":1,"uuid":"628e868a-9884-42e6-9331-caef97cb6967","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":44089},{"type":"v1","addr":"172.21.15.96:6801","nonce":44089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":44089},{"type":"v1","addr":"172.21.15.96:6803","nonce":44089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":44089},{"type":"v1","addr":"172.21.15.96:6807","nonce":44089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":44089},{"type":"v1","addr":"172.21.15.96:6805","nonce":44089}]},"public_addr":"172.21.15.96:6801/44089","cluster_addr":"172.21.15.96:6803/44089","heartbeat_back_addr":"172.21.15.96:6807/44089","heartbeat_front_addr":"172.21.15.96:6805/44089","state":["exists","up"]},{"osd":2,"uuid":"46de3f65-87a8-460c-b57a-4468d9c89a57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":44090},{"type":"v1","addr":"172.21.15.96:6809","nonce":44090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":44090},{"type":"v1","addr":"172.21.15.96:6811","nonce":44090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":44090},{"type":"v1","addr":"172.21.15.96:6815","nonce":44090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":44090},{"type":"v1","addr":"172.21.15.96:6813","nonce":44090}]},"public_addr":"172.21.15.96:6809/44090","cluster_addr":"172.21.15.96:6811/44090","heartbeat_back_addr":"172.21.15.96:6815/44090","heartbeat_front_addr":"172.21.15.96:6813/44090","state":["exists","up"]},{"osd":3,"uuid":"5507e063-10bc-4e32-b1ec-00aa3e47d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":44121},{"type":"v1","addr":"172.21.15.96:6825","nonce":44121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":44121},{"type":"v1","addr":"172.21.15.96:6827","nonce":44121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":44121},{"type":"v1","addr":"172.21.15.96:6831","nonce":44121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":44121},{"type":"v1","addr":"172.21.15.96:6829","nonce":44121}]},"public_addr":"172.21.15.96:6825/44121","cluster_addr":"172.21.15.96:6827/44121","heartbeat_back_addr":"172.21.15.96:6831/44121","heartbeat_front_addr":"172.21.15.96:6829/44121","state":["exists","up"]},{"osd":4,"uuid":"6a74d453-1048-44ef-a5b0-4277583d76c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":44025},{"type":"v1","addr":"172.21.15.151:6817","nonce":44025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":44025},{"type":"v1","addr":"172.21.15.151:6819","nonce":44025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":44025},{"type":"v1","addr":"172.21.15.151:6823","nonce":44025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":44025},{"type":"v1","addr":"172.21.15.151:6821","nonce":44025}]},"public_addr":"172.21.15.151:6817/44025","cluster_addr":"172.21.15.151:6819/44025","heartbeat_back_addr":"172.21.15.151:6823/44025","heartbeat_front_addr":"172.21.15.151:6821/44025","state":["exists","up"]},{"osd":5,"uuid":"3645ef18-5574-49f8-ab81-b40651208309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":44023},{"type":"v1","addr":"172.21.15.151:6801","nonce":44023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":44023},{"type":"v1","addr":"172.21.15.151:6803","nonce":44023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":44023},{"type":"v1","addr":"172.21.15.151:6807","nonce":44023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":44023},{"type":"v1","addr":"172.21.15.151:6805","nonce":44023}]},"public_addr":"172.21.15.151:6801/44023","cluster_addr":"172.21.15.151:6803/44023","heartbeat_back_addr":"172.21.15.151:6807/44023","heartbeat_front_addr":"172.21.15.151:6805/44023","state":["exists","up"]},{"osd":6,"uuid":"6b8b0b36-9246-4241-a197-c69fa48a6202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":75,"up_from":104,"up_thru":135,"down_at":76,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":59996},{"type":"v1","addr":"172.21.15.151:6825","nonce":59996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":59996},{"type":"v1","addr":"172.21.15.151:6827","nonce":59996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":59996},{"type":"v1","addr":"172.21.15.151:6831","nonce":59996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":59996},{"type":"v1","addr":"172.21.15.151:6829","nonce":59996}]},"public_addr":"172.21.15.151:6825/59996","cluster_addr":"172.21.15.151:6827/59996","heartbeat_back_addr":"172.21.15.151:6831/59996","heartbeat_front_addr":"172.21.15.151:6829/59996","state":["exists","up"]},{"osd":7,"uuid":"4501cd9f-e437-4ddd-af3d-129fdf1ee61a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":44024},{"type":"v1","addr":"172.21.15.151:6809","nonce":44024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":44024},{"type":"v1","addr":"172.21.15.151:6811","nonce":44024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":44024},{"type":"v1","addr":"172.21.15.151:6815","nonce":44024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":44024},{"type":"v1","addr":"172.21.15.151:6813","nonce":44024}]},"public_addr":"172.21.15.151:6809/44024","cluster_addr":"172.21.15.151:6811/44024","heartbeat_back_addr":"172.21.15.151:6815/44024","heartbeat_front_addr":"172.21.15.151:6813/44024","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T15:31:56.300910+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T15:31:56.202064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T15:31:56.198961+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T15:31:56.346777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T15:31:56.282585+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T15:31:56.257943+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T15:31:56.260479+0000","dead_epoch":76},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T15:31:56.242093+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.0","mappings":[{"from":3,"to":2}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T15:34:33.554 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-24T15:34:33.554 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T15:34:33.867 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:34:33.868 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-04-24T15:34:33.879 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":220,"stamp":"2024-04-24T15:34:33.580033+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":51,"num_read":114923,"num_read_kb":727642,"num_write":20449,"num_write_kb":358744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4420,"num_bytes_recovered":106356266,"num_keys_recovered":235,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17371,"ondisk_log_size":17371,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2440640,"kb_used_data":14024,"kb_used_omap":55,"kb_used_meta":2426440,"kb_avail":752534080,"statfs":{"total":773094113280,"available":770594897920,"internally_reserved":0,"allocated":14360576,"data_stored":8434480,"data_compressed":10519,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":56663,"internal_metadata":2484675241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.729940"},"pg_stats":[{"pgid":"4.1f","version":"131'1485","reported_seq":2550,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698777+0000","last_change":"2024-04-24T15:34:15.652525+0000","last_active":"2024-04-24T15:34:28.698777+0000","last_peered":"2024-04-24T15:34:28.698777+0000","last_clean":"2024-04-24T15:34:28.698777+0000","last_became_active":"2024-04-24T15:34:15.650850+0000","last_became_peered":"2024-04-24T15:34:15.650850+0000","last_unstale":"2024-04-24T15:34:28.698777+0000","last_undegraded":"2024-04-24T15:34:28.698777+0000","last_fullsized":"2024-04-24T15:34:28.698777+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":1485,"log_dups_size":0,"ondisk_log_size":1485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:06:34.577605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001749947,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":8143,"num_read_kb":14722,"num_write":2575,"num_write_kb":16077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":5952278,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"129'507","reported_seq":6418,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698830+0000","last_change":"2024-04-24T15:34:15.669801+0000","last_active":"2024-04-24T15:34:28.698830+0000","last_peered":"2024-04-24T15:34:28.698830+0000","last_clean":"2024-04-24T15:34:28.698830+0000","last_became_active":"2024-04-24T15:34:15.668657+0000","last_became_peered":"2024-04-24T15:34:15.668657+0000","last_unstale":"2024-04-24T15:34:28.698830+0000","last_undegraded":"2024-04-24T15:34:28.698830+0000","last_fullsized":"2024-04-24T15:34:28.698830+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":507,"log_dups_size":0,"ondisk_log_size":507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:32:57.253624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029386599999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":7278,"num_read_kb":39113,"num_write":560,"num_write_kb":6592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":4495932,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"129'529","reported_seq":3745,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861949+0000","last_change":"2024-04-24T15:34:16.660701+0000","last_active":"2024-04-24T15:34:27.861949+0000","last_peered":"2024-04-24T15:34:27.861949+0000","last_clean":"2024-04-24T15:34:27.861949+0000","last_became_active":"2024-04-24T15:34:16.659202+0000","last_became_peered":"2024-04-24T15:34:16.659202+0000","last_unstale":"2024-04-24T15:34:27.861949+0000","last_undegraded":"2024-04-24T15:34:27.861949+0000","last_fullsized":"2024-04-24T15:34:27.861949+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":529,"log_dups_size":0,"ondisk_log_size":529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:31:57.762122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015488839999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":4427,"num_read_kb":51896,"num_write":603,"num_write_kb":18759,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":580671,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"131'526","reported_seq":2700,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.699161+0000","last_change":"2024-04-24T15:34:19.807684+0000","last_active":"2024-04-24T15:34:28.699161+0000","last_peered":"2024-04-24T15:34:28.699161+0000","last_clean":"2024-04-24T15:34:28.699161+0000","last_became_active":"2024-04-24T15:34:19.806783+0000","last_became_peered":"2024-04-24T15:34:19.806783+0000","last_unstale":"2024-04-24T15:34:28.699161+0000","last_undegraded":"2024-04-24T15:34:28.699161+0000","last_fullsized":"2024-04-24T15:34:28.699161+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":526,"log_dups_size":0,"ondisk_log_size":526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:06:01.349645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001413493,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":3716,"num_read_kb":10078,"num_write":547,"num_write_kb":2888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"129'385","reported_seq":2359,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861581+0000","last_change":"2024-04-24T15:34:23.827263+0000","last_active":"2024-04-24T15:34:27.861581+0000","last_peered":"2024-04-24T15:34:27.861581+0000","last_clean":"2024-04-24T15:34:27.861581+0000","last_became_active":"2024-04-24T15:34:23.825472+0000","last_became_peered":"2024-04-24T15:34:23.825472+0000","last_unstale":"2024-04-24T15:34:27.861581+0000","last_undegraded":"2024-04-24T15:34:27.861581+0000","last_fullsized":"2024-04-24T15:34:27.861581+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":385,"log_dups_size":0,"ondisk_log_size":385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:19:37.470876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011107479999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":2335,"num_read_kb":25405,"num_write":329,"num_write_kb":6118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":79,"num_bytes_recovered":2143880,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"129'449","reported_seq":6610,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861569+0000","last_change":"2024-04-24T15:34:23.827554+0000","last_active":"2024-04-24T15:34:27.861569+0000","last_peered":"2024-04-24T15:34:27.861569+0000","last_clean":"2024-04-24T15:34:27.861569+0000","last_became_active":"2024-04-24T15:34:23.824861+0000","last_became_peered":"2024-04-24T15:34:23.824861+0000","last_unstale":"2024-04-24T15:34:27.861569+0000","last_undegraded":"2024-04-24T15:34:27.861569+0000","last_fullsized":"2024-04-24T15:34:27.861569+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:42:20.554334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097376600000000004,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":6368,"num_read_kb":19805,"num_write":475,"num_write_kb":10193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":89,"num_bytes_recovered":1137673,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"129'537","reported_seq":2680,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861805+0000","last_change":"2024-04-24T15:34:08.756894+0000","last_active":"2024-04-24T15:34:27.861805+0000","last_peered":"2024-04-24T15:34:27.861805+0000","last_clean":"2024-04-24T15:34:27.861805+0000","last_became_active":"2024-04-24T15:32:17.047842+0000","last_became_peered":"2024-04-24T15:32:17.047842+0000","last_unstale":"2024-04-24T15:34:27.861805+0000","last_undegraded":"2024-04-24T15:34:27.861805+0000","last_fullsized":"2024-04-24T15:34:27.861805+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":537,"log_dups_size":0,"ondisk_log_size":537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:09:52.065772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081826100000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":4119,"num_read_kb":22694,"num_write":527,"num_write_kb":11193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"130'444","reported_seq":1968,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861537+0000","last_change":"2024-04-24T15:34:08.758067+0000","last_active":"2024-04-24T15:34:27.861537+0000","last_peered":"2024-04-24T15:34:27.861537+0000","last_clean":"2024-04-24T15:34:27.861537+0000","last_became_active":"2024-04-24T15:33:42.641679+0000","last_became_peered":"2024-04-24T15:33:42.641679+0000","last_unstale":"2024-04-24T15:34:27.861537+0000","last_undegraded":"2024-04-24T15:34:27.861537+0000","last_fullsized":"2024-04-24T15:34:27.861537+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":444,"log_dups_size":0,"ondisk_log_size":444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:29:58.781375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077716999999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":1887,"num_read_kb":15406,"num_write":547,"num_write_kb":8155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":4099152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"129'523","reported_seq":2447,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861936+0000","last_change":"2024-04-24T15:34:08.758924+0000","last_active":"2024-04-24T15:34:27.861936+0000","last_peered":"2024-04-24T15:34:27.861936+0000","last_clean":"2024-04-24T15:34:27.861936+0000","last_became_active":"2024-04-24T15:33:56.699629+0000","last_became_peered":"2024-04-24T15:33:56.699629+0000","last_unstale":"2024-04-24T15:34:27.861936+0000","last_undegraded":"2024-04-24T15:34:27.861936+0000","last_fullsized":"2024-04-24T15:34:27.861936+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":523,"log_dups_size":0,"ondisk_log_size":523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:53:30.709042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00212766,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":3353,"num_read_kb":16260,"num_write":594,"num_write_kb":12753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":352256,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"129'316","reported_seq":2577,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861623+0000","last_change":"2024-04-24T15:34:23.829028+0000","last_active":"2024-04-24T15:34:27.861623+0000","last_peered":"2024-04-24T15:34:27.861623+0000","last_clean":"2024-04-24T15:34:27.861623+0000","last_became_active":"2024-04-24T15:34:23.827917+0000","last_became_peered":"2024-04-24T15:34:23.827917+0000","last_unstale":"2024-04-24T15:34:27.861623+0000","last_undegraded":"2024-04-24T15:34:27.861623+0000","last_fullsized":"2024-04-24T15:34:27.861623+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":316,"log_dups_size":0,"ondisk_log_size":316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:11:49.794260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065534800000000004,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":1633,"num_read_kb":26736,"num_write":303,"num_write_kb":13822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":9304722,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":121,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698692+0000","last_change":"2024-04-24T15:33:53.644249+0000","last_active":"2024-04-24T15:34:28.698692+0000","last_peered":"2024-04-24T15:34:28.698692+0000","last_clean":"2024-04-24T15:34:28.698692+0000","last_became_active":"2024-04-24T15:33:53.643941+0000","last_became_peered":"2024-04-24T15:33:53.643941+0000","last_unstale":"2024-04-24T15:34:28.698692+0000","last_undegraded":"2024-04-24T15:34:28.698692+0000","last_fullsized":"2024-04-24T15:34:28.698692+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:42:05.771051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"130'862","reported_seq":3009,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698843+0000","last_change":"2024-04-24T15:34:19.810827+0000","last_active":"2024-04-24T15:34:28.698843+0000","last_peered":"2024-04-24T15:34:28.698843+0000","last_clean":"2024-04-24T15:34:28.698843+0000","last_became_active":"2024-04-24T15:34:19.809619+0000","last_became_peered":"2024-04-24T15:34:19.809619+0000","last_unstale":"2024-04-24T15:34:28.698843+0000","last_undegraded":"2024-04-24T15:34:28.698843+0000","last_fullsized":"2024-04-24T15:34:28.698843+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:42:29.391778+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00033353700000000002,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":4689,"num_read_kb":23870,"num_write":1044,"num_write_kb":13092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":176,"num_bytes_recovered":5601476,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":24,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.699509+0000","last_change":"2024-04-24T15:34:12.652580+0000","last_active":"2024-04-24T15:34:28.699509+0000","last_peered":"2024-04-24T15:34:28.699509+0000","last_clean":"2024-04-24T15:34:28.699509+0000","last_became_active":"2024-04-24T15:34:12.652298+0000","last_became_peered":"2024-04-24T15:34:12.652298+0000","last_unstale":"2024-04-24T15:34:28.699509+0000","last_undegraded":"2024-04-24T15:34:28.699509+0000","last_fullsized":"2024-04-24T15:34:28.699509+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:22:59.160275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":41,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.862035+0000","last_change":"2024-04-24T15:33:53.645875+0000","last_active":"2024-04-24T15:34:27.862035+0000","last_peered":"2024-04-24T15:34:27.862035+0000","last_clean":"2024-04-24T15:34:27.862035+0000","last_became_active":"2024-04-24T15:33:53.645696+0000","last_became_peered":"2024-04-24T15:33:53.645696+0000","last_unstale":"2024-04-24T15:34:27.862035+0000","last_undegraded":"2024-04-24T15:34:27.862035+0000","last_fullsized":"2024-04-24T15:34:27.862035+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:29:35.345558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"131'584","reported_seq":5172,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698983+0000","last_change":"2024-04-24T15:34:17.655614+0000","last_active":"2024-04-24T15:34:28.698983+0000","last_peered":"2024-04-24T15:34:28.698983+0000","last_clean":"2024-04-24T15:34:28.698983+0000","last_became_active":"2024-04-24T15:34:17.654759+0000","last_became_peered":"2024-04-24T15:34:17.654759+0000","last_unstale":"2024-04-24T15:34:28.698983+0000","last_undegraded":"2024-04-24T15:34:28.698983+0000","last_fullsized":"2024-04-24T15:34:28.698983+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":584,"log_dups_size":0,"ondisk_log_size":584,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:42:40.438352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061779100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3592,"num_read_kb":14899,"num_write":598,"num_write_kb":11359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":23598,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"60'1","reported_seq":80,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698947+0000","last_change":"2024-04-24T15:33:47.663123+0000","last_active":"2024-04-24T15:34:28.698947+0000","last_peered":"2024-04-24T15:34:28.698947+0000","last_clean":"2024-04-24T15:34:28.698947+0000","last_became_active":"2024-04-24T15:33:47.662685+0000","last_became_peered":"2024-04-24T15:33:47.662685+0000","last_unstale":"2024-04-24T15:34:28.698947+0000","last_undegraded":"2024-04-24T15:34:28.698947+0000","last_fullsized":"2024-04-24T15:34:28.698947+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_clean_scrub_stamp":"2024-04-24T15:32:06.972928+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:05:35.654980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":50,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861972+0000","last_change":"2024-04-24T15:34:12.653126+0000","last_active":"2024-04-24T15:34:27.861972+0000","last_peered":"2024-04-24T15:34:27.861972+0000","last_clean":"2024-04-24T15:34:27.861972+0000","last_became_active":"2024-04-24T15:34:12.652805+0000","last_became_peered":"2024-04-24T15:34:12.652805+0000","last_unstale":"2024-04-24T15:34:27.861972+0000","last_undegraded":"2024-04-24T15:34:27.861972+0000","last_fullsized":"2024-04-24T15:34:27.861972+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:44:23.434720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"131'615","reported_seq":2369,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698926+0000","last_change":"2024-04-24T15:34:16.760282+0000","last_active":"2024-04-24T15:34:28.698926+0000","last_peered":"2024-04-24T15:34:28.698926+0000","last_clean":"2024-04-24T15:34:28.698926+0000","last_became_active":"2024-04-24T15:34:16.655390+0000","last_became_peered":"2024-04-24T15:34:16.655390+0000","last_unstale":"2024-04-24T15:34:28.698926+0000","last_undegraded":"2024-04-24T15:34:28.698926+0000","last_fullsized":"2024-04-24T15:34:28.698926+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":615,"log_dups_size":0,"ondisk_log_size":615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:01:32.241481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042917399999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":3015,"num_read_kb":19823,"num_write":696,"num_write_kb":5112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"90'2","reported_seq":182,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861981+0000","last_change":"2024-04-24T15:34:13.666629+0000","last_active":"2024-04-24T15:34:27.861981+0000","last_peered":"2024-04-24T15:34:27.861981+0000","last_clean":"2024-04-24T15:34:27.861981+0000","last_became_active":"2024-04-24T15:34:13.662363+0000","last_became_peered":"2024-04-24T15:34:13.662363+0000","last_unstale":"2024-04-24T15:34:27.861981+0000","last_undegraded":"2024-04-24T15:34:27.861981+0000","last_fullsized":"2024-04-24T15:34:27.861981+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_clean_scrub_stamp":"2024-04-24T15:32:06.972928+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:09:38.698619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":251,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.862026+0000","last_change":"2024-04-24T15:33:57.644653+0000","last_active":"2024-04-24T15:34:27.862026+0000","last_peered":"2024-04-24T15:34:27.862026+0000","last_clean":"2024-04-24T15:34:27.862026+0000","last_became_active":"2024-04-24T15:33:57.644312+0000","last_became_peered":"2024-04-24T15:33:57.644312+0000","last_unstale":"2024-04-24T15:34:27.862026+0000","last_undegraded":"2024-04-24T15:34:27.862026+0000","last_fullsized":"2024-04-24T15:34:27.862026+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:31:59.935287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:31:59.935287+0000","last_clean_scrub_stamp":"2024-04-24T15:31:59.935287+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:21:46.534664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698724+0000","last_change":"2024-04-24T15:34:13.650533+0000","last_active":"2024-04-24T15:34:28.698724+0000","last_peered":"2024-04-24T15:34:28.698724+0000","last_clean":"2024-04-24T15:34:28.698724+0000","last_became_active":"2024-04-24T15:34:13.650174+0000","last_became_peered":"2024-04-24T15:34:13.650174+0000","last_unstale":"2024-04-24T15:34:28.698724+0000","last_undegraded":"2024-04-24T15:34:28.698724+0000","last_fullsized":"2024-04-24T15:34:28.698724+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:01:43.442578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"129'591","reported_seq":2456,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698898+0000","last_change":"2024-04-24T15:34:08.805854+0000","last_active":"2024-04-24T15:34:28.698898+0000","last_peered":"2024-04-24T15:34:28.698898+0000","last_clean":"2024-04-24T15:34:28.698898+0000","last_became_active":"2024-04-24T15:33:45.658207+0000","last_became_peered":"2024-04-24T15:33:45.658207+0000","last_unstale":"2024-04-24T15:34:28.698898+0000","last_undegraded":"2024-04-24T15:34:28.698898+0000","last_fullsized":"2024-04-24T15:34:28.698898+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":591,"log_dups_size":0,"ondisk_log_size":591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:59:01.456518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010111199999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":3352,"num_read_kb":26205,"num_write":791,"num_write_kb":11816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":8821091,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":24,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.699087+0000","last_change":"2024-04-24T15:34:12.655959+0000","last_active":"2024-04-24T15:34:28.699087+0000","last_peered":"2024-04-24T15:34:28.699087+0000","last_clean":"2024-04-24T15:34:28.699087+0000","last_became_active":"2024-04-24T15:34:12.655508+0000","last_became_peered":"2024-04-24T15:34:12.655508+0000","last_unstale":"2024-04-24T15:34:28.699087+0000","last_undegraded":"2024-04-24T15:34:28.699087+0000","last_fullsized":"2024-04-24T15:34:28.699087+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_clean_scrub_stamp":"2024-04-24T15:32:06.972928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:13:50.464460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":59,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.699249+0000","last_change":"2024-04-24T15:34:19.838358+0000","last_active":"2024-04-24T15:34:28.699249+0000","last_peered":"2024-04-24T15:34:28.699249+0000","last_clean":"2024-04-24T15:34:28.699249+0000","last_became_active":"2024-04-24T15:34:19.838058+0000","last_became_peered":"2024-04-24T15:34:19.838058+0000","last_unstale":"2024-04-24T15:34:28.699249+0000","last_undegraded":"2024-04-24T15:34:28.699249+0000","last_fullsized":"2024-04-24T15:34:28.699249+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:29:43.725200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"131'430","reported_seq":1231,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.699369+0000","last_change":"2024-04-24T15:34:15.660049+0000","last_active":"2024-04-24T15:34:28.699369+0000","last_peered":"2024-04-24T15:34:28.699369+0000","last_clean":"2024-04-24T15:34:28.699369+0000","last_became_active":"2024-04-24T15:34:15.658368+0000","last_became_peered":"2024-04-24T15:34:15.658368+0000","last_unstale":"2024-04-24T15:34:28.699369+0000","last_undegraded":"2024-04-24T15:34:28.699369+0000","last_fullsized":"2024-04-24T15:34:28.699369+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:29:46.353757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015045289999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":1970,"num_read_kb":12710,"num_write":463,"num_write_kb":8069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":208,"num_bytes_recovered":107133,"num_keys_recovered":27,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":93,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861700+0000","last_change":"2024-04-24T15:34:24.428957+0000","last_active":"2024-04-24T15:34:27.861700+0000","last_peered":"2024-04-24T15:34:27.861700+0000","last_clean":"2024-04-24T15:34:27.861700+0000","last_became_active":"2024-04-24T15:34:24.428510+0000","last_became_peered":"2024-04-24T15:34:24.428510+0000","last_unstale":"2024-04-24T15:34:27.861700+0000","last_undegraded":"2024-04-24T15:34:27.861700+0000","last_fullsized":"2024-04-24T15:34:27.861700+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_clean_scrub_stamp":"2024-04-24T15:32:06.972928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:36:35.138337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698664+0000","last_change":"2024-04-24T15:34:13.689898+0000","last_active":"2024-04-24T15:34:28.698664+0000","last_peered":"2024-04-24T15:34:28.698664+0000","last_clean":"2024-04-24T15:34:28.698664+0000","last_became_active":"2024-04-24T15:34:13.689685+0000","last_became_peered":"2024-04-24T15:34:13.689685+0000","last_unstale":"2024-04-24T15:34:28.698664+0000","last_undegraded":"2024-04-24T15:34:28.698664+0000","last_fullsized":"2024-04-24T15:34:28.698664+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:24:27.251519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"130'457","reported_seq":1995,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.699625+0000","last_change":"2024-04-24T15:34:16.654083+0000","last_active":"2024-04-24T15:34:28.699625+0000","last_peered":"2024-04-24T15:34:28.699625+0000","last_clean":"2024-04-24T15:34:28.699625+0000","last_became_active":"2024-04-24T15:34:16.651906+0000","last_became_peered":"2024-04-24T15:34:16.651906+0000","last_unstale":"2024-04-24T15:34:28.699625+0000","last_undegraded":"2024-04-24T15:34:28.699625+0000","last_fullsized":"2024-04-24T15:34:28.699625+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":457,"log_dups_size":0,"ondisk_log_size":457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:54.315154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010565220000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2138,"num_read_kb":24032,"num_write":423,"num_write_kb":14900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"130'408","reported_seq":1974,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.699200+0000","last_change":"2024-04-24T15:34:16.653875+0000","last_active":"2024-04-24T15:34:28.699200+0000","last_peered":"2024-04-24T15:34:28.699200+0000","last_clean":"2024-04-24T15:34:28.699200+0000","last_became_active":"2024-04-24T15:34:16.651931+0000","last_became_peered":"2024-04-24T15:34:16.651931+0000","last_unstale":"2024-04-24T15:34:28.699200+0000","last_undegraded":"2024-04-24T15:34:28.699200+0000","last_fullsized":"2024-04-24T15:34:28.699200+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:59:24.823763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039757799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2388,"num_read_kb":22956,"num_write":379,"num_write_kb":16060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":578400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.699581+0000","last_change":"2024-04-24T15:34:12.653777+0000","last_active":"2024-04-24T15:34:28.699581+0000","last_peered":"2024-04-24T15:34:28.699581+0000","last_clean":"2024-04-24T15:34:28.699581+0000","last_became_active":"2024-04-24T15:34:12.653529+0000","last_became_peered":"2024-04-24T15:34:12.653529+0000","last_unstale":"2024-04-24T15:34:28.699581+0000","last_undegraded":"2024-04-24T15:34:28.699581+0000","last_fullsized":"2024-04-24T15:34:28.699581+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:21:58.411545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"129'479","reported_seq":1997,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698576+0000","last_change":"2024-04-24T15:34:16.656437+0000","last_active":"2024-04-24T15:34:28.698576+0000","last_peered":"2024-04-24T15:34:28.698576+0000","last_clean":"2024-04-24T15:34:28.698576+0000","last_became_active":"2024-04-24T15:34:16.654252+0000","last_became_peered":"2024-04-24T15:34:16.654252+0000","last_unstale":"2024-04-24T15:34:28.698576+0000","last_undegraded":"2024-04-24T15:34:28.698576+0000","last_fullsized":"2024-04-24T15:34:28.698576+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:53:07.507328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035787500000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":2434,"num_read_kb":14959,"num_write":545,"num_write_kb":12213,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":335,"num_bytes_recovered":6920328,"num_keys_recovered":43,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"130'616","reported_seq":2734,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861566+0000","last_change":"2024-04-24T15:34:23.827845+0000","last_active":"2024-04-24T15:34:27.861566+0000","last_peered":"2024-04-24T15:34:27.861566+0000","last_clean":"2024-04-24T15:34:27.861566+0000","last_became_active":"2024-04-24T15:34:23.825759+0000","last_became_peered":"2024-04-24T15:34:23.825759+0000","last_unstale":"2024-04-24T15:34:27.861566+0000","last_undegraded":"2024-04-24T15:34:27.861566+0000","last_fullsized":"2024-04-24T15:34:27.861566+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:49:46.424217+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0055639879999999997,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":4324,"num_read_kb":21128,"num_write":677,"num_write_kb":10933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":5713612,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"130'485","reported_seq":3231,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698742+0000","last_change":"2024-04-24T15:34:15.659323+0000","last_active":"2024-04-24T15:34:28.698742+0000","last_peered":"2024-04-24T15:34:28.698742+0000","last_clean":"2024-04-24T15:34:28.698742+0000","last_became_active":"2024-04-24T15:34:15.657660+0000","last_became_peered":"2024-04-24T15:34:15.657660+0000","last_unstale":"2024-04-24T15:34:28.698742+0000","last_undegraded":"2024-04-24T15:34:28.698742+0000","last_fullsized":"2024-04-24T15:34:28.698742+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:09:16.071953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066763,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3120,"num_read_kb":27458,"num_write":430,"num_write_kb":12267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":237,"num_bytes_recovered":5299136,"num_keys_recovered":49,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"129'551","reported_seq":1976,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698866+0000","last_change":"2024-04-24T15:34:15.669951+0000","last_active":"2024-04-24T15:34:28.698866+0000","last_peered":"2024-04-24T15:34:28.698866+0000","last_clean":"2024-04-24T15:34:28.698866+0000","last_became_active":"2024-04-24T15:34:15.668717+0000","last_became_peered":"2024-04-24T15:34:15.668717+0000","last_unstale":"2024-04-24T15:34:28.698866+0000","last_undegraded":"2024-04-24T15:34:28.698866+0000","last_fullsized":"2024-04-24T15:34:28.698866+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":551,"log_dups_size":0,"ondisk_log_size":551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:46:28.552856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039775200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2839,"num_read_kb":21248,"num_write":721,"num_write_kb":8473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":141,"num_bytes_recovered":1490474,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"127'1046","reported_seq":2505,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861716+0000","last_change":"2024-04-24T15:34:23.826944+0000","last_active":"2024-04-24T15:34:27.861716+0000","last_peered":"2024-04-24T15:34:27.861716+0000","last_clean":"2024-04-24T15:34:27.861716+0000","last_became_active":"2024-04-24T15:34:23.825039+0000","last_became_peered":"2024-04-24T15:34:23.825039+0000","last_unstale":"2024-04-24T15:34:27.861716+0000","last_undegraded":"2024-04-24T15:34:27.861716+0000","last_fullsized":"2024-04-24T15:34:27.861716+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":1046,"log_dups_size":0,"ondisk_log_size":1046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:15:05.349996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029127150000000002,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":6311,"num_read_kb":16558,"num_write":1677,"num_write_kb":12129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":54,"num_bytes_recovered":1219753,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.d","version":"127'298","reported_seq":2572,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861837+0000","last_change":"2024-04-24T15:34:23.825797+0000","last_active":"2024-04-24T15:34:27.861837+0000","last_peered":"2024-04-24T15:34:27.861837+0000","last_clean":"2024-04-24T15:34:27.861837+0000","last_became_active":"2024-04-24T15:34:23.823763+0000","last_became_peered":"2024-04-24T15:34:23.823763+0000","last_unstale":"2024-04-24T15:34:27.861837+0000","last_undegraded":"2024-04-24T15:34:27.861837+0000","last_fullsized":"2024-04-24T15:34:27.861837+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":298,"log_dups_size":0,"ondisk_log_size":298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:24:00.110009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047837299999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":2121,"num_read_kb":46483,"num_write":303,"num_write_kb":20603,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":4246835,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"129'563","reported_seq":2395,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861756+0000","last_change":"2024-04-24T15:34:23.826137+0000","last_active":"2024-04-24T15:34:27.861756+0000","last_peered":"2024-04-24T15:34:27.861756+0000","last_clean":"2024-04-24T15:34:27.861756+0000","last_became_active":"2024-04-24T15:34:23.824037+0000","last_became_peered":"2024-04-24T15:34:23.824037+0000","last_unstale":"2024-04-24T15:34:27.861756+0000","last_undegraded":"2024-04-24T15:34:27.861756+0000","last_fullsized":"2024-04-24T15:34:27.861756+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":563,"log_dups_size":0,"ondisk_log_size":563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:30:41.572188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001084025,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3022,"num_read_kb":13469,"num_write":696,"num_write_kb":6185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":113,"num_bytes_recovered":1676557,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"130'907","reported_seq":3306,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.699413+0000","last_change":"2024-04-24T15:34:16.657695+0000","last_active":"2024-04-24T15:34:28.699413+0000","last_peered":"2024-04-24T15:34:28.699413+0000","last_clean":"2024-04-24T15:34:28.699413+0000","last_became_active":"2024-04-24T15:34:16.655733+0000","last_became_peered":"2024-04-24T15:34:16.655733+0000","last_unstale":"2024-04-24T15:34:28.699413+0000","last_undegraded":"2024-04-24T15:34:28.699413+0000","last_fullsized":"2024-04-24T15:34:28.699413+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":907,"log_dups_size":0,"ondisk_log_size":907,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:59:41.606707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018480180000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":5733,"num_read_kb":14488,"num_write":1011,"num_write_kb":8995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":223,"num_bytes_recovered":630687,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"130'244","reported_seq":1061,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861808+0000","last_change":"2024-04-24T15:34:15.664032+0000","last_active":"2024-04-24T15:34:27.861808+0000","last_peered":"2024-04-24T15:34:27.861808+0000","last_clean":"2024-04-24T15:34:27.861808+0000","last_became_active":"2024-04-24T15:34:15.663081+0000","last_became_peered":"2024-04-24T15:34:15.663081+0000","last_unstale":"2024-04-24T15:34:27.861808+0000","last_undegraded":"2024-04-24T15:34:27.861808+0000","last_fullsized":"2024-04-24T15:34:27.861808+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":244,"log_dups_size":0,"ondisk_log_size":244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:33:57.743630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010945530000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":384,"num_read_kb":4092,"num_write":235,"num_write_kb":5715,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":1989443,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.11","version":"129'705","reported_seq":2162,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861883+0000","last_change":"2024-04-24T15:34:08.757586+0000","last_active":"2024-04-24T15:34:27.861883+0000","last_peered":"2024-04-24T15:34:27.861883+0000","last_clean":"2024-04-24T15:34:27.861883+0000","last_became_active":"2024-04-24T15:33:57.644309+0000","last_became_peered":"2024-04-24T15:33:57.644309+0000","last_unstale":"2024-04-24T15:34:27.861883+0000","last_undegraded":"2024-04-24T15:34:27.861883+0000","last_fullsized":"2024-04-24T15:34:27.861883+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":705,"log_dups_size":0,"ondisk_log_size":705,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:26:21.749588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012982219999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":2615,"num_read_kb":21488,"num_write":891,"num_write_kb":14014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":5651177,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"129'331","reported_seq":5228,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.699265+0000","last_change":"2024-04-24T15:34:15.655138+0000","last_active":"2024-04-24T15:34:28.699265+0000","last_peered":"2024-04-24T15:34:28.699265+0000","last_clean":"2024-04-24T15:34:28.699265+0000","last_became_active":"2024-04-24T15:34:15.653205+0000","last_became_peered":"2024-04-24T15:34:15.653205+0000","last_unstale":"2024-04-24T15:34:28.699265+0000","last_undegraded":"2024-04-24T15:34:28.699265+0000","last_fullsized":"2024-04-24T15:34:28.699265+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":331,"log_dups_size":0,"ondisk_log_size":331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:28:59.539282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013576079999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":4982,"num_read_kb":42072,"num_write":333,"num_write_kb":14625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":3758016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"129'241","reported_seq":2003,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861679+0000","last_change":"2024-04-24T15:34:23.825462+0000","last_active":"2024-04-24T15:34:27.861679+0000","last_peered":"2024-04-24T15:34:27.861679+0000","last_clean":"2024-04-24T15:34:27.861679+0000","last_became_active":"2024-04-24T15:34:23.823962+0000","last_became_peered":"2024-04-24T15:34:23.823962+0000","last_unstale":"2024-04-24T15:34:27.861679+0000","last_undegraded":"2024-04-24T15:34:27.861679+0000","last_fullsized":"2024-04-24T15:34:27.861679+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":241,"log_dups_size":0,"ondisk_log_size":241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:52:34.277987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000635438,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":1312,"num_read_kb":18878,"num_write":253,"num_write_kb":9844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":108,"num_bytes_recovered":2694373,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"131'450","reported_seq":6595,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.862055+0000","last_change":"2024-04-24T15:34:16.660466+0000","last_active":"2024-04-24T15:34:27.862055+0000","last_peered":"2024-04-24T15:34:27.862055+0000","last_clean":"2024-04-24T15:34:27.862055+0000","last_became_active":"2024-04-24T15:34:16.659071+0000","last_became_peered":"2024-04-24T15:34:16.659071+0000","last_unstale":"2024-04-24T15:34:27.862055+0000","last_undegraded":"2024-04-24T15:34:27.862055+0000","last_fullsized":"2024-04-24T15:34:27.862055+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:10:28.933829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013356920000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":7366,"num_read_kb":44142,"num_write":407,"num_write_kb":9850,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":233,"num_bytes_recovered":4458876,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"130'312","reported_seq":1575,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698823+0000","last_change":"2024-04-24T15:34:16.657117+0000","last_active":"2024-04-24T15:34:28.698823+0000","last_peered":"2024-04-24T15:34:28.698823+0000","last_clean":"2024-04-24T15:34:28.698823+0000","last_became_active":"2024-04-24T15:34:16.655084+0000","last_became_peered":"2024-04-24T15:34:16.655084+0000","last_unstale":"2024-04-24T15:34:28.698823+0000","last_undegraded":"2024-04-24T15:34:28.698823+0000","last_fullsized":"2024-04-24T15:34:28.698823+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:26:41.362344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003384743,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":837,"num_read_kb":16692,"num_write":319,"num_write_kb":15488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":277,"num_bytes_recovered":16228001,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.16","version":"129'375","reported_seq":2454,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.699026+0000","last_change":"2024-04-24T15:34:15.669825+0000","last_active":"2024-04-24T15:34:28.699026+0000","last_peered":"2024-04-24T15:34:28.699026+0000","last_clean":"2024-04-24T15:34:28.699026+0000","last_became_active":"2024-04-24T15:34:15.668678+0000","last_became_peered":"2024-04-24T15:34:15.668678+0000","last_unstale":"2024-04-24T15:34:28.699026+0000","last_undegraded":"2024-04-24T15:34:28.699026+0000","last_fullsized":"2024-04-24T15:34:28.699026+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:43:39.484677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018886770000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":3004,"num_read_kb":17777,"num_write":338,"num_write_kb":8036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":48,"num_read":114807,"num_read_kb":727542,"num_write":20290,"num_write_kb":356328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4413,"num_bytes_recovered":105175530,"num_keys_recovered":235,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15242,"internal_metadata":0},"log_size":17251,"ondisk_log_size":17251,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":8,"acting":8,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":16196,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574882,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299136,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072704,"statfs":{"total":96636764160,"available":96330448896,"internally_reserved":0,"allocated":1818624,"data_stored":1494500,"data_compressed":5677,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.46000000000000002,"15min":0.46200000000000002},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.60099999999999998,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.54200000000000004,"15min":0.55600000000000005},"min":{"1min":0.33900000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.67200000000000004,"5min":2.4710000000000001,"15min":2.4710000000000001},"last":0.73999999999999999}]},{"osd":1,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.57399999999999995,"15min":0.59299999999999997},"min":{"1min":0.28100000000000003,"5min":0.248,"15min":0.248},"max":{"1min":0.63600000000000001,"5min":2.4729999999999999,"15min":2.4729999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.51500000000000001,"15min":0.52400000000000002},"min":{"1min":0.29999999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.56899999999999995,"5min":2.4020000000000001,"15min":2.4020000000000001},"last":0.61099999999999999}]},{"osd":2,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.45700000000000002,"15min":0.45800000000000002},"min":{"1min":0.35499999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.59699999999999998,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.503,"5min":0.54800000000000004,"15min":0.55600000000000005},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.82199999999999995,"5min":2.363,"15min":2.363},"last":0.752}]},{"osd":3,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.52200000000000002,"15min":0.51800000000000002},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.53800000000000003,"15min":0.53200000000000003},"min":{"1min":0.38600000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.63600000000000001}]},{"osd":4,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.41399999999999998,"15min":0.41999999999999998},"min":{"1min":0.26100000000000001,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.66100000000000003,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.42299999999999999,"15min":0.42499999999999999},"min":{"1min":0.30599999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.73499999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.441}]},{"osd":5,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.45100000000000001,"15min":0.45500000000000002},"min":{"1min":0.307,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.51600000000000001,"15min":0.52000000000000002},"min":{"1min":0.36399999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.80500000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.67900000000000005}]},{"osd":6,"last update":"Wed Apr 24 15:32:58 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.65700000000000003}]}]},{"osd":6,"up_from":104,"seq":446676598796,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":305896,"kb_used_data":1748,"kb_used_omap":43,"kb_used_meta":304084,"kb_avail":94065944,"statfs":{"total":96636764160,"available":96323526656,"internally_reserved":0,"allocated":1789952,"data_stored":910351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":44735,"internal_metadata":311382337},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574881,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307280,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":305534,"kb_avail":94064560,"statfs":{"total":96636764160,"available":96322109440,"internally_reserved":0,"allocated":1773568,"data_stored":903176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312867160},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 15:34:01 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.35599999999999998,"15min":0.35799999999999998},"min":{"1min":0.20699999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.51100000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.377,"15min":0.376},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":1.353,"5min":1.375,"15min":1.375},"last":0.376}]},{"osd":2,"last update":"Wed Apr 24 15:34:01 2024","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.33600000000000002,"15min":0.34000000000000002},"min":{"1min":0.224,"5min":0.223,"15min":0.223},"max":{"1min":0.46100000000000002,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.255},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.41499999999999998,"15min":0.42699999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.52800000000000002,"5min":1.494,"15min":1.494},"last":0.30399999999999999}]},{"osd":3,"last update":"Wed Apr 24 15:34:01 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.42799999999999999,"15min":0.433},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.97499999999999998,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.377,"15min":0.372},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.40799999999999997}]},{"osd":4,"last update":"Wed Apr 24 15:34:01 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.53500000000000003,"15min":0.54100000000000004},"min":{"1min":0.34799999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.72199999999999998,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.495},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.57799999999999996,"15min":0.58199999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.52100000000000002}]},{"osd":5,"last update":"Wed Apr 24 15:34:01 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.379,"5min":1.454,"15min":1.454},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.53800000000000003,"15min":0.54200000000000004},"min":{"1min":0.35999999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.67100000000000004,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.56799999999999995}]},{"osd":6,"last update":"Wed Apr 24 15:33:01 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.548,"5min":1.548,"15min":1.548},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.51300000000000001}]},{"osd":7,"last update":"Wed Apr 24 15:34:01 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.56899999999999995,"15min":0.57099999999999995},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.55100000000000005,"15min":0.55700000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.69599999999999995,"5min":0.999,"15min":0.999},"last":0.53600000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574882,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303260,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94068580,"statfs":{"total":96636764160,"available":96326225920,"internally_reserved":0,"allocated":1847296,"data_stored":1500719,"data_compressed":4842,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.30499999999999999,"5min":0.45700000000000002,"15min":0.48299999999999998},"min":{"1min":0.17899999999999999,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.47999999999999998,"5min":3.7890000000000001,"15min":3.7890000000000001},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.46700000000000003,"15min":0.48899999999999999},"min":{"1min":0.224,"5min":0.191,"15min":0.191},"max":{"1min":0.59899999999999998,"5min":2.7890000000000001,"15min":2.7890000000000001},"last":0.35099999999999998}]},{"osd":2,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.53100000000000003,"15min":0.55900000000000005},"min":{"1min":0.22700000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.61399999999999999,"5min":3.9950000000000001,"15min":3.9950000000000001},"last":0.312},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.42299999999999999,"15min":0.436},"min":{"1min":0.26000000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.56799999999999995,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":0.28299999999999997}]},{"osd":3,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.48499999999999999,"15min":0.505},"min":{"1min":0.24099999999999999,"5min":0.223,"15min":0.223},"max":{"1min":0.54000000000000004,"5min":2.097,"15min":2.097},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.622,"15min":0.65700000000000003},"min":{"1min":0.30499999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.79500000000000004,"5min":3.2109999999999999,"15min":3.2109999999999999},"last":0.38800000000000001}]},{"osd":4,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.55700000000000005,"15min":0.56699999999999995},"min":{"1min":0.31,"5min":0.308,"15min":0.308},"max":{"1min":0.69099999999999995,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.65000000000000002,"15min":0.67300000000000004},"min":{"1min":0.34899999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.85599999999999998,"5min":3.282,"15min":3.282},"last":0.5}]},{"osd":5,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.60099999999999998,"15min":0.61299999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.79100000000000004,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.64600000000000002,"15min":0.66200000000000003},"min":{"1min":0.38700000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.78000000000000003,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":0.55400000000000005}]},{"osd":6,"last update":"Wed Apr 24 15:33:00 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":3.2509999999999999,"5min":3.2509999999999999,"15min":3.2509999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":2.1909999999999998,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":0.57999999999999996}]},{"osd":7,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.60299999999999998,"15min":0.60899999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.075,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.63200000000000001,"15min":0.64400000000000002},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.876,"5min":2.206,"15min":2.206},"last":0.52800000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574882,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303200,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94068640,"statfs":{"total":96636764160,"available":96326287360,"internally_reserved":0,"allocated":1785856,"data_stored":909395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 15:34:06 2024","interfaces":[{"interface":"back","average":{"1min":0.29199999999999998,"5min":0.312,"15min":0.315},"min":{"1min":0.16200000000000001,"5min":0.16200000000000001,"15min":0.16200000000000001},"max":{"1min":0.41399999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.30599999999999999,"5min":0.33700000000000002,"15min":0.34200000000000003},"min":{"1min":0.185,"5min":0.185,"15min":0.185},"max":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"last":0.442}]},{"osd":1,"last update":"Wed Apr 24 15:34:06 2024","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.36899999999999999,"15min":0.377},"min":{"1min":0.23300000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.47399999999999998,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.46100000000000002,"15min":0.48199999999999998},"min":{"1min":0.24099999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.441,"5min":3.1110000000000002,"15min":3.1110000000000002},"last":0.48999999999999999}]},{"osd":3,"last update":"Wed Apr 24 15:34:06 2024","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.40500000000000003,"15min":0.41899999999999998},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.41399999999999998,"5min":2.056,"15min":2.056},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.39900000000000002,"15min":0.40500000000000003},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.52900000000000003,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.54100000000000004}]},{"osd":4,"last update":"Wed Apr 24 15:34:06 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.51800000000000002,"15min":0.52400000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.53600000000000003,"15min":0.54500000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.70399999999999996,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.57299999999999995}]},{"osd":5,"last update":"Wed Apr 24 15:34:06 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.54300000000000004,"15min":0.54600000000000004},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.63700000000000001,"15min":0.65400000000000003},"min":{"1min":0.41399999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.80500000000000005,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":0.621}]},{"osd":6,"last update":"Wed Apr 24 15:33:03 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.1000000000000001,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.54900000000000004}]},{"osd":7,"last update":"Wed Apr 24 15:34:06 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.63500000000000001,"15min":0.65700000000000003},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.63800000000000001,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.56999999999999995,"15min":0.57799999999999996},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73999999999999999,"5min":1.502,"15min":1.502},"last":0.67400000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574882,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303204,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94068636,"statfs":{"total":96636764160,"available":96326283264,"internally_reserved":0,"allocated":1794048,"data_stored":910370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 15:34:04 2024","interfaces":[{"interface":"back","average":{"1min":0.29399999999999998,"5min":0.34599999999999997,"15min":0.35499999999999998},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.39900000000000002,"5min":1.224,"15min":1.224},"last":0.27600000000000002},{"interface":"front","average":{"1min":0.30199999999999999,"5min":0.34000000000000002,"15min":0.34599999999999997},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.41999999999999998,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.30099999999999999}]},{"osd":1,"last update":"Wed Apr 24 15:34:04 2024","interfaces":[{"interface":"back","average":{"1min":0.30299999999999999,"5min":0.33400000000000002,"15min":0.33900000000000002},"min":{"1min":0.19600000000000001,"5min":0.187,"15min":0.187},"max":{"1min":0.437,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.42199999999999999,"15min":0.434},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.71299999999999997,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.39400000000000002}]},{"osd":2,"last update":"Wed Apr 24 15:34:04 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.39400000000000002,"15min":0.40300000000000002},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.46500000000000002,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.376,"5min":0.41299999999999998,"15min":0.41899999999999998},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.53000000000000003,"5min":1.252,"15min":1.252},"last":0.44800000000000001}]},{"osd":4,"last update":"Wed Apr 24 15:34:04 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.52200000000000002,"15min":0.53400000000000003},"min":{"1min":0.28599999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.56499999999999995,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.52100000000000002,"15min":0.53300000000000003},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.60699999999999998,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.69999999999999996}]},{"osd":5,"last update":"Wed Apr 24 15:34:04 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.55600000000000005,"15min":0.56999999999999995},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.61199999999999999,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.5,"5min":0.58799999999999997,"15min":0.60199999999999998},"min":{"1min":0.38300000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.81100000000000005,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.621}]},{"osd":6,"last update":"Wed Apr 24 15:33:00 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.60199999999999998}]},{"osd":7,"last update":"Wed Apr 24 15:34:04 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.59099999999999997,"15min":0.60499999999999998},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.66800000000000004,"5min":1.635,"15min":1.635},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.495,"5min":0.55600000000000005,"15min":0.56599999999999995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.59699999999999998,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.64500000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574883,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307284,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":305534,"kb_avail":94064556,"statfs":{"total":96636764160,"available":96322105344,"internally_reserved":0,"allocated":1773568,"data_stored":901837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312867160},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 15:34:00 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.505,"15min":0.50900000000000001},"min":{"1min":0.249,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":1.8620000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.626,"5min":0.56599999999999995,"15min":0.55600000000000005},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":5.6239999999999997,"5min":5.6239999999999997,"15min":5.6239999999999997},"last":0.64100000000000001}]},{"osd":1,"last update":"Wed Apr 24 15:34:00 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.58699999999999997,"15min":0.57299999999999995},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":5.7149999999999999,"5min":5.7149999999999999,"15min":5.7149999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.35499999999999998,"5min":0.31,"15min":0.31},"max":{"1min":1.913,"5min":1.913,"15min":1.913},"last":0.53100000000000003}]},{"osd":2,"last update":"Wed Apr 24 15:34:00 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.54600000000000004,"15min":0.52800000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":5.6660000000000004,"5min":5.6660000000000004,"15min":5.6660000000000004},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.53400000000000003,"15min":0.54400000000000004},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.819,"5min":1.819,"15min":1.819},"last":0.46800000000000003}]},{"osd":3,"last update":"Wed Apr 24 15:34:00 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.53400000000000003,"15min":0.53600000000000003},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.9550000000000001,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":0.502},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.55500000000000005,"15min":0.55000000000000004},"min":{"1min":0.36799999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":2.3149999999999999,"5min":2.3149999999999999,"15min":2.3149999999999999},"last":0.55400000000000005}]},{"osd":5,"last update":"Wed Apr 24 15:34:00 2024","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.58299999999999996,"15min":0.51700000000000002},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":14.667999999999999,"5min":14.667999999999999,"15min":14.667999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.434,"5min":0.497,"15min":0.50700000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.9890000000000001,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":0.76400000000000001}]},{"osd":6,"last update":"Wed Apr 24 15:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.73399999999999999}]},{"osd":7,"last update":"Wed Apr 24 15:34:00 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.54000000000000004,"15min":0.53100000000000003},"min":{"1min":0.28599999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":5.2759999999999998,"5min":5.2759999999999998,"15min":5.2759999999999998},"last":0.71999999999999997},{"interface":"front","average":{"1min":1.0469999999999999,"5min":0.64100000000000001,"15min":0.57299999999999995},"min":{"1min":0.29399999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":14.773,"5min":14.773,"15min":14.773},"last":0.78700000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574882,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311380,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":309630,"kb_avail":94060460,"statfs":{"total":96636764160,"available":96317911040,"internally_reserved":0,"allocated":1777664,"data_stored":904132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317061464},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.47899999999999998,"15min":0.48499999999999999},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.46800000000000003,"15min":0.47599999999999998},"min":{"1min":0.246,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.629,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.55300000000000005}]},{"osd":1,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.46300000000000002,"15min":0.47099999999999997},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.64900000000000002,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.46899999999999997,"15min":0.47099999999999997},"min":{"1min":0.317,"5min":0.313,"15min":0.313},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.67300000000000004}]},{"osd":2,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.47499999999999998,"15min":0.46600000000000003},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":2.4169999999999998,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.48599999999999999,"15min":0.49199999999999999},"min":{"1min":0.30099999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.64200000000000002}]},{"osd":3,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.48899999999999999,"15min":0.495},"min":{"1min":0.308,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.70999999999999996,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":2.3370000000000002,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":0.65900000000000003}]},{"osd":4,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.44800000000000001,"15min":0.45700000000000002},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.624,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.752},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.48399999999999999,"15min":0.48599999999999999},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.59899999999999998}]},{"osd":6,"last update":"Wed Apr 24 15:33:02 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.71599999999999997}]},{"osd":7,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.52600000000000002,"15min":0.53100000000000003},"min":{"1min":0.30299999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.78600000000000003,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.496,"5min":0.53200000000000003,"15min":0.53800000000000003},"min":{"1min":0.34499999999999997,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.77100000000000002,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.76500000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4842,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":5677,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":5677,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15242,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T15:34:33.882 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T15:34:34.195 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:34:34.196 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-04-24T15:34:34.208 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":220,"stamp":"2024-04-24T15:34:33.580033+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":51,"num_read":114923,"num_read_kb":727642,"num_write":20449,"num_write_kb":358744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4420,"num_bytes_recovered":106356266,"num_keys_recovered":235,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17371,"ondisk_log_size":17371,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2440640,"kb_used_data":14024,"kb_used_omap":55,"kb_used_meta":2426440,"kb_avail":752534080,"statfs":{"total":773094113280,"available":770594897920,"internally_reserved":0,"allocated":14360576,"data_stored":8434480,"data_compressed":10519,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":56663,"internal_metadata":2484675241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.729940"},"pg_stats":[{"pgid":"4.1f","version":"131'1485","reported_seq":2550,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698777+0000","last_change":"2024-04-24T15:34:15.652525+0000","last_active":"2024-04-24T15:34:28.698777+0000","last_peered":"2024-04-24T15:34:28.698777+0000","last_clean":"2024-04-24T15:34:28.698777+0000","last_became_active":"2024-04-24T15:34:15.650850+0000","last_became_peered":"2024-04-24T15:34:15.650850+0000","last_unstale":"2024-04-24T15:34:28.698777+0000","last_undegraded":"2024-04-24T15:34:28.698777+0000","last_fullsized":"2024-04-24T15:34:28.698777+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":1485,"log_dups_size":0,"ondisk_log_size":1485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:06:34.577605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001749947,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":8143,"num_read_kb":14722,"num_write":2575,"num_write_kb":16077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":5952278,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"129'507","reported_seq":6418,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698830+0000","last_change":"2024-04-24T15:34:15.669801+0000","last_active":"2024-04-24T15:34:28.698830+0000","last_peered":"2024-04-24T15:34:28.698830+0000","last_clean":"2024-04-24T15:34:28.698830+0000","last_became_active":"2024-04-24T15:34:15.668657+0000","last_became_peered":"2024-04-24T15:34:15.668657+0000","last_unstale":"2024-04-24T15:34:28.698830+0000","last_undegraded":"2024-04-24T15:34:28.698830+0000","last_fullsized":"2024-04-24T15:34:28.698830+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":507,"log_dups_size":0,"ondisk_log_size":507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:32:57.253624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029386599999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":7278,"num_read_kb":39113,"num_write":560,"num_write_kb":6592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":4495932,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"129'529","reported_seq":3745,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861949+0000","last_change":"2024-04-24T15:34:16.660701+0000","last_active":"2024-04-24T15:34:27.861949+0000","last_peered":"2024-04-24T15:34:27.861949+0000","last_clean":"2024-04-24T15:34:27.861949+0000","last_became_active":"2024-04-24T15:34:16.659202+0000","last_became_peered":"2024-04-24T15:34:16.659202+0000","last_unstale":"2024-04-24T15:34:27.861949+0000","last_undegraded":"2024-04-24T15:34:27.861949+0000","last_fullsized":"2024-04-24T15:34:27.861949+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":529,"log_dups_size":0,"ondisk_log_size":529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:31:57.762122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015488839999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":4427,"num_read_kb":51896,"num_write":603,"num_write_kb":18759,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":580671,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"131'526","reported_seq":2700,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.699161+0000","last_change":"2024-04-24T15:34:19.807684+0000","last_active":"2024-04-24T15:34:28.699161+0000","last_peered":"2024-04-24T15:34:28.699161+0000","last_clean":"2024-04-24T15:34:28.699161+0000","last_became_active":"2024-04-24T15:34:19.806783+0000","last_became_peered":"2024-04-24T15:34:19.806783+0000","last_unstale":"2024-04-24T15:34:28.699161+0000","last_undegraded":"2024-04-24T15:34:28.699161+0000","last_fullsized":"2024-04-24T15:34:28.699161+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":526,"log_dups_size":0,"ondisk_log_size":526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:06:01.349645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001413493,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":3716,"num_read_kb":10078,"num_write":547,"num_write_kb":2888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"129'385","reported_seq":2359,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861581+0000","last_change":"2024-04-24T15:34:23.827263+0000","last_active":"2024-04-24T15:34:27.861581+0000","last_peered":"2024-04-24T15:34:27.861581+0000","last_clean":"2024-04-24T15:34:27.861581+0000","last_became_active":"2024-04-24T15:34:23.825472+0000","last_became_peered":"2024-04-24T15:34:23.825472+0000","last_unstale":"2024-04-24T15:34:27.861581+0000","last_undegraded":"2024-04-24T15:34:27.861581+0000","last_fullsized":"2024-04-24T15:34:27.861581+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":385,"log_dups_size":0,"ondisk_log_size":385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:19:37.470876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011107479999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":2335,"num_read_kb":25405,"num_write":329,"num_write_kb":6118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":79,"num_bytes_recovered":2143880,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"129'449","reported_seq":6610,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861569+0000","last_change":"2024-04-24T15:34:23.827554+0000","last_active":"2024-04-24T15:34:27.861569+0000","last_peered":"2024-04-24T15:34:27.861569+0000","last_clean":"2024-04-24T15:34:27.861569+0000","last_became_active":"2024-04-24T15:34:23.824861+0000","last_became_peered":"2024-04-24T15:34:23.824861+0000","last_unstale":"2024-04-24T15:34:27.861569+0000","last_undegraded":"2024-04-24T15:34:27.861569+0000","last_fullsized":"2024-04-24T15:34:27.861569+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:42:20.554334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097376600000000004,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":6368,"num_read_kb":19805,"num_write":475,"num_write_kb":10193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":89,"num_bytes_recovered":1137673,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"129'537","reported_seq":2680,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861805+0000","last_change":"2024-04-24T15:34:08.756894+0000","last_active":"2024-04-24T15:34:27.861805+0000","last_peered":"2024-04-24T15:34:27.861805+0000","last_clean":"2024-04-24T15:34:27.861805+0000","last_became_active":"2024-04-24T15:32:17.047842+0000","last_became_peered":"2024-04-24T15:32:17.047842+0000","last_unstale":"2024-04-24T15:34:27.861805+0000","last_undegraded":"2024-04-24T15:34:27.861805+0000","last_fullsized":"2024-04-24T15:34:27.861805+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":537,"log_dups_size":0,"ondisk_log_size":537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:09:52.065772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081826100000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":4119,"num_read_kb":22694,"num_write":527,"num_write_kb":11193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"130'444","reported_seq":1968,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861537+0000","last_change":"2024-04-24T15:34:08.758067+0000","last_active":"2024-04-24T15:34:27.861537+0000","last_peered":"2024-04-24T15:34:27.861537+0000","last_clean":"2024-04-24T15:34:27.861537+0000","last_became_active":"2024-04-24T15:33:42.641679+0000","last_became_peered":"2024-04-24T15:33:42.641679+0000","last_unstale":"2024-04-24T15:34:27.861537+0000","last_undegraded":"2024-04-24T15:34:27.861537+0000","last_fullsized":"2024-04-24T15:34:27.861537+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":444,"log_dups_size":0,"ondisk_log_size":444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:29:58.781375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077716999999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":1887,"num_read_kb":15406,"num_write":547,"num_write_kb":8155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":4099152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"129'523","reported_seq":2447,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861936+0000","last_change":"2024-04-24T15:34:08.758924+0000","last_active":"2024-04-24T15:34:27.861936+0000","last_peered":"2024-04-24T15:34:27.861936+0000","last_clean":"2024-04-24T15:34:27.861936+0000","last_became_active":"2024-04-24T15:33:56.699629+0000","last_became_peered":"2024-04-24T15:33:56.699629+0000","last_unstale":"2024-04-24T15:34:27.861936+0000","last_undegraded":"2024-04-24T15:34:27.861936+0000","last_fullsized":"2024-04-24T15:34:27.861936+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":523,"log_dups_size":0,"ondisk_log_size":523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:53:30.709042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00212766,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":3353,"num_read_kb":16260,"num_write":594,"num_write_kb":12753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":352256,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"129'316","reported_seq":2577,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861623+0000","last_change":"2024-04-24T15:34:23.829028+0000","last_active":"2024-04-24T15:34:27.861623+0000","last_peered":"2024-04-24T15:34:27.861623+0000","last_clean":"2024-04-24T15:34:27.861623+0000","last_became_active":"2024-04-24T15:34:23.827917+0000","last_became_peered":"2024-04-24T15:34:23.827917+0000","last_unstale":"2024-04-24T15:34:27.861623+0000","last_undegraded":"2024-04-24T15:34:27.861623+0000","last_fullsized":"2024-04-24T15:34:27.861623+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":316,"log_dups_size":0,"ondisk_log_size":316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:11:49.794260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065534800000000004,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":1633,"num_read_kb":26736,"num_write":303,"num_write_kb":13822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":9304722,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":121,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698692+0000","last_change":"2024-04-24T15:33:53.644249+0000","last_active":"2024-04-24T15:34:28.698692+0000","last_peered":"2024-04-24T15:34:28.698692+0000","last_clean":"2024-04-24T15:34:28.698692+0000","last_became_active":"2024-04-24T15:33:53.643941+0000","last_became_peered":"2024-04-24T15:33:53.643941+0000","last_unstale":"2024-04-24T15:34:28.698692+0000","last_undegraded":"2024-04-24T15:34:28.698692+0000","last_fullsized":"2024-04-24T15:34:28.698692+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:42:05.771051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"130'862","reported_seq":3009,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698843+0000","last_change":"2024-04-24T15:34:19.810827+0000","last_active":"2024-04-24T15:34:28.698843+0000","last_peered":"2024-04-24T15:34:28.698843+0000","last_clean":"2024-04-24T15:34:28.698843+0000","last_became_active":"2024-04-24T15:34:19.809619+0000","last_became_peered":"2024-04-24T15:34:19.809619+0000","last_unstale":"2024-04-24T15:34:28.698843+0000","last_undegraded":"2024-04-24T15:34:28.698843+0000","last_fullsized":"2024-04-24T15:34:28.698843+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:42:29.391778+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00033353700000000002,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":4689,"num_read_kb":23870,"num_write":1044,"num_write_kb":13092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":176,"num_bytes_recovered":5601476,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":24,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.699509+0000","last_change":"2024-04-24T15:34:12.652580+0000","last_active":"2024-04-24T15:34:28.699509+0000","last_peered":"2024-04-24T15:34:28.699509+0000","last_clean":"2024-04-24T15:34:28.699509+0000","last_became_active":"2024-04-24T15:34:12.652298+0000","last_became_peered":"2024-04-24T15:34:12.652298+0000","last_unstale":"2024-04-24T15:34:28.699509+0000","last_undegraded":"2024-04-24T15:34:28.699509+0000","last_fullsized":"2024-04-24T15:34:28.699509+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:22:59.160275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":41,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.862035+0000","last_change":"2024-04-24T15:33:53.645875+0000","last_active":"2024-04-24T15:34:27.862035+0000","last_peered":"2024-04-24T15:34:27.862035+0000","last_clean":"2024-04-24T15:34:27.862035+0000","last_became_active":"2024-04-24T15:33:53.645696+0000","last_became_peered":"2024-04-24T15:33:53.645696+0000","last_unstale":"2024-04-24T15:34:27.862035+0000","last_undegraded":"2024-04-24T15:34:27.862035+0000","last_fullsized":"2024-04-24T15:34:27.862035+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:29:35.345558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"131'584","reported_seq":5172,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698983+0000","last_change":"2024-04-24T15:34:17.655614+0000","last_active":"2024-04-24T15:34:28.698983+0000","last_peered":"2024-04-24T15:34:28.698983+0000","last_clean":"2024-04-24T15:34:28.698983+0000","last_became_active":"2024-04-24T15:34:17.654759+0000","last_became_peered":"2024-04-24T15:34:17.654759+0000","last_unstale":"2024-04-24T15:34:28.698983+0000","last_undegraded":"2024-04-24T15:34:28.698983+0000","last_fullsized":"2024-04-24T15:34:28.698983+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":584,"log_dups_size":0,"ondisk_log_size":584,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:42:40.438352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061779100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3592,"num_read_kb":14899,"num_write":598,"num_write_kb":11359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":23598,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"60'1","reported_seq":80,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698947+0000","last_change":"2024-04-24T15:33:47.663123+0000","last_active":"2024-04-24T15:34:28.698947+0000","last_peered":"2024-04-24T15:34:28.698947+0000","last_clean":"2024-04-24T15:34:28.698947+0000","last_became_active":"2024-04-24T15:33:47.662685+0000","last_became_peered":"2024-04-24T15:33:47.662685+0000","last_unstale":"2024-04-24T15:34:28.698947+0000","last_undegraded":"2024-04-24T15:34:28.698947+0000","last_fullsized":"2024-04-24T15:34:28.698947+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_clean_scrub_stamp":"2024-04-24T15:32:06.972928+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:05:35.654980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":50,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861972+0000","last_change":"2024-04-24T15:34:12.653126+0000","last_active":"2024-04-24T15:34:27.861972+0000","last_peered":"2024-04-24T15:34:27.861972+0000","last_clean":"2024-04-24T15:34:27.861972+0000","last_became_active":"2024-04-24T15:34:12.652805+0000","last_became_peered":"2024-04-24T15:34:12.652805+0000","last_unstale":"2024-04-24T15:34:27.861972+0000","last_undegraded":"2024-04-24T15:34:27.861972+0000","last_fullsized":"2024-04-24T15:34:27.861972+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:44:23.434720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"131'615","reported_seq":2369,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698926+0000","last_change":"2024-04-24T15:34:16.760282+0000","last_active":"2024-04-24T15:34:28.698926+0000","last_peered":"2024-04-24T15:34:28.698926+0000","last_clean":"2024-04-24T15:34:28.698926+0000","last_became_active":"2024-04-24T15:34:16.655390+0000","last_became_peered":"2024-04-24T15:34:16.655390+0000","last_unstale":"2024-04-24T15:34:28.698926+0000","last_undegraded":"2024-04-24T15:34:28.698926+0000","last_fullsized":"2024-04-24T15:34:28.698926+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":615,"log_dups_size":0,"ondisk_log_size":615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:01:32.241481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042917399999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":3015,"num_read_kb":19823,"num_write":696,"num_write_kb":5112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"90'2","reported_seq":182,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861981+0000","last_change":"2024-04-24T15:34:13.666629+0000","last_active":"2024-04-24T15:34:27.861981+0000","last_peered":"2024-04-24T15:34:27.861981+0000","last_clean":"2024-04-24T15:34:27.861981+0000","last_became_active":"2024-04-24T15:34:13.662363+0000","last_became_peered":"2024-04-24T15:34:13.662363+0000","last_unstale":"2024-04-24T15:34:27.861981+0000","last_undegraded":"2024-04-24T15:34:27.861981+0000","last_fullsized":"2024-04-24T15:34:27.861981+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_clean_scrub_stamp":"2024-04-24T15:32:06.972928+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:09:38.698619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":251,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.862026+0000","last_change":"2024-04-24T15:33:57.644653+0000","last_active":"2024-04-24T15:34:27.862026+0000","last_peered":"2024-04-24T15:34:27.862026+0000","last_clean":"2024-04-24T15:34:27.862026+0000","last_became_active":"2024-04-24T15:33:57.644312+0000","last_became_peered":"2024-04-24T15:33:57.644312+0000","last_unstale":"2024-04-24T15:34:27.862026+0000","last_undegraded":"2024-04-24T15:34:27.862026+0000","last_fullsized":"2024-04-24T15:34:27.862026+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:31:59.935287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:31:59.935287+0000","last_clean_scrub_stamp":"2024-04-24T15:31:59.935287+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:21:46.534664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698724+0000","last_change":"2024-04-24T15:34:13.650533+0000","last_active":"2024-04-24T15:34:28.698724+0000","last_peered":"2024-04-24T15:34:28.698724+0000","last_clean":"2024-04-24T15:34:28.698724+0000","last_became_active":"2024-04-24T15:34:13.650174+0000","last_became_peered":"2024-04-24T15:34:13.650174+0000","last_unstale":"2024-04-24T15:34:28.698724+0000","last_undegraded":"2024-04-24T15:34:28.698724+0000","last_fullsized":"2024-04-24T15:34:28.698724+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:01:43.442578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"129'591","reported_seq":2456,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698898+0000","last_change":"2024-04-24T15:34:08.805854+0000","last_active":"2024-04-24T15:34:28.698898+0000","last_peered":"2024-04-24T15:34:28.698898+0000","last_clean":"2024-04-24T15:34:28.698898+0000","last_became_active":"2024-04-24T15:33:45.658207+0000","last_became_peered":"2024-04-24T15:33:45.658207+0000","last_unstale":"2024-04-24T15:34:28.698898+0000","last_undegraded":"2024-04-24T15:34:28.698898+0000","last_fullsized":"2024-04-24T15:34:28.698898+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":591,"log_dups_size":0,"ondisk_log_size":591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:59:01.456518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010111199999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":3352,"num_read_kb":26205,"num_write":791,"num_write_kb":11816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":8821091,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":24,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.699087+0000","last_change":"2024-04-24T15:34:12.655959+0000","last_active":"2024-04-24T15:34:28.699087+0000","last_peered":"2024-04-24T15:34:28.699087+0000","last_clean":"2024-04-24T15:34:28.699087+0000","last_became_active":"2024-04-24T15:34:12.655508+0000","last_became_peered":"2024-04-24T15:34:12.655508+0000","last_unstale":"2024-04-24T15:34:28.699087+0000","last_undegraded":"2024-04-24T15:34:28.699087+0000","last_fullsized":"2024-04-24T15:34:28.699087+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_clean_scrub_stamp":"2024-04-24T15:32:06.972928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:13:50.464460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":59,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.699249+0000","last_change":"2024-04-24T15:34:19.838358+0000","last_active":"2024-04-24T15:34:28.699249+0000","last_peered":"2024-04-24T15:34:28.699249+0000","last_clean":"2024-04-24T15:34:28.699249+0000","last_became_active":"2024-04-24T15:34:19.838058+0000","last_became_peered":"2024-04-24T15:34:19.838058+0000","last_unstale":"2024-04-24T15:34:28.699249+0000","last_undegraded":"2024-04-24T15:34:28.699249+0000","last_fullsized":"2024-04-24T15:34:28.699249+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:29:43.725200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"131'430","reported_seq":1231,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.699369+0000","last_change":"2024-04-24T15:34:15.660049+0000","last_active":"2024-04-24T15:34:28.699369+0000","last_peered":"2024-04-24T15:34:28.699369+0000","last_clean":"2024-04-24T15:34:28.699369+0000","last_became_active":"2024-04-24T15:34:15.658368+0000","last_became_peered":"2024-04-24T15:34:15.658368+0000","last_unstale":"2024-04-24T15:34:28.699369+0000","last_undegraded":"2024-04-24T15:34:28.699369+0000","last_fullsized":"2024-04-24T15:34:28.699369+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:29:46.353757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015045289999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":1970,"num_read_kb":12710,"num_write":463,"num_write_kb":8069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":208,"num_bytes_recovered":107133,"num_keys_recovered":27,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":93,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861700+0000","last_change":"2024-04-24T15:34:24.428957+0000","last_active":"2024-04-24T15:34:27.861700+0000","last_peered":"2024-04-24T15:34:27.861700+0000","last_clean":"2024-04-24T15:34:27.861700+0000","last_became_active":"2024-04-24T15:34:24.428510+0000","last_became_peered":"2024-04-24T15:34:24.428510+0000","last_unstale":"2024-04-24T15:34:27.861700+0000","last_undegraded":"2024-04-24T15:34:27.861700+0000","last_fullsized":"2024-04-24T15:34:27.861700+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_clean_scrub_stamp":"2024-04-24T15:32:06.972928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:36:35.138337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698664+0000","last_change":"2024-04-24T15:34:13.689898+0000","last_active":"2024-04-24T15:34:28.698664+0000","last_peered":"2024-04-24T15:34:28.698664+0000","last_clean":"2024-04-24T15:34:28.698664+0000","last_became_active":"2024-04-24T15:34:13.689685+0000","last_became_peered":"2024-04-24T15:34:13.689685+0000","last_unstale":"2024-04-24T15:34:28.698664+0000","last_undegraded":"2024-04-24T15:34:28.698664+0000","last_fullsized":"2024-04-24T15:34:28.698664+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:24:27.251519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"130'457","reported_seq":1995,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.699625+0000","last_change":"2024-04-24T15:34:16.654083+0000","last_active":"2024-04-24T15:34:28.699625+0000","last_peered":"2024-04-24T15:34:28.699625+0000","last_clean":"2024-04-24T15:34:28.699625+0000","last_became_active":"2024-04-24T15:34:16.651906+0000","last_became_peered":"2024-04-24T15:34:16.651906+0000","last_unstale":"2024-04-24T15:34:28.699625+0000","last_undegraded":"2024-04-24T15:34:28.699625+0000","last_fullsized":"2024-04-24T15:34:28.699625+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":457,"log_dups_size":0,"ondisk_log_size":457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:54.315154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010565220000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2138,"num_read_kb":24032,"num_write":423,"num_write_kb":14900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"130'408","reported_seq":1974,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.699200+0000","last_change":"2024-04-24T15:34:16.653875+0000","last_active":"2024-04-24T15:34:28.699200+0000","last_peered":"2024-04-24T15:34:28.699200+0000","last_clean":"2024-04-24T15:34:28.699200+0000","last_became_active":"2024-04-24T15:34:16.651931+0000","last_became_peered":"2024-04-24T15:34:16.651931+0000","last_unstale":"2024-04-24T15:34:28.699200+0000","last_undegraded":"2024-04-24T15:34:28.699200+0000","last_fullsized":"2024-04-24T15:34:28.699200+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:59:24.823763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039757799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2388,"num_read_kb":22956,"num_write":379,"num_write_kb":16060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":578400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.699581+0000","last_change":"2024-04-24T15:34:12.653777+0000","last_active":"2024-04-24T15:34:28.699581+0000","last_peered":"2024-04-24T15:34:28.699581+0000","last_clean":"2024-04-24T15:34:28.699581+0000","last_became_active":"2024-04-24T15:34:12.653529+0000","last_became_peered":"2024-04-24T15:34:12.653529+0000","last_unstale":"2024-04-24T15:34:28.699581+0000","last_undegraded":"2024-04-24T15:34:28.699581+0000","last_fullsized":"2024-04-24T15:34:28.699581+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:21:58.411545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"129'479","reported_seq":1997,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698576+0000","last_change":"2024-04-24T15:34:16.656437+0000","last_active":"2024-04-24T15:34:28.698576+0000","last_peered":"2024-04-24T15:34:28.698576+0000","last_clean":"2024-04-24T15:34:28.698576+0000","last_became_active":"2024-04-24T15:34:16.654252+0000","last_became_peered":"2024-04-24T15:34:16.654252+0000","last_unstale":"2024-04-24T15:34:28.698576+0000","last_undegraded":"2024-04-24T15:34:28.698576+0000","last_fullsized":"2024-04-24T15:34:28.698576+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:53:07.507328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035787500000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":2434,"num_read_kb":14959,"num_write":545,"num_write_kb":12213,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":335,"num_bytes_recovered":6920328,"num_keys_recovered":43,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"130'616","reported_seq":2734,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861566+0000","last_change":"2024-04-24T15:34:23.827845+0000","last_active":"2024-04-24T15:34:27.861566+0000","last_peered":"2024-04-24T15:34:27.861566+0000","last_clean":"2024-04-24T15:34:27.861566+0000","last_became_active":"2024-04-24T15:34:23.825759+0000","last_became_peered":"2024-04-24T15:34:23.825759+0000","last_unstale":"2024-04-24T15:34:27.861566+0000","last_undegraded":"2024-04-24T15:34:27.861566+0000","last_fullsized":"2024-04-24T15:34:27.861566+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:49:46.424217+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0055639879999999997,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":4324,"num_read_kb":21128,"num_write":677,"num_write_kb":10933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":5713612,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"130'485","reported_seq":3231,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698742+0000","last_change":"2024-04-24T15:34:15.659323+0000","last_active":"2024-04-24T15:34:28.698742+0000","last_peered":"2024-04-24T15:34:28.698742+0000","last_clean":"2024-04-24T15:34:28.698742+0000","last_became_active":"2024-04-24T15:34:15.657660+0000","last_became_peered":"2024-04-24T15:34:15.657660+0000","last_unstale":"2024-04-24T15:34:28.698742+0000","last_undegraded":"2024-04-24T15:34:28.698742+0000","last_fullsized":"2024-04-24T15:34:28.698742+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:09:16.071953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066763,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3120,"num_read_kb":27458,"num_write":430,"num_write_kb":12267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":237,"num_bytes_recovered":5299136,"num_keys_recovered":49,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"129'551","reported_seq":1976,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698866+0000","last_change":"2024-04-24T15:34:15.669951+0000","last_active":"2024-04-24T15:34:28.698866+0000","last_peered":"2024-04-24T15:34:28.698866+0000","last_clean":"2024-04-24T15:34:28.698866+0000","last_became_active":"2024-04-24T15:34:15.668717+0000","last_became_peered":"2024-04-24T15:34:15.668717+0000","last_unstale":"2024-04-24T15:34:28.698866+0000","last_undegraded":"2024-04-24T15:34:28.698866+0000","last_fullsized":"2024-04-24T15:34:28.698866+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":551,"log_dups_size":0,"ondisk_log_size":551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:46:28.552856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039775200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2839,"num_read_kb":21248,"num_write":721,"num_write_kb":8473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":141,"num_bytes_recovered":1490474,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"127'1046","reported_seq":2505,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861716+0000","last_change":"2024-04-24T15:34:23.826944+0000","last_active":"2024-04-24T15:34:27.861716+0000","last_peered":"2024-04-24T15:34:27.861716+0000","last_clean":"2024-04-24T15:34:27.861716+0000","last_became_active":"2024-04-24T15:34:23.825039+0000","last_became_peered":"2024-04-24T15:34:23.825039+0000","last_unstale":"2024-04-24T15:34:27.861716+0000","last_undegraded":"2024-04-24T15:34:27.861716+0000","last_fullsized":"2024-04-24T15:34:27.861716+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":1046,"log_dups_size":0,"ondisk_log_size":1046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:15:05.349996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029127150000000002,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":6311,"num_read_kb":16558,"num_write":1677,"num_write_kb":12129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":54,"num_bytes_recovered":1219753,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.d","version":"127'298","reported_seq":2572,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861837+0000","last_change":"2024-04-24T15:34:23.825797+0000","last_active":"2024-04-24T15:34:27.861837+0000","last_peered":"2024-04-24T15:34:27.861837+0000","last_clean":"2024-04-24T15:34:27.861837+0000","last_became_active":"2024-04-24T15:34:23.823763+0000","last_became_peered":"2024-04-24T15:34:23.823763+0000","last_unstale":"2024-04-24T15:34:27.861837+0000","last_undegraded":"2024-04-24T15:34:27.861837+0000","last_fullsized":"2024-04-24T15:34:27.861837+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":298,"log_dups_size":0,"ondisk_log_size":298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:24:00.110009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047837299999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":2121,"num_read_kb":46483,"num_write":303,"num_write_kb":20603,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":4246835,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"129'563","reported_seq":2395,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861756+0000","last_change":"2024-04-24T15:34:23.826137+0000","last_active":"2024-04-24T15:34:27.861756+0000","last_peered":"2024-04-24T15:34:27.861756+0000","last_clean":"2024-04-24T15:34:27.861756+0000","last_became_active":"2024-04-24T15:34:23.824037+0000","last_became_peered":"2024-04-24T15:34:23.824037+0000","last_unstale":"2024-04-24T15:34:27.861756+0000","last_undegraded":"2024-04-24T15:34:27.861756+0000","last_fullsized":"2024-04-24T15:34:27.861756+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":563,"log_dups_size":0,"ondisk_log_size":563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:30:41.572188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001084025,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3022,"num_read_kb":13469,"num_write":696,"num_write_kb":6185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":113,"num_bytes_recovered":1676557,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"130'907","reported_seq":3306,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.699413+0000","last_change":"2024-04-24T15:34:16.657695+0000","last_active":"2024-04-24T15:34:28.699413+0000","last_peered":"2024-04-24T15:34:28.699413+0000","last_clean":"2024-04-24T15:34:28.699413+0000","last_became_active":"2024-04-24T15:34:16.655733+0000","last_became_peered":"2024-04-24T15:34:16.655733+0000","last_unstale":"2024-04-24T15:34:28.699413+0000","last_undegraded":"2024-04-24T15:34:28.699413+0000","last_fullsized":"2024-04-24T15:34:28.699413+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":907,"log_dups_size":0,"ondisk_log_size":907,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:59:41.606707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018480180000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":5733,"num_read_kb":14488,"num_write":1011,"num_write_kb":8995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":223,"num_bytes_recovered":630687,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"130'244","reported_seq":1061,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861808+0000","last_change":"2024-04-24T15:34:15.664032+0000","last_active":"2024-04-24T15:34:27.861808+0000","last_peered":"2024-04-24T15:34:27.861808+0000","last_clean":"2024-04-24T15:34:27.861808+0000","last_became_active":"2024-04-24T15:34:15.663081+0000","last_became_peered":"2024-04-24T15:34:15.663081+0000","last_unstale":"2024-04-24T15:34:27.861808+0000","last_undegraded":"2024-04-24T15:34:27.861808+0000","last_fullsized":"2024-04-24T15:34:27.861808+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":244,"log_dups_size":0,"ondisk_log_size":244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:33:57.743630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010945530000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":384,"num_read_kb":4092,"num_write":235,"num_write_kb":5715,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":1989443,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.11","version":"129'705","reported_seq":2162,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861883+0000","last_change":"2024-04-24T15:34:08.757586+0000","last_active":"2024-04-24T15:34:27.861883+0000","last_peered":"2024-04-24T15:34:27.861883+0000","last_clean":"2024-04-24T15:34:27.861883+0000","last_became_active":"2024-04-24T15:33:57.644309+0000","last_became_peered":"2024-04-24T15:33:57.644309+0000","last_unstale":"2024-04-24T15:34:27.861883+0000","last_undegraded":"2024-04-24T15:34:27.861883+0000","last_fullsized":"2024-04-24T15:34:27.861883+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":705,"log_dups_size":0,"ondisk_log_size":705,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:26:21.749588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012982219999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":2615,"num_read_kb":21488,"num_write":891,"num_write_kb":14014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":5651177,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"129'331","reported_seq":5228,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.699265+0000","last_change":"2024-04-24T15:34:15.655138+0000","last_active":"2024-04-24T15:34:28.699265+0000","last_peered":"2024-04-24T15:34:28.699265+0000","last_clean":"2024-04-24T15:34:28.699265+0000","last_became_active":"2024-04-24T15:34:15.653205+0000","last_became_peered":"2024-04-24T15:34:15.653205+0000","last_unstale":"2024-04-24T15:34:28.699265+0000","last_undegraded":"2024-04-24T15:34:28.699265+0000","last_fullsized":"2024-04-24T15:34:28.699265+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":331,"log_dups_size":0,"ondisk_log_size":331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:28:59.539282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013576079999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":4982,"num_read_kb":42072,"num_write":333,"num_write_kb":14625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":3758016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"129'241","reported_seq":2003,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861679+0000","last_change":"2024-04-24T15:34:23.825462+0000","last_active":"2024-04-24T15:34:27.861679+0000","last_peered":"2024-04-24T15:34:27.861679+0000","last_clean":"2024-04-24T15:34:27.861679+0000","last_became_active":"2024-04-24T15:34:23.823962+0000","last_became_peered":"2024-04-24T15:34:23.823962+0000","last_unstale":"2024-04-24T15:34:27.861679+0000","last_undegraded":"2024-04-24T15:34:27.861679+0000","last_fullsized":"2024-04-24T15:34:27.861679+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":241,"log_dups_size":0,"ondisk_log_size":241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:52:34.277987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000635438,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":1312,"num_read_kb":18878,"num_write":253,"num_write_kb":9844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":108,"num_bytes_recovered":2694373,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"131'450","reported_seq":6595,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.862055+0000","last_change":"2024-04-24T15:34:16.660466+0000","last_active":"2024-04-24T15:34:27.862055+0000","last_peered":"2024-04-24T15:34:27.862055+0000","last_clean":"2024-04-24T15:34:27.862055+0000","last_became_active":"2024-04-24T15:34:16.659071+0000","last_became_peered":"2024-04-24T15:34:16.659071+0000","last_unstale":"2024-04-24T15:34:27.862055+0000","last_undegraded":"2024-04-24T15:34:27.862055+0000","last_fullsized":"2024-04-24T15:34:27.862055+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:10:28.933829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013356920000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":7366,"num_read_kb":44142,"num_write":407,"num_write_kb":9850,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":233,"num_bytes_recovered":4458876,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"130'312","reported_seq":1575,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698823+0000","last_change":"2024-04-24T15:34:16.657117+0000","last_active":"2024-04-24T15:34:28.698823+0000","last_peered":"2024-04-24T15:34:28.698823+0000","last_clean":"2024-04-24T15:34:28.698823+0000","last_became_active":"2024-04-24T15:34:16.655084+0000","last_became_peered":"2024-04-24T15:34:16.655084+0000","last_unstale":"2024-04-24T15:34:28.698823+0000","last_undegraded":"2024-04-24T15:34:28.698823+0000","last_fullsized":"2024-04-24T15:34:28.698823+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:26:41.362344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003384743,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":837,"num_read_kb":16692,"num_write":319,"num_write_kb":15488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":277,"num_bytes_recovered":16228001,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.16","version":"129'375","reported_seq":2454,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.699026+0000","last_change":"2024-04-24T15:34:15.669825+0000","last_active":"2024-04-24T15:34:28.699026+0000","last_peered":"2024-04-24T15:34:28.699026+0000","last_clean":"2024-04-24T15:34:28.699026+0000","last_became_active":"2024-04-24T15:34:15.668678+0000","last_became_peered":"2024-04-24T15:34:15.668678+0000","last_unstale":"2024-04-24T15:34:28.699026+0000","last_undegraded":"2024-04-24T15:34:28.699026+0000","last_fullsized":"2024-04-24T15:34:28.699026+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:43:39.484677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018886770000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":3004,"num_read_kb":17777,"num_write":338,"num_write_kb":8036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":48,"num_read":114807,"num_read_kb":727542,"num_write":20290,"num_write_kb":356328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4413,"num_bytes_recovered":105175530,"num_keys_recovered":235,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15242,"internal_metadata":0},"log_size":17251,"ondisk_log_size":17251,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":8,"acting":8,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":16196,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574882,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299136,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072704,"statfs":{"total":96636764160,"available":96330448896,"internally_reserved":0,"allocated":1818624,"data_stored":1494500,"data_compressed":5677,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.46000000000000002,"15min":0.46200000000000002},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.60099999999999998,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.54200000000000004,"15min":0.55600000000000005},"min":{"1min":0.33900000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.67200000000000004,"5min":2.4710000000000001,"15min":2.4710000000000001},"last":0.73999999999999999}]},{"osd":1,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.57399999999999995,"15min":0.59299999999999997},"min":{"1min":0.28100000000000003,"5min":0.248,"15min":0.248},"max":{"1min":0.63600000000000001,"5min":2.4729999999999999,"15min":2.4729999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.51500000000000001,"15min":0.52400000000000002},"min":{"1min":0.29999999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.56899999999999995,"5min":2.4020000000000001,"15min":2.4020000000000001},"last":0.61099999999999999}]},{"osd":2,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.45700000000000002,"15min":0.45800000000000002},"min":{"1min":0.35499999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.59699999999999998,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.503,"5min":0.54800000000000004,"15min":0.55600000000000005},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.82199999999999995,"5min":2.363,"15min":2.363},"last":0.752}]},{"osd":3,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.52200000000000002,"15min":0.51800000000000002},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.53800000000000003,"15min":0.53200000000000003},"min":{"1min":0.38600000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.63600000000000001}]},{"osd":4,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.41399999999999998,"15min":0.41999999999999998},"min":{"1min":0.26100000000000001,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.66100000000000003,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.42299999999999999,"15min":0.42499999999999999},"min":{"1min":0.30599999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.73499999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.441}]},{"osd":5,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.45100000000000001,"15min":0.45500000000000002},"min":{"1min":0.307,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.51600000000000001,"15min":0.52000000000000002},"min":{"1min":0.36399999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.80500000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.67900000000000005}]},{"osd":6,"last update":"Wed Apr 24 15:32:58 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.65700000000000003}]}]},{"osd":6,"up_from":104,"seq":446676598796,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":305896,"kb_used_data":1748,"kb_used_omap":43,"kb_used_meta":304084,"kb_avail":94065944,"statfs":{"total":96636764160,"available":96323526656,"internally_reserved":0,"allocated":1789952,"data_stored":910351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":44735,"internal_metadata":311382337},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574881,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307280,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":305534,"kb_avail":94064560,"statfs":{"total":96636764160,"available":96322109440,"internally_reserved":0,"allocated":1773568,"data_stored":903176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312867160},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 15:34:01 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.35599999999999998,"15min":0.35799999999999998},"min":{"1min":0.20699999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.51100000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.377,"15min":0.376},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":1.353,"5min":1.375,"15min":1.375},"last":0.376}]},{"osd":2,"last update":"Wed Apr 24 15:34:01 2024","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.33600000000000002,"15min":0.34000000000000002},"min":{"1min":0.224,"5min":0.223,"15min":0.223},"max":{"1min":0.46100000000000002,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.255},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.41499999999999998,"15min":0.42699999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.52800000000000002,"5min":1.494,"15min":1.494},"last":0.30399999999999999}]},{"osd":3,"last update":"Wed Apr 24 15:34:01 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.42799999999999999,"15min":0.433},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.97499999999999998,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.377,"15min":0.372},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.40799999999999997}]},{"osd":4,"last update":"Wed Apr 24 15:34:01 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.53500000000000003,"15min":0.54100000000000004},"min":{"1min":0.34799999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.72199999999999998,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.495},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.57799999999999996,"15min":0.58199999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.52100000000000002}]},{"osd":5,"last update":"Wed Apr 24 15:34:01 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.379,"5min":1.454,"15min":1.454},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.53800000000000003,"15min":0.54200000000000004},"min":{"1min":0.35999999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.67100000000000004,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.56799999999999995}]},{"osd":6,"last update":"Wed Apr 24 15:33:01 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.548,"5min":1.548,"15min":1.548},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.51300000000000001}]},{"osd":7,"last update":"Wed Apr 24 15:34:01 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.56899999999999995,"15min":0.57099999999999995},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.55100000000000005,"15min":0.55700000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.69599999999999995,"5min":0.999,"15min":0.999},"last":0.53600000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574882,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303260,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94068580,"statfs":{"total":96636764160,"available":96326225920,"internally_reserved":0,"allocated":1847296,"data_stored":1500719,"data_compressed":4842,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.30499999999999999,"5min":0.45700000000000002,"15min":0.48299999999999998},"min":{"1min":0.17899999999999999,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.47999999999999998,"5min":3.7890000000000001,"15min":3.7890000000000001},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.46700000000000003,"15min":0.48899999999999999},"min":{"1min":0.224,"5min":0.191,"15min":0.191},"max":{"1min":0.59899999999999998,"5min":2.7890000000000001,"15min":2.7890000000000001},"last":0.35099999999999998}]},{"osd":2,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.53100000000000003,"15min":0.55900000000000005},"min":{"1min":0.22700000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.61399999999999999,"5min":3.9950000000000001,"15min":3.9950000000000001},"last":0.312},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.42299999999999999,"15min":0.436},"min":{"1min":0.26000000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.56799999999999995,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":0.28299999999999997}]},{"osd":3,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.48499999999999999,"15min":0.505},"min":{"1min":0.24099999999999999,"5min":0.223,"15min":0.223},"max":{"1min":0.54000000000000004,"5min":2.097,"15min":2.097},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.622,"15min":0.65700000000000003},"min":{"1min":0.30499999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.79500000000000004,"5min":3.2109999999999999,"15min":3.2109999999999999},"last":0.38800000000000001}]},{"osd":4,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.55700000000000005,"15min":0.56699999999999995},"min":{"1min":0.31,"5min":0.308,"15min":0.308},"max":{"1min":0.69099999999999995,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.65000000000000002,"15min":0.67300000000000004},"min":{"1min":0.34899999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.85599999999999998,"5min":3.282,"15min":3.282},"last":0.5}]},{"osd":5,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.60099999999999998,"15min":0.61299999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.79100000000000004,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.64600000000000002,"15min":0.66200000000000003},"min":{"1min":0.38700000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.78000000000000003,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":0.55400000000000005}]},{"osd":6,"last update":"Wed Apr 24 15:33:00 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":3.2509999999999999,"5min":3.2509999999999999,"15min":3.2509999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":2.1909999999999998,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":0.57999999999999996}]},{"osd":7,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.60299999999999998,"15min":0.60899999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.075,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.63200000000000001,"15min":0.64400000000000002},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.876,"5min":2.206,"15min":2.206},"last":0.52800000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574882,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303200,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94068640,"statfs":{"total":96636764160,"available":96326287360,"internally_reserved":0,"allocated":1785856,"data_stored":909395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 15:34:06 2024","interfaces":[{"interface":"back","average":{"1min":0.29199999999999998,"5min":0.312,"15min":0.315},"min":{"1min":0.16200000000000001,"5min":0.16200000000000001,"15min":0.16200000000000001},"max":{"1min":0.41399999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.30599999999999999,"5min":0.33700000000000002,"15min":0.34200000000000003},"min":{"1min":0.185,"5min":0.185,"15min":0.185},"max":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"last":0.442}]},{"osd":1,"last update":"Wed Apr 24 15:34:06 2024","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.36899999999999999,"15min":0.377},"min":{"1min":0.23300000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.47399999999999998,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.46100000000000002,"15min":0.48199999999999998},"min":{"1min":0.24099999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.441,"5min":3.1110000000000002,"15min":3.1110000000000002},"last":0.48999999999999999}]},{"osd":3,"last update":"Wed Apr 24 15:34:06 2024","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.40500000000000003,"15min":0.41899999999999998},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.41399999999999998,"5min":2.056,"15min":2.056},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.39900000000000002,"15min":0.40500000000000003},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.52900000000000003,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.54100000000000004}]},{"osd":4,"last update":"Wed Apr 24 15:34:06 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.51800000000000002,"15min":0.52400000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.53600000000000003,"15min":0.54500000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.70399999999999996,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.57299999999999995}]},{"osd":5,"last update":"Wed Apr 24 15:34:06 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.54300000000000004,"15min":0.54600000000000004},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.63700000000000001,"15min":0.65400000000000003},"min":{"1min":0.41399999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.80500000000000005,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":0.621}]},{"osd":6,"last update":"Wed Apr 24 15:33:03 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.1000000000000001,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.54900000000000004}]},{"osd":7,"last update":"Wed Apr 24 15:34:06 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.63500000000000001,"15min":0.65700000000000003},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.63800000000000001,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.56999999999999995,"15min":0.57799999999999996},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73999999999999999,"5min":1.502,"15min":1.502},"last":0.67400000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574882,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303204,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94068636,"statfs":{"total":96636764160,"available":96326283264,"internally_reserved":0,"allocated":1794048,"data_stored":910370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 15:34:04 2024","interfaces":[{"interface":"back","average":{"1min":0.29399999999999998,"5min":0.34599999999999997,"15min":0.35499999999999998},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.39900000000000002,"5min":1.224,"15min":1.224},"last":0.27600000000000002},{"interface":"front","average":{"1min":0.30199999999999999,"5min":0.34000000000000002,"15min":0.34599999999999997},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.41999999999999998,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.30099999999999999}]},{"osd":1,"last update":"Wed Apr 24 15:34:04 2024","interfaces":[{"interface":"back","average":{"1min":0.30299999999999999,"5min":0.33400000000000002,"15min":0.33900000000000002},"min":{"1min":0.19600000000000001,"5min":0.187,"15min":0.187},"max":{"1min":0.437,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.42199999999999999,"15min":0.434},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.71299999999999997,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.39400000000000002}]},{"osd":2,"last update":"Wed Apr 24 15:34:04 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.39400000000000002,"15min":0.40300000000000002},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.46500000000000002,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.376,"5min":0.41299999999999998,"15min":0.41899999999999998},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.53000000000000003,"5min":1.252,"15min":1.252},"last":0.44800000000000001}]},{"osd":4,"last update":"Wed Apr 24 15:34:04 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.52200000000000002,"15min":0.53400000000000003},"min":{"1min":0.28599999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.56499999999999995,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.52100000000000002,"15min":0.53300000000000003},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.60699999999999998,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.69999999999999996}]},{"osd":5,"last update":"Wed Apr 24 15:34:04 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.55600000000000005,"15min":0.56999999999999995},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.61199999999999999,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.5,"5min":0.58799999999999997,"15min":0.60199999999999998},"min":{"1min":0.38300000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.81100000000000005,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.621}]},{"osd":6,"last update":"Wed Apr 24 15:33:00 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.60199999999999998}]},{"osd":7,"last update":"Wed Apr 24 15:34:04 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.59099999999999997,"15min":0.60499999999999998},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.66800000000000004,"5min":1.635,"15min":1.635},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.495,"5min":0.55600000000000005,"15min":0.56599999999999995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.59699999999999998,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.64500000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574883,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307284,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":305534,"kb_avail":94064556,"statfs":{"total":96636764160,"available":96322105344,"internally_reserved":0,"allocated":1773568,"data_stored":901837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312867160},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 15:34:00 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.505,"15min":0.50900000000000001},"min":{"1min":0.249,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":1.8620000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.626,"5min":0.56599999999999995,"15min":0.55600000000000005},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":5.6239999999999997,"5min":5.6239999999999997,"15min":5.6239999999999997},"last":0.64100000000000001}]},{"osd":1,"last update":"Wed Apr 24 15:34:00 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.58699999999999997,"15min":0.57299999999999995},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":5.7149999999999999,"5min":5.7149999999999999,"15min":5.7149999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.35499999999999998,"5min":0.31,"15min":0.31},"max":{"1min":1.913,"5min":1.913,"15min":1.913},"last":0.53100000000000003}]},{"osd":2,"last update":"Wed Apr 24 15:34:00 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.54600000000000004,"15min":0.52800000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":5.6660000000000004,"5min":5.6660000000000004,"15min":5.6660000000000004},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.53400000000000003,"15min":0.54400000000000004},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.819,"5min":1.819,"15min":1.819},"last":0.46800000000000003}]},{"osd":3,"last update":"Wed Apr 24 15:34:00 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.53400000000000003,"15min":0.53600000000000003},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.9550000000000001,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":0.502},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.55500000000000005,"15min":0.55000000000000004},"min":{"1min":0.36799999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":2.3149999999999999,"5min":2.3149999999999999,"15min":2.3149999999999999},"last":0.55400000000000005}]},{"osd":5,"last update":"Wed Apr 24 15:34:00 2024","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.58299999999999996,"15min":0.51700000000000002},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":14.667999999999999,"5min":14.667999999999999,"15min":14.667999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.434,"5min":0.497,"15min":0.50700000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.9890000000000001,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":0.76400000000000001}]},{"osd":6,"last update":"Wed Apr 24 15:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.73399999999999999}]},{"osd":7,"last update":"Wed Apr 24 15:34:00 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.54000000000000004,"15min":0.53100000000000003},"min":{"1min":0.28599999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":5.2759999999999998,"5min":5.2759999999999998,"15min":5.2759999999999998},"last":0.71999999999999997},{"interface":"front","average":{"1min":1.0469999999999999,"5min":0.64100000000000001,"15min":0.57299999999999995},"min":{"1min":0.29399999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":14.773,"5min":14.773,"15min":14.773},"last":0.78700000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574882,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311380,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":309630,"kb_avail":94060460,"statfs":{"total":96636764160,"available":96317911040,"internally_reserved":0,"allocated":1777664,"data_stored":904132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317061464},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.47899999999999998,"15min":0.48499999999999999},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.46800000000000003,"15min":0.47599999999999998},"min":{"1min":0.246,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.629,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.55300000000000005}]},{"osd":1,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.46300000000000002,"15min":0.47099999999999997},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.64900000000000002,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.46899999999999997,"15min":0.47099999999999997},"min":{"1min":0.317,"5min":0.313,"15min":0.313},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.67300000000000004}]},{"osd":2,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.47499999999999998,"15min":0.46600000000000003},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":2.4169999999999998,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.48599999999999999,"15min":0.49199999999999999},"min":{"1min":0.30099999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.64200000000000002}]},{"osd":3,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.48899999999999999,"15min":0.495},"min":{"1min":0.308,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.70999999999999996,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":2.3370000000000002,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":0.65900000000000003}]},{"osd":4,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.44800000000000001,"15min":0.45700000000000002},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.624,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.752},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.48399999999999999,"15min":0.48599999999999999},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.59899999999999998}]},{"osd":6,"last update":"Wed Apr 24 15:33:02 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.71599999999999997}]},{"osd":7,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.52600000000000002,"15min":0.53100000000000003},"min":{"1min":0.30299999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.78600000000000003,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.496,"5min":0.53200000000000003,"15min":0.53800000000000003},"min":{"1min":0.34499999999999997,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.77100000000000002,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.76500000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4842,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":5677,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":5677,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15242,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T15:34:34.211 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-24T15:34:34.212 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T15:34:34.525 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:34:34.527 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-04-24T15:34:34.537 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":220,"stamp":"2024-04-24T15:34:33.580033+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":51,"num_read":114923,"num_read_kb":727642,"num_write":20449,"num_write_kb":358744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4420,"num_bytes_recovered":106356266,"num_keys_recovered":235,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17371,"ondisk_log_size":17371,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2440640,"kb_used_data":14024,"kb_used_omap":55,"kb_used_meta":2426440,"kb_avail":752534080,"statfs":{"total":773094113280,"available":770594897920,"internally_reserved":0,"allocated":14360576,"data_stored":8434480,"data_compressed":10519,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":56663,"internal_metadata":2484675241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.729940"},"pg_stats":[{"pgid":"4.1f","version":"131'1485","reported_seq":2550,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698777+0000","last_change":"2024-04-24T15:34:15.652525+0000","last_active":"2024-04-24T15:34:28.698777+0000","last_peered":"2024-04-24T15:34:28.698777+0000","last_clean":"2024-04-24T15:34:28.698777+0000","last_became_active":"2024-04-24T15:34:15.650850+0000","last_became_peered":"2024-04-24T15:34:15.650850+0000","last_unstale":"2024-04-24T15:34:28.698777+0000","last_undegraded":"2024-04-24T15:34:28.698777+0000","last_fullsized":"2024-04-24T15:34:28.698777+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":1485,"log_dups_size":0,"ondisk_log_size":1485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:06:34.577605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001749947,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":8143,"num_read_kb":14722,"num_write":2575,"num_write_kb":16077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":5952278,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"129'507","reported_seq":6418,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698830+0000","last_change":"2024-04-24T15:34:15.669801+0000","last_active":"2024-04-24T15:34:28.698830+0000","last_peered":"2024-04-24T15:34:28.698830+0000","last_clean":"2024-04-24T15:34:28.698830+0000","last_became_active":"2024-04-24T15:34:15.668657+0000","last_became_peered":"2024-04-24T15:34:15.668657+0000","last_unstale":"2024-04-24T15:34:28.698830+0000","last_undegraded":"2024-04-24T15:34:28.698830+0000","last_fullsized":"2024-04-24T15:34:28.698830+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":507,"log_dups_size":0,"ondisk_log_size":507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:32:57.253624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029386599999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":7278,"num_read_kb":39113,"num_write":560,"num_write_kb":6592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":4495932,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"129'529","reported_seq":3745,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861949+0000","last_change":"2024-04-24T15:34:16.660701+0000","last_active":"2024-04-24T15:34:27.861949+0000","last_peered":"2024-04-24T15:34:27.861949+0000","last_clean":"2024-04-24T15:34:27.861949+0000","last_became_active":"2024-04-24T15:34:16.659202+0000","last_became_peered":"2024-04-24T15:34:16.659202+0000","last_unstale":"2024-04-24T15:34:27.861949+0000","last_undegraded":"2024-04-24T15:34:27.861949+0000","last_fullsized":"2024-04-24T15:34:27.861949+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":529,"log_dups_size":0,"ondisk_log_size":529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:31:57.762122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015488839999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":4427,"num_read_kb":51896,"num_write":603,"num_write_kb":18759,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":580671,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"131'526","reported_seq":2700,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.699161+0000","last_change":"2024-04-24T15:34:19.807684+0000","last_active":"2024-04-24T15:34:28.699161+0000","last_peered":"2024-04-24T15:34:28.699161+0000","last_clean":"2024-04-24T15:34:28.699161+0000","last_became_active":"2024-04-24T15:34:19.806783+0000","last_became_peered":"2024-04-24T15:34:19.806783+0000","last_unstale":"2024-04-24T15:34:28.699161+0000","last_undegraded":"2024-04-24T15:34:28.699161+0000","last_fullsized":"2024-04-24T15:34:28.699161+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":526,"log_dups_size":0,"ondisk_log_size":526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:06:01.349645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001413493,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":3716,"num_read_kb":10078,"num_write":547,"num_write_kb":2888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"129'385","reported_seq":2359,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861581+0000","last_change":"2024-04-24T15:34:23.827263+0000","last_active":"2024-04-24T15:34:27.861581+0000","last_peered":"2024-04-24T15:34:27.861581+0000","last_clean":"2024-04-24T15:34:27.861581+0000","last_became_active":"2024-04-24T15:34:23.825472+0000","last_became_peered":"2024-04-24T15:34:23.825472+0000","last_unstale":"2024-04-24T15:34:27.861581+0000","last_undegraded":"2024-04-24T15:34:27.861581+0000","last_fullsized":"2024-04-24T15:34:27.861581+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":385,"log_dups_size":0,"ondisk_log_size":385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:19:37.470876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011107479999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":2335,"num_read_kb":25405,"num_write":329,"num_write_kb":6118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":79,"num_bytes_recovered":2143880,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"129'449","reported_seq":6610,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861569+0000","last_change":"2024-04-24T15:34:23.827554+0000","last_active":"2024-04-24T15:34:27.861569+0000","last_peered":"2024-04-24T15:34:27.861569+0000","last_clean":"2024-04-24T15:34:27.861569+0000","last_became_active":"2024-04-24T15:34:23.824861+0000","last_became_peered":"2024-04-24T15:34:23.824861+0000","last_unstale":"2024-04-24T15:34:27.861569+0000","last_undegraded":"2024-04-24T15:34:27.861569+0000","last_fullsized":"2024-04-24T15:34:27.861569+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:42:20.554334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097376600000000004,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":6368,"num_read_kb":19805,"num_write":475,"num_write_kb":10193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":89,"num_bytes_recovered":1137673,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"129'537","reported_seq":2680,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861805+0000","last_change":"2024-04-24T15:34:08.756894+0000","last_active":"2024-04-24T15:34:27.861805+0000","last_peered":"2024-04-24T15:34:27.861805+0000","last_clean":"2024-04-24T15:34:27.861805+0000","last_became_active":"2024-04-24T15:32:17.047842+0000","last_became_peered":"2024-04-24T15:32:17.047842+0000","last_unstale":"2024-04-24T15:34:27.861805+0000","last_undegraded":"2024-04-24T15:34:27.861805+0000","last_fullsized":"2024-04-24T15:34:27.861805+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":537,"log_dups_size":0,"ondisk_log_size":537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:09:52.065772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081826100000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":4119,"num_read_kb":22694,"num_write":527,"num_write_kb":11193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"130'444","reported_seq":1968,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861537+0000","last_change":"2024-04-24T15:34:08.758067+0000","last_active":"2024-04-24T15:34:27.861537+0000","last_peered":"2024-04-24T15:34:27.861537+0000","last_clean":"2024-04-24T15:34:27.861537+0000","last_became_active":"2024-04-24T15:33:42.641679+0000","last_became_peered":"2024-04-24T15:33:42.641679+0000","last_unstale":"2024-04-24T15:34:27.861537+0000","last_undegraded":"2024-04-24T15:34:27.861537+0000","last_fullsized":"2024-04-24T15:34:27.861537+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":444,"log_dups_size":0,"ondisk_log_size":444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:29:58.781375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077716999999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":1887,"num_read_kb":15406,"num_write":547,"num_write_kb":8155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":4099152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"129'523","reported_seq":2447,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861936+0000","last_change":"2024-04-24T15:34:08.758924+0000","last_active":"2024-04-24T15:34:27.861936+0000","last_peered":"2024-04-24T15:34:27.861936+0000","last_clean":"2024-04-24T15:34:27.861936+0000","last_became_active":"2024-04-24T15:33:56.699629+0000","last_became_peered":"2024-04-24T15:33:56.699629+0000","last_unstale":"2024-04-24T15:34:27.861936+0000","last_undegraded":"2024-04-24T15:34:27.861936+0000","last_fullsized":"2024-04-24T15:34:27.861936+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":523,"log_dups_size":0,"ondisk_log_size":523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:53:30.709042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00212766,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":3353,"num_read_kb":16260,"num_write":594,"num_write_kb":12753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":352256,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"129'316","reported_seq":2577,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861623+0000","last_change":"2024-04-24T15:34:23.829028+0000","last_active":"2024-04-24T15:34:27.861623+0000","last_peered":"2024-04-24T15:34:27.861623+0000","last_clean":"2024-04-24T15:34:27.861623+0000","last_became_active":"2024-04-24T15:34:23.827917+0000","last_became_peered":"2024-04-24T15:34:23.827917+0000","last_unstale":"2024-04-24T15:34:27.861623+0000","last_undegraded":"2024-04-24T15:34:27.861623+0000","last_fullsized":"2024-04-24T15:34:27.861623+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":316,"log_dups_size":0,"ondisk_log_size":316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:11:49.794260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065534800000000004,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":1633,"num_read_kb":26736,"num_write":303,"num_write_kb":13822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":9304722,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":121,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698692+0000","last_change":"2024-04-24T15:33:53.644249+0000","last_active":"2024-04-24T15:34:28.698692+0000","last_peered":"2024-04-24T15:34:28.698692+0000","last_clean":"2024-04-24T15:34:28.698692+0000","last_became_active":"2024-04-24T15:33:53.643941+0000","last_became_peered":"2024-04-24T15:33:53.643941+0000","last_unstale":"2024-04-24T15:34:28.698692+0000","last_undegraded":"2024-04-24T15:34:28.698692+0000","last_fullsized":"2024-04-24T15:34:28.698692+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:42:05.771051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"130'862","reported_seq":3009,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698843+0000","last_change":"2024-04-24T15:34:19.810827+0000","last_active":"2024-04-24T15:34:28.698843+0000","last_peered":"2024-04-24T15:34:28.698843+0000","last_clean":"2024-04-24T15:34:28.698843+0000","last_became_active":"2024-04-24T15:34:19.809619+0000","last_became_peered":"2024-04-24T15:34:19.809619+0000","last_unstale":"2024-04-24T15:34:28.698843+0000","last_undegraded":"2024-04-24T15:34:28.698843+0000","last_fullsized":"2024-04-24T15:34:28.698843+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:42:29.391778+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00033353700000000002,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":4689,"num_read_kb":23870,"num_write":1044,"num_write_kb":13092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":176,"num_bytes_recovered":5601476,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":24,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.699509+0000","last_change":"2024-04-24T15:34:12.652580+0000","last_active":"2024-04-24T15:34:28.699509+0000","last_peered":"2024-04-24T15:34:28.699509+0000","last_clean":"2024-04-24T15:34:28.699509+0000","last_became_active":"2024-04-24T15:34:12.652298+0000","last_became_peered":"2024-04-24T15:34:12.652298+0000","last_unstale":"2024-04-24T15:34:28.699509+0000","last_undegraded":"2024-04-24T15:34:28.699509+0000","last_fullsized":"2024-04-24T15:34:28.699509+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:22:59.160275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":41,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.862035+0000","last_change":"2024-04-24T15:33:53.645875+0000","last_active":"2024-04-24T15:34:27.862035+0000","last_peered":"2024-04-24T15:34:27.862035+0000","last_clean":"2024-04-24T15:34:27.862035+0000","last_became_active":"2024-04-24T15:33:53.645696+0000","last_became_peered":"2024-04-24T15:33:53.645696+0000","last_unstale":"2024-04-24T15:34:27.862035+0000","last_undegraded":"2024-04-24T15:34:27.862035+0000","last_fullsized":"2024-04-24T15:34:27.862035+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:29:35.345558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"131'584","reported_seq":5172,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698983+0000","last_change":"2024-04-24T15:34:17.655614+0000","last_active":"2024-04-24T15:34:28.698983+0000","last_peered":"2024-04-24T15:34:28.698983+0000","last_clean":"2024-04-24T15:34:28.698983+0000","last_became_active":"2024-04-24T15:34:17.654759+0000","last_became_peered":"2024-04-24T15:34:17.654759+0000","last_unstale":"2024-04-24T15:34:28.698983+0000","last_undegraded":"2024-04-24T15:34:28.698983+0000","last_fullsized":"2024-04-24T15:34:28.698983+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":584,"log_dups_size":0,"ondisk_log_size":584,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:42:40.438352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061779100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3592,"num_read_kb":14899,"num_write":598,"num_write_kb":11359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":23598,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"60'1","reported_seq":80,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698947+0000","last_change":"2024-04-24T15:33:47.663123+0000","last_active":"2024-04-24T15:34:28.698947+0000","last_peered":"2024-04-24T15:34:28.698947+0000","last_clean":"2024-04-24T15:34:28.698947+0000","last_became_active":"2024-04-24T15:33:47.662685+0000","last_became_peered":"2024-04-24T15:33:47.662685+0000","last_unstale":"2024-04-24T15:34:28.698947+0000","last_undegraded":"2024-04-24T15:34:28.698947+0000","last_fullsized":"2024-04-24T15:34:28.698947+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_clean_scrub_stamp":"2024-04-24T15:32:06.972928+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:05:35.654980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":50,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861972+0000","last_change":"2024-04-24T15:34:12.653126+0000","last_active":"2024-04-24T15:34:27.861972+0000","last_peered":"2024-04-24T15:34:27.861972+0000","last_clean":"2024-04-24T15:34:27.861972+0000","last_became_active":"2024-04-24T15:34:12.652805+0000","last_became_peered":"2024-04-24T15:34:12.652805+0000","last_unstale":"2024-04-24T15:34:27.861972+0000","last_undegraded":"2024-04-24T15:34:27.861972+0000","last_fullsized":"2024-04-24T15:34:27.861972+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:44:23.434720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"131'615","reported_seq":2369,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698926+0000","last_change":"2024-04-24T15:34:16.760282+0000","last_active":"2024-04-24T15:34:28.698926+0000","last_peered":"2024-04-24T15:34:28.698926+0000","last_clean":"2024-04-24T15:34:28.698926+0000","last_became_active":"2024-04-24T15:34:16.655390+0000","last_became_peered":"2024-04-24T15:34:16.655390+0000","last_unstale":"2024-04-24T15:34:28.698926+0000","last_undegraded":"2024-04-24T15:34:28.698926+0000","last_fullsized":"2024-04-24T15:34:28.698926+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":615,"log_dups_size":0,"ondisk_log_size":615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:01:32.241481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042917399999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":3015,"num_read_kb":19823,"num_write":696,"num_write_kb":5112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"90'2","reported_seq":182,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861981+0000","last_change":"2024-04-24T15:34:13.666629+0000","last_active":"2024-04-24T15:34:27.861981+0000","last_peered":"2024-04-24T15:34:27.861981+0000","last_clean":"2024-04-24T15:34:27.861981+0000","last_became_active":"2024-04-24T15:34:13.662363+0000","last_became_peered":"2024-04-24T15:34:13.662363+0000","last_unstale":"2024-04-24T15:34:27.861981+0000","last_undegraded":"2024-04-24T15:34:27.861981+0000","last_fullsized":"2024-04-24T15:34:27.861981+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_clean_scrub_stamp":"2024-04-24T15:32:06.972928+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:09:38.698619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":251,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.862026+0000","last_change":"2024-04-24T15:33:57.644653+0000","last_active":"2024-04-24T15:34:27.862026+0000","last_peered":"2024-04-24T15:34:27.862026+0000","last_clean":"2024-04-24T15:34:27.862026+0000","last_became_active":"2024-04-24T15:33:57.644312+0000","last_became_peered":"2024-04-24T15:33:57.644312+0000","last_unstale":"2024-04-24T15:34:27.862026+0000","last_undegraded":"2024-04-24T15:34:27.862026+0000","last_fullsized":"2024-04-24T15:34:27.862026+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:31:59.935287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:31:59.935287+0000","last_clean_scrub_stamp":"2024-04-24T15:31:59.935287+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:21:46.534664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698724+0000","last_change":"2024-04-24T15:34:13.650533+0000","last_active":"2024-04-24T15:34:28.698724+0000","last_peered":"2024-04-24T15:34:28.698724+0000","last_clean":"2024-04-24T15:34:28.698724+0000","last_became_active":"2024-04-24T15:34:13.650174+0000","last_became_peered":"2024-04-24T15:34:13.650174+0000","last_unstale":"2024-04-24T15:34:28.698724+0000","last_undegraded":"2024-04-24T15:34:28.698724+0000","last_fullsized":"2024-04-24T15:34:28.698724+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:01:43.442578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"129'591","reported_seq":2456,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698898+0000","last_change":"2024-04-24T15:34:08.805854+0000","last_active":"2024-04-24T15:34:28.698898+0000","last_peered":"2024-04-24T15:34:28.698898+0000","last_clean":"2024-04-24T15:34:28.698898+0000","last_became_active":"2024-04-24T15:33:45.658207+0000","last_became_peered":"2024-04-24T15:33:45.658207+0000","last_unstale":"2024-04-24T15:34:28.698898+0000","last_undegraded":"2024-04-24T15:34:28.698898+0000","last_fullsized":"2024-04-24T15:34:28.698898+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":591,"log_dups_size":0,"ondisk_log_size":591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:59:01.456518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010111199999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":3352,"num_read_kb":26205,"num_write":791,"num_write_kb":11816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":8821091,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":24,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.699087+0000","last_change":"2024-04-24T15:34:12.655959+0000","last_active":"2024-04-24T15:34:28.699087+0000","last_peered":"2024-04-24T15:34:28.699087+0000","last_clean":"2024-04-24T15:34:28.699087+0000","last_became_active":"2024-04-24T15:34:12.655508+0000","last_became_peered":"2024-04-24T15:34:12.655508+0000","last_unstale":"2024-04-24T15:34:28.699087+0000","last_undegraded":"2024-04-24T15:34:28.699087+0000","last_fullsized":"2024-04-24T15:34:28.699087+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_clean_scrub_stamp":"2024-04-24T15:32:06.972928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:13:50.464460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":59,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.699249+0000","last_change":"2024-04-24T15:34:19.838358+0000","last_active":"2024-04-24T15:34:28.699249+0000","last_peered":"2024-04-24T15:34:28.699249+0000","last_clean":"2024-04-24T15:34:28.699249+0000","last_became_active":"2024-04-24T15:34:19.838058+0000","last_became_peered":"2024-04-24T15:34:19.838058+0000","last_unstale":"2024-04-24T15:34:28.699249+0000","last_undegraded":"2024-04-24T15:34:28.699249+0000","last_fullsized":"2024-04-24T15:34:28.699249+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:29:43.725200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"131'430","reported_seq":1231,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.699369+0000","last_change":"2024-04-24T15:34:15.660049+0000","last_active":"2024-04-24T15:34:28.699369+0000","last_peered":"2024-04-24T15:34:28.699369+0000","last_clean":"2024-04-24T15:34:28.699369+0000","last_became_active":"2024-04-24T15:34:15.658368+0000","last_became_peered":"2024-04-24T15:34:15.658368+0000","last_unstale":"2024-04-24T15:34:28.699369+0000","last_undegraded":"2024-04-24T15:34:28.699369+0000","last_fullsized":"2024-04-24T15:34:28.699369+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:29:46.353757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015045289999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":1970,"num_read_kb":12710,"num_write":463,"num_write_kb":8069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":208,"num_bytes_recovered":107133,"num_keys_recovered":27,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":93,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861700+0000","last_change":"2024-04-24T15:34:24.428957+0000","last_active":"2024-04-24T15:34:27.861700+0000","last_peered":"2024-04-24T15:34:27.861700+0000","last_clean":"2024-04-24T15:34:27.861700+0000","last_became_active":"2024-04-24T15:34:24.428510+0000","last_became_peered":"2024-04-24T15:34:24.428510+0000","last_unstale":"2024-04-24T15:34:27.861700+0000","last_undegraded":"2024-04-24T15:34:27.861700+0000","last_fullsized":"2024-04-24T15:34:27.861700+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_clean_scrub_stamp":"2024-04-24T15:32:06.972928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:36:35.138337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698664+0000","last_change":"2024-04-24T15:34:13.689898+0000","last_active":"2024-04-24T15:34:28.698664+0000","last_peered":"2024-04-24T15:34:28.698664+0000","last_clean":"2024-04-24T15:34:28.698664+0000","last_became_active":"2024-04-24T15:34:13.689685+0000","last_became_peered":"2024-04-24T15:34:13.689685+0000","last_unstale":"2024-04-24T15:34:28.698664+0000","last_undegraded":"2024-04-24T15:34:28.698664+0000","last_fullsized":"2024-04-24T15:34:28.698664+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:24:27.251519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"130'457","reported_seq":1995,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.699625+0000","last_change":"2024-04-24T15:34:16.654083+0000","last_active":"2024-04-24T15:34:28.699625+0000","last_peered":"2024-04-24T15:34:28.699625+0000","last_clean":"2024-04-24T15:34:28.699625+0000","last_became_active":"2024-04-24T15:34:16.651906+0000","last_became_peered":"2024-04-24T15:34:16.651906+0000","last_unstale":"2024-04-24T15:34:28.699625+0000","last_undegraded":"2024-04-24T15:34:28.699625+0000","last_fullsized":"2024-04-24T15:34:28.699625+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":457,"log_dups_size":0,"ondisk_log_size":457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:54.315154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010565220000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2138,"num_read_kb":24032,"num_write":423,"num_write_kb":14900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"130'408","reported_seq":1974,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.699200+0000","last_change":"2024-04-24T15:34:16.653875+0000","last_active":"2024-04-24T15:34:28.699200+0000","last_peered":"2024-04-24T15:34:28.699200+0000","last_clean":"2024-04-24T15:34:28.699200+0000","last_became_active":"2024-04-24T15:34:16.651931+0000","last_became_peered":"2024-04-24T15:34:16.651931+0000","last_unstale":"2024-04-24T15:34:28.699200+0000","last_undegraded":"2024-04-24T15:34:28.699200+0000","last_fullsized":"2024-04-24T15:34:28.699200+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:59:24.823763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039757799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2388,"num_read_kb":22956,"num_write":379,"num_write_kb":16060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":578400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.699581+0000","last_change":"2024-04-24T15:34:12.653777+0000","last_active":"2024-04-24T15:34:28.699581+0000","last_peered":"2024-04-24T15:34:28.699581+0000","last_clean":"2024-04-24T15:34:28.699581+0000","last_became_active":"2024-04-24T15:34:12.653529+0000","last_became_peered":"2024-04-24T15:34:12.653529+0000","last_unstale":"2024-04-24T15:34:28.699581+0000","last_undegraded":"2024-04-24T15:34:28.699581+0000","last_fullsized":"2024-04-24T15:34:28.699581+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:21:58.411545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"129'479","reported_seq":1997,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698576+0000","last_change":"2024-04-24T15:34:16.656437+0000","last_active":"2024-04-24T15:34:28.698576+0000","last_peered":"2024-04-24T15:34:28.698576+0000","last_clean":"2024-04-24T15:34:28.698576+0000","last_became_active":"2024-04-24T15:34:16.654252+0000","last_became_peered":"2024-04-24T15:34:16.654252+0000","last_unstale":"2024-04-24T15:34:28.698576+0000","last_undegraded":"2024-04-24T15:34:28.698576+0000","last_fullsized":"2024-04-24T15:34:28.698576+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:53:07.507328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035787500000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":2434,"num_read_kb":14959,"num_write":545,"num_write_kb":12213,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":335,"num_bytes_recovered":6920328,"num_keys_recovered":43,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"130'616","reported_seq":2734,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861566+0000","last_change":"2024-04-24T15:34:23.827845+0000","last_active":"2024-04-24T15:34:27.861566+0000","last_peered":"2024-04-24T15:34:27.861566+0000","last_clean":"2024-04-24T15:34:27.861566+0000","last_became_active":"2024-04-24T15:34:23.825759+0000","last_became_peered":"2024-04-24T15:34:23.825759+0000","last_unstale":"2024-04-24T15:34:27.861566+0000","last_undegraded":"2024-04-24T15:34:27.861566+0000","last_fullsized":"2024-04-24T15:34:27.861566+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:49:46.424217+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0055639879999999997,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":4324,"num_read_kb":21128,"num_write":677,"num_write_kb":10933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":5713612,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"130'485","reported_seq":3231,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698742+0000","last_change":"2024-04-24T15:34:15.659323+0000","last_active":"2024-04-24T15:34:28.698742+0000","last_peered":"2024-04-24T15:34:28.698742+0000","last_clean":"2024-04-24T15:34:28.698742+0000","last_became_active":"2024-04-24T15:34:15.657660+0000","last_became_peered":"2024-04-24T15:34:15.657660+0000","last_unstale":"2024-04-24T15:34:28.698742+0000","last_undegraded":"2024-04-24T15:34:28.698742+0000","last_fullsized":"2024-04-24T15:34:28.698742+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:09:16.071953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066763,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3120,"num_read_kb":27458,"num_write":430,"num_write_kb":12267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":237,"num_bytes_recovered":5299136,"num_keys_recovered":49,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"129'551","reported_seq":1976,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698866+0000","last_change":"2024-04-24T15:34:15.669951+0000","last_active":"2024-04-24T15:34:28.698866+0000","last_peered":"2024-04-24T15:34:28.698866+0000","last_clean":"2024-04-24T15:34:28.698866+0000","last_became_active":"2024-04-24T15:34:15.668717+0000","last_became_peered":"2024-04-24T15:34:15.668717+0000","last_unstale":"2024-04-24T15:34:28.698866+0000","last_undegraded":"2024-04-24T15:34:28.698866+0000","last_fullsized":"2024-04-24T15:34:28.698866+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":551,"log_dups_size":0,"ondisk_log_size":551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:46:28.552856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039775200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2839,"num_read_kb":21248,"num_write":721,"num_write_kb":8473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":141,"num_bytes_recovered":1490474,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"127'1046","reported_seq":2505,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861716+0000","last_change":"2024-04-24T15:34:23.826944+0000","last_active":"2024-04-24T15:34:27.861716+0000","last_peered":"2024-04-24T15:34:27.861716+0000","last_clean":"2024-04-24T15:34:27.861716+0000","last_became_active":"2024-04-24T15:34:23.825039+0000","last_became_peered":"2024-04-24T15:34:23.825039+0000","last_unstale":"2024-04-24T15:34:27.861716+0000","last_undegraded":"2024-04-24T15:34:27.861716+0000","last_fullsized":"2024-04-24T15:34:27.861716+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":1046,"log_dups_size":0,"ondisk_log_size":1046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:15:05.349996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029127150000000002,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":6311,"num_read_kb":16558,"num_write":1677,"num_write_kb":12129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":54,"num_bytes_recovered":1219753,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.d","version":"127'298","reported_seq":2572,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861837+0000","last_change":"2024-04-24T15:34:23.825797+0000","last_active":"2024-04-24T15:34:27.861837+0000","last_peered":"2024-04-24T15:34:27.861837+0000","last_clean":"2024-04-24T15:34:27.861837+0000","last_became_active":"2024-04-24T15:34:23.823763+0000","last_became_peered":"2024-04-24T15:34:23.823763+0000","last_unstale":"2024-04-24T15:34:27.861837+0000","last_undegraded":"2024-04-24T15:34:27.861837+0000","last_fullsized":"2024-04-24T15:34:27.861837+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":298,"log_dups_size":0,"ondisk_log_size":298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:24:00.110009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047837299999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":2121,"num_read_kb":46483,"num_write":303,"num_write_kb":20603,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":4246835,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"129'563","reported_seq":2395,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861756+0000","last_change":"2024-04-24T15:34:23.826137+0000","last_active":"2024-04-24T15:34:27.861756+0000","last_peered":"2024-04-24T15:34:27.861756+0000","last_clean":"2024-04-24T15:34:27.861756+0000","last_became_active":"2024-04-24T15:34:23.824037+0000","last_became_peered":"2024-04-24T15:34:23.824037+0000","last_unstale":"2024-04-24T15:34:27.861756+0000","last_undegraded":"2024-04-24T15:34:27.861756+0000","last_fullsized":"2024-04-24T15:34:27.861756+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":563,"log_dups_size":0,"ondisk_log_size":563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:30:41.572188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001084025,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3022,"num_read_kb":13469,"num_write":696,"num_write_kb":6185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":113,"num_bytes_recovered":1676557,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"130'907","reported_seq":3306,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.699413+0000","last_change":"2024-04-24T15:34:16.657695+0000","last_active":"2024-04-24T15:34:28.699413+0000","last_peered":"2024-04-24T15:34:28.699413+0000","last_clean":"2024-04-24T15:34:28.699413+0000","last_became_active":"2024-04-24T15:34:16.655733+0000","last_became_peered":"2024-04-24T15:34:16.655733+0000","last_unstale":"2024-04-24T15:34:28.699413+0000","last_undegraded":"2024-04-24T15:34:28.699413+0000","last_fullsized":"2024-04-24T15:34:28.699413+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":907,"log_dups_size":0,"ondisk_log_size":907,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:59:41.606707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018480180000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":5733,"num_read_kb":14488,"num_write":1011,"num_write_kb":8995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":223,"num_bytes_recovered":630687,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"130'244","reported_seq":1061,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861808+0000","last_change":"2024-04-24T15:34:15.664032+0000","last_active":"2024-04-24T15:34:27.861808+0000","last_peered":"2024-04-24T15:34:27.861808+0000","last_clean":"2024-04-24T15:34:27.861808+0000","last_became_active":"2024-04-24T15:34:15.663081+0000","last_became_peered":"2024-04-24T15:34:15.663081+0000","last_unstale":"2024-04-24T15:34:27.861808+0000","last_undegraded":"2024-04-24T15:34:27.861808+0000","last_fullsized":"2024-04-24T15:34:27.861808+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":244,"log_dups_size":0,"ondisk_log_size":244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:33:57.743630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010945530000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":384,"num_read_kb":4092,"num_write":235,"num_write_kb":5715,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":1989443,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.11","version":"129'705","reported_seq":2162,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861883+0000","last_change":"2024-04-24T15:34:08.757586+0000","last_active":"2024-04-24T15:34:27.861883+0000","last_peered":"2024-04-24T15:34:27.861883+0000","last_clean":"2024-04-24T15:34:27.861883+0000","last_became_active":"2024-04-24T15:33:57.644309+0000","last_became_peered":"2024-04-24T15:33:57.644309+0000","last_unstale":"2024-04-24T15:34:27.861883+0000","last_undegraded":"2024-04-24T15:34:27.861883+0000","last_fullsized":"2024-04-24T15:34:27.861883+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":705,"log_dups_size":0,"ondisk_log_size":705,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:26:21.749588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012982219999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":2615,"num_read_kb":21488,"num_write":891,"num_write_kb":14014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":5651177,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"129'331","reported_seq":5228,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.699265+0000","last_change":"2024-04-24T15:34:15.655138+0000","last_active":"2024-04-24T15:34:28.699265+0000","last_peered":"2024-04-24T15:34:28.699265+0000","last_clean":"2024-04-24T15:34:28.699265+0000","last_became_active":"2024-04-24T15:34:15.653205+0000","last_became_peered":"2024-04-24T15:34:15.653205+0000","last_unstale":"2024-04-24T15:34:28.699265+0000","last_undegraded":"2024-04-24T15:34:28.699265+0000","last_fullsized":"2024-04-24T15:34:28.699265+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":331,"log_dups_size":0,"ondisk_log_size":331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:28:59.539282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013576079999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":4982,"num_read_kb":42072,"num_write":333,"num_write_kb":14625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":3758016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"129'241","reported_seq":2003,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861679+0000","last_change":"2024-04-24T15:34:23.825462+0000","last_active":"2024-04-24T15:34:27.861679+0000","last_peered":"2024-04-24T15:34:27.861679+0000","last_clean":"2024-04-24T15:34:27.861679+0000","last_became_active":"2024-04-24T15:34:23.823962+0000","last_became_peered":"2024-04-24T15:34:23.823962+0000","last_unstale":"2024-04-24T15:34:27.861679+0000","last_undegraded":"2024-04-24T15:34:27.861679+0000","last_fullsized":"2024-04-24T15:34:27.861679+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":241,"log_dups_size":0,"ondisk_log_size":241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:52:34.277987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000635438,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":1312,"num_read_kb":18878,"num_write":253,"num_write_kb":9844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":108,"num_bytes_recovered":2694373,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"131'450","reported_seq":6595,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.862055+0000","last_change":"2024-04-24T15:34:16.660466+0000","last_active":"2024-04-24T15:34:27.862055+0000","last_peered":"2024-04-24T15:34:27.862055+0000","last_clean":"2024-04-24T15:34:27.862055+0000","last_became_active":"2024-04-24T15:34:16.659071+0000","last_became_peered":"2024-04-24T15:34:16.659071+0000","last_unstale":"2024-04-24T15:34:27.862055+0000","last_undegraded":"2024-04-24T15:34:27.862055+0000","last_fullsized":"2024-04-24T15:34:27.862055+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:10:28.933829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013356920000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":7366,"num_read_kb":44142,"num_write":407,"num_write_kb":9850,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":233,"num_bytes_recovered":4458876,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"130'312","reported_seq":1575,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698823+0000","last_change":"2024-04-24T15:34:16.657117+0000","last_active":"2024-04-24T15:34:28.698823+0000","last_peered":"2024-04-24T15:34:28.698823+0000","last_clean":"2024-04-24T15:34:28.698823+0000","last_became_active":"2024-04-24T15:34:16.655084+0000","last_became_peered":"2024-04-24T15:34:16.655084+0000","last_unstale":"2024-04-24T15:34:28.698823+0000","last_undegraded":"2024-04-24T15:34:28.698823+0000","last_fullsized":"2024-04-24T15:34:28.698823+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:26:41.362344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003384743,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":837,"num_read_kb":16692,"num_write":319,"num_write_kb":15488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":277,"num_bytes_recovered":16228001,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.16","version":"129'375","reported_seq":2454,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.699026+0000","last_change":"2024-04-24T15:34:15.669825+0000","last_active":"2024-04-24T15:34:28.699026+0000","last_peered":"2024-04-24T15:34:28.699026+0000","last_clean":"2024-04-24T15:34:28.699026+0000","last_became_active":"2024-04-24T15:34:15.668678+0000","last_became_peered":"2024-04-24T15:34:15.668678+0000","last_unstale":"2024-04-24T15:34:28.699026+0000","last_undegraded":"2024-04-24T15:34:28.699026+0000","last_fullsized":"2024-04-24T15:34:28.699026+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:43:39.484677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018886770000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":3004,"num_read_kb":17777,"num_write":338,"num_write_kb":8036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":48,"num_read":114807,"num_read_kb":727542,"num_write":20290,"num_write_kb":356328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4413,"num_bytes_recovered":105175530,"num_keys_recovered":235,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15242,"internal_metadata":0},"log_size":17251,"ondisk_log_size":17251,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":8,"acting":8,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":16196,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574882,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299136,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072704,"statfs":{"total":96636764160,"available":96330448896,"internally_reserved":0,"allocated":1818624,"data_stored":1494500,"data_compressed":5677,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.46000000000000002,"15min":0.46200000000000002},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.60099999999999998,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.54200000000000004,"15min":0.55600000000000005},"min":{"1min":0.33900000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.67200000000000004,"5min":2.4710000000000001,"15min":2.4710000000000001},"last":0.73999999999999999}]},{"osd":1,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.57399999999999995,"15min":0.59299999999999997},"min":{"1min":0.28100000000000003,"5min":0.248,"15min":0.248},"max":{"1min":0.63600000000000001,"5min":2.4729999999999999,"15min":2.4729999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.51500000000000001,"15min":0.52400000000000002},"min":{"1min":0.29999999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.56899999999999995,"5min":2.4020000000000001,"15min":2.4020000000000001},"last":0.61099999999999999}]},{"osd":2,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.45700000000000002,"15min":0.45800000000000002},"min":{"1min":0.35499999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.59699999999999998,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.503,"5min":0.54800000000000004,"15min":0.55600000000000005},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.82199999999999995,"5min":2.363,"15min":2.363},"last":0.752}]},{"osd":3,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.52200000000000002,"15min":0.51800000000000002},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.53800000000000003,"15min":0.53200000000000003},"min":{"1min":0.38600000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.63600000000000001}]},{"osd":4,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.41399999999999998,"15min":0.41999999999999998},"min":{"1min":0.26100000000000001,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.66100000000000003,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.42299999999999999,"15min":0.42499999999999999},"min":{"1min":0.30599999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.73499999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.441}]},{"osd":5,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.45100000000000001,"15min":0.45500000000000002},"min":{"1min":0.307,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.51600000000000001,"15min":0.52000000000000002},"min":{"1min":0.36399999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.80500000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.67900000000000005}]},{"osd":6,"last update":"Wed Apr 24 15:32:58 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.65700000000000003}]}]},{"osd":6,"up_from":104,"seq":446676598796,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":305896,"kb_used_data":1748,"kb_used_omap":43,"kb_used_meta":304084,"kb_avail":94065944,"statfs":{"total":96636764160,"available":96323526656,"internally_reserved":0,"allocated":1789952,"data_stored":910351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":44735,"internal_metadata":311382337},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574881,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307280,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":305534,"kb_avail":94064560,"statfs":{"total":96636764160,"available":96322109440,"internally_reserved":0,"allocated":1773568,"data_stored":903176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312867160},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 15:34:01 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.35599999999999998,"15min":0.35799999999999998},"min":{"1min":0.20699999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.51100000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.377,"15min":0.376},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":1.353,"5min":1.375,"15min":1.375},"last":0.376}]},{"osd":2,"last update":"Wed Apr 24 15:34:01 2024","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.33600000000000002,"15min":0.34000000000000002},"min":{"1min":0.224,"5min":0.223,"15min":0.223},"max":{"1min":0.46100000000000002,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.255},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.41499999999999998,"15min":0.42699999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.52800000000000002,"5min":1.494,"15min":1.494},"last":0.30399999999999999}]},{"osd":3,"last update":"Wed Apr 24 15:34:01 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.42799999999999999,"15min":0.433},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.97499999999999998,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.377,"15min":0.372},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.40799999999999997}]},{"osd":4,"last update":"Wed Apr 24 15:34:01 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.53500000000000003,"15min":0.54100000000000004},"min":{"1min":0.34799999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.72199999999999998,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.495},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.57799999999999996,"15min":0.58199999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.52100000000000002}]},{"osd":5,"last update":"Wed Apr 24 15:34:01 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.379,"5min":1.454,"15min":1.454},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.53800000000000003,"15min":0.54200000000000004},"min":{"1min":0.35999999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.67100000000000004,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.56799999999999995}]},{"osd":6,"last update":"Wed Apr 24 15:33:01 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.548,"5min":1.548,"15min":1.548},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.51300000000000001}]},{"osd":7,"last update":"Wed Apr 24 15:34:01 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.56899999999999995,"15min":0.57099999999999995},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.55100000000000005,"15min":0.55700000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.69599999999999995,"5min":0.999,"15min":0.999},"last":0.53600000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574882,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303260,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94068580,"statfs":{"total":96636764160,"available":96326225920,"internally_reserved":0,"allocated":1847296,"data_stored":1500719,"data_compressed":4842,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.30499999999999999,"5min":0.45700000000000002,"15min":0.48299999999999998},"min":{"1min":0.17899999999999999,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.47999999999999998,"5min":3.7890000000000001,"15min":3.7890000000000001},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.46700000000000003,"15min":0.48899999999999999},"min":{"1min":0.224,"5min":0.191,"15min":0.191},"max":{"1min":0.59899999999999998,"5min":2.7890000000000001,"15min":2.7890000000000001},"last":0.35099999999999998}]},{"osd":2,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.53100000000000003,"15min":0.55900000000000005},"min":{"1min":0.22700000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.61399999999999999,"5min":3.9950000000000001,"15min":3.9950000000000001},"last":0.312},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.42299999999999999,"15min":0.436},"min":{"1min":0.26000000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.56799999999999995,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":0.28299999999999997}]},{"osd":3,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.48499999999999999,"15min":0.505},"min":{"1min":0.24099999999999999,"5min":0.223,"15min":0.223},"max":{"1min":0.54000000000000004,"5min":2.097,"15min":2.097},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.622,"15min":0.65700000000000003},"min":{"1min":0.30499999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.79500000000000004,"5min":3.2109999999999999,"15min":3.2109999999999999},"last":0.38800000000000001}]},{"osd":4,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.55700000000000005,"15min":0.56699999999999995},"min":{"1min":0.31,"5min":0.308,"15min":0.308},"max":{"1min":0.69099999999999995,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.65000000000000002,"15min":0.67300000000000004},"min":{"1min":0.34899999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.85599999999999998,"5min":3.282,"15min":3.282},"last":0.5}]},{"osd":5,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.60099999999999998,"15min":0.61299999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.79100000000000004,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.64600000000000002,"15min":0.66200000000000003},"min":{"1min":0.38700000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.78000000000000003,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":0.55400000000000005}]},{"osd":6,"last update":"Wed Apr 24 15:33:00 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":3.2509999999999999,"5min":3.2509999999999999,"15min":3.2509999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":2.1909999999999998,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":0.57999999999999996}]},{"osd":7,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.60299999999999998,"15min":0.60899999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.075,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.63200000000000001,"15min":0.64400000000000002},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.876,"5min":2.206,"15min":2.206},"last":0.52800000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574882,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303200,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94068640,"statfs":{"total":96636764160,"available":96326287360,"internally_reserved":0,"allocated":1785856,"data_stored":909395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 15:34:06 2024","interfaces":[{"interface":"back","average":{"1min":0.29199999999999998,"5min":0.312,"15min":0.315},"min":{"1min":0.16200000000000001,"5min":0.16200000000000001,"15min":0.16200000000000001},"max":{"1min":0.41399999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.30599999999999999,"5min":0.33700000000000002,"15min":0.34200000000000003},"min":{"1min":0.185,"5min":0.185,"15min":0.185},"max":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"last":0.442}]},{"osd":1,"last update":"Wed Apr 24 15:34:06 2024","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.36899999999999999,"15min":0.377},"min":{"1min":0.23300000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.47399999999999998,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.46100000000000002,"15min":0.48199999999999998},"min":{"1min":0.24099999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.441,"5min":3.1110000000000002,"15min":3.1110000000000002},"last":0.48999999999999999}]},{"osd":3,"last update":"Wed Apr 24 15:34:06 2024","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.40500000000000003,"15min":0.41899999999999998},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.41399999999999998,"5min":2.056,"15min":2.056},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.39900000000000002,"15min":0.40500000000000003},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.52900000000000003,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.54100000000000004}]},{"osd":4,"last update":"Wed Apr 24 15:34:06 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.51800000000000002,"15min":0.52400000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.53600000000000003,"15min":0.54500000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.70399999999999996,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.57299999999999995}]},{"osd":5,"last update":"Wed Apr 24 15:34:06 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.54300000000000004,"15min":0.54600000000000004},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.63700000000000001,"15min":0.65400000000000003},"min":{"1min":0.41399999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.80500000000000005,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":0.621}]},{"osd":6,"last update":"Wed Apr 24 15:33:03 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.1000000000000001,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.54900000000000004}]},{"osd":7,"last update":"Wed Apr 24 15:34:06 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.63500000000000001,"15min":0.65700000000000003},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.63800000000000001,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.56999999999999995,"15min":0.57799999999999996},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73999999999999999,"5min":1.502,"15min":1.502},"last":0.67400000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574882,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303204,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94068636,"statfs":{"total":96636764160,"available":96326283264,"internally_reserved":0,"allocated":1794048,"data_stored":910370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 15:34:04 2024","interfaces":[{"interface":"back","average":{"1min":0.29399999999999998,"5min":0.34599999999999997,"15min":0.35499999999999998},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.39900000000000002,"5min":1.224,"15min":1.224},"last":0.27600000000000002},{"interface":"front","average":{"1min":0.30199999999999999,"5min":0.34000000000000002,"15min":0.34599999999999997},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.41999999999999998,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.30099999999999999}]},{"osd":1,"last update":"Wed Apr 24 15:34:04 2024","interfaces":[{"interface":"back","average":{"1min":0.30299999999999999,"5min":0.33400000000000002,"15min":0.33900000000000002},"min":{"1min":0.19600000000000001,"5min":0.187,"15min":0.187},"max":{"1min":0.437,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.42199999999999999,"15min":0.434},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.71299999999999997,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.39400000000000002}]},{"osd":2,"last update":"Wed Apr 24 15:34:04 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.39400000000000002,"15min":0.40300000000000002},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.46500000000000002,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.376,"5min":0.41299999999999998,"15min":0.41899999999999998},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.53000000000000003,"5min":1.252,"15min":1.252},"last":0.44800000000000001}]},{"osd":4,"last update":"Wed Apr 24 15:34:04 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.52200000000000002,"15min":0.53400000000000003},"min":{"1min":0.28599999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.56499999999999995,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.52100000000000002,"15min":0.53300000000000003},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.60699999999999998,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.69999999999999996}]},{"osd":5,"last update":"Wed Apr 24 15:34:04 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.55600000000000005,"15min":0.56999999999999995},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.61199999999999999,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.5,"5min":0.58799999999999997,"15min":0.60199999999999998},"min":{"1min":0.38300000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.81100000000000005,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.621}]},{"osd":6,"last update":"Wed Apr 24 15:33:00 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.60199999999999998}]},{"osd":7,"last update":"Wed Apr 24 15:34:04 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.59099999999999997,"15min":0.60499999999999998},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.66800000000000004,"5min":1.635,"15min":1.635},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.495,"5min":0.55600000000000005,"15min":0.56599999999999995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.59699999999999998,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.64500000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574883,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307284,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":305534,"kb_avail":94064556,"statfs":{"total":96636764160,"available":96322105344,"internally_reserved":0,"allocated":1773568,"data_stored":901837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312867160},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 15:34:00 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.505,"15min":0.50900000000000001},"min":{"1min":0.249,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":1.8620000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.626,"5min":0.56599999999999995,"15min":0.55600000000000005},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":5.6239999999999997,"5min":5.6239999999999997,"15min":5.6239999999999997},"last":0.64100000000000001}]},{"osd":1,"last update":"Wed Apr 24 15:34:00 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.58699999999999997,"15min":0.57299999999999995},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":5.7149999999999999,"5min":5.7149999999999999,"15min":5.7149999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.35499999999999998,"5min":0.31,"15min":0.31},"max":{"1min":1.913,"5min":1.913,"15min":1.913},"last":0.53100000000000003}]},{"osd":2,"last update":"Wed Apr 24 15:34:00 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.54600000000000004,"15min":0.52800000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":5.6660000000000004,"5min":5.6660000000000004,"15min":5.6660000000000004},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.53400000000000003,"15min":0.54400000000000004},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.819,"5min":1.819,"15min":1.819},"last":0.46800000000000003}]},{"osd":3,"last update":"Wed Apr 24 15:34:00 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.53400000000000003,"15min":0.53600000000000003},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.9550000000000001,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":0.502},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.55500000000000005,"15min":0.55000000000000004},"min":{"1min":0.36799999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":2.3149999999999999,"5min":2.3149999999999999,"15min":2.3149999999999999},"last":0.55400000000000005}]},{"osd":5,"last update":"Wed Apr 24 15:34:00 2024","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.58299999999999996,"15min":0.51700000000000002},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":14.667999999999999,"5min":14.667999999999999,"15min":14.667999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.434,"5min":0.497,"15min":0.50700000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.9890000000000001,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":0.76400000000000001}]},{"osd":6,"last update":"Wed Apr 24 15:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.73399999999999999}]},{"osd":7,"last update":"Wed Apr 24 15:34:00 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.54000000000000004,"15min":0.53100000000000003},"min":{"1min":0.28599999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":5.2759999999999998,"5min":5.2759999999999998,"15min":5.2759999999999998},"last":0.71999999999999997},{"interface":"front","average":{"1min":1.0469999999999999,"5min":0.64100000000000001,"15min":0.57299999999999995},"min":{"1min":0.29399999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":14.773,"5min":14.773,"15min":14.773},"last":0.78700000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574882,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311380,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":309630,"kb_avail":94060460,"statfs":{"total":96636764160,"available":96317911040,"internally_reserved":0,"allocated":1777664,"data_stored":904132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317061464},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.47899999999999998,"15min":0.48499999999999999},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.46800000000000003,"15min":0.47599999999999998},"min":{"1min":0.246,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.629,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.55300000000000005}]},{"osd":1,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.46300000000000002,"15min":0.47099999999999997},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.64900000000000002,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.46899999999999997,"15min":0.47099999999999997},"min":{"1min":0.317,"5min":0.313,"15min":0.313},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.67300000000000004}]},{"osd":2,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.47499999999999998,"15min":0.46600000000000003},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":2.4169999999999998,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.48599999999999999,"15min":0.49199999999999999},"min":{"1min":0.30099999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.64200000000000002}]},{"osd":3,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.48899999999999999,"15min":0.495},"min":{"1min":0.308,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.70999999999999996,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":2.3370000000000002,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":0.65900000000000003}]},{"osd":4,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.44800000000000001,"15min":0.45700000000000002},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.624,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.752},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.48399999999999999,"15min":0.48599999999999999},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.59899999999999998}]},{"osd":6,"last update":"Wed Apr 24 15:33:02 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.71599999999999997}]},{"osd":7,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.52600000000000002,"15min":0.53100000000000003},"min":{"1min":0.30299999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.78600000000000003,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.496,"5min":0.53200000000000003,"15min":0.53800000000000003},"min":{"1min":0.34499999999999997,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.77100000000000002,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.76500000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4842,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":5677,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":5677,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15242,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T15:34:34.540 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T15:34:34.854 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:34:34.854 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":149,"fsid":"253d426c-62cb-4194-8514-6cd970fd85c7","created":"2024-04-24T15:31:47.870327+0000","modified":"2024-04-24T15:34:28.859605+0000","last_up_change":"2024-04-24T15:33:41.967503+0000","last_in_change":"2024-04-24T15:34:12.639389+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T15:31:59.589762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-24T15:31:59.971601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"21","last_force_op_resend_prenautilus":"21","last_force_op_resend_preluminous":"21","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[3],"tier_of":-1,"read_tier":3,"write_tier":3,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cache","create_time":"2024-04-24T15:32:06.914463+0000","flags":9,"flags_names":"hashpspool,incomplete_clones","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":4,"pg_placement_num":4,"pg_placement_num_target":4,"pg_num_target":4,"pg_num_pending":4,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"21","last_force_op_resend_prenautilus":"21","last_force_op_resend_preluminous":"21","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":2,"read_tier":-1,"write_tier":-1,"cache_mode":"writeback","target_max_bytes":0,"target_max_objects":250,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"bloom","false_positive_probability":0.050000000000000003,"target_size":0,"seed":0},"hit_set_period":60,"hit_set_count":8,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"pool_client.0","create_time":"2024-04-24T15:32:15.929114+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":69,"snap_epoch":131,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"92941fb9-96ee-4b2b-8147-3c1d61192f94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":44106},{"type":"v1","addr":"172.21.15.96:6817","nonce":44106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":44106},{"type":"v1","addr":"172.21.15.96:6819","nonce":44106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":44106},{"type":"v1","addr":"172.21.15.96:6823","nonce":44106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":44106},{"type":"v1","addr":"172.21.15.96:6821","nonce":44106}]},"public_addr":"172.21.15.96:6817/44106","cluster_addr":"172.21.15.96:6819/44106","heartbeat_back_addr":"172.21.15.96:6823/44106","heartbeat_front_addr":"172.21.15.96:6821/44106","state":["exists","up"]},{"osd":1,"uuid":"628e868a-9884-42e6-9331-caef97cb6967","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":44089},{"type":"v1","addr":"172.21.15.96:6801","nonce":44089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":44089},{"type":"v1","addr":"172.21.15.96:6803","nonce":44089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":44089},{"type":"v1","addr":"172.21.15.96:6807","nonce":44089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":44089},{"type":"v1","addr":"172.21.15.96:6805","nonce":44089}]},"public_addr":"172.21.15.96:6801/44089","cluster_addr":"172.21.15.96:6803/44089","heartbeat_back_addr":"172.21.15.96:6807/44089","heartbeat_front_addr":"172.21.15.96:6805/44089","state":["exists","up"]},{"osd":2,"uuid":"46de3f65-87a8-460c-b57a-4468d9c89a57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":44090},{"type":"v1","addr":"172.21.15.96:6809","nonce":44090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":44090},{"type":"v1","addr":"172.21.15.96:6811","nonce":44090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":44090},{"type":"v1","addr":"172.21.15.96:6815","nonce":44090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":44090},{"type":"v1","addr":"172.21.15.96:6813","nonce":44090}]},"public_addr":"172.21.15.96:6809/44090","cluster_addr":"172.21.15.96:6811/44090","heartbeat_back_addr":"172.21.15.96:6815/44090","heartbeat_front_addr":"172.21.15.96:6813/44090","state":["exists","up"]},{"osd":3,"uuid":"5507e063-10bc-4e32-b1ec-00aa3e47d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":44121},{"type":"v1","addr":"172.21.15.96:6825","nonce":44121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":44121},{"type":"v1","addr":"172.21.15.96:6827","nonce":44121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":44121},{"type":"v1","addr":"172.21.15.96:6831","nonce":44121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":44121},{"type":"v1","addr":"172.21.15.96:6829","nonce":44121}]},"public_addr":"172.21.15.96:6825/44121","cluster_addr":"172.21.15.96:6827/44121","heartbeat_back_addr":"172.21.15.96:6831/44121","heartbeat_front_addr":"172.21.15.96:6829/44121","state":["exists","up"]},{"osd":4,"uuid":"6a74d453-1048-44ef-a5b0-4277583d76c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":44025},{"type":"v1","addr":"172.21.15.151:6817","nonce":44025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":44025},{"type":"v1","addr":"172.21.15.151:6819","nonce":44025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":44025},{"type":"v1","addr":"172.21.15.151:6823","nonce":44025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":44025},{"type":"v1","addr":"172.21.15.151:6821","nonce":44025}]},"public_addr":"172.21.15.151:6817/44025","cluster_addr":"172.21.15.151:6819/44025","heartbeat_back_addr":"172.21.15.151:6823/44025","heartbeat_front_addr":"172.21.15.151:6821/44025","state":["exists","up"]},{"osd":5,"uuid":"3645ef18-5574-49f8-ab81-b40651208309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":44023},{"type":"v1","addr":"172.21.15.151:6801","nonce":44023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":44023},{"type":"v1","addr":"172.21.15.151:6803","nonce":44023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":44023},{"type":"v1","addr":"172.21.15.151:6807","nonce":44023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":44023},{"type":"v1","addr":"172.21.15.151:6805","nonce":44023}]},"public_addr":"172.21.15.151:6801/44023","cluster_addr":"172.21.15.151:6803/44023","heartbeat_back_addr":"172.21.15.151:6807/44023","heartbeat_front_addr":"172.21.15.151:6805/44023","state":["exists","up"]},{"osd":6,"uuid":"6b8b0b36-9246-4241-a197-c69fa48a6202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":75,"up_from":104,"up_thru":135,"down_at":76,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":59996},{"type":"v1","addr":"172.21.15.151:6825","nonce":59996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":59996},{"type":"v1","addr":"172.21.15.151:6827","nonce":59996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":59996},{"type":"v1","addr":"172.21.15.151:6831","nonce":59996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":59996},{"type":"v1","addr":"172.21.15.151:6829","nonce":59996}]},"public_addr":"172.21.15.151:6825/59996","cluster_addr":"172.21.15.151:6827/59996","heartbeat_back_addr":"172.21.15.151:6831/59996","heartbeat_front_addr":"172.21.15.151:6829/59996","state":["exists","up"]},{"osd":7,"uuid":"4501cd9f-e437-4ddd-af3d-129fdf1ee61a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":44024},{"type":"v1","addr":"172.21.15.151:6809","nonce":44024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":44024},{"type":"v1","addr":"172.21.15.151:6811","nonce":44024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":44024},{"type":"v1","addr":"172.21.15.151:6815","nonce":44024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":44024},{"type":"v1","addr":"172.21.15.151:6813","nonce":44024}]},"public_addr":"172.21.15.151:6809/44024","cluster_addr":"172.21.15.151:6811/44024","heartbeat_back_addr":"172.21.15.151:6815/44024","heartbeat_front_addr":"172.21.15.151:6813/44024","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T15:31:56.300910+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T15:31:56.202064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T15:31:56.198961+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T15:31:56.346777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T15:31:56.282585+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T15:31:56.257943+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T15:31:56.260479+0000","dead_epoch":76},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T15:31:56.242093+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.0","mappings":[{"from":3,"to":2}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T15:34:35.868 INFO:tasks.ceph:Scrubbing osd.0 2024-04-24T15:34:35.868 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-24T15:34:36.023 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:34:36.023 INFO:teuthology.orchestra.run.smithi096.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-24T15:34:36.023 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:34:36.034 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-24T15:34:36.347 INFO:teuthology.orchestra.run.smithi096.stderr:instructed osd(s) 0 to deep-scrub 2024-04-24T15:34:36.357 INFO:tasks.ceph:Scrubbing osd.1 2024-04-24T15:34:36.358 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-24T15:34:36.498 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:34:36.498 INFO:teuthology.orchestra.run.smithi096.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-24T15:34:36.498 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:34:36.509 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-24T15:34:36.825 INFO:teuthology.orchestra.run.smithi096.stderr:instructed osd(s) 1 to deep-scrub 2024-04-24T15:34:36.835 INFO:tasks.ceph:Scrubbing osd.2 2024-04-24T15:34:36.836 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-24T15:34:36.978 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:34:36.978 INFO:teuthology.orchestra.run.smithi096.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-24T15:34:36.978 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:34:36.989 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-24T15:34:37.305 INFO:teuthology.orchestra.run.smithi096.stderr:instructed osd(s) 2 to deep-scrub 2024-04-24T15:34:37.317 INFO:tasks.ceph:Scrubbing osd.3 2024-04-24T15:34:37.317 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-24T15:34:37.458 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:34:37.458 INFO:teuthology.orchestra.run.smithi096.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-24T15:34:37.458 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:34:37.470 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-24T15:34:37.788 INFO:teuthology.orchestra.run.smithi096.stderr:instructed osd(s) 3 to deep-scrub 2024-04-24T15:34:37.798 INFO:tasks.ceph:Scrubbing osd.4 2024-04-24T15:34:37.799 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-24T15:34:37.942 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:34:37.942 INFO:teuthology.orchestra.run.smithi096.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-24T15:34:37.943 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:34:37.954 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-24T15:34:38.268 INFO:teuthology.orchestra.run.smithi096.stderr:instructed osd(s) 4 to deep-scrub 2024-04-24T15:34:38.279 INFO:tasks.ceph:Scrubbing osd.5 2024-04-24T15:34:38.280 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-24T15:34:38.423 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:34:38.423 INFO:teuthology.orchestra.run.smithi096.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-24T15:34:38.423 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:34:38.434 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-24T15:34:38.750 INFO:teuthology.orchestra.run.smithi096.stderr:instructed osd(s) 5 to deep-scrub 2024-04-24T15:34:38.761 INFO:tasks.ceph:Scrubbing osd.6 2024-04-24T15:34:38.761 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-24T15:34:38.903 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:34:38.904 INFO:teuthology.orchestra.run.smithi096.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-24T15:34:38.904 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:34:38.915 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-24T15:34:39.227 INFO:teuthology.orchestra.run.smithi096.stderr:instructed osd(s) 6 to deep-scrub 2024-04-24T15:34:39.238 INFO:tasks.ceph:Scrubbing osd.7 2024-04-24T15:34:39.238 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-24T15:34:39.380 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-24T15:34:39.381 INFO:teuthology.orchestra.run.smithi096.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-24T15:34:39.381 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-24T15:34:39.392 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-24T15:34:39.706 INFO:teuthology.orchestra.run.smithi096.stderr:instructed osd(s) 7 to deep-scrub 2024-04-24T15:34:39.717 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T15:34:40.033 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:34:40.034 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-04-24T15:34:40.045 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":223,"stamp":"2024-04-24T15:34:39.581155+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":51,"num_read":114923,"num_read_kb":727642,"num_write":20449,"num_write_kb":358744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4420,"num_bytes_recovered":106356266,"num_keys_recovered":235,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17379,"ondisk_log_size":17379,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2440656,"kb_used_data":14072,"kb_used_omap":55,"kb_used_meta":2426440,"kb_avail":752534064,"statfs":{"total":773094113280,"available":770594881536,"internally_reserved":0,"allocated":14409728,"data_stored":8459356,"data_compressed":10519,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":56663,"internal_metadata":2484675241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.718622"},"pg_stats":[{"pgid":"4.1f","version":"131'1485","reported_seq":2550,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698777+0000","last_change":"2024-04-24T15:34:15.652525+0000","last_active":"2024-04-24T15:34:28.698777+0000","last_peered":"2024-04-24T15:34:28.698777+0000","last_clean":"2024-04-24T15:34:28.698777+0000","last_became_active":"2024-04-24T15:34:15.650850+0000","last_became_peered":"2024-04-24T15:34:15.650850+0000","last_unstale":"2024-04-24T15:34:28.698777+0000","last_undegraded":"2024-04-24T15:34:28.698777+0000","last_fullsized":"2024-04-24T15:34:28.698777+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":1485,"log_dups_size":0,"ondisk_log_size":1485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:06:34.577605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001749947,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":8143,"num_read_kb":14722,"num_write":2575,"num_write_kb":16077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":5952278,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"129'507","reported_seq":6419,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.862878+0000","last_change":"2024-04-24T15:34:15.669801+0000","last_active":"2024-04-24T15:34:28.862878+0000","last_peered":"2024-04-24T15:34:28.862878+0000","last_clean":"2024-04-24T15:34:28.862878+0000","last_became_active":"2024-04-24T15:34:15.668657+0000","last_became_peered":"2024-04-24T15:34:15.668657+0000","last_unstale":"2024-04-24T15:34:28.862878+0000","last_undegraded":"2024-04-24T15:34:28.862878+0000","last_fullsized":"2024-04-24T15:34:28.862878+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":507,"log_dups_size":0,"ondisk_log_size":507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029386599999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":7278,"num_read_kb":39113,"num_write":560,"num_write_kb":6592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":4495932,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"129'529","reported_seq":3746,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.864316+0000","last_change":"2024-04-24T15:34:16.660701+0000","last_active":"2024-04-24T15:34:28.864316+0000","last_peered":"2024-04-24T15:34:28.864316+0000","last_clean":"2024-04-24T15:34:28.864316+0000","last_became_active":"2024-04-24T15:34:16.659202+0000","last_became_peered":"2024-04-24T15:34:16.659202+0000","last_unstale":"2024-04-24T15:34:28.864316+0000","last_undegraded":"2024-04-24T15:34:28.864316+0000","last_fullsized":"2024-04-24T15:34:28.864316+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":529,"log_dups_size":0,"ondisk_log_size":529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015488839999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":4427,"num_read_kb":51896,"num_write":603,"num_write_kb":18759,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":580671,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"131'526","reported_seq":2701,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:29.418221+0000","last_change":"2024-04-24T15:34:19.807684+0000","last_active":"2024-04-24T15:34:29.418221+0000","last_peered":"2024-04-24T15:34:29.418221+0000","last_clean":"2024-04-24T15:34:29.418221+0000","last_became_active":"2024-04-24T15:34:19.806783+0000","last_became_peered":"2024-04-24T15:34:19.806783+0000","last_unstale":"2024-04-24T15:34:29.418221+0000","last_undegraded":"2024-04-24T15:34:29.418221+0000","last_fullsized":"2024-04-24T15:34:29.418221+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":526,"log_dups_size":0,"ondisk_log_size":526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001413493,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":3716,"num_read_kb":10078,"num_write":547,"num_write_kb":2888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"129'385","reported_seq":2360,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:29.418941+0000","last_change":"2024-04-24T15:34:23.827263+0000","last_active":"2024-04-24T15:34:29.418941+0000","last_peered":"2024-04-24T15:34:29.418941+0000","last_clean":"2024-04-24T15:34:29.418941+0000","last_became_active":"2024-04-24T15:34:23.825472+0000","last_became_peered":"2024-04-24T15:34:23.825472+0000","last_unstale":"2024-04-24T15:34:29.418941+0000","last_undegraded":"2024-04-24T15:34:29.418941+0000","last_fullsized":"2024-04-24T15:34:29.418941+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":385,"log_dups_size":0,"ondisk_log_size":385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011107479999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":2335,"num_read_kb":25405,"num_write":329,"num_write_kb":6118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":79,"num_bytes_recovered":2143880,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"129'449","reported_seq":6611,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:29.418887+0000","last_change":"2024-04-24T15:34:23.827554+0000","last_active":"2024-04-24T15:34:29.418887+0000","last_peered":"2024-04-24T15:34:29.418887+0000","last_clean":"2024-04-24T15:34:29.418887+0000","last_became_active":"2024-04-24T15:34:23.824861+0000","last_became_peered":"2024-04-24T15:34:23.824861+0000","last_unstale":"2024-04-24T15:34:29.418887+0000","last_undegraded":"2024-04-24T15:34:29.418887+0000","last_fullsized":"2024-04-24T15:34:29.418887+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097376600000000004,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":6368,"num_read_kb":19805,"num_write":475,"num_write_kb":10193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":89,"num_bytes_recovered":1137673,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"129'537","reported_seq":2681,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.864054+0000","last_change":"2024-04-24T15:34:08.756894+0000","last_active":"2024-04-24T15:34:28.864054+0000","last_peered":"2024-04-24T15:34:28.864054+0000","last_clean":"2024-04-24T15:34:28.864054+0000","last_became_active":"2024-04-24T15:32:17.047842+0000","last_became_peered":"2024-04-24T15:32:17.047842+0000","last_unstale":"2024-04-24T15:34:28.864054+0000","last_undegraded":"2024-04-24T15:34:28.864054+0000","last_fullsized":"2024-04-24T15:34:28.864054+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":537,"log_dups_size":0,"ondisk_log_size":537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081826100000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":4119,"num_read_kb":22694,"num_write":527,"num_write_kb":11193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"130'444","reported_seq":1969,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:29.418668+0000","last_change":"2024-04-24T15:34:08.758067+0000","last_active":"2024-04-24T15:34:29.418668+0000","last_peered":"2024-04-24T15:34:29.418668+0000","last_clean":"2024-04-24T15:34:29.418668+0000","last_became_active":"2024-04-24T15:33:42.641679+0000","last_became_peered":"2024-04-24T15:33:42.641679+0000","last_unstale":"2024-04-24T15:34:29.418668+0000","last_undegraded":"2024-04-24T15:34:29.418668+0000","last_fullsized":"2024-04-24T15:34:29.418668+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":444,"log_dups_size":0,"ondisk_log_size":444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077716999999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":1887,"num_read_kb":15406,"num_write":547,"num_write_kb":8155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":4099152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"129'523","reported_seq":2448,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.864377+0000","last_change":"2024-04-24T15:34:08.758924+0000","last_active":"2024-04-24T15:34:28.864377+0000","last_peered":"2024-04-24T15:34:28.864377+0000","last_clean":"2024-04-24T15:34:28.864377+0000","last_became_active":"2024-04-24T15:33:56.699629+0000","last_became_peered":"2024-04-24T15:33:56.699629+0000","last_unstale":"2024-04-24T15:34:28.864377+0000","last_undegraded":"2024-04-24T15:34:28.864377+0000","last_fullsized":"2024-04-24T15:34:28.864377+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":523,"log_dups_size":0,"ondisk_log_size":523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00212766,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":3353,"num_read_kb":16260,"num_write":594,"num_write_kb":12753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":352256,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"129'316","reported_seq":2578,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:29.418823+0000","last_change":"2024-04-24T15:34:23.829028+0000","last_active":"2024-04-24T15:34:29.418823+0000","last_peered":"2024-04-24T15:34:29.418823+0000","last_clean":"2024-04-24T15:34:29.418823+0000","last_became_active":"2024-04-24T15:34:23.827917+0000","last_became_peered":"2024-04-24T15:34:23.827917+0000","last_unstale":"2024-04-24T15:34:29.418823+0000","last_undegraded":"2024-04-24T15:34:29.418823+0000","last_fullsized":"2024-04-24T15:34:29.418823+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":316,"log_dups_size":0,"ondisk_log_size":316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065534800000000004,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":1633,"num_read_kb":26736,"num_write":303,"num_write_kb":13822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":9304722,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":121,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698692+0000","last_change":"2024-04-24T15:33:53.644249+0000","last_active":"2024-04-24T15:34:28.698692+0000","last_peered":"2024-04-24T15:34:28.698692+0000","last_clean":"2024-04-24T15:34:28.698692+0000","last_became_active":"2024-04-24T15:33:53.643941+0000","last_became_peered":"2024-04-24T15:33:53.643941+0000","last_unstale":"2024-04-24T15:34:28.698692+0000","last_undegraded":"2024-04-24T15:34:28.698692+0000","last_fullsized":"2024-04-24T15:34:28.698692+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:42:05.771051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"149'867","reported_seq":3025,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:37.530105+0000","last_change":"2024-04-24T15:34:37.530071+0000","last_active":"2024-04-24T15:34:37.530105+0000","last_peered":"2024-04-24T15:34:37.530105+0000","last_clean":"2024-04-24T15:34:37.530105+0000","last_became_active":"2024-04-24T15:34:19.809619+0000","last_became_peered":"2024-04-24T15:34:19.809619+0000","last_unstale":"2024-04-24T15:34:37.530105+0000","last_undegraded":"2024-04-24T15:34:37.530105+0000","last_fullsized":"2024-04-24T15:34:37.530105+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"149'867","last_scrub_stamp":"2024-04-24T15:34:37.529969+0000","last_deep_scrub":"149'867","last_deep_scrub_stamp":"2024-04-24T15:34:37.529969+0000","last_clean_scrub_stamp":"2024-04-24T15:34:37.529969+0000","objects_scrubbed":5,"log_size":867,"log_dups_size":0,"ondisk_log_size":867,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:05:19.302172+0000","scrub_duration":0.0058171969999999996,"objects_trimmed":2,"snaptrim_duration":0.00033353700000000002,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":4689,"num_read_kb":23870,"num_write":1044,"num_write_kb":13092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":176,"num_bytes_recovered":5601476,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":25,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:29.418463+0000","last_change":"2024-04-24T15:34:12.652580+0000","last_active":"2024-04-24T15:34:29.418463+0000","last_peered":"2024-04-24T15:34:29.418463+0000","last_clean":"2024-04-24T15:34:29.418463+0000","last_became_active":"2024-04-24T15:34:12.652298+0000","last_became_peered":"2024-04-24T15:34:12.652298+0000","last_unstale":"2024-04-24T15:34:29.418463+0000","last_undegraded":"2024-04-24T15:34:29.418463+0000","last_fullsized":"2024-04-24T15:34:29.418463+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":41,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.862035+0000","last_change":"2024-04-24T15:33:53.645875+0000","last_active":"2024-04-24T15:34:27.862035+0000","last_peered":"2024-04-24T15:34:27.862035+0000","last_clean":"2024-04-24T15:34:27.862035+0000","last_became_active":"2024-04-24T15:33:53.645696+0000","last_became_peered":"2024-04-24T15:33:53.645696+0000","last_unstale":"2024-04-24T15:34:27.862035+0000","last_undegraded":"2024-04-24T15:34:27.862035+0000","last_fullsized":"2024-04-24T15:34:27.862035+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:29:35.345558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"131'584","reported_seq":5173,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.863023+0000","last_change":"2024-04-24T15:34:17.655614+0000","last_active":"2024-04-24T15:34:28.863023+0000","last_peered":"2024-04-24T15:34:28.863023+0000","last_clean":"2024-04-24T15:34:28.863023+0000","last_became_active":"2024-04-24T15:34:17.654759+0000","last_became_peered":"2024-04-24T15:34:17.654759+0000","last_unstale":"2024-04-24T15:34:28.863023+0000","last_undegraded":"2024-04-24T15:34:28.863023+0000","last_fullsized":"2024-04-24T15:34:28.863023+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":584,"log_dups_size":0,"ondisk_log_size":584,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061779100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3592,"num_read_kb":14899,"num_write":598,"num_write_kb":11359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":23598,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"149'2","reported_seq":92,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:36.366432+0000","last_change":"2024-04-24T15:34:36.366408+0000","last_active":"2024-04-24T15:34:36.366432+0000","last_peered":"2024-04-24T15:34:36.366432+0000","last_clean":"2024-04-24T15:34:36.366432+0000","last_became_active":"2024-04-24T15:33:47.662685+0000","last_became_peered":"2024-04-24T15:33:47.662685+0000","last_unstale":"2024-04-24T15:34:36.366432+0000","last_undegraded":"2024-04-24T15:34:36.366432+0000","last_fullsized":"2024-04-24T15:34:36.366432+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"149'2","last_scrub_stamp":"2024-04-24T15:34:36.366339+0000","last_deep_scrub":"149'2","last_deep_scrub_stamp":"2024-04-24T15:34:36.366339+0000","last_clean_scrub_stamp":"2024-04-24T15:34:36.366339+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:57:31.467325+0000","scrub_duration":0.0030479940000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":50,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.861972+0000","last_change":"2024-04-24T15:34:12.653126+0000","last_active":"2024-04-24T15:34:27.861972+0000","last_peered":"2024-04-24T15:34:27.861972+0000","last_clean":"2024-04-24T15:34:27.861972+0000","last_became_active":"2024-04-24T15:34:12.652805+0000","last_became_peered":"2024-04-24T15:34:12.652805+0000","last_unstale":"2024-04-24T15:34:27.861972+0000","last_undegraded":"2024-04-24T15:34:27.861972+0000","last_fullsized":"2024-04-24T15:34:27.861972+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:44:23.434720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"131'615","reported_seq":2370,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.862969+0000","last_change":"2024-04-24T15:34:16.760282+0000","last_active":"2024-04-24T15:34:28.862969+0000","last_peered":"2024-04-24T15:34:28.862969+0000","last_clean":"2024-04-24T15:34:28.862969+0000","last_became_active":"2024-04-24T15:34:16.655390+0000","last_became_peered":"2024-04-24T15:34:16.655390+0000","last_unstale":"2024-04-24T15:34:28.862969+0000","last_undegraded":"2024-04-24T15:34:28.862969+0000","last_fullsized":"2024-04-24T15:34:28.862969+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":615,"log_dups_size":0,"ondisk_log_size":615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042917399999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":3015,"num_read_kb":19823,"num_write":696,"num_write_kb":5112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"149'4","reported_seq":195,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:38.813383+0000","last_change":"2024-04-24T15:34:38.813338+0000","last_active":"2024-04-24T15:34:38.813383+0000","last_peered":"2024-04-24T15:34:38.813383+0000","last_clean":"2024-04-24T15:34:38.813383+0000","last_became_active":"2024-04-24T15:34:13.662363+0000","last_became_peered":"2024-04-24T15:34:13.662363+0000","last_unstale":"2024-04-24T15:34:38.813383+0000","last_undegraded":"2024-04-24T15:34:38.813383+0000","last_fullsized":"2024-04-24T15:34:38.813383+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"149'4","last_scrub_stamp":"2024-04-24T15:34:38.813254+0000","last_deep_scrub":"149'4","last_deep_scrub_stamp":"2024-04-24T15:34:38.813254+0000","last_clean_scrub_stamp":"2024-04-24T15:34:38.813254+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:56:01.218000+0000","scrub_duration":0.006372244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":251,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:27.862026+0000","last_change":"2024-04-24T15:33:57.644653+0000","last_active":"2024-04-24T15:34:27.862026+0000","last_peered":"2024-04-24T15:34:27.862026+0000","last_clean":"2024-04-24T15:34:27.862026+0000","last_became_active":"2024-04-24T15:33:57.644312+0000","last_became_peered":"2024-04-24T15:33:57.644312+0000","last_unstale":"2024-04-24T15:34:27.862026+0000","last_undegraded":"2024-04-24T15:34:27.862026+0000","last_fullsized":"2024-04-24T15:34:27.862026+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:31:59.935287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:31:59.935287+0000","last_clean_scrub_stamp":"2024-04-24T15:31:59.935287+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:21:46.534664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:29.418894+0000","last_change":"2024-04-24T15:34:13.650533+0000","last_active":"2024-04-24T15:34:29.418894+0000","last_peered":"2024-04-24T15:34:29.418894+0000","last_clean":"2024-04-24T15:34:29.418894+0000","last_became_active":"2024-04-24T15:34:13.650174+0000","last_became_peered":"2024-04-24T15:34:13.650174+0000","last_unstale":"2024-04-24T15:34:29.418894+0000","last_undegraded":"2024-04-24T15:34:29.418894+0000","last_fullsized":"2024-04-24T15:34:29.418894+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"129'591","reported_seq":2456,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698898+0000","last_change":"2024-04-24T15:34:08.805854+0000","last_active":"2024-04-24T15:34:28.698898+0000","last_peered":"2024-04-24T15:34:28.698898+0000","last_clean":"2024-04-24T15:34:28.698898+0000","last_became_active":"2024-04-24T15:33:45.658207+0000","last_became_peered":"2024-04-24T15:33:45.658207+0000","last_unstale":"2024-04-24T15:34:28.698898+0000","last_undegraded":"2024-04-24T15:34:28.698898+0000","last_fullsized":"2024-04-24T15:34:28.698898+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":591,"log_dups_size":0,"ondisk_log_size":591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:59:01.456518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010111199999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":3352,"num_read_kb":26205,"num_write":791,"num_write_kb":11816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":8821091,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":35,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:38.329944+0000","last_change":"2024-04-24T15:34:38.329904+0000","last_active":"2024-04-24T15:34:38.329944+0000","last_peered":"2024-04-24T15:34:38.329944+0000","last_clean":"2024-04-24T15:34:38.329944+0000","last_became_active":"2024-04-24T15:34:12.655508+0000","last_became_peered":"2024-04-24T15:34:12.655508+0000","last_unstale":"2024-04-24T15:34:38.329944+0000","last_undegraded":"2024-04-24T15:34:38.329944+0000","last_fullsized":"2024-04-24T15:34:38.329944+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:34:38.329808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:34:38.329808+0000","last_clean_scrub_stamp":"2024-04-24T15:34:38.329808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:03:40.453877+0000","scrub_duration":0.001431858,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":70,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:38.557893+0000","last_change":"2024-04-24T15:34:38.557860+0000","last_active":"2024-04-24T15:34:38.557893+0000","last_peered":"2024-04-24T15:34:38.557893+0000","last_clean":"2024-04-24T15:34:38.557893+0000","last_became_active":"2024-04-24T15:34:19.838058+0000","last_became_peered":"2024-04-24T15:34:19.838058+0000","last_unstale":"2024-04-24T15:34:38.557893+0000","last_undegraded":"2024-04-24T15:34:38.557893+0000","last_fullsized":"2024-04-24T15:34:38.557893+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:34:38.557748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:34:38.557748+0000","last_clean_scrub_stamp":"2024-04-24T15:34:38.557748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:53:13.458240+0000","scrub_duration":0.001481909,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"131'430","reported_seq":1232,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:29.418405+0000","last_change":"2024-04-24T15:34:15.660049+0000","last_active":"2024-04-24T15:34:29.418405+0000","last_peered":"2024-04-24T15:34:29.418405+0000","last_clean":"2024-04-24T15:34:29.418405+0000","last_became_active":"2024-04-24T15:34:15.658368+0000","last_became_peered":"2024-04-24T15:34:15.658368+0000","last_unstale":"2024-04-24T15:34:29.418405+0000","last_undegraded":"2024-04-24T15:34:29.418405+0000","last_fullsized":"2024-04-24T15:34:29.418405+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015045289999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":1970,"num_read_kb":12710,"num_write":463,"num_write_kb":8069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":208,"num_bytes_recovered":107133,"num_keys_recovered":27,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":94,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:29.419029+0000","last_change":"2024-04-24T15:34:24.428957+0000","last_active":"2024-04-24T15:34:29.419029+0000","last_peered":"2024-04-24T15:34:29.419029+0000","last_clean":"2024-04-24T15:34:29.419029+0000","last_became_active":"2024-04-24T15:34:24.428510+0000","last_became_peered":"2024-04-24T15:34:24.428510+0000","last_unstale":"2024-04-24T15:34:29.419029+0000","last_undegraded":"2024-04-24T15:34:29.419029+0000","last_fullsized":"2024-04-24T15:34:29.419029+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_clean_scrub_stamp":"2024-04-24T15:32:06.972928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:29.418805+0000","last_change":"2024-04-24T15:34:13.689898+0000","last_active":"2024-04-24T15:34:29.418805+0000","last_peered":"2024-04-24T15:34:29.418805+0000","last_clean":"2024-04-24T15:34:29.418805+0000","last_became_active":"2024-04-24T15:34:13.689685+0000","last_became_peered":"2024-04-24T15:34:13.689685+0000","last_unstale":"2024-04-24T15:34:29.418805+0000","last_undegraded":"2024-04-24T15:34:29.418805+0000","last_fullsized":"2024-04-24T15:34:29.418805+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"130'457","reported_seq":1996,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:29.418565+0000","last_change":"2024-04-24T15:34:16.654083+0000","last_active":"2024-04-24T15:34:29.418565+0000","last_peered":"2024-04-24T15:34:29.418565+0000","last_clean":"2024-04-24T15:34:29.418565+0000","last_became_active":"2024-04-24T15:34:16.651906+0000","last_became_peered":"2024-04-24T15:34:16.651906+0000","last_unstale":"2024-04-24T15:34:29.418565+0000","last_undegraded":"2024-04-24T15:34:29.418565+0000","last_fullsized":"2024-04-24T15:34:29.418565+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":457,"log_dups_size":0,"ondisk_log_size":457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010565220000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2138,"num_read_kb":24032,"num_write":423,"num_write_kb":14900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"130'408","reported_seq":1975,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:29.418428+0000","last_change":"2024-04-24T15:34:16.653875+0000","last_active":"2024-04-24T15:34:29.418428+0000","last_peered":"2024-04-24T15:34:29.418428+0000","last_clean":"2024-04-24T15:34:29.418428+0000","last_became_active":"2024-04-24T15:34:16.651931+0000","last_became_peered":"2024-04-24T15:34:16.651931+0000","last_unstale":"2024-04-24T15:34:29.418428+0000","last_undegraded":"2024-04-24T15:34:29.418428+0000","last_fullsized":"2024-04-24T15:34:29.418428+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039757799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2388,"num_read_kb":22956,"num_write":379,"num_write_kb":16060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":578400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":25,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:29.418466+0000","last_change":"2024-04-24T15:34:12.653777+0000","last_active":"2024-04-24T15:34:29.418466+0000","last_peered":"2024-04-24T15:34:29.418466+0000","last_clean":"2024-04-24T15:34:29.418466+0000","last_became_active":"2024-04-24T15:34:12.653529+0000","last_became_peered":"2024-04-24T15:34:12.653529+0000","last_unstale":"2024-04-24T15:34:29.418466+0000","last_undegraded":"2024-04-24T15:34:29.418466+0000","last_fullsized":"2024-04-24T15:34:29.418466+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:00.938537+0000","last_clean_scrub_stamp":"2024-04-24T15:32:00.938537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"129'479","reported_seq":1998,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:29.419063+0000","last_change":"2024-04-24T15:34:16.656437+0000","last_active":"2024-04-24T15:34:29.419063+0000","last_peered":"2024-04-24T15:34:29.419063+0000","last_clean":"2024-04-24T15:34:29.419063+0000","last_became_active":"2024-04-24T15:34:16.654252+0000","last_became_peered":"2024-04-24T15:34:16.654252+0000","last_unstale":"2024-04-24T15:34:29.419063+0000","last_undegraded":"2024-04-24T15:34:29.419063+0000","last_fullsized":"2024-04-24T15:34:29.419063+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035787500000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":2434,"num_read_kb":14959,"num_write":545,"num_write_kb":12213,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":335,"num_bytes_recovered":6920328,"num_keys_recovered":43,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"130'616","reported_seq":2735,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:29.418778+0000","last_change":"2024-04-24T15:34:23.827845+0000","last_active":"2024-04-24T15:34:29.418778+0000","last_peered":"2024-04-24T15:34:29.418778+0000","last_clean":"2024-04-24T15:34:29.418778+0000","last_became_active":"2024-04-24T15:34:23.825759+0000","last_became_peered":"2024-04-24T15:34:23.825759+0000","last_unstale":"2024-04-24T15:34:29.418778+0000","last_undegraded":"2024-04-24T15:34:29.418778+0000","last_fullsized":"2024-04-24T15:34:29.418778+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0055639879999999997,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":4324,"num_read_kb":21128,"num_write":677,"num_write_kb":10933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":5713612,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"130'485","reported_seq":3231,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.698742+0000","last_change":"2024-04-24T15:34:15.659323+0000","last_active":"2024-04-24T15:34:28.698742+0000","last_peered":"2024-04-24T15:34:28.698742+0000","last_clean":"2024-04-24T15:34:28.698742+0000","last_became_active":"2024-04-24T15:34:15.657660+0000","last_became_peered":"2024-04-24T15:34:15.657660+0000","last_unstale":"2024-04-24T15:34:28.698742+0000","last_undegraded":"2024-04-24T15:34:28.698742+0000","last_fullsized":"2024-04-24T15:34:28.698742+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:09:16.071953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066763,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3120,"num_read_kb":27458,"num_write":430,"num_write_kb":12267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":237,"num_bytes_recovered":5299136,"num_keys_recovered":49,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"129'551","reported_seq":1987,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:38.371801+0000","last_change":"2024-04-24T15:34:38.371767+0000","last_active":"2024-04-24T15:34:38.371801+0000","last_peered":"2024-04-24T15:34:38.371801+0000","last_clean":"2024-04-24T15:34:38.371801+0000","last_became_active":"2024-04-24T15:34:15.668717+0000","last_became_peered":"2024-04-24T15:34:15.668717+0000","last_unstale":"2024-04-24T15:34:38.371801+0000","last_undegraded":"2024-04-24T15:34:38.371801+0000","last_fullsized":"2024-04-24T15:34:38.371801+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"129'551","last_scrub_stamp":"2024-04-24T15:34:38.371640+0000","last_deep_scrub":"129'551","last_deep_scrub_stamp":"2024-04-24T15:34:38.371640+0000","last_clean_scrub_stamp":"2024-04-24T15:34:38.371640+0000","objects_scrubbed":0,"log_size":551,"log_dups_size":0,"ondisk_log_size":551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:19:54.567694+0000","scrub_duration":0.001344725,"objects_trimmed":0,"snaptrim_duration":0.00039775200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2839,"num_read_kb":21248,"num_write":721,"num_write_kb":8473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":141,"num_bytes_recovered":1490474,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"127'1046","reported_seq":2506,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:29.419089+0000","last_change":"2024-04-24T15:34:23.826944+0000","last_active":"2024-04-24T15:34:29.419089+0000","last_peered":"2024-04-24T15:34:29.419089+0000","last_clean":"2024-04-24T15:34:29.419089+0000","last_became_active":"2024-04-24T15:34:23.825039+0000","last_became_peered":"2024-04-24T15:34:23.825039+0000","last_unstale":"2024-04-24T15:34:29.419089+0000","last_undegraded":"2024-04-24T15:34:29.419089+0000","last_fullsized":"2024-04-24T15:34:29.419089+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":1046,"log_dups_size":0,"ondisk_log_size":1046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029127150000000002,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":6311,"num_read_kb":16558,"num_write":1677,"num_write_kb":12129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":54,"num_bytes_recovered":1219753,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.d","version":"127'298","reported_seq":2573,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:29.419384+0000","last_change":"2024-04-24T15:34:23.825797+0000","last_active":"2024-04-24T15:34:29.419384+0000","last_peered":"2024-04-24T15:34:29.419384+0000","last_clean":"2024-04-24T15:34:29.419384+0000","last_became_active":"2024-04-24T15:34:23.823763+0000","last_became_peered":"2024-04-24T15:34:23.823763+0000","last_unstale":"2024-04-24T15:34:29.419384+0000","last_undegraded":"2024-04-24T15:34:29.419384+0000","last_fullsized":"2024-04-24T15:34:29.419384+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":298,"log_dups_size":0,"ondisk_log_size":298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047837299999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":2121,"num_read_kb":46483,"num_write":303,"num_write_kb":20603,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":4246835,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"129'563","reported_seq":2396,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:29.419242+0000","last_change":"2024-04-24T15:34:23.826137+0000","last_active":"2024-04-24T15:34:29.419242+0000","last_peered":"2024-04-24T15:34:29.419242+0000","last_clean":"2024-04-24T15:34:29.419242+0000","last_became_active":"2024-04-24T15:34:23.824037+0000","last_became_peered":"2024-04-24T15:34:23.824037+0000","last_unstale":"2024-04-24T15:34:29.419242+0000","last_undegraded":"2024-04-24T15:34:29.419242+0000","last_fullsized":"2024-04-24T15:34:29.419242+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":563,"log_dups_size":0,"ondisk_log_size":563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001084025,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3022,"num_read_kb":13469,"num_write":696,"num_write_kb":6185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":113,"num_bytes_recovered":1676557,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"130'907","reported_seq":3307,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:29.418481+0000","last_change":"2024-04-24T15:34:16.657695+0000","last_active":"2024-04-24T15:34:29.418481+0000","last_peered":"2024-04-24T15:34:29.418481+0000","last_clean":"2024-04-24T15:34:29.418481+0000","last_became_active":"2024-04-24T15:34:16.655733+0000","last_became_peered":"2024-04-24T15:34:16.655733+0000","last_unstale":"2024-04-24T15:34:29.418481+0000","last_undegraded":"2024-04-24T15:34:29.418481+0000","last_fullsized":"2024-04-24T15:34:29.418481+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":907,"log_dups_size":0,"ondisk_log_size":907,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018480180000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":5733,"num_read_kb":14488,"num_write":1011,"num_write_kb":8995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":223,"num_bytes_recovered":630687,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"130'244","reported_seq":1062,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.864052+0000","last_change":"2024-04-24T15:34:15.664032+0000","last_active":"2024-04-24T15:34:28.864052+0000","last_peered":"2024-04-24T15:34:28.864052+0000","last_clean":"2024-04-24T15:34:28.864052+0000","last_became_active":"2024-04-24T15:34:15.663081+0000","last_became_peered":"2024-04-24T15:34:15.663081+0000","last_unstale":"2024-04-24T15:34:28.864052+0000","last_undegraded":"2024-04-24T15:34:28.864052+0000","last_fullsized":"2024-04-24T15:34:28.864052+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":244,"log_dups_size":0,"ondisk_log_size":244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010945530000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":384,"num_read_kb":4092,"num_write":235,"num_write_kb":5715,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":1989443,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.11","version":"129'705","reported_seq":2163,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.864196+0000","last_change":"2024-04-24T15:34:08.757586+0000","last_active":"2024-04-24T15:34:28.864196+0000","last_peered":"2024-04-24T15:34:28.864196+0000","last_clean":"2024-04-24T15:34:28.864196+0000","last_became_active":"2024-04-24T15:33:57.644309+0000","last_became_peered":"2024-04-24T15:33:57.644309+0000","last_unstale":"2024-04-24T15:34:28.864196+0000","last_undegraded":"2024-04-24T15:34:28.864196+0000","last_fullsized":"2024-04-24T15:34:28.864196+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":705,"log_dups_size":0,"ondisk_log_size":705,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012982219999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":2615,"num_read_kb":21488,"num_write":891,"num_write_kb":14014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":5651177,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"129'331","reported_seq":5229,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:29.418515+0000","last_change":"2024-04-24T15:34:15.655138+0000","last_active":"2024-04-24T15:34:29.418515+0000","last_peered":"2024-04-24T15:34:29.418515+0000","last_clean":"2024-04-24T15:34:29.418515+0000","last_became_active":"2024-04-24T15:34:15.653205+0000","last_became_peered":"2024-04-24T15:34:15.653205+0000","last_unstale":"2024-04-24T15:34:29.418515+0000","last_undegraded":"2024-04-24T15:34:29.418515+0000","last_fullsized":"2024-04-24T15:34:29.418515+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":331,"log_dups_size":0,"ondisk_log_size":331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013576079999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":4982,"num_read_kb":42072,"num_write":333,"num_write_kb":14625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":3758016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"129'241","reported_seq":2004,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:29.418986+0000","last_change":"2024-04-24T15:34:23.825462+0000","last_active":"2024-04-24T15:34:29.418986+0000","last_peered":"2024-04-24T15:34:29.418986+0000","last_clean":"2024-04-24T15:34:29.418986+0000","last_became_active":"2024-04-24T15:34:23.823962+0000","last_became_peered":"2024-04-24T15:34:23.823962+0000","last_unstale":"2024-04-24T15:34:29.418986+0000","last_undegraded":"2024-04-24T15:34:29.418986+0000","last_fullsized":"2024-04-24T15:34:29.418986+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":241,"log_dups_size":0,"ondisk_log_size":241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000635438,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":1312,"num_read_kb":18878,"num_write":253,"num_write_kb":9844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":108,"num_bytes_recovered":2694373,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"131'450","reported_seq":6596,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.864339+0000","last_change":"2024-04-24T15:34:16.660466+0000","last_active":"2024-04-24T15:34:28.864339+0000","last_peered":"2024-04-24T15:34:28.864339+0000","last_clean":"2024-04-24T15:34:28.864339+0000","last_became_active":"2024-04-24T15:34:16.659071+0000","last_became_peered":"2024-04-24T15:34:16.659071+0000","last_unstale":"2024-04-24T15:34:28.864339+0000","last_undegraded":"2024-04-24T15:34:28.864339+0000","last_fullsized":"2024-04-24T15:34:28.864339+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013356920000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":7366,"num_read_kb":44142,"num_write":407,"num_write_kb":9850,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":233,"num_bytes_recovered":4458876,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"130'312","reported_seq":1576,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:29.419166+0000","last_change":"2024-04-24T15:34:16.657117+0000","last_active":"2024-04-24T15:34:29.419166+0000","last_peered":"2024-04-24T15:34:29.419166+0000","last_clean":"2024-04-24T15:34:29.419166+0000","last_became_active":"2024-04-24T15:34:16.655084+0000","last_became_peered":"2024-04-24T15:34:16.655084+0000","last_unstale":"2024-04-24T15:34:29.419166+0000","last_undegraded":"2024-04-24T15:34:29.419166+0000","last_fullsized":"2024-04-24T15:34:29.419166+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003384743,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":837,"num_read_kb":16692,"num_write":319,"num_write_kb":15488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":277,"num_bytes_recovered":16228001,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.16","version":"129'375","reported_seq":2455,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:28.863021+0000","last_change":"2024-04-24T15:34:15.669825+0000","last_active":"2024-04-24T15:34:28.863021+0000","last_peered":"2024-04-24T15:34:28.863021+0000","last_clean":"2024-04-24T15:34:28.863021+0000","last_became_active":"2024-04-24T15:34:15.668678+0000","last_became_peered":"2024-04-24T15:34:15.668678+0000","last_unstale":"2024-04-24T15:34:28.863021+0000","last_undegraded":"2024-04-24T15:34:28.863021+0000","last_fullsized":"2024-04-24T15:34:28.863021+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018886770000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":3004,"num_read_kb":17777,"num_write":338,"num_write_kb":8036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":48,"num_read":114807,"num_read_kb":727542,"num_write":20290,"num_write_kb":356328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4413,"num_bytes_recovered":105175530,"num_keys_recovered":235,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15242,"internal_metadata":0},"log_size":17256,"ondisk_log_size":17256,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":8,"acting":8,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":16196,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574884,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299140,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072700,"statfs":{"total":96636764160,"available":96330444800,"internally_reserved":0,"allocated":1830912,"data_stored":1500719,"data_compressed":5677,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.46000000000000002,"15min":0.46200000000000002},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.60099999999999998,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.499},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.54200000000000004,"15min":0.55600000000000005},"min":{"1min":0.33900000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.67200000000000004,"5min":2.4710000000000001,"15min":2.4710000000000001},"last":0.48199999999999998}]},{"osd":1,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.57399999999999995,"15min":0.59299999999999997},"min":{"1min":0.28100000000000003,"5min":0.248,"15min":0.248},"max":{"1min":0.63600000000000001,"5min":2.4729999999999999,"15min":2.4729999999999999},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.51500000000000001,"15min":0.52400000000000002},"min":{"1min":0.29999999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.56899999999999995,"5min":2.4020000000000001,"15min":2.4020000000000001},"last":0.53800000000000003}]},{"osd":2,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.45700000000000002,"15min":0.45800000000000002},"min":{"1min":0.35499999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.59699999999999998,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.503,"5min":0.54800000000000004,"15min":0.55600000000000005},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.82199999999999995,"5min":2.363,"15min":2.363},"last":0.39900000000000002}]},{"osd":3,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.52200000000000002,"15min":0.51800000000000002},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.53800000000000003,"15min":0.53200000000000003},"min":{"1min":0.38600000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.56000000000000005}]},{"osd":4,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.41399999999999998,"15min":0.41999999999999998},"min":{"1min":0.26100000000000001,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.66100000000000003,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.42299999999999999,"15min":0.42499999999999999},"min":{"1min":0.30599999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.73499999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.52100000000000002}]},{"osd":5,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.45100000000000001,"15min":0.45500000000000002},"min":{"1min":0.307,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.51600000000000001,"15min":0.52000000000000002},"min":{"1min":0.36399999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.80500000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.629}]},{"osd":6,"last update":"Wed Apr 24 15:32:58 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.57299999999999995}]}]},{"osd":6,"up_from":104,"seq":446676598797,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":305896,"kb_used_data":1748,"kb_used_omap":43,"kb_used_meta":304084,"kb_avail":94065944,"statfs":{"total":96636764160,"available":96323526656,"internally_reserved":0,"allocated":1789952,"data_stored":910351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":44735,"internal_metadata":311382337},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574883,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307296,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":305534,"kb_avail":94064544,"statfs":{"total":96636764160,"available":96322093056,"internally_reserved":0,"allocated":1785856,"data_stored":909395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312867160},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 15:34:01 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.35599999999999998,"15min":0.35799999999999998},"min":{"1min":0.20699999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.51100000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.317},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.377,"15min":0.376},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":1.353,"5min":1.375,"15min":1.375},"last":0.28999999999999998}]},{"osd":2,"last update":"Wed Apr 24 15:34:01 2024","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.33600000000000002,"15min":0.34000000000000002},"min":{"1min":0.224,"5min":0.223,"15min":0.223},"max":{"1min":0.46100000000000002,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.41499999999999998,"15min":0.42699999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.52800000000000002,"5min":1.494,"15min":1.494},"last":0.35899999999999999}]},{"osd":3,"last update":"Wed Apr 24 15:34:01 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.42799999999999999,"15min":0.433},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.97499999999999998,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.25800000000000001},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.377,"15min":0.372},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.436}]},{"osd":4,"last update":"Wed Apr 24 15:34:01 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.53500000000000003,"15min":0.54100000000000004},"min":{"1min":0.34799999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.72199999999999998,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.496},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.57799999999999996,"15min":0.58199999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.61099999999999999}]},{"osd":5,"last update":"Wed Apr 24 15:34:01 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.379,"5min":1.454,"15min":1.454},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.53800000000000003,"15min":0.54200000000000004},"min":{"1min":0.35999999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.67100000000000004,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.51300000000000001}]},{"osd":6,"last update":"Wed Apr 24 15:33:01 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.548,"5min":1.548,"15min":1.548},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.56399999999999995}]},{"osd":7,"last update":"Wed Apr 24 15:34:01 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.56899999999999995,"15min":0.57099999999999995},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.55100000000000005,"15min":0.55700000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.69599999999999995,"5min":0.999,"15min":0.999},"last":0.52900000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574884,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303252,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94068588,"statfs":{"total":96636764160,"available":96326234112,"internally_reserved":0,"allocated":1847296,"data_stored":1500719,"data_compressed":4842,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.30499999999999999,"5min":0.45700000000000002,"15min":0.48299999999999998},"min":{"1min":0.17899999999999999,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.47999999999999998,"5min":3.7890000000000001,"15min":3.7890000000000001},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.46700000000000003,"15min":0.48899999999999999},"min":{"1min":0.224,"5min":0.191,"15min":0.191},"max":{"1min":0.59899999999999998,"5min":2.7890000000000001,"15min":2.7890000000000001},"last":0.60899999999999999}]},{"osd":2,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.53100000000000003,"15min":0.55900000000000005},"min":{"1min":0.22700000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.61399999999999999,"5min":3.9950000000000001,"15min":3.9950000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.42299999999999999,"15min":0.436},"min":{"1min":0.26000000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.56799999999999995,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":0.64300000000000002}]},{"osd":3,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.48499999999999999,"15min":0.505},"min":{"1min":0.24099999999999999,"5min":0.223,"15min":0.223},"max":{"1min":0.54000000000000004,"5min":2.097,"15min":2.097},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.622,"15min":0.65700000000000003},"min":{"1min":0.30499999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.79500000000000004,"5min":3.2109999999999999,"15min":3.2109999999999999},"last":0.57699999999999996}]},{"osd":4,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.55700000000000005,"15min":0.56699999999999995},"min":{"1min":0.31,"5min":0.308,"15min":0.308},"max":{"1min":0.69099999999999995,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.65000000000000002,"15min":0.67300000000000004},"min":{"1min":0.34899999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.85599999999999998,"5min":3.282,"15min":3.282},"last":0.81100000000000005}]},{"osd":5,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.60099999999999998,"15min":0.61299999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.79100000000000004,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.64600000000000002,"15min":0.66200000000000003},"min":{"1min":0.38700000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.78000000000000003,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":0.83499999999999996}]},{"osd":6,"last update":"Wed Apr 24 15:33:00 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":3.2509999999999999,"5min":3.2509999999999999,"15min":3.2509999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":2.1909999999999998,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":0.85799999999999998}]},{"osd":7,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.60299999999999998,"15min":0.60899999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.075,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.63200000000000001,"15min":0.64400000000000002},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.876,"5min":2.206,"15min":2.206},"last":0.79700000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574884,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303192,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94068648,"statfs":{"total":96636764160,"available":96326295552,"internally_reserved":0,"allocated":1785856,"data_stored":909395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 15:34:06 2024","interfaces":[{"interface":"back","average":{"1min":0.29199999999999998,"5min":0.312,"15min":0.315},"min":{"1min":0.16200000000000001,"5min":0.16200000000000001,"15min":0.16200000000000001},"max":{"1min":0.41399999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"last":0.23999999999999999},{"interface":"front","average":{"1min":0.30599999999999999,"5min":0.33700000000000002,"15min":0.34200000000000003},"min":{"1min":0.185,"5min":0.185,"15min":0.185},"max":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"last":0.36699999999999999}]},{"osd":1,"last update":"Wed Apr 24 15:34:06 2024","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.36899999999999999,"15min":0.377},"min":{"1min":0.23300000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.47399999999999998,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.46100000000000002,"15min":0.48199999999999998},"min":{"1min":0.24099999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.441,"5min":3.1110000000000002,"15min":3.1110000000000002},"last":0.30199999999999999}]},{"osd":3,"last update":"Wed Apr 24 15:34:06 2024","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.40500000000000003,"15min":0.41899999999999998},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.41399999999999998,"5min":2.056,"15min":2.056},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.39900000000000002,"15min":0.40500000000000003},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.52900000000000003,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.31900000000000001}]},{"osd":4,"last update":"Wed Apr 24 15:34:06 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.51800000000000002,"15min":0.52400000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.53600000000000003,"15min":0.54500000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.70399999999999996,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.64300000000000002}]},{"osd":5,"last update":"Wed Apr 24 15:34:06 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.54300000000000004,"15min":0.54600000000000004},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.63700000000000001,"15min":0.65400000000000003},"min":{"1min":0.41399999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.80500000000000005,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":0.60699999999999998}]},{"osd":6,"last update":"Wed Apr 24 15:33:03 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.1000000000000001,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.52800000000000002}]},{"osd":7,"last update":"Wed Apr 24 15:34:06 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.63500000000000001,"15min":0.65700000000000003},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.63800000000000001,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":0.623},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.56999999999999995,"15min":0.57799999999999996},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73999999999999999,"5min":1.502,"15min":1.502},"last":0.72199999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574884,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303204,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94068636,"statfs":{"total":96636764160,"available":96326283264,"internally_reserved":0,"allocated":1794048,"data_stored":910370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 15:34:04 2024","interfaces":[{"interface":"back","average":{"1min":0.29399999999999998,"5min":0.34599999999999997,"15min":0.35499999999999998},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.39900000000000002,"5min":1.224,"15min":1.224},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.30199999999999999,"5min":0.34000000000000002,"15min":0.34599999999999997},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.41999999999999998,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.25800000000000001}]},{"osd":1,"last update":"Wed Apr 24 15:34:04 2024","interfaces":[{"interface":"back","average":{"1min":0.30299999999999999,"5min":0.33400000000000002,"15min":0.33900000000000002},"min":{"1min":0.19600000000000001,"5min":0.187,"15min":0.187},"max":{"1min":0.437,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.23300000000000001},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.42199999999999999,"15min":0.434},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.71299999999999997,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.28399999999999997}]},{"osd":2,"last update":"Wed Apr 24 15:34:04 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.39400000000000002,"15min":0.40300000000000002},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.46500000000000002,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.376,"5min":0.41299999999999998,"15min":0.41899999999999998},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.53000000000000003,"5min":1.252,"15min":1.252},"last":0.33700000000000002}]},{"osd":4,"last update":"Wed Apr 24 15:34:04 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.52200000000000002,"15min":0.53400000000000003},"min":{"1min":0.28599999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.56499999999999995,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.52100000000000002,"15min":0.53300000000000003},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.60699999999999998,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.371}]},{"osd":5,"last update":"Wed Apr 24 15:34:04 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.55600000000000005,"15min":0.56999999999999995},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.61199999999999999,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.5,"5min":0.58799999999999997,"15min":0.60199999999999998},"min":{"1min":0.38300000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.81100000000000005,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.496}]},{"osd":6,"last update":"Wed Apr 24 15:33:00 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.48599999999999999}]},{"osd":7,"last update":"Wed Apr 24 15:34:04 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.59099999999999997,"15min":0.60499999999999998},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.66800000000000004,"5min":1.635,"15min":1.635},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.495,"5min":0.55600000000000005,"15min":0.56599999999999995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.59699999999999998,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.45400000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574885,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307288,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":305534,"kb_avail":94064552,"statfs":{"total":96636764160,"available":96322101248,"internally_reserved":0,"allocated":1785856,"data_stored":908056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312867160},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 15:34:00 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.505,"15min":0.50900000000000001},"min":{"1min":0.249,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":1.8620000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.56599999999999995,"15min":0.55600000000000005},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":5.6239999999999997,"5min":5.6239999999999997,"15min":5.6239999999999997},"last":0.496}]},{"osd":1,"last update":"Wed Apr 24 15:34:00 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.58699999999999997,"15min":0.57299999999999995},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":5.7149999999999999,"5min":5.7149999999999999,"15min":5.7149999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.35499999999999998,"5min":0.31,"15min":0.31},"max":{"1min":1.913,"5min":1.913,"15min":1.913},"last":0.60699999999999998}]},{"osd":2,"last update":"Wed Apr 24 15:34:00 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.54600000000000004,"15min":0.52800000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":5.6660000000000004,"5min":5.6660000000000004,"15min":5.6660000000000004},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.53400000000000003,"15min":0.54400000000000004},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.819,"5min":1.819,"15min":1.819},"last":0.36599999999999999}]},{"osd":3,"last update":"Wed Apr 24 15:34:00 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.53400000000000003,"15min":0.53600000000000003},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.9550000000000001,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.55500000000000005,"15min":0.55000000000000004},"min":{"1min":0.36799999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":2.3149999999999999,"5min":2.3149999999999999,"15min":2.3149999999999999},"last":0.52400000000000002}]},{"osd":5,"last update":"Wed Apr 24 15:34:00 2024","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.58299999999999996,"15min":0.51700000000000002},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":14.667999999999999,"5min":14.667999999999999,"15min":14.667999999999999},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.434,"5min":0.497,"15min":0.50700000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.9890000000000001,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":0.437}]},{"osd":6,"last update":"Wed Apr 24 15:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.53800000000000003}]},{"osd":7,"last update":"Wed Apr 24 15:34:00 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.54000000000000004,"15min":0.53100000000000003},"min":{"1min":0.28599999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":5.2759999999999998,"5min":5.2759999999999998,"15min":5.2759999999999998},"last":0.41699999999999998},{"interface":"front","average":{"1min":1.0469999999999999,"5min":0.64100000000000001,"15min":0.57299999999999995},"min":{"1min":0.29399999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":14.773,"5min":14.773,"15min":14.773},"last":0.64100000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574884,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311388,"kb_used_data":1748,"kb_used_omap":1,"kb_used_meta":309630,"kb_avail":94060452,"statfs":{"total":96636764160,"available":96317902848,"internally_reserved":0,"allocated":1789952,"data_stored":910351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317061464},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.47899999999999998,"15min":0.48499999999999999},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.46800000000000003,"15min":0.47599999999999998},"min":{"1min":0.246,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.629,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.53500000000000003}]},{"osd":1,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.46300000000000002,"15min":0.47099999999999997},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.64900000000000002,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.46899999999999997,"15min":0.47099999999999997},"min":{"1min":0.317,"5min":0.313,"15min":0.313},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.56799999999999995}]},{"osd":2,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.47499999999999998,"15min":0.46600000000000003},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":2.4169999999999998,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.48599999999999999,"15min":0.49199999999999999},"min":{"1min":0.30099999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.39600000000000002}]},{"osd":3,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.48899999999999999,"15min":0.495},"min":{"1min":0.308,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.70999999999999996,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":2.3370000000000002,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":0.57799999999999996}]},{"osd":4,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.44800000000000001,"15min":0.45700000000000002},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.624,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.48399999999999999,"15min":0.48599999999999999},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.433}]},{"osd":6,"last update":"Wed Apr 24 15:33:02 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.38},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.46200000000000002}]},{"osd":7,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.52600000000000002,"15min":0.53100000000000003},"min":{"1min":0.30299999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.78600000000000003,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.496,"5min":0.53200000000000003,"15min":0.53800000000000003},"min":{"1min":0.34499999999999997,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.77100000000000002,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.50800000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4842,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":5677,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":5677,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15242,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T15:34:40.052 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2024-04-24T15:32:16.022683+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=32, tm_sec=16, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=34, tm_sec=34, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T15:34:40.052 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2024-04-24T15:32:16.022683+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=32, tm_sec=16, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=34, tm_sec=34, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T15:34:40.052 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2024-04-24T15:32:16.022683+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=32, tm_sec=16, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=34, tm_sec=34, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T15:34:40.052 INFO:tasks.ceph:pgid 4.1c last_scrub_stamp 2024-04-24T15:32:16.022683+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=32, tm_sec=16, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=34, tm_sec=34, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T15:34:40.052 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2024-04-24T15:32:16.022683+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=32, tm_sec=16, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=34, tm_sec=34, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T15:34:40.053 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2024-04-24T15:32:16.022683+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=32, tm_sec=16, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=34, tm_sec=34, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T15:34:40.053 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2024-04-24T15:32:16.022683+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=32, tm_sec=16, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=34, tm_sec=34, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T15:34:40.053 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2024-04-24T15:32:16.022683+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=32, tm_sec=16, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=34, tm_sec=34, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T15:34:40.053 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2024-04-24T15:32:16.022683+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=32, tm_sec=16, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=34, tm_sec=34, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T15:34:40.053 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2024-04-24T15:32:16.022683+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=32, tm_sec=16, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=34, tm_sec=34, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T15:34:40.053 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-04-24T15:32:00.938537+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=32, tm_sec=0, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=34, tm_sec=34, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T15:34:40.053 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-04-24T15:32:00.938537+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=32, tm_sec=0, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=34, tm_sec=34, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T15:34:40.053 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-04-24T15:32:00.938537+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=32, tm_sec=0, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=34, tm_sec=34, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T15:34:40.054 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2024-04-24T15:32:16.022683+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=32, tm_sec=16, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=34, tm_sec=34, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T15:34:40.054 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-04-24T15:32:00.938537+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=32, tm_sec=0, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=34, tm_sec=34, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T15:34:40.054 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2024-04-24T15:32:16.022683+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=32, tm_sec=16, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=34, tm_sec=34, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T15:34:40.054 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-24T15:31:59.935287+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=31, tm_sec=59, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=34, tm_sec=34, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T15:34:40.054 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-04-24T15:32:00.938537+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=32, tm_sec=0, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=34, tm_sec=34, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T15:34:40.054 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2024-04-24T15:32:16.022683+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=32, tm_sec=16, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=34, tm_sec=34, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T15:34:40.054 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2024-04-24T15:32:16.022683+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=32, tm_sec=16, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=34, tm_sec=34, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T15:34:40.055 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-04-24T15:32:06.972928+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=32, tm_sec=6, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=34, tm_sec=34, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T15:34:40.055 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-04-24T15:32:00.938537+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=32, tm_sec=0, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=34, tm_sec=34, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T15:34:40.055 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2024-04-24T15:32:16.022683+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=32, tm_sec=16, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=34, tm_sec=34, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T15:34:40.055 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2024-04-24T15:32:16.022683+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=32, tm_sec=16, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=34, tm_sec=34, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T15:34:40.055 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-04-24T15:32:00.938537+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=32, tm_sec=0, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=34, tm_sec=34, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T15:34:40.055 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2024-04-24T15:32:16.022683+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=32, tm_sec=16, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=34, tm_sec=34, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T15:34:40.055 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2024-04-24T15:32:16.022683+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=32, tm_sec=16, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=34, tm_sec=34, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T15:34:40.055 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2024-04-24T15:32:16.022683+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=32, tm_sec=16, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=34, tm_sec=34, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T15:34:40.056 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2024-04-24T15:32:16.022683+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=32, tm_sec=16, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=34, tm_sec=34, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T15:34:40.056 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2024-04-24T15:32:16.022683+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=32, tm_sec=16, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=34, tm_sec=34, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T15:34:40.056 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2024-04-24T15:32:16.022683+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=32, tm_sec=16, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=34, tm_sec=34, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T15:34:40.056 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2024-04-24T15:32:16.022683+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=32, tm_sec=16, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=34, tm_sec=34, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T15:34:40.056 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2024-04-24T15:32:16.022683+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=32, tm_sec=16, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=34, tm_sec=34, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T15:34:40.056 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2024-04-24T15:32:16.022683+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=32, tm_sec=16, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=34, tm_sec=34, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T15:34:40.056 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2024-04-24T15:32:16.022683+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=32, tm_sec=16, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=34, tm_sec=34, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T15:34:40.056 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2024-04-24T15:32:16.022683+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=32, tm_sec=16, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=34, tm_sec=34, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T15:34:40.057 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2024-04-24T15:32:16.022683+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=32, tm_sec=16, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=34, tm_sec=34, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T15:34:40.057 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2024-04-24T15:32:16.022683+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=32, tm_sec=16, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=34, tm_sec=34, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T15:34:40.057 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2024-04-24T15:32:16.022683+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=32, tm_sec=16, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=34, tm_sec=34, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T15:34:40.057 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-24T15:35:00.058 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T15:35:00.386 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:35:00.387 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-04-24T15:35:00.397 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":233,"stamp":"2024-04-24T15:34:59.585703+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590470,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":51,"num_read":114923,"num_read_kb":727642,"num_write":20450,"num_write_kb":358745,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4420,"num_bytes_recovered":106356266,"num_keys_recovered":235,"num_objects_omap":3,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":83,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17423,"ondisk_log_size":17423,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2440640,"kb_used_data":14080,"kb_used_omap":55,"kb_used_meta":2426440,"kb_avail":752534080,"statfs":{"total":773094113280,"available":770594897920,"internally_reserved":0,"allocated":14417920,"data_stored":8459522,"data_compressed":10519,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":56663,"internal_metadata":2484675241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":83,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":83,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002622"},"pg_stats":[{"pgid":"4.1f","version":"149'1486","reported_seq":2562,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:50.673275+0000","last_change":"2024-04-24T15:34:50.673238+0000","last_active":"2024-04-24T15:34:50.673275+0000","last_peered":"2024-04-24T15:34:50.673275+0000","last_clean":"2024-04-24T15:34:50.673275+0000","last_became_active":"2024-04-24T15:34:15.650850+0000","last_became_peered":"2024-04-24T15:34:15.650850+0000","last_unstale":"2024-04-24T15:34:50.673275+0000","last_undegraded":"2024-04-24T15:34:50.673275+0000","last_fullsized":"2024-04-24T15:34:50.673275+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"149'1486","last_scrub_stamp":"2024-04-24T15:34:50.673098+0000","last_deep_scrub":"149'1486","last_deep_scrub_stamp":"2024-04-24T15:34:50.673098+0000","last_clean_scrub_stamp":"2024-04-24T15:34:50.673098+0000","objects_scrubbed":1,"log_size":1486,"log_dups_size":0,"ondisk_log_size":1486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:16:54.402785+0000","scrub_duration":0.003073887,"objects_trimmed":0,"snaptrim_duration":0.001749947,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":8143,"num_read_kb":14722,"num_write":2575,"num_write_kb":16077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":5952278,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"149'508","reported_seq":6430,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:47.349997+0000","last_change":"2024-04-24T15:34:47.349948+0000","last_active":"2024-04-24T15:34:47.349997+0000","last_peered":"2024-04-24T15:34:47.349997+0000","last_clean":"2024-04-24T15:34:47.349997+0000","last_became_active":"2024-04-24T15:34:15.668657+0000","last_became_peered":"2024-04-24T15:34:15.668657+0000","last_unstale":"2024-04-24T15:34:47.349997+0000","last_undegraded":"2024-04-24T15:34:47.349997+0000","last_fullsized":"2024-04-24T15:34:47.349997+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"149'508","last_scrub_stamp":"2024-04-24T15:34:47.349854+0000","last_deep_scrub":"149'508","last_deep_scrub_stamp":"2024-04-24T15:34:47.349854+0000","last_clean_scrub_stamp":"2024-04-24T15:34:47.349854+0000","objects_scrubbed":1,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:31:51.178405+0000","scrub_duration":0.0041498170000000001,"objects_trimmed":0,"snaptrim_duration":0.00029386599999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":7278,"num_read_kb":39113,"num_write":560,"num_write_kb":6592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":4495932,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"149'531","reported_seq":3758,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:46.824854+0000","last_change":"2024-04-24T15:34:46.824821+0000","last_active":"2024-04-24T15:34:46.824854+0000","last_peered":"2024-04-24T15:34:46.824854+0000","last_clean":"2024-04-24T15:34:46.824854+0000","last_became_active":"2024-04-24T15:34:16.659202+0000","last_became_peered":"2024-04-24T15:34:16.659202+0000","last_unstale":"2024-04-24T15:34:46.824854+0000","last_undegraded":"2024-04-24T15:34:46.824854+0000","last_fullsized":"2024-04-24T15:34:46.824854+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"149'531","last_scrub_stamp":"2024-04-24T15:34:46.824709+0000","last_deep_scrub":"149'531","last_deep_scrub_stamp":"2024-04-24T15:34:46.824709+0000","last_clean_scrub_stamp":"2024-04-24T15:34:46.824709+0000","objects_scrubbed":2,"log_size":531,"log_dups_size":0,"ondisk_log_size":531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:30:24.464225+0000","scrub_duration":0.005707068,"objects_trimmed":0,"snaptrim_duration":0.0015488839999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":4427,"num_read_kb":51896,"num_write":603,"num_write_kb":18759,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":580671,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"149'528","reported_seq":2713,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:40.527438+0000","last_change":"2024-04-24T15:34:40.527414+0000","last_active":"2024-04-24T15:34:40.527438+0000","last_peered":"2024-04-24T15:34:40.527438+0000","last_clean":"2024-04-24T15:34:40.527438+0000","last_became_active":"2024-04-24T15:34:19.806783+0000","last_became_peered":"2024-04-24T15:34:19.806783+0000","last_unstale":"2024-04-24T15:34:40.527438+0000","last_undegraded":"2024-04-24T15:34:40.527438+0000","last_fullsized":"2024-04-24T15:34:40.527438+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"149'528","last_scrub_stamp":"2024-04-24T15:34:40.527334+0000","last_deep_scrub":"149'528","last_deep_scrub_stamp":"2024-04-24T15:34:40.527334+0000","last_clean_scrub_stamp":"2024-04-24T15:34:40.527334+0000","objects_scrubbed":2,"log_size":528,"log_dups_size":0,"ondisk_log_size":528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:53:26.967300+0000","scrub_duration":0.002933373,"objects_trimmed":0,"snaptrim_duration":0.001413493,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":3716,"num_read_kb":10078,"num_write":547,"num_write_kb":2888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"149'386","reported_seq":2371,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:42.476749+0000","last_change":"2024-04-24T15:34:42.476712+0000","last_active":"2024-04-24T15:34:42.476749+0000","last_peered":"2024-04-24T15:34:42.476749+0000","last_clean":"2024-04-24T15:34:42.476749+0000","last_became_active":"2024-04-24T15:34:23.825472+0000","last_became_peered":"2024-04-24T15:34:23.825472+0000","last_unstale":"2024-04-24T15:34:42.476749+0000","last_undegraded":"2024-04-24T15:34:42.476749+0000","last_fullsized":"2024-04-24T15:34:42.476749+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"149'386","last_scrub_stamp":"2024-04-24T15:34:42.476583+0000","last_deep_scrub":"149'386","last_deep_scrub_stamp":"2024-04-24T15:34:42.476583+0000","last_clean_scrub_stamp":"2024-04-24T15:34:42.476583+0000","objects_scrubbed":1,"log_size":386,"log_dups_size":0,"ondisk_log_size":386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:14:33.494375+0000","scrub_duration":0.0039126279999999996,"objects_trimmed":0,"snaptrim_duration":0.0011107479999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":2335,"num_read_kb":25405,"num_write":329,"num_write_kb":6118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":79,"num_bytes_recovered":2143880,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"149'451","reported_seq":6623,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:41.444924+0000","last_change":"2024-04-24T15:34:41.444877+0000","last_active":"2024-04-24T15:34:41.444924+0000","last_peered":"2024-04-24T15:34:41.444924+0000","last_clean":"2024-04-24T15:34:41.444924+0000","last_became_active":"2024-04-24T15:34:23.824861+0000","last_became_peered":"2024-04-24T15:34:23.824861+0000","last_unstale":"2024-04-24T15:34:41.444924+0000","last_undegraded":"2024-04-24T15:34:41.444924+0000","last_fullsized":"2024-04-24T15:34:41.444924+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"149'451","last_scrub_stamp":"2024-04-24T15:34:41.444781+0000","last_deep_scrub":"149'451","last_deep_scrub_stamp":"2024-04-24T15:34:41.444781+0000","last_clean_scrub_stamp":"2024-04-24T15:34:41.444781+0000","objects_scrubbed":2,"log_size":451,"log_dups_size":0,"ondisk_log_size":451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:56:48.021882+0000","scrub_duration":0.0047438239999999998,"objects_trimmed":0,"snaptrim_duration":0.00097376600000000004,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":6368,"num_read_kb":19805,"num_write":475,"num_write_kb":10193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":89,"num_bytes_recovered":1137673,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"149'538","reported_seq":2692,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:54.856624+0000","last_change":"2024-04-24T15:34:54.856577+0000","last_active":"2024-04-24T15:34:54.856624+0000","last_peered":"2024-04-24T15:34:54.856624+0000","last_clean":"2024-04-24T15:34:54.856624+0000","last_became_active":"2024-04-24T15:32:17.047842+0000","last_became_peered":"2024-04-24T15:32:17.047842+0000","last_unstale":"2024-04-24T15:34:54.856624+0000","last_undegraded":"2024-04-24T15:34:54.856624+0000","last_fullsized":"2024-04-24T15:34:54.856624+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"149'538","last_scrub_stamp":"2024-04-24T15:34:54.856475+0000","last_deep_scrub":"149'538","last_deep_scrub_stamp":"2024-04-24T15:34:54.856475+0000","last_clean_scrub_stamp":"2024-04-24T15:34:54.856475+0000","objects_scrubbed":1,"log_size":538,"log_dups_size":0,"ondisk_log_size":538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:47:25.664161+0000","scrub_duration":0.0034313270000000001,"objects_trimmed":0,"snaptrim_duration":0.00081826100000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":4119,"num_read_kb":22694,"num_write":527,"num_write_kb":11193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"149'449","reported_seq":1984,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:40.467325+0000","last_change":"2024-04-24T15:34:40.467271+0000","last_active":"2024-04-24T15:34:40.467325+0000","last_peered":"2024-04-24T15:34:40.467325+0000","last_clean":"2024-04-24T15:34:40.467325+0000","last_became_active":"2024-04-24T15:33:42.641679+0000","last_became_peered":"2024-04-24T15:33:42.641679+0000","last_unstale":"2024-04-24T15:34:40.467325+0000","last_undegraded":"2024-04-24T15:34:40.467325+0000","last_fullsized":"2024-04-24T15:34:40.467325+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"149'449","last_scrub_stamp":"2024-04-24T15:34:40.467105+0000","last_deep_scrub":"149'449","last_deep_scrub_stamp":"2024-04-24T15:34:40.467105+0000","last_clean_scrub_stamp":"2024-04-24T15:34:40.467105+0000","objects_scrubbed":5,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:24:09.528219+0000","scrub_duration":0.011387028,"objects_trimmed":0,"snaptrim_duration":0.00077716999999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":1887,"num_read_kb":15406,"num_write":547,"num_write_kb":8155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":4099152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"149'525","reported_seq":2460,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:47.871962+0000","last_change":"2024-04-24T15:34:47.871933+0000","last_active":"2024-04-24T15:34:47.871962+0000","last_peered":"2024-04-24T15:34:47.871962+0000","last_clean":"2024-04-24T15:34:47.871962+0000","last_became_active":"2024-04-24T15:33:56.699629+0000","last_became_peered":"2024-04-24T15:33:56.699629+0000","last_unstale":"2024-04-24T15:34:47.871962+0000","last_undegraded":"2024-04-24T15:34:47.871962+0000","last_fullsized":"2024-04-24T15:34:47.871962+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"149'525","last_scrub_stamp":"2024-04-24T15:34:47.871840+0000","last_deep_scrub":"149'525","last_deep_scrub_stamp":"2024-04-24T15:34:47.871840+0000","last_clean_scrub_stamp":"2024-04-24T15:34:47.871840+0000","objects_scrubbed":2,"log_size":525,"log_dups_size":0,"ondisk_log_size":525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:09:43.201751+0000","scrub_duration":0.0066328519999999998,"objects_trimmed":0,"snaptrim_duration":0.00212766,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":3353,"num_read_kb":16260,"num_write":594,"num_write_kb":12753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":352256,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"149'318","reported_seq":2590,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:43.501751+0000","last_change":"2024-04-24T15:34:43.501722+0000","last_active":"2024-04-24T15:34:43.501751+0000","last_peered":"2024-04-24T15:34:43.501751+0000","last_clean":"2024-04-24T15:34:43.501751+0000","last_became_active":"2024-04-24T15:34:23.827917+0000","last_became_peered":"2024-04-24T15:34:23.827917+0000","last_unstale":"2024-04-24T15:34:43.501751+0000","last_undegraded":"2024-04-24T15:34:43.501751+0000","last_fullsized":"2024-04-24T15:34:43.501751+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"149'318","last_scrub_stamp":"2024-04-24T15:34:43.501627+0000","last_deep_scrub":"149'318","last_deep_scrub_stamp":"2024-04-24T15:34:43.501627+0000","last_clean_scrub_stamp":"2024-04-24T15:34:43.501627+0000","objects_scrubbed":2,"log_size":318,"log_dups_size":0,"ondisk_log_size":318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:32:18.592700+0000","scrub_duration":0.0035660290000000001,"objects_trimmed":0,"snaptrim_duration":0.00065534800000000004,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":1633,"num_read_kb":26736,"num_write":303,"num_write_kb":13822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":9304722,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":132,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:41.718271+0000","last_change":"2024-04-24T15:34:41.718227+0000","last_active":"2024-04-24T15:34:41.718271+0000","last_peered":"2024-04-24T15:34:41.718271+0000","last_clean":"2024-04-24T15:34:41.718271+0000","last_became_active":"2024-04-24T15:33:53.643941+0000","last_became_peered":"2024-04-24T15:33:53.643941+0000","last_unstale":"2024-04-24T15:34:41.718271+0000","last_undegraded":"2024-04-24T15:34:41.718271+0000","last_fullsized":"2024-04-24T15:34:41.718271+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:34:41.718086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:34:41.718086+0000","last_clean_scrub_stamp":"2024-04-24T15:34:41.718086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:11:48.998969+0000","scrub_duration":0.0017191350000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"149'867","reported_seq":3025,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:37.530105+0000","last_change":"2024-04-24T15:34:37.530071+0000","last_active":"2024-04-24T15:34:37.530105+0000","last_peered":"2024-04-24T15:34:37.530105+0000","last_clean":"2024-04-24T15:34:37.530105+0000","last_became_active":"2024-04-24T15:34:19.809619+0000","last_became_peered":"2024-04-24T15:34:19.809619+0000","last_unstale":"2024-04-24T15:34:37.530105+0000","last_undegraded":"2024-04-24T15:34:37.530105+0000","last_fullsized":"2024-04-24T15:34:37.530105+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"149'867","last_scrub_stamp":"2024-04-24T15:34:37.529969+0000","last_deep_scrub":"149'867","last_deep_scrub_stamp":"2024-04-24T15:34:37.529969+0000","last_clean_scrub_stamp":"2024-04-24T15:34:37.529969+0000","objects_scrubbed":5,"log_size":867,"log_dups_size":0,"ondisk_log_size":867,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:05:19.302172+0000","scrub_duration":0.0058171969999999996,"objects_trimmed":2,"snaptrim_duration":0.00033353700000000002,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":4689,"num_read_kb":23870,"num_write":1044,"num_write_kb":13092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":176,"num_bytes_recovered":5601476,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":35,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:44.274028+0000","last_change":"2024-04-24T15:34:44.274001+0000","last_active":"2024-04-24T15:34:44.274028+0000","last_peered":"2024-04-24T15:34:44.274028+0000","last_clean":"2024-04-24T15:34:44.274028+0000","last_became_active":"2024-04-24T15:34:12.652298+0000","last_became_peered":"2024-04-24T15:34:12.652298+0000","last_unstale":"2024-04-24T15:34:44.274028+0000","last_undegraded":"2024-04-24T15:34:44.274028+0000","last_fullsized":"2024-04-24T15:34:44.274028+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:34:44.273900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:34:44.273900+0000","last_clean_scrub_stamp":"2024-04-24T15:34:44.273900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:24:59.154244+0000","scrub_duration":0.0019815879999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":52,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:40.791816+0000","last_change":"2024-04-24T15:34:40.791767+0000","last_active":"2024-04-24T15:34:40.791816+0000","last_peered":"2024-04-24T15:34:40.791816+0000","last_clean":"2024-04-24T15:34:40.791816+0000","last_became_active":"2024-04-24T15:33:53.645696+0000","last_became_peered":"2024-04-24T15:33:53.645696+0000","last_unstale":"2024-04-24T15:34:40.791816+0000","last_undegraded":"2024-04-24T15:34:40.791816+0000","last_fullsized":"2024-04-24T15:34:40.791816+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:34:40.791674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:34:40.791674+0000","last_clean_scrub_stamp":"2024-04-24T15:34:40.791674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:10:48.332079+0000","scrub_duration":0.0017782620000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"149'585","reported_seq":5184,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:40.349759+0000","last_change":"2024-04-24T15:34:40.349734+0000","last_active":"2024-04-24T15:34:40.349759+0000","last_peered":"2024-04-24T15:34:40.349759+0000","last_clean":"2024-04-24T15:34:40.349759+0000","last_became_active":"2024-04-24T15:34:17.654759+0000","last_became_peered":"2024-04-24T15:34:17.654759+0000","last_unstale":"2024-04-24T15:34:40.349759+0000","last_undegraded":"2024-04-24T15:34:40.349759+0000","last_fullsized":"2024-04-24T15:34:40.349759+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"149'585","last_scrub_stamp":"2024-04-24T15:34:40.349659+0000","last_deep_scrub":"149'585","last_deep_scrub_stamp":"2024-04-24T15:34:40.349659+0000","last_clean_scrub_stamp":"2024-04-24T15:34:40.349659+0000","objects_scrubbed":1,"log_size":585,"log_dups_size":0,"ondisk_log_size":585,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:47:17.470137+0000","scrub_duration":0.0034538360000000001,"objects_trimmed":0,"snaptrim_duration":0.00061779100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3592,"num_read_kb":14899,"num_write":598,"num_write_kb":11359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":23598,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"149'3","reported_seq":93,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:36.366452+0000","last_change":"2024-04-24T15:34:36.366408+0000","last_active":"2024-04-24T15:34:36.366452+0000","last_peered":"2024-04-24T15:34:36.366452+0000","last_clean":"2024-04-24T15:34:36.366452+0000","last_became_active":"2024-04-24T15:33:47.662685+0000","last_became_peered":"2024-04-24T15:33:47.662685+0000","last_unstale":"2024-04-24T15:34:36.366452+0000","last_undegraded":"2024-04-24T15:34:36.366452+0000","last_fullsized":"2024-04-24T15:34:36.366452+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"149'2","last_scrub_stamp":"2024-04-24T15:34:36.366339+0000","last_deep_scrub":"149'2","last_deep_scrub_stamp":"2024-04-24T15:34:36.366339+0000","last_clean_scrub_stamp":"2024-04-24T15:34:36.366339+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:57:31.467325+0000","scrub_duration":0.0030479940000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":83,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":61,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:41.805246+0000","last_change":"2024-04-24T15:34:41.805209+0000","last_active":"2024-04-24T15:34:41.805246+0000","last_peered":"2024-04-24T15:34:41.805246+0000","last_clean":"2024-04-24T15:34:41.805246+0000","last_became_active":"2024-04-24T15:34:12.652805+0000","last_became_peered":"2024-04-24T15:34:12.652805+0000","last_unstale":"2024-04-24T15:34:41.805246+0000","last_undegraded":"2024-04-24T15:34:41.805246+0000","last_fullsized":"2024-04-24T15:34:41.805246+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:34:41.805118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:34:41.805118+0000","last_clean_scrub_stamp":"2024-04-24T15:34:41.805118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:28:34.952739+0000","scrub_duration":0.001661591,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"149'616","reported_seq":2381,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:39.346173+0000","last_change":"2024-04-24T15:34:39.346151+0000","last_active":"2024-04-24T15:34:39.346173+0000","last_peered":"2024-04-24T15:34:39.346173+0000","last_clean":"2024-04-24T15:34:39.346173+0000","last_became_active":"2024-04-24T15:34:16.655390+0000","last_became_peered":"2024-04-24T15:34:16.655390+0000","last_unstale":"2024-04-24T15:34:39.346173+0000","last_undegraded":"2024-04-24T15:34:39.346173+0000","last_fullsized":"2024-04-24T15:34:39.346173+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"149'616","last_scrub_stamp":"2024-04-24T15:34:39.346081+0000","last_deep_scrub":"149'616","last_deep_scrub_stamp":"2024-04-24T15:34:39.346081+0000","last_clean_scrub_stamp":"2024-04-24T15:34:39.346081+0000","objects_scrubbed":1,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:37:29.323750+0000","scrub_duration":0.0025279489999999998,"objects_trimmed":0,"snaptrim_duration":0.00042917399999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":3015,"num_read_kb":19823,"num_write":696,"num_write_kb":5112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"149'4","reported_seq":195,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:38.813383+0000","last_change":"2024-04-24T15:34:38.813338+0000","last_active":"2024-04-24T15:34:38.813383+0000","last_peered":"2024-04-24T15:34:38.813383+0000","last_clean":"2024-04-24T15:34:38.813383+0000","last_became_active":"2024-04-24T15:34:13.662363+0000","last_became_peered":"2024-04-24T15:34:13.662363+0000","last_unstale":"2024-04-24T15:34:38.813383+0000","last_undegraded":"2024-04-24T15:34:38.813383+0000","last_fullsized":"2024-04-24T15:34:38.813383+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"149'4","last_scrub_stamp":"2024-04-24T15:34:38.813254+0000","last_deep_scrub":"149'4","last_deep_scrub_stamp":"2024-04-24T15:34:38.813254+0000","last_clean_scrub_stamp":"2024-04-24T15:34:38.813254+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:56:01.218000+0000","scrub_duration":0.006372244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"149'119","reported_seq":264,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:43.742101+0000","last_change":"2024-04-24T15:34:43.742050+0000","last_active":"2024-04-24T15:34:43.742101+0000","last_peered":"2024-04-24T15:34:43.742101+0000","last_clean":"2024-04-24T15:34:43.742101+0000","last_became_active":"2024-04-24T15:33:57.644312+0000","last_became_peered":"2024-04-24T15:33:57.644312+0000","last_unstale":"2024-04-24T15:34:43.742101+0000","last_undegraded":"2024-04-24T15:34:43.742101+0000","last_fullsized":"2024-04-24T15:34:43.742101+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"149'119","last_scrub_stamp":"2024-04-24T15:34:43.741949+0000","last_deep_scrub":"149'119","last_deep_scrub_stamp":"2024-04-24T15:34:43.741949+0000","last_clean_scrub_stamp":"2024-04-24T15:34:43.741949+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:37:17.783904+0000","scrub_duration":0.0054802949999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":34,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:47.897301+0000","last_change":"2024-04-24T15:34:47.897272+0000","last_active":"2024-04-24T15:34:47.897301+0000","last_peered":"2024-04-24T15:34:47.897301+0000","last_clean":"2024-04-24T15:34:47.897301+0000","last_became_active":"2024-04-24T15:34:13.650174+0000","last_became_peered":"2024-04-24T15:34:13.650174+0000","last_unstale":"2024-04-24T15:34:47.897301+0000","last_undegraded":"2024-04-24T15:34:47.897301+0000","last_fullsized":"2024-04-24T15:34:47.897301+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:34:47.897162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:34:47.897162+0000","last_clean_scrub_stamp":"2024-04-24T15:34:47.897162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:20:29.591212+0000","scrub_duration":0.001724338,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"149'592","reported_seq":2468,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:48.615780+0000","last_change":"2024-04-24T15:34:48.615749+0000","last_active":"2024-04-24T15:34:48.615780+0000","last_peered":"2024-04-24T15:34:48.615780+0000","last_clean":"2024-04-24T15:34:48.615780+0000","last_became_active":"2024-04-24T15:33:45.658207+0000","last_became_peered":"2024-04-24T15:33:45.658207+0000","last_unstale":"2024-04-24T15:34:48.615780+0000","last_undegraded":"2024-04-24T15:34:48.615780+0000","last_fullsized":"2024-04-24T15:34:48.615780+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"149'592","last_scrub_stamp":"2024-04-24T15:34:48.615646+0000","last_deep_scrub":"149'592","last_deep_scrub_stamp":"2024-04-24T15:34:48.615646+0000","last_clean_scrub_stamp":"2024-04-24T15:34:48.615646+0000","objects_scrubbed":1,"log_size":592,"log_dups_size":0,"ondisk_log_size":592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:36:52.015734+0000","scrub_duration":0.0035351150000000001,"objects_trimmed":0,"snaptrim_duration":0.0010111199999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":3352,"num_read_kb":26205,"num_write":791,"num_write_kb":11816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":8821091,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":35,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:38.329944+0000","last_change":"2024-04-24T15:34:38.329904+0000","last_active":"2024-04-24T15:34:38.329944+0000","last_peered":"2024-04-24T15:34:38.329944+0000","last_clean":"2024-04-24T15:34:38.329944+0000","last_became_active":"2024-04-24T15:34:12.655508+0000","last_became_peered":"2024-04-24T15:34:12.655508+0000","last_unstale":"2024-04-24T15:34:38.329944+0000","last_undegraded":"2024-04-24T15:34:38.329944+0000","last_fullsized":"2024-04-24T15:34:38.329944+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:34:38.329808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:34:38.329808+0000","last_clean_scrub_stamp":"2024-04-24T15:34:38.329808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:03:40.453877+0000","scrub_duration":0.001431858,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":70,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:38.557893+0000","last_change":"2024-04-24T15:34:38.557860+0000","last_active":"2024-04-24T15:34:38.557893+0000","last_peered":"2024-04-24T15:34:38.557893+0000","last_clean":"2024-04-24T15:34:38.557893+0000","last_became_active":"2024-04-24T15:34:19.838058+0000","last_became_peered":"2024-04-24T15:34:19.838058+0000","last_unstale":"2024-04-24T15:34:38.557893+0000","last_undegraded":"2024-04-24T15:34:38.557893+0000","last_fullsized":"2024-04-24T15:34:38.557893+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:34:38.557748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:34:38.557748+0000","last_clean_scrub_stamp":"2024-04-24T15:34:38.557748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:53:13.458240+0000","scrub_duration":0.001481909,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"149'433","reported_seq":1245,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:49.273658+0000","last_change":"2024-04-24T15:34:49.273632+0000","last_active":"2024-04-24T15:34:49.273658+0000","last_peered":"2024-04-24T15:34:49.273658+0000","last_clean":"2024-04-24T15:34:49.273658+0000","last_became_active":"2024-04-24T15:34:15.658368+0000","last_became_peered":"2024-04-24T15:34:15.658368+0000","last_unstale":"2024-04-24T15:34:49.273658+0000","last_undegraded":"2024-04-24T15:34:49.273658+0000","last_fullsized":"2024-04-24T15:34:49.273658+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"149'433","last_scrub_stamp":"2024-04-24T15:34:49.273531+0000","last_deep_scrub":"149'433","last_deep_scrub_stamp":"2024-04-24T15:34:49.273531+0000","last_clean_scrub_stamp":"2024-04-24T15:34:49.273531+0000","objects_scrubbed":3,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:56:17.249810+0000","scrub_duration":0.0041417190000000003,"objects_trimmed":0,"snaptrim_duration":0.0015045289999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":1970,"num_read_kb":12710,"num_write":463,"num_write_kb":8069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":208,"num_bytes_recovered":107133,"num_keys_recovered":27,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":94,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:29.419029+0000","last_change":"2024-04-24T15:34:24.428957+0000","last_active":"2024-04-24T15:34:29.419029+0000","last_peered":"2024-04-24T15:34:29.419029+0000","last_clean":"2024-04-24T15:34:29.419029+0000","last_became_active":"2024-04-24T15:34:24.428510+0000","last_became_peered":"2024-04-24T15:34:24.428510+0000","last_unstale":"2024-04-24T15:34:29.419029+0000","last_undegraded":"2024-04-24T15:34:29.419029+0000","last_fullsized":"2024-04-24T15:34:29.419029+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:06.972928+0000","last_clean_scrub_stamp":"2024-04-24T15:32:06.972928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":34,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:49.923775+0000","last_change":"2024-04-24T15:34:49.923738+0000","last_active":"2024-04-24T15:34:49.923775+0000","last_peered":"2024-04-24T15:34:49.923775+0000","last_clean":"2024-04-24T15:34:49.923775+0000","last_became_active":"2024-04-24T15:34:13.689685+0000","last_became_peered":"2024-04-24T15:34:13.689685+0000","last_unstale":"2024-04-24T15:34:49.923775+0000","last_undegraded":"2024-04-24T15:34:49.923775+0000","last_fullsized":"2024-04-24T15:34:49.923775+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:34:49.923635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:34:49.923635+0000","last_clean_scrub_stamp":"2024-04-24T15:34:49.923635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:53:47.338193+0000","scrub_duration":0.0018300860000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"130'457","reported_seq":2006,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:54.278887+0000","last_change":"2024-04-24T15:34:54.278853+0000","last_active":"2024-04-24T15:34:54.278887+0000","last_peered":"2024-04-24T15:34:54.278887+0000","last_clean":"2024-04-24T15:34:54.278887+0000","last_became_active":"2024-04-24T15:34:16.651906+0000","last_became_peered":"2024-04-24T15:34:16.651906+0000","last_unstale":"2024-04-24T15:34:54.278887+0000","last_undegraded":"2024-04-24T15:34:54.278887+0000","last_fullsized":"2024-04-24T15:34:54.278887+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"130'457","last_scrub_stamp":"2024-04-24T15:34:54.278739+0000","last_deep_scrub":"130'457","last_deep_scrub_stamp":"2024-04-24T15:34:54.278739+0000","last_clean_scrub_stamp":"2024-04-24T15:34:54.278739+0000","objects_scrubbed":0,"log_size":457,"log_dups_size":0,"ondisk_log_size":457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:29:38.898067+0000","scrub_duration":0.001758279,"objects_trimmed":0,"snaptrim_duration":0.0010565220000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2138,"num_read_kb":24032,"num_write":423,"num_write_kb":14900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"130'408","reported_seq":1985,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:50.238546+0000","last_change":"2024-04-24T15:34:50.238512+0000","last_active":"2024-04-24T15:34:50.238546+0000","last_peered":"2024-04-24T15:34:50.238546+0000","last_clean":"2024-04-24T15:34:50.238546+0000","last_became_active":"2024-04-24T15:34:16.651931+0000","last_became_peered":"2024-04-24T15:34:16.651931+0000","last_unstale":"2024-04-24T15:34:50.238546+0000","last_undegraded":"2024-04-24T15:34:50.238546+0000","last_fullsized":"2024-04-24T15:34:50.238546+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"130'408","last_scrub_stamp":"2024-04-24T15:34:50.238356+0000","last_deep_scrub":"130'408","last_deep_scrub_stamp":"2024-04-24T15:34:50.238356+0000","last_clean_scrub_stamp":"2024-04-24T15:34:50.238356+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:48:49.804142+0000","scrub_duration":0.0019868109999999998,"objects_trimmed":0,"snaptrim_duration":0.00039757799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2388,"num_read_kb":22956,"num_write":379,"num_write_kb":16060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":578400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":35,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:45.281361+0000","last_change":"2024-04-24T15:34:45.281331+0000","last_active":"2024-04-24T15:34:45.281361+0000","last_peered":"2024-04-24T15:34:45.281361+0000","last_clean":"2024-04-24T15:34:45.281361+0000","last_became_active":"2024-04-24T15:34:12.653529+0000","last_became_peered":"2024-04-24T15:34:12.653529+0000","last_unstale":"2024-04-24T15:34:45.281361+0000","last_undegraded":"2024-04-24T15:34:45.281361+0000","last_fullsized":"2024-04-24T15:34:45.281361+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:34:45.281222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:34:45.281222+0000","last_clean_scrub_stamp":"2024-04-24T15:34:45.281222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:06:49.812307+0000","scrub_duration":0.00148835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"149'481","reported_seq":2010,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:39.783116+0000","last_change":"2024-04-24T15:34:39.783087+0000","last_active":"2024-04-24T15:34:39.783116+0000","last_peered":"2024-04-24T15:34:39.783116+0000","last_clean":"2024-04-24T15:34:39.783116+0000","last_became_active":"2024-04-24T15:34:16.654252+0000","last_became_peered":"2024-04-24T15:34:16.654252+0000","last_unstale":"2024-04-24T15:34:39.783116+0000","last_undegraded":"2024-04-24T15:34:39.783116+0000","last_fullsized":"2024-04-24T15:34:39.783116+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"149'481","last_scrub_stamp":"2024-04-24T15:34:39.782995+0000","last_deep_scrub":"149'481","last_deep_scrub_stamp":"2024-04-24T15:34:39.782995+0000","last_clean_scrub_stamp":"2024-04-24T15:34:39.782995+0000","objects_scrubbed":2,"log_size":481,"log_dups_size":0,"ondisk_log_size":481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:01:17.010816+0000","scrub_duration":0.003456808,"objects_trimmed":0,"snaptrim_duration":0.00035787500000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":2434,"num_read_kb":14959,"num_write":545,"num_write_kb":12213,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":335,"num_bytes_recovered":6920328,"num_keys_recovered":43,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"130'616","reported_seq":2735,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:29.418778+0000","last_change":"2024-04-24T15:34:23.827845+0000","last_active":"2024-04-24T15:34:29.418778+0000","last_peered":"2024-04-24T15:34:29.418778+0000","last_clean":"2024-04-24T15:34:29.418778+0000","last_became_active":"2024-04-24T15:34:23.825759+0000","last_became_peered":"2024-04-24T15:34:23.825759+0000","last_unstale":"2024-04-24T15:34:29.418778+0000","last_undegraded":"2024-04-24T15:34:29.418778+0000","last_fullsized":"2024-04-24T15:34:29.418778+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:32:16.022683+0000","last_clean_scrub_stamp":"2024-04-24T15:32:16.022683+0000","objects_scrubbed":0,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0055639879999999997,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":4324,"num_read_kb":21128,"num_write":677,"num_write_kb":10933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":5713612,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"149'486","reported_seq":3243,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:46.639033+0000","last_change":"2024-04-24T15:34:46.638989+0000","last_active":"2024-04-24T15:34:46.639033+0000","last_peered":"2024-04-24T15:34:46.639033+0000","last_clean":"2024-04-24T15:34:46.639033+0000","last_became_active":"2024-04-24T15:34:15.657660+0000","last_became_peered":"2024-04-24T15:34:15.657660+0000","last_unstale":"2024-04-24T15:34:46.639033+0000","last_undegraded":"2024-04-24T15:34:46.639033+0000","last_fullsized":"2024-04-24T15:34:46.639033+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"149'486","last_scrub_stamp":"2024-04-24T15:34:46.638853+0000","last_deep_scrub":"149'486","last_deep_scrub_stamp":"2024-04-24T15:34:46.638853+0000","last_clean_scrub_stamp":"2024-04-24T15:34:46.638853+0000","objects_scrubbed":1,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:11:13.324969+0000","scrub_duration":0.003648166,"objects_trimmed":0,"snaptrim_duration":0.00066763,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3120,"num_read_kb":27458,"num_write":430,"num_write_kb":12267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":237,"num_bytes_recovered":5299136,"num_keys_recovered":49,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"129'551","reported_seq":1987,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:38.371801+0000","last_change":"2024-04-24T15:34:38.371767+0000","last_active":"2024-04-24T15:34:38.371801+0000","last_peered":"2024-04-24T15:34:38.371801+0000","last_clean":"2024-04-24T15:34:38.371801+0000","last_became_active":"2024-04-24T15:34:15.668717+0000","last_became_peered":"2024-04-24T15:34:15.668717+0000","last_unstale":"2024-04-24T15:34:38.371801+0000","last_undegraded":"2024-04-24T15:34:38.371801+0000","last_fullsized":"2024-04-24T15:34:38.371801+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"129'551","last_scrub_stamp":"2024-04-24T15:34:38.371640+0000","last_deep_scrub":"129'551","last_deep_scrub_stamp":"2024-04-24T15:34:38.371640+0000","last_clean_scrub_stamp":"2024-04-24T15:34:38.371640+0000","objects_scrubbed":0,"log_size":551,"log_dups_size":0,"ondisk_log_size":551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:19:54.567694+0000","scrub_duration":0.001344725,"objects_trimmed":0,"snaptrim_duration":0.00039775200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2839,"num_read_kb":21248,"num_write":721,"num_write_kb":8473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":141,"num_bytes_recovered":1490474,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"149'1049","reported_seq":2519,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:52.641192+0000","last_change":"2024-04-24T15:34:52.641161+0000","last_active":"2024-04-24T15:34:52.641192+0000","last_peered":"2024-04-24T15:34:52.641192+0000","last_clean":"2024-04-24T15:34:52.641192+0000","last_became_active":"2024-04-24T15:34:23.825039+0000","last_became_peered":"2024-04-24T15:34:23.825039+0000","last_unstale":"2024-04-24T15:34:52.641192+0000","last_undegraded":"2024-04-24T15:34:52.641192+0000","last_fullsized":"2024-04-24T15:34:52.641192+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"149'1049","last_scrub_stamp":"2024-04-24T15:34:52.641062+0000","last_deep_scrub":"149'1049","last_deep_scrub_stamp":"2024-04-24T15:34:52.641062+0000","last_clean_scrub_stamp":"2024-04-24T15:34:52.641062+0000","objects_scrubbed":3,"log_size":1049,"log_dups_size":0,"ondisk_log_size":1049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:09:37.232015+0000","scrub_duration":0.00545701,"objects_trimmed":0,"snaptrim_duration":0.0029127150000000002,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":6311,"num_read_kb":16558,"num_write":1677,"num_write_kb":12129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":54,"num_bytes_recovered":1219753,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.d","version":"149'299","reported_seq":2584,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:54.716410+0000","last_change":"2024-04-24T15:34:54.716378+0000","last_active":"2024-04-24T15:34:54.716410+0000","last_peered":"2024-04-24T15:34:54.716410+0000","last_clean":"2024-04-24T15:34:54.716410+0000","last_became_active":"2024-04-24T15:34:23.823763+0000","last_became_peered":"2024-04-24T15:34:23.823763+0000","last_unstale":"2024-04-24T15:34:54.716410+0000","last_undegraded":"2024-04-24T15:34:54.716410+0000","last_fullsized":"2024-04-24T15:34:54.716410+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"149'299","last_scrub_stamp":"2024-04-24T15:34:54.716280+0000","last_deep_scrub":"149'299","last_deep_scrub_stamp":"2024-04-24T15:34:54.716280+0000","last_clean_scrub_stamp":"2024-04-24T15:34:54.716280+0000","objects_scrubbed":1,"log_size":299,"log_dups_size":0,"ondisk_log_size":299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:19:20.347461+0000","scrub_duration":0.003803912,"objects_trimmed":0,"snaptrim_duration":0.00047837299999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":2121,"num_read_kb":46483,"num_write":303,"num_write_kb":20603,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":4246835,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"129'563","reported_seq":2406,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:44.539365+0000","last_change":"2024-04-24T15:34:44.539331+0000","last_active":"2024-04-24T15:34:44.539365+0000","last_peered":"2024-04-24T15:34:44.539365+0000","last_clean":"2024-04-24T15:34:44.539365+0000","last_became_active":"2024-04-24T15:34:23.824037+0000","last_became_peered":"2024-04-24T15:34:23.824037+0000","last_unstale":"2024-04-24T15:34:44.539365+0000","last_undegraded":"2024-04-24T15:34:44.539365+0000","last_fullsized":"2024-04-24T15:34:44.539365+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"129'563","last_scrub_stamp":"2024-04-24T15:34:44.539210+0000","last_deep_scrub":"129'563","last_deep_scrub_stamp":"2024-04-24T15:34:44.539210+0000","last_clean_scrub_stamp":"2024-04-24T15:34:44.539210+0000","objects_scrubbed":0,"log_size":563,"log_dups_size":0,"ondisk_log_size":563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:42:19.986755+0000","scrub_duration":0.001982106,"objects_trimmed":0,"snaptrim_duration":0.001084025,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3022,"num_read_kb":13469,"num_write":696,"num_write_kb":6185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":113,"num_bytes_recovered":1676557,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"149'909","reported_seq":3319,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:52.269827+0000","last_change":"2024-04-24T15:34:52.269796+0000","last_active":"2024-04-24T15:34:52.269827+0000","last_peered":"2024-04-24T15:34:52.269827+0000","last_clean":"2024-04-24T15:34:52.269827+0000","last_became_active":"2024-04-24T15:34:16.655733+0000","last_became_peered":"2024-04-24T15:34:16.655733+0000","last_unstale":"2024-04-24T15:34:52.269827+0000","last_undegraded":"2024-04-24T15:34:52.269827+0000","last_fullsized":"2024-04-24T15:34:52.269827+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"149'909","last_scrub_stamp":"2024-04-24T15:34:52.269703+0000","last_deep_scrub":"149'909","last_deep_scrub_stamp":"2024-04-24T15:34:52.269703+0000","last_clean_scrub_stamp":"2024-04-24T15:34:52.269703+0000","objects_scrubbed":2,"log_size":909,"log_dups_size":0,"ondisk_log_size":909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:54:41.487566+0000","scrub_duration":0.0037041299999999999,"objects_trimmed":0,"snaptrim_duration":0.0018480180000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":5733,"num_read_kb":14488,"num_write":1011,"num_write_kb":8995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":223,"num_bytes_recovered":630687,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"130'244","reported_seq":1072,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:52.850262+0000","last_change":"2024-04-24T15:34:52.850233+0000","last_active":"2024-04-24T15:34:52.850262+0000","last_peered":"2024-04-24T15:34:52.850262+0000","last_clean":"2024-04-24T15:34:52.850262+0000","last_became_active":"2024-04-24T15:34:15.663081+0000","last_became_peered":"2024-04-24T15:34:15.663081+0000","last_unstale":"2024-04-24T15:34:52.850262+0000","last_undegraded":"2024-04-24T15:34:52.850262+0000","last_fullsized":"2024-04-24T15:34:52.850262+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"130'244","last_scrub_stamp":"2024-04-24T15:34:52.850135+0000","last_deep_scrub":"130'244","last_deep_scrub_stamp":"2024-04-24T15:34:52.850135+0000","last_clean_scrub_stamp":"2024-04-24T15:34:52.850135+0000","objects_scrubbed":0,"log_size":244,"log_dups_size":0,"ondisk_log_size":244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:06:40.625024+0000","scrub_duration":0.0016600580000000001,"objects_trimmed":0,"snaptrim_duration":0.0010945530000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":384,"num_read_kb":4092,"num_write":235,"num_write_kb":5715,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":1989443,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.11","version":"149'707","reported_seq":2175,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:42.781526+0000","last_change":"2024-04-24T15:34:42.781482+0000","last_active":"2024-04-24T15:34:42.781526+0000","last_peered":"2024-04-24T15:34:42.781526+0000","last_clean":"2024-04-24T15:34:42.781526+0000","last_became_active":"2024-04-24T15:33:57.644309+0000","last_became_peered":"2024-04-24T15:33:57.644309+0000","last_unstale":"2024-04-24T15:34:42.781526+0000","last_undegraded":"2024-04-24T15:34:42.781526+0000","last_fullsized":"2024-04-24T15:34:42.781526+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"149'707","last_scrub_stamp":"2024-04-24T15:34:42.781389+0000","last_deep_scrub":"149'707","last_deep_scrub_stamp":"2024-04-24T15:34:42.781389+0000","last_clean_scrub_stamp":"2024-04-24T15:34:42.781389+0000","objects_scrubbed":2,"log_size":707,"log_dups_size":0,"ondisk_log_size":707,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:56:03.138605+0000","scrub_duration":0.005289134,"objects_trimmed":0,"snaptrim_duration":0.0012982219999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":2615,"num_read_kb":21488,"num_write":891,"num_write_kb":14014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":5651177,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"149'332","reported_seq":5240,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:46.291941+0000","last_change":"2024-04-24T15:34:46.291914+0000","last_active":"2024-04-24T15:34:46.291941+0000","last_peered":"2024-04-24T15:34:46.291941+0000","last_clean":"2024-04-24T15:34:46.291941+0000","last_became_active":"2024-04-24T15:34:15.653205+0000","last_became_peered":"2024-04-24T15:34:15.653205+0000","last_unstale":"2024-04-24T15:34:46.291941+0000","last_undegraded":"2024-04-24T15:34:46.291941+0000","last_fullsized":"2024-04-24T15:34:46.291941+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"149'332","last_scrub_stamp":"2024-04-24T15:34:46.291812+0000","last_deep_scrub":"149'332","last_deep_scrub_stamp":"2024-04-24T15:34:46.291812+0000","last_clean_scrub_stamp":"2024-04-24T15:34:46.291812+0000","objects_scrubbed":1,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:01:55.256353+0000","scrub_duration":0.003426279,"objects_trimmed":0,"snaptrim_duration":0.0013576079999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":4982,"num_read_kb":42072,"num_write":333,"num_write_kb":14625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":3758016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"149'242","reported_seq":2015,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:55.668018+0000","last_change":"2024-04-24T15:34:55.667990+0000","last_active":"2024-04-24T15:34:55.668018+0000","last_peered":"2024-04-24T15:34:55.668018+0000","last_clean":"2024-04-24T15:34:55.668018+0000","last_became_active":"2024-04-24T15:34:23.823962+0000","last_became_peered":"2024-04-24T15:34:23.823962+0000","last_unstale":"2024-04-24T15:34:55.668018+0000","last_undegraded":"2024-04-24T15:34:55.668018+0000","last_fullsized":"2024-04-24T15:34:55.668018+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"149'242","last_scrub_stamp":"2024-04-24T15:34:55.667892+0000","last_deep_scrub":"149'242","last_deep_scrub_stamp":"2024-04-24T15:34:55.667892+0000","last_clean_scrub_stamp":"2024-04-24T15:34:55.667892+0000","objects_scrubbed":1,"log_size":242,"log_dups_size":0,"ondisk_log_size":242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:42:05.504651+0000","scrub_duration":0.0028992699999999998,"objects_trimmed":0,"snaptrim_duration":0.000635438,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":1312,"num_read_kb":18878,"num_write":253,"num_write_kb":9844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":108,"num_bytes_recovered":2694373,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"149'451","reported_seq":6607,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:45.807425+0000","last_change":"2024-04-24T15:34:45.807385+0000","last_active":"2024-04-24T15:34:45.807425+0000","last_peered":"2024-04-24T15:34:45.807425+0000","last_clean":"2024-04-24T15:34:45.807425+0000","last_became_active":"2024-04-24T15:34:16.659071+0000","last_became_peered":"2024-04-24T15:34:16.659071+0000","last_unstale":"2024-04-24T15:34:45.807425+0000","last_undegraded":"2024-04-24T15:34:45.807425+0000","last_fullsized":"2024-04-24T15:34:45.807425+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"149'451","last_scrub_stamp":"2024-04-24T15:34:45.807282+0000","last_deep_scrub":"149'451","last_deep_scrub_stamp":"2024-04-24T15:34:45.807282+0000","last_clean_scrub_stamp":"2024-04-24T15:34:45.807282+0000","objects_scrubbed":1,"log_size":451,"log_dups_size":0,"ondisk_log_size":451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:49:42.830953+0000","scrub_duration":0.0030214170000000002,"objects_trimmed":0,"snaptrim_duration":0.0013356920000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":7366,"num_read_kb":44142,"num_write":407,"num_write_kb":9850,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":233,"num_bytes_recovered":4458876,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"130'312","reported_seq":1586,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:50.936162+0000","last_change":"2024-04-24T15:34:50.936113+0000","last_active":"2024-04-24T15:34:50.936162+0000","last_peered":"2024-04-24T15:34:50.936162+0000","last_clean":"2024-04-24T15:34:50.936162+0000","last_became_active":"2024-04-24T15:34:16.655084+0000","last_became_peered":"2024-04-24T15:34:16.655084+0000","last_unstale":"2024-04-24T15:34:50.936162+0000","last_undegraded":"2024-04-24T15:34:50.936162+0000","last_fullsized":"2024-04-24T15:34:50.936162+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"130'312","last_scrub_stamp":"2024-04-24T15:34:50.936005+0000","last_deep_scrub":"130'312","last_deep_scrub_stamp":"2024-04-24T15:34:50.936005+0000","last_clean_scrub_stamp":"2024-04-24T15:34:50.936005+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:30.062675+0000","scrub_duration":0.0015637629999999999,"objects_trimmed":0,"snaptrim_duration":0.003384743,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":837,"num_read_kb":16692,"num_write":319,"num_write_kb":15488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":277,"num_bytes_recovered":16228001,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.16","version":"149'377","reported_seq":2467,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:48.393753+0000","last_change":"2024-04-24T15:34:48.393706+0000","last_active":"2024-04-24T15:34:48.393753+0000","last_peered":"2024-04-24T15:34:48.393753+0000","last_clean":"2024-04-24T15:34:48.393753+0000","last_became_active":"2024-04-24T15:34:15.668678+0000","last_became_peered":"2024-04-24T15:34:15.668678+0000","last_unstale":"2024-04-24T15:34:48.393753+0000","last_undegraded":"2024-04-24T15:34:48.393753+0000","last_fullsized":"2024-04-24T15:34:48.393753+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"149'377","last_scrub_stamp":"2024-04-24T15:34:48.393610+0000","last_deep_scrub":"149'377","last_deep_scrub_stamp":"2024-04-24T15:34:48.393610+0000","last_clean_scrub_stamp":"2024-04-24T15:34:48.393610+0000","objects_scrubbed":2,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:52:36.292086+0000","scrub_duration":0.003892805,"objects_trimmed":0,"snaptrim_duration":0.0018886770000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":3004,"num_read_kb":17777,"num_write":338,"num_write_kb":8036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":48,"num_read":114807,"num_read_kb":727542,"num_write":20290,"num_write_kb":356328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4413,"num_bytes_recovered":105175530,"num_keys_recovered":235,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15242,"internal_metadata":0},"log_size":17297,"ondisk_log_size":17297,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":83,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":83,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":8,"acting":8,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":16196,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574888,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299140,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072700,"statfs":{"total":96636764160,"available":96330444800,"internally_reserved":0,"allocated":1830912,"data_stored":1500719,"data_compressed":5677,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.46000000000000002,"15min":0.46200000000000002},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.60099999999999998,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.54200000000000004,"15min":0.55600000000000005},"min":{"1min":0.33900000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.67200000000000004,"5min":2.4710000000000001,"15min":2.4710000000000001},"last":0.44600000000000001}]},{"osd":1,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.57399999999999995,"15min":0.59299999999999997},"min":{"1min":0.28100000000000003,"5min":0.248,"15min":0.248},"max":{"1min":0.63600000000000001,"5min":2.4729999999999999,"15min":2.4729999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.51500000000000001,"15min":0.52400000000000002},"min":{"1min":0.29999999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.56899999999999995,"5min":2.4020000000000001,"15min":2.4020000000000001},"last":0.55000000000000004}]},{"osd":2,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.45700000000000002,"15min":0.45800000000000002},"min":{"1min":0.35499999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.59699999999999998,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.503,"5min":0.54800000000000004,"15min":0.55600000000000005},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.82199999999999995,"5min":2.363,"15min":2.363},"last":0.67000000000000004}]},{"osd":3,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.52200000000000002,"15min":0.51800000000000002},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.53800000000000003,"15min":0.53200000000000003},"min":{"1min":0.38600000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.56899999999999995}]},{"osd":4,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.41399999999999998,"15min":0.41999999999999998},"min":{"1min":0.26100000000000001,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.66100000000000003,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.498},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.42299999999999999,"15min":0.42499999999999999},"min":{"1min":0.30599999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.73499999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.52800000000000002}]},{"osd":5,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.45100000000000001,"15min":0.45500000000000002},"min":{"1min":0.307,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.51600000000000001,"15min":0.52000000000000002},"min":{"1min":0.36399999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.80500000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.61399999999999999}]},{"osd":6,"last update":"Wed Apr 24 15:34:43 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.626}]}]},{"osd":6,"up_from":104,"seq":446676598801,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":305884,"kb_used_data":1748,"kb_used_omap":43,"kb_used_meta":304084,"kb_avail":94065956,"statfs":{"total":96636764160,"available":96323538944,"internally_reserved":0,"allocated":1789952,"data_stored":910351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":44735,"internal_metadata":311382337},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 15:34:43 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.749}]},{"osd":1,"last update":"Wed Apr 24 15:34:43 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.499},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.8640000000000001,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":0.60899999999999999}]},{"osd":2,"last update":"Wed Apr 24 15:34:43 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.623}]},{"osd":3,"last update":"Wed Apr 24 15:34:43 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":2.6389999999999998,"5min":2.6389999999999998,"15min":2.6389999999999998},"last":0.57299999999999995}]},{"osd":4,"last update":"Wed Apr 24 15:34:43 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.35799999999999998}]},{"osd":5,"last update":"Wed Apr 24 15:34:43 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.44400000000000001}]},{"osd":7,"last update":"Wed Apr 24 15:34:43 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.34200000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574887,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307288,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":305534,"kb_avail":94064552,"statfs":{"total":96636764160,"available":96322101248,"internally_reserved":0,"allocated":1785856,"data_stored":909395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312867160},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 15:34:01 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.35599999999999998,"15min":0.35799999999999998},"min":{"1min":0.20699999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.51100000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.377,"15min":0.376},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":1.353,"5min":1.375,"15min":1.375},"last":0.39100000000000001}]},{"osd":2,"last update":"Wed Apr 24 15:34:01 2024","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.33600000000000002,"15min":0.34000000000000002},"min":{"1min":0.224,"5min":0.223,"15min":0.223},"max":{"1min":0.46100000000000002,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.41499999999999998,"15min":0.42699999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.52800000000000002,"5min":1.494,"15min":1.494},"last":0.47599999999999998}]},{"osd":3,"last update":"Wed Apr 24 15:34:01 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.42799999999999999,"15min":0.433},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.97499999999999998,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.377,"15min":0.372},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.44700000000000001}]},{"osd":4,"last update":"Wed Apr 24 15:34:01 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.53500000000000003,"15min":0.54100000000000004},"min":{"1min":0.34799999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.72199999999999998,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.57799999999999996,"15min":0.58199999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.52900000000000003}]},{"osd":5,"last update":"Wed Apr 24 15:34:01 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.379,"5min":1.454,"15min":1.454},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.53800000000000003,"15min":0.54200000000000004},"min":{"1min":0.35999999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.67100000000000004,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.58999999999999997}]},{"osd":6,"last update":"Wed Apr 24 15:34:44 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.57699999999999996}]},{"osd":7,"last update":"Wed Apr 24 15:34:01 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.56899999999999995,"15min":0.57099999999999995},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.55100000000000005,"15min":0.55700000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.69599999999999995,"5min":0.999,"15min":0.999},"last":0.499}]}]},{"osd":0,"up_from":13,"seq":55834574888,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303256,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94068584,"statfs":{"total":96636764160,"available":96326230016,"internally_reserved":0,"allocated":1851392,"data_stored":1500802,"data_compressed":4842,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.30499999999999999,"5min":0.45700000000000002,"15min":0.48299999999999998},"min":{"1min":0.17899999999999999,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.47999999999999998,"5min":3.7890000000000001,"15min":3.7890000000000001},"last":0.435},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.46700000000000003,"15min":0.48899999999999999},"min":{"1min":0.224,"5min":0.191,"15min":0.191},"max":{"1min":0.59899999999999998,"5min":2.7890000000000001,"15min":2.7890000000000001},"last":0.51500000000000001}]},{"osd":2,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.53100000000000003,"15min":0.55900000000000005},"min":{"1min":0.22700000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.61399999999999999,"5min":3.9950000000000001,"15min":3.9950000000000001},"last":0.495},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.42299999999999999,"15min":0.436},"min":{"1min":0.26000000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.56799999999999995,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":0.46800000000000003}]},{"osd":3,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.48499999999999999,"15min":0.505},"min":{"1min":0.24099999999999999,"5min":0.223,"15min":0.223},"max":{"1min":0.54000000000000004,"5min":2.097,"15min":2.097},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.622,"15min":0.65700000000000003},"min":{"1min":0.30499999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.79500000000000004,"5min":3.2109999999999999,"15min":3.2109999999999999},"last":0.53400000000000003}]},{"osd":4,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.55700000000000005,"15min":0.56699999999999995},"min":{"1min":0.31,"5min":0.308,"15min":0.308},"max":{"1min":0.69099999999999995,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.65000000000000002,"15min":0.67300000000000004},"min":{"1min":0.34899999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.85599999999999998,"5min":3.282,"15min":3.282},"last":0.58599999999999997}]},{"osd":5,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.60099999999999998,"15min":0.61299999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.79100000000000004,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.64600000000000002,"15min":0.66200000000000003},"min":{"1min":0.38700000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.78000000000000003,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":0.85699999999999998}]},{"osd":6,"last update":"Wed Apr 24 15:34:43 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":8.0619999999999994,"5min":8.0619999999999994,"15min":8.0619999999999994},"last":0.83199999999999996}]},{"osd":7,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.60299999999999998,"15min":0.60899999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.075,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":0.749},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.63200000000000001,"15min":0.64400000000000002},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.876,"5min":2.206,"15min":2.206},"last":0.79400000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574888,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303192,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94068648,"statfs":{"total":96636764160,"available":96326295552,"internally_reserved":0,"allocated":1785856,"data_stored":909395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 15:34:06 2024","interfaces":[{"interface":"back","average":{"1min":0.29199999999999998,"5min":0.312,"15min":0.315},"min":{"1min":0.16200000000000001,"5min":0.16200000000000001,"15min":0.16200000000000001},"max":{"1min":0.41399999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"last":0.373},{"interface":"front","average":{"1min":0.30599999999999999,"5min":0.33700000000000002,"15min":0.34200000000000003},"min":{"1min":0.185,"5min":0.185,"15min":0.185},"max":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"last":0.34399999999999997}]},{"osd":1,"last update":"Wed Apr 24 15:34:06 2024","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.36899999999999999,"15min":0.377},"min":{"1min":0.23300000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.47399999999999998,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.46100000000000002,"15min":0.48199999999999998},"min":{"1min":0.24099999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.441,"5min":3.1110000000000002,"15min":3.1110000000000002},"last":0.441}]},{"osd":3,"last update":"Wed Apr 24 15:34:06 2024","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.40500000000000003,"15min":0.41899999999999998},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.41399999999999998,"5min":2.056,"15min":2.056},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.39900000000000002,"15min":0.40500000000000003},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.52900000000000003,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.45800000000000002}]},{"osd":4,"last update":"Wed Apr 24 15:34:06 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.51800000000000002,"15min":0.52400000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.53600000000000003,"15min":0.54500000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.70399999999999996,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.50700000000000001}]},{"osd":5,"last update":"Wed Apr 24 15:34:06 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.54300000000000004,"15min":0.54600000000000004},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.63700000000000001,"15min":0.65400000000000003},"min":{"1min":0.41399999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.80500000000000005,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":0.621}]},{"osd":6,"last update":"Wed Apr 24 15:34:46 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.56699999999999995}]},{"osd":7,"last update":"Wed Apr 24 15:34:06 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.63500000000000001,"15min":0.65700000000000003},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.63800000000000001,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.56999999999999995,"15min":0.57799999999999996},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73999999999999999,"5min":1.502,"15min":1.502},"last":0.64600000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574888,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303200,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94068640,"statfs":{"total":96636764160,"available":96326287360,"internally_reserved":0,"allocated":1794048,"data_stored":910370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 15:34:04 2024","interfaces":[{"interface":"back","average":{"1min":0.29399999999999998,"5min":0.34599999999999997,"15min":0.35499999999999998},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.39900000000000002,"5min":1.224,"15min":1.224},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.30199999999999999,"5min":0.34000000000000002,"15min":0.34599999999999997},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.41999999999999998,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.41299999999999998}]},{"osd":1,"last update":"Wed Apr 24 15:34:04 2024","interfaces":[{"interface":"back","average":{"1min":0.30299999999999999,"5min":0.33400000000000002,"15min":0.33900000000000002},"min":{"1min":0.19600000000000001,"5min":0.187,"15min":0.187},"max":{"1min":0.437,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.42199999999999999,"15min":0.434},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.71299999999999997,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.51700000000000002}]},{"osd":2,"last update":"Wed Apr 24 15:34:04 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.39400000000000002,"15min":0.40300000000000002},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.46500000000000002,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.376,"5min":0.41299999999999998,"15min":0.41899999999999998},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.53000000000000003,"5min":1.252,"15min":1.252},"last":0.56899999999999995}]},{"osd":4,"last update":"Wed Apr 24 15:34:04 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.52200000000000002,"15min":0.53400000000000003},"min":{"1min":0.28599999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.56499999999999995,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.52100000000000002,"15min":0.53300000000000003},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.60699999999999998,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.70799999999999996}]},{"osd":5,"last update":"Wed Apr 24 15:34:04 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.55600000000000005,"15min":0.56999999999999995},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.61199999999999999,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.5,"5min":0.58799999999999997,"15min":0.60199999999999998},"min":{"1min":0.38300000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.81100000000000005,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.82499999999999996}]},{"osd":6,"last update":"Wed Apr 24 15:34:45 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.86799999999999999}]},{"osd":7,"last update":"Wed Apr 24 15:34:04 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.59099999999999997,"15min":0.60499999999999998},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.66800000000000004,"5min":1.635,"15min":1.635},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.495,"5min":0.55600000000000005,"15min":0.56599999999999995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.59699999999999998,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.84299999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574889,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307292,"kb_used_data":1748,"kb_used_omap":1,"kb_used_meta":305534,"kb_avail":94064548,"statfs":{"total":96636764160,"available":96322097152,"internally_reserved":0,"allocated":1789952,"data_stored":908139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312867160},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 15:34:00 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.505,"15min":0.50900000000000001},"min":{"1min":0.249,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":1.8620000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.626,"5min":0.56599999999999995,"15min":0.55600000000000005},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":5.6239999999999997,"5min":5.6239999999999997,"15min":5.6239999999999997},"last":0.48299999999999998}]},{"osd":1,"last update":"Wed Apr 24 15:34:00 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.58699999999999997,"15min":0.57299999999999995},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":5.7149999999999999,"5min":5.7149999999999999,"15min":5.7149999999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.35499999999999998,"5min":0.31,"15min":0.31},"max":{"1min":1.913,"5min":1.913,"15min":1.913},"last":0.58999999999999997}]},{"osd":2,"last update":"Wed Apr 24 15:34:00 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.54600000000000004,"15min":0.52800000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":5.6660000000000004,"5min":5.6660000000000004,"15min":5.6660000000000004},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.53400000000000003,"15min":0.54400000000000004},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.819,"5min":1.819,"15min":1.819},"last":0.497}]},{"osd":3,"last update":"Wed Apr 24 15:34:00 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.53400000000000003,"15min":0.53600000000000003},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.9550000000000001,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.55500000000000005,"15min":0.55000000000000004},"min":{"1min":0.36799999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":2.3149999999999999,"5min":2.3149999999999999,"15min":2.3149999999999999},"last":0.47499999999999998}]},{"osd":5,"last update":"Wed Apr 24 15:34:00 2024","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.58299999999999996,"15min":0.51700000000000002},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":14.667999999999999,"5min":14.667999999999999,"15min":14.667999999999999},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.434,"5min":0.497,"15min":0.50700000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.9890000000000001,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":0.36399999999999999}]},{"osd":6,"last update":"Wed Apr 24 15:34:43 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.44900000000000001}]},{"osd":7,"last update":"Wed Apr 24 15:34:00 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.54000000000000004,"15min":0.53100000000000003},"min":{"1min":0.28599999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":5.2759999999999998,"5min":5.2759999999999998,"15min":5.2759999999999998},"last":0.41499999999999998},{"interface":"front","average":{"1min":1.0469999999999999,"5min":0.64100000000000001,"15min":0.57299999999999995},"min":{"1min":0.29399999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":14.773,"5min":14.773,"15min":14.773},"last":0.40300000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574888,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311388,"kb_used_data":1748,"kb_used_omap":1,"kb_used_meta":309630,"kb_avail":94060452,"statfs":{"total":96636764160,"available":96317902848,"internally_reserved":0,"allocated":1789952,"data_stored":910351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317061464},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.47899999999999998,"15min":0.48499999999999999},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.46800000000000003,"15min":0.47599999999999998},"min":{"1min":0.246,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.629,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.54200000000000004}]},{"osd":1,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.46300000000000002,"15min":0.47099999999999997},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.64900000000000002,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.504},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.46899999999999997,"15min":0.47099999999999997},"min":{"1min":0.317,"5min":0.313,"15min":0.313},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.65500000000000003}]},{"osd":2,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.47499999999999998,"15min":0.46600000000000003},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":2.4169999999999998,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.48599999999999999,"15min":0.49199999999999999},"min":{"1min":0.30099999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.442}]},{"osd":3,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.48899999999999999,"15min":0.495},"min":{"1min":0.308,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.70999999999999996,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":2.3370000000000002,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":0.60099999999999998}]},{"osd":4,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.44800000000000001,"15min":0.45700000000000002},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.624,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.48399999999999999,"15min":0.48599999999999999},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.57599999999999996}]},{"osd":6,"last update":"Wed Apr 24 15:34:45 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.69699999999999995}]},{"osd":7,"last update":"Wed Apr 24 15:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.52600000000000002,"15min":0.53100000000000003},"min":{"1min":0.30299999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.78600000000000003,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.496,"5min":0.53200000000000003,"15min":0.53800000000000003},"min":{"1min":0.34499999999999997,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.77100000000000002,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.67200000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4842,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":5677,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":5677,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15242,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T15:35:00.401 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-04-24T15:32:06.972928+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=32, tm_sec=6, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=34, tm_sec=34, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T15:35:00.402 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2024-04-24T15:32:16.022683+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=32, tm_sec=16, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=15, tm_min=34, tm_sec=34, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T15:35:00.402 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-24T15:35:20.403 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T15:35:20.732 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-24T15:35:20.733 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-04-24T15:35:20.743 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":243,"stamp":"2024-04-24T15:35:19.590261+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590470,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":51,"num_read":114923,"num_read_kb":727642,"num_write":20450,"num_write_kb":358745,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4420,"num_bytes_recovered":106356266,"num_keys_recovered":235,"num_objects_omap":3,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":83,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17429,"ondisk_log_size":17429,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2440640,"kb_used_data":14080,"kb_used_omap":55,"kb_used_meta":2426440,"kb_avail":752534080,"statfs":{"total":773094113280,"available":770594897920,"internally_reserved":0,"allocated":14417920,"data_stored":8459522,"data_compressed":10519,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":56663,"internal_metadata":2484675241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002674"},"pg_stats":[{"pgid":"4.1f","version":"149'1486","reported_seq":2562,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:50.673275+0000","last_change":"2024-04-24T15:34:50.673238+0000","last_active":"2024-04-24T15:34:50.673275+0000","last_peered":"2024-04-24T15:34:50.673275+0000","last_clean":"2024-04-24T15:34:50.673275+0000","last_became_active":"2024-04-24T15:34:15.650850+0000","last_became_peered":"2024-04-24T15:34:15.650850+0000","last_unstale":"2024-04-24T15:34:50.673275+0000","last_undegraded":"2024-04-24T15:34:50.673275+0000","last_fullsized":"2024-04-24T15:34:50.673275+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"149'1486","last_scrub_stamp":"2024-04-24T15:34:50.673098+0000","last_deep_scrub":"149'1486","last_deep_scrub_stamp":"2024-04-24T15:34:50.673098+0000","last_clean_scrub_stamp":"2024-04-24T15:34:50.673098+0000","objects_scrubbed":1,"log_size":1486,"log_dups_size":0,"ondisk_log_size":1486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:16:54.402785+0000","scrub_duration":0.003073887,"objects_trimmed":0,"snaptrim_duration":0.001749947,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":8143,"num_read_kb":14722,"num_write":2575,"num_write_kb":16077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":5952278,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"149'508","reported_seq":6430,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:47.349997+0000","last_change":"2024-04-24T15:34:47.349948+0000","last_active":"2024-04-24T15:34:47.349997+0000","last_peered":"2024-04-24T15:34:47.349997+0000","last_clean":"2024-04-24T15:34:47.349997+0000","last_became_active":"2024-04-24T15:34:15.668657+0000","last_became_peered":"2024-04-24T15:34:15.668657+0000","last_unstale":"2024-04-24T15:34:47.349997+0000","last_undegraded":"2024-04-24T15:34:47.349997+0000","last_fullsized":"2024-04-24T15:34:47.349997+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"149'508","last_scrub_stamp":"2024-04-24T15:34:47.349854+0000","last_deep_scrub":"149'508","last_deep_scrub_stamp":"2024-04-24T15:34:47.349854+0000","last_clean_scrub_stamp":"2024-04-24T15:34:47.349854+0000","objects_scrubbed":1,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:31:51.178405+0000","scrub_duration":0.0041498170000000001,"objects_trimmed":0,"snaptrim_duration":0.00029386599999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":7278,"num_read_kb":39113,"num_write":560,"num_write_kb":6592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":4495932,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"149'531","reported_seq":3758,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:46.824854+0000","last_change":"2024-04-24T15:34:46.824821+0000","last_active":"2024-04-24T15:34:46.824854+0000","last_peered":"2024-04-24T15:34:46.824854+0000","last_clean":"2024-04-24T15:34:46.824854+0000","last_became_active":"2024-04-24T15:34:16.659202+0000","last_became_peered":"2024-04-24T15:34:16.659202+0000","last_unstale":"2024-04-24T15:34:46.824854+0000","last_undegraded":"2024-04-24T15:34:46.824854+0000","last_fullsized":"2024-04-24T15:34:46.824854+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"149'531","last_scrub_stamp":"2024-04-24T15:34:46.824709+0000","last_deep_scrub":"149'531","last_deep_scrub_stamp":"2024-04-24T15:34:46.824709+0000","last_clean_scrub_stamp":"2024-04-24T15:34:46.824709+0000","objects_scrubbed":2,"log_size":531,"log_dups_size":0,"ondisk_log_size":531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:30:24.464225+0000","scrub_duration":0.005707068,"objects_trimmed":0,"snaptrim_duration":0.0015488839999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":4427,"num_read_kb":51896,"num_write":603,"num_write_kb":18759,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":580671,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"149'528","reported_seq":2713,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:40.527438+0000","last_change":"2024-04-24T15:34:40.527414+0000","last_active":"2024-04-24T15:34:40.527438+0000","last_peered":"2024-04-24T15:34:40.527438+0000","last_clean":"2024-04-24T15:34:40.527438+0000","last_became_active":"2024-04-24T15:34:19.806783+0000","last_became_peered":"2024-04-24T15:34:19.806783+0000","last_unstale":"2024-04-24T15:34:40.527438+0000","last_undegraded":"2024-04-24T15:34:40.527438+0000","last_fullsized":"2024-04-24T15:34:40.527438+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"149'528","last_scrub_stamp":"2024-04-24T15:34:40.527334+0000","last_deep_scrub":"149'528","last_deep_scrub_stamp":"2024-04-24T15:34:40.527334+0000","last_clean_scrub_stamp":"2024-04-24T15:34:40.527334+0000","objects_scrubbed":2,"log_size":528,"log_dups_size":0,"ondisk_log_size":528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:53:26.967300+0000","scrub_duration":0.002933373,"objects_trimmed":0,"snaptrim_duration":0.001413493,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":3716,"num_read_kb":10078,"num_write":547,"num_write_kb":2888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"149'386","reported_seq":2371,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:42.476749+0000","last_change":"2024-04-24T15:34:42.476712+0000","last_active":"2024-04-24T15:34:42.476749+0000","last_peered":"2024-04-24T15:34:42.476749+0000","last_clean":"2024-04-24T15:34:42.476749+0000","last_became_active":"2024-04-24T15:34:23.825472+0000","last_became_peered":"2024-04-24T15:34:23.825472+0000","last_unstale":"2024-04-24T15:34:42.476749+0000","last_undegraded":"2024-04-24T15:34:42.476749+0000","last_fullsized":"2024-04-24T15:34:42.476749+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"149'386","last_scrub_stamp":"2024-04-24T15:34:42.476583+0000","last_deep_scrub":"149'386","last_deep_scrub_stamp":"2024-04-24T15:34:42.476583+0000","last_clean_scrub_stamp":"2024-04-24T15:34:42.476583+0000","objects_scrubbed":1,"log_size":386,"log_dups_size":0,"ondisk_log_size":386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:14:33.494375+0000","scrub_duration":0.0039126279999999996,"objects_trimmed":0,"snaptrim_duration":0.0011107479999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":2335,"num_read_kb":25405,"num_write":329,"num_write_kb":6118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":79,"num_bytes_recovered":2143880,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"149'451","reported_seq":6623,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:41.444924+0000","last_change":"2024-04-24T15:34:41.444877+0000","last_active":"2024-04-24T15:34:41.444924+0000","last_peered":"2024-04-24T15:34:41.444924+0000","last_clean":"2024-04-24T15:34:41.444924+0000","last_became_active":"2024-04-24T15:34:23.824861+0000","last_became_peered":"2024-04-24T15:34:23.824861+0000","last_unstale":"2024-04-24T15:34:41.444924+0000","last_undegraded":"2024-04-24T15:34:41.444924+0000","last_fullsized":"2024-04-24T15:34:41.444924+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"149'451","last_scrub_stamp":"2024-04-24T15:34:41.444781+0000","last_deep_scrub":"149'451","last_deep_scrub_stamp":"2024-04-24T15:34:41.444781+0000","last_clean_scrub_stamp":"2024-04-24T15:34:41.444781+0000","objects_scrubbed":2,"log_size":451,"log_dups_size":0,"ondisk_log_size":451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:56:48.021882+0000","scrub_duration":0.0047438239999999998,"objects_trimmed":0,"snaptrim_duration":0.00097376600000000004,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":6368,"num_read_kb":19805,"num_write":475,"num_write_kb":10193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":89,"num_bytes_recovered":1137673,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"149'538","reported_seq":2692,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:54.856624+0000","last_change":"2024-04-24T15:34:54.856577+0000","last_active":"2024-04-24T15:34:54.856624+0000","last_peered":"2024-04-24T15:34:54.856624+0000","last_clean":"2024-04-24T15:34:54.856624+0000","last_became_active":"2024-04-24T15:32:17.047842+0000","last_became_peered":"2024-04-24T15:32:17.047842+0000","last_unstale":"2024-04-24T15:34:54.856624+0000","last_undegraded":"2024-04-24T15:34:54.856624+0000","last_fullsized":"2024-04-24T15:34:54.856624+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"149'538","last_scrub_stamp":"2024-04-24T15:34:54.856475+0000","last_deep_scrub":"149'538","last_deep_scrub_stamp":"2024-04-24T15:34:54.856475+0000","last_clean_scrub_stamp":"2024-04-24T15:34:54.856475+0000","objects_scrubbed":1,"log_size":538,"log_dups_size":0,"ondisk_log_size":538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:47:25.664161+0000","scrub_duration":0.0034313270000000001,"objects_trimmed":0,"snaptrim_duration":0.00081826100000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":4119,"num_read_kb":22694,"num_write":527,"num_write_kb":11193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"149'449","reported_seq":1984,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:40.467325+0000","last_change":"2024-04-24T15:34:40.467271+0000","last_active":"2024-04-24T15:34:40.467325+0000","last_peered":"2024-04-24T15:34:40.467325+0000","last_clean":"2024-04-24T15:34:40.467325+0000","last_became_active":"2024-04-24T15:33:42.641679+0000","last_became_peered":"2024-04-24T15:33:42.641679+0000","last_unstale":"2024-04-24T15:34:40.467325+0000","last_undegraded":"2024-04-24T15:34:40.467325+0000","last_fullsized":"2024-04-24T15:34:40.467325+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"149'449","last_scrub_stamp":"2024-04-24T15:34:40.467105+0000","last_deep_scrub":"149'449","last_deep_scrub_stamp":"2024-04-24T15:34:40.467105+0000","last_clean_scrub_stamp":"2024-04-24T15:34:40.467105+0000","objects_scrubbed":5,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:24:09.528219+0000","scrub_duration":0.011387028,"objects_trimmed":0,"snaptrim_duration":0.00077716999999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":1887,"num_read_kb":15406,"num_write":547,"num_write_kb":8155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":4099152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"149'525","reported_seq":2460,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:47.871962+0000","last_change":"2024-04-24T15:34:47.871933+0000","last_active":"2024-04-24T15:34:47.871962+0000","last_peered":"2024-04-24T15:34:47.871962+0000","last_clean":"2024-04-24T15:34:47.871962+0000","last_became_active":"2024-04-24T15:33:56.699629+0000","last_became_peered":"2024-04-24T15:33:56.699629+0000","last_unstale":"2024-04-24T15:34:47.871962+0000","last_undegraded":"2024-04-24T15:34:47.871962+0000","last_fullsized":"2024-04-24T15:34:47.871962+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"149'525","last_scrub_stamp":"2024-04-24T15:34:47.871840+0000","last_deep_scrub":"149'525","last_deep_scrub_stamp":"2024-04-24T15:34:47.871840+0000","last_clean_scrub_stamp":"2024-04-24T15:34:47.871840+0000","objects_scrubbed":2,"log_size":525,"log_dups_size":0,"ondisk_log_size":525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:09:43.201751+0000","scrub_duration":0.0066328519999999998,"objects_trimmed":0,"snaptrim_duration":0.00212766,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":3353,"num_read_kb":16260,"num_write":594,"num_write_kb":12753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":352256,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"149'318","reported_seq":2590,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:43.501751+0000","last_change":"2024-04-24T15:34:43.501722+0000","last_active":"2024-04-24T15:34:43.501751+0000","last_peered":"2024-04-24T15:34:43.501751+0000","last_clean":"2024-04-24T15:34:43.501751+0000","last_became_active":"2024-04-24T15:34:23.827917+0000","last_became_peered":"2024-04-24T15:34:23.827917+0000","last_unstale":"2024-04-24T15:34:43.501751+0000","last_undegraded":"2024-04-24T15:34:43.501751+0000","last_fullsized":"2024-04-24T15:34:43.501751+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"149'318","last_scrub_stamp":"2024-04-24T15:34:43.501627+0000","last_deep_scrub":"149'318","last_deep_scrub_stamp":"2024-04-24T15:34:43.501627+0000","last_clean_scrub_stamp":"2024-04-24T15:34:43.501627+0000","objects_scrubbed":2,"log_size":318,"log_dups_size":0,"ondisk_log_size":318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:32:18.592700+0000","scrub_duration":0.0035660290000000001,"objects_trimmed":0,"snaptrim_duration":0.00065534800000000004,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":1633,"num_read_kb":26736,"num_write":303,"num_write_kb":13822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":9304722,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":132,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:41.718271+0000","last_change":"2024-04-24T15:34:41.718227+0000","last_active":"2024-04-24T15:34:41.718271+0000","last_peered":"2024-04-24T15:34:41.718271+0000","last_clean":"2024-04-24T15:34:41.718271+0000","last_became_active":"2024-04-24T15:33:53.643941+0000","last_became_peered":"2024-04-24T15:33:53.643941+0000","last_unstale":"2024-04-24T15:34:41.718271+0000","last_undegraded":"2024-04-24T15:34:41.718271+0000","last_fullsized":"2024-04-24T15:34:41.718271+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:34:41.718086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:34:41.718086+0000","last_clean_scrub_stamp":"2024-04-24T15:34:41.718086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:11:48.998969+0000","scrub_duration":0.0017191350000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"149'867","reported_seq":3025,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:37.530105+0000","last_change":"2024-04-24T15:34:37.530071+0000","last_active":"2024-04-24T15:34:37.530105+0000","last_peered":"2024-04-24T15:34:37.530105+0000","last_clean":"2024-04-24T15:34:37.530105+0000","last_became_active":"2024-04-24T15:34:19.809619+0000","last_became_peered":"2024-04-24T15:34:19.809619+0000","last_unstale":"2024-04-24T15:34:37.530105+0000","last_undegraded":"2024-04-24T15:34:37.530105+0000","last_fullsized":"2024-04-24T15:34:37.530105+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"149'867","last_scrub_stamp":"2024-04-24T15:34:37.529969+0000","last_deep_scrub":"149'867","last_deep_scrub_stamp":"2024-04-24T15:34:37.529969+0000","last_clean_scrub_stamp":"2024-04-24T15:34:37.529969+0000","objects_scrubbed":5,"log_size":867,"log_dups_size":0,"ondisk_log_size":867,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:05:19.302172+0000","scrub_duration":0.0058171969999999996,"objects_trimmed":2,"snaptrim_duration":0.00033353700000000002,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":4689,"num_read_kb":23870,"num_write":1044,"num_write_kb":13092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":176,"num_bytes_recovered":5601476,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":35,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:44.274028+0000","last_change":"2024-04-24T15:34:44.274001+0000","last_active":"2024-04-24T15:34:44.274028+0000","last_peered":"2024-04-24T15:34:44.274028+0000","last_clean":"2024-04-24T15:34:44.274028+0000","last_became_active":"2024-04-24T15:34:12.652298+0000","last_became_peered":"2024-04-24T15:34:12.652298+0000","last_unstale":"2024-04-24T15:34:44.274028+0000","last_undegraded":"2024-04-24T15:34:44.274028+0000","last_fullsized":"2024-04-24T15:34:44.274028+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:34:44.273900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:34:44.273900+0000","last_clean_scrub_stamp":"2024-04-24T15:34:44.273900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:24:59.154244+0000","scrub_duration":0.0019815879999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":52,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:40.791816+0000","last_change":"2024-04-24T15:34:40.791767+0000","last_active":"2024-04-24T15:34:40.791816+0000","last_peered":"2024-04-24T15:34:40.791816+0000","last_clean":"2024-04-24T15:34:40.791816+0000","last_became_active":"2024-04-24T15:33:53.645696+0000","last_became_peered":"2024-04-24T15:33:53.645696+0000","last_unstale":"2024-04-24T15:34:40.791816+0000","last_undegraded":"2024-04-24T15:34:40.791816+0000","last_fullsized":"2024-04-24T15:34:40.791816+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:34:40.791674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:34:40.791674+0000","last_clean_scrub_stamp":"2024-04-24T15:34:40.791674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:10:48.332079+0000","scrub_duration":0.0017782620000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"149'585","reported_seq":5184,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:40.349759+0000","last_change":"2024-04-24T15:34:40.349734+0000","last_active":"2024-04-24T15:34:40.349759+0000","last_peered":"2024-04-24T15:34:40.349759+0000","last_clean":"2024-04-24T15:34:40.349759+0000","last_became_active":"2024-04-24T15:34:17.654759+0000","last_became_peered":"2024-04-24T15:34:17.654759+0000","last_unstale":"2024-04-24T15:34:40.349759+0000","last_undegraded":"2024-04-24T15:34:40.349759+0000","last_fullsized":"2024-04-24T15:34:40.349759+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"149'585","last_scrub_stamp":"2024-04-24T15:34:40.349659+0000","last_deep_scrub":"149'585","last_deep_scrub_stamp":"2024-04-24T15:34:40.349659+0000","last_clean_scrub_stamp":"2024-04-24T15:34:40.349659+0000","objects_scrubbed":1,"log_size":585,"log_dups_size":0,"ondisk_log_size":585,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:47:17.470137+0000","scrub_duration":0.0034538360000000001,"objects_trimmed":0,"snaptrim_duration":0.00061779100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3592,"num_read_kb":14899,"num_write":598,"num_write_kb":11359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":23598,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"149'3","reported_seq":93,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:36.366452+0000","last_change":"2024-04-24T15:34:36.366408+0000","last_active":"2024-04-24T15:34:36.366452+0000","last_peered":"2024-04-24T15:34:36.366452+0000","last_clean":"2024-04-24T15:34:36.366452+0000","last_became_active":"2024-04-24T15:33:47.662685+0000","last_became_peered":"2024-04-24T15:33:47.662685+0000","last_unstale":"2024-04-24T15:34:36.366452+0000","last_undegraded":"2024-04-24T15:34:36.366452+0000","last_fullsized":"2024-04-24T15:34:36.366452+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"149'2","last_scrub_stamp":"2024-04-24T15:34:36.366339+0000","last_deep_scrub":"149'2","last_deep_scrub_stamp":"2024-04-24T15:34:36.366339+0000","last_clean_scrub_stamp":"2024-04-24T15:34:36.366339+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:57:31.467325+0000","scrub_duration":0.0030479940000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":83,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":61,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:41.805246+0000","last_change":"2024-04-24T15:34:41.805209+0000","last_active":"2024-04-24T15:34:41.805246+0000","last_peered":"2024-04-24T15:34:41.805246+0000","last_clean":"2024-04-24T15:34:41.805246+0000","last_became_active":"2024-04-24T15:34:12.652805+0000","last_became_peered":"2024-04-24T15:34:12.652805+0000","last_unstale":"2024-04-24T15:34:41.805246+0000","last_undegraded":"2024-04-24T15:34:41.805246+0000","last_fullsized":"2024-04-24T15:34:41.805246+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:34:41.805118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:34:41.805118+0000","last_clean_scrub_stamp":"2024-04-24T15:34:41.805118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:28:34.952739+0000","scrub_duration":0.001661591,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"149'616","reported_seq":2381,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:39.346173+0000","last_change":"2024-04-24T15:34:39.346151+0000","last_active":"2024-04-24T15:34:39.346173+0000","last_peered":"2024-04-24T15:34:39.346173+0000","last_clean":"2024-04-24T15:34:39.346173+0000","last_became_active":"2024-04-24T15:34:16.655390+0000","last_became_peered":"2024-04-24T15:34:16.655390+0000","last_unstale":"2024-04-24T15:34:39.346173+0000","last_undegraded":"2024-04-24T15:34:39.346173+0000","last_fullsized":"2024-04-24T15:34:39.346173+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"149'616","last_scrub_stamp":"2024-04-24T15:34:39.346081+0000","last_deep_scrub":"149'616","last_deep_scrub_stamp":"2024-04-24T15:34:39.346081+0000","last_clean_scrub_stamp":"2024-04-24T15:34:39.346081+0000","objects_scrubbed":1,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:37:29.323750+0000","scrub_duration":0.0025279489999999998,"objects_trimmed":0,"snaptrim_duration":0.00042917399999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":3015,"num_read_kb":19823,"num_write":696,"num_write_kb":5112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"149'4","reported_seq":195,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:38.813383+0000","last_change":"2024-04-24T15:34:38.813338+0000","last_active":"2024-04-24T15:34:38.813383+0000","last_peered":"2024-04-24T15:34:38.813383+0000","last_clean":"2024-04-24T15:34:38.813383+0000","last_became_active":"2024-04-24T15:34:13.662363+0000","last_became_peered":"2024-04-24T15:34:13.662363+0000","last_unstale":"2024-04-24T15:34:38.813383+0000","last_undegraded":"2024-04-24T15:34:38.813383+0000","last_fullsized":"2024-04-24T15:34:38.813383+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"149'4","last_scrub_stamp":"2024-04-24T15:34:38.813254+0000","last_deep_scrub":"149'4","last_deep_scrub_stamp":"2024-04-24T15:34:38.813254+0000","last_clean_scrub_stamp":"2024-04-24T15:34:38.813254+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:56:01.218000+0000","scrub_duration":0.006372244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"149'119","reported_seq":264,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:43.742101+0000","last_change":"2024-04-24T15:34:43.742050+0000","last_active":"2024-04-24T15:34:43.742101+0000","last_peered":"2024-04-24T15:34:43.742101+0000","last_clean":"2024-04-24T15:34:43.742101+0000","last_became_active":"2024-04-24T15:33:57.644312+0000","last_became_peered":"2024-04-24T15:33:57.644312+0000","last_unstale":"2024-04-24T15:34:43.742101+0000","last_undegraded":"2024-04-24T15:34:43.742101+0000","last_fullsized":"2024-04-24T15:34:43.742101+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"149'119","last_scrub_stamp":"2024-04-24T15:34:43.741949+0000","last_deep_scrub":"149'119","last_deep_scrub_stamp":"2024-04-24T15:34:43.741949+0000","last_clean_scrub_stamp":"2024-04-24T15:34:43.741949+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:37:17.783904+0000","scrub_duration":0.0054802949999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":34,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:47.897301+0000","last_change":"2024-04-24T15:34:47.897272+0000","last_active":"2024-04-24T15:34:47.897301+0000","last_peered":"2024-04-24T15:34:47.897301+0000","last_clean":"2024-04-24T15:34:47.897301+0000","last_became_active":"2024-04-24T15:34:13.650174+0000","last_became_peered":"2024-04-24T15:34:13.650174+0000","last_unstale":"2024-04-24T15:34:47.897301+0000","last_undegraded":"2024-04-24T15:34:47.897301+0000","last_fullsized":"2024-04-24T15:34:47.897301+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:34:47.897162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:34:47.897162+0000","last_clean_scrub_stamp":"2024-04-24T15:34:47.897162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:20:29.591212+0000","scrub_duration":0.001724338,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"149'592","reported_seq":2468,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:48.615780+0000","last_change":"2024-04-24T15:34:48.615749+0000","last_active":"2024-04-24T15:34:48.615780+0000","last_peered":"2024-04-24T15:34:48.615780+0000","last_clean":"2024-04-24T15:34:48.615780+0000","last_became_active":"2024-04-24T15:33:45.658207+0000","last_became_peered":"2024-04-24T15:33:45.658207+0000","last_unstale":"2024-04-24T15:34:48.615780+0000","last_undegraded":"2024-04-24T15:34:48.615780+0000","last_fullsized":"2024-04-24T15:34:48.615780+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"149'592","last_scrub_stamp":"2024-04-24T15:34:48.615646+0000","last_deep_scrub":"149'592","last_deep_scrub_stamp":"2024-04-24T15:34:48.615646+0000","last_clean_scrub_stamp":"2024-04-24T15:34:48.615646+0000","objects_scrubbed":1,"log_size":592,"log_dups_size":0,"ondisk_log_size":592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:36:52.015734+0000","scrub_duration":0.0035351150000000001,"objects_trimmed":0,"snaptrim_duration":0.0010111199999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":3352,"num_read_kb":26205,"num_write":791,"num_write_kb":11816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":8821091,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":35,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:38.329944+0000","last_change":"2024-04-24T15:34:38.329904+0000","last_active":"2024-04-24T15:34:38.329944+0000","last_peered":"2024-04-24T15:34:38.329944+0000","last_clean":"2024-04-24T15:34:38.329944+0000","last_became_active":"2024-04-24T15:34:12.655508+0000","last_became_peered":"2024-04-24T15:34:12.655508+0000","last_unstale":"2024-04-24T15:34:38.329944+0000","last_undegraded":"2024-04-24T15:34:38.329944+0000","last_fullsized":"2024-04-24T15:34:38.329944+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:34:38.329808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:34:38.329808+0000","last_clean_scrub_stamp":"2024-04-24T15:34:38.329808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:03:40.453877+0000","scrub_duration":0.001431858,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":70,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:38.557893+0000","last_change":"2024-04-24T15:34:38.557860+0000","last_active":"2024-04-24T15:34:38.557893+0000","last_peered":"2024-04-24T15:34:38.557893+0000","last_clean":"2024-04-24T15:34:38.557893+0000","last_became_active":"2024-04-24T15:34:19.838058+0000","last_became_peered":"2024-04-24T15:34:19.838058+0000","last_unstale":"2024-04-24T15:34:38.557893+0000","last_undegraded":"2024-04-24T15:34:38.557893+0000","last_fullsized":"2024-04-24T15:34:38.557893+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:34:38.557748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:34:38.557748+0000","last_clean_scrub_stamp":"2024-04-24T15:34:38.557748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:53:13.458240+0000","scrub_duration":0.001481909,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"149'433","reported_seq":1245,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:49.273658+0000","last_change":"2024-04-24T15:34:49.273632+0000","last_active":"2024-04-24T15:34:49.273658+0000","last_peered":"2024-04-24T15:34:49.273658+0000","last_clean":"2024-04-24T15:34:49.273658+0000","last_became_active":"2024-04-24T15:34:15.658368+0000","last_became_peered":"2024-04-24T15:34:15.658368+0000","last_unstale":"2024-04-24T15:34:49.273658+0000","last_undegraded":"2024-04-24T15:34:49.273658+0000","last_fullsized":"2024-04-24T15:34:49.273658+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"149'433","last_scrub_stamp":"2024-04-24T15:34:49.273531+0000","last_deep_scrub":"149'433","last_deep_scrub_stamp":"2024-04-24T15:34:49.273531+0000","last_clean_scrub_stamp":"2024-04-24T15:34:49.273531+0000","objects_scrubbed":3,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:56:17.249810+0000","scrub_duration":0.0041417190000000003,"objects_trimmed":0,"snaptrim_duration":0.0015045289999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":1970,"num_read_kb":12710,"num_write":463,"num_write_kb":8069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":208,"num_bytes_recovered":107133,"num_keys_recovered":27,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":104,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:35:06.534405+0000","last_change":"2024-04-24T15:35:06.534375+0000","last_active":"2024-04-24T15:35:06.534405+0000","last_peered":"2024-04-24T15:35:06.534405+0000","last_clean":"2024-04-24T15:35:06.534405+0000","last_became_active":"2024-04-24T15:34:24.428510+0000","last_became_peered":"2024-04-24T15:34:24.428510+0000","last_unstale":"2024-04-24T15:35:06.534405+0000","last_undegraded":"2024-04-24T15:35:06.534405+0000","last_fullsized":"2024-04-24T15:35:06.534405+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:35:06.534260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:35:06.534260+0000","last_clean_scrub_stamp":"2024-04-24T15:35:06.534260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:01:22.878099+0000","scrub_duration":0.002578178,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":34,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:49.923775+0000","last_change":"2024-04-24T15:34:49.923738+0000","last_active":"2024-04-24T15:34:49.923775+0000","last_peered":"2024-04-24T15:34:49.923775+0000","last_clean":"2024-04-24T15:34:49.923775+0000","last_became_active":"2024-04-24T15:34:13.689685+0000","last_became_peered":"2024-04-24T15:34:13.689685+0000","last_unstale":"2024-04-24T15:34:49.923775+0000","last_undegraded":"2024-04-24T15:34:49.923775+0000","last_fullsized":"2024-04-24T15:34:49.923775+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:34:49.923635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:34:49.923635+0000","last_clean_scrub_stamp":"2024-04-24T15:34:49.923635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:53:47.338193+0000","scrub_duration":0.0018300860000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"130'457","reported_seq":2006,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:54.278887+0000","last_change":"2024-04-24T15:34:54.278853+0000","last_active":"2024-04-24T15:34:54.278887+0000","last_peered":"2024-04-24T15:34:54.278887+0000","last_clean":"2024-04-24T15:34:54.278887+0000","last_became_active":"2024-04-24T15:34:16.651906+0000","last_became_peered":"2024-04-24T15:34:16.651906+0000","last_unstale":"2024-04-24T15:34:54.278887+0000","last_undegraded":"2024-04-24T15:34:54.278887+0000","last_fullsized":"2024-04-24T15:34:54.278887+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"130'457","last_scrub_stamp":"2024-04-24T15:34:54.278739+0000","last_deep_scrub":"130'457","last_deep_scrub_stamp":"2024-04-24T15:34:54.278739+0000","last_clean_scrub_stamp":"2024-04-24T15:34:54.278739+0000","objects_scrubbed":0,"log_size":457,"log_dups_size":0,"ondisk_log_size":457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:29:38.898067+0000","scrub_duration":0.001758279,"objects_trimmed":0,"snaptrim_duration":0.0010565220000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2138,"num_read_kb":24032,"num_write":423,"num_write_kb":14900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"130'408","reported_seq":1985,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:50.238546+0000","last_change":"2024-04-24T15:34:50.238512+0000","last_active":"2024-04-24T15:34:50.238546+0000","last_peered":"2024-04-24T15:34:50.238546+0000","last_clean":"2024-04-24T15:34:50.238546+0000","last_became_active":"2024-04-24T15:34:16.651931+0000","last_became_peered":"2024-04-24T15:34:16.651931+0000","last_unstale":"2024-04-24T15:34:50.238546+0000","last_undegraded":"2024-04-24T15:34:50.238546+0000","last_fullsized":"2024-04-24T15:34:50.238546+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"130'408","last_scrub_stamp":"2024-04-24T15:34:50.238356+0000","last_deep_scrub":"130'408","last_deep_scrub_stamp":"2024-04-24T15:34:50.238356+0000","last_clean_scrub_stamp":"2024-04-24T15:34:50.238356+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:48:49.804142+0000","scrub_duration":0.0019868109999999998,"objects_trimmed":0,"snaptrim_duration":0.00039757799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2388,"num_read_kb":22956,"num_write":379,"num_write_kb":16060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":578400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":35,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:45.281361+0000","last_change":"2024-04-24T15:34:45.281331+0000","last_active":"2024-04-24T15:34:45.281361+0000","last_peered":"2024-04-24T15:34:45.281361+0000","last_clean":"2024-04-24T15:34:45.281361+0000","last_became_active":"2024-04-24T15:34:12.653529+0000","last_became_peered":"2024-04-24T15:34:12.653529+0000","last_unstale":"2024-04-24T15:34:45.281361+0000","last_undegraded":"2024-04-24T15:34:45.281361+0000","last_fullsized":"2024-04-24T15:34:45.281361+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T15:34:45.281222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T15:34:45.281222+0000","last_clean_scrub_stamp":"2024-04-24T15:34:45.281222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:06:49.812307+0000","scrub_duration":0.00148835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"149'481","reported_seq":2010,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:39.783116+0000","last_change":"2024-04-24T15:34:39.783087+0000","last_active":"2024-04-24T15:34:39.783116+0000","last_peered":"2024-04-24T15:34:39.783116+0000","last_clean":"2024-04-24T15:34:39.783116+0000","last_became_active":"2024-04-24T15:34:16.654252+0000","last_became_peered":"2024-04-24T15:34:16.654252+0000","last_unstale":"2024-04-24T15:34:39.783116+0000","last_undegraded":"2024-04-24T15:34:39.783116+0000","last_fullsized":"2024-04-24T15:34:39.783116+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"149'481","last_scrub_stamp":"2024-04-24T15:34:39.782995+0000","last_deep_scrub":"149'481","last_deep_scrub_stamp":"2024-04-24T15:34:39.782995+0000","last_clean_scrub_stamp":"2024-04-24T15:34:39.782995+0000","objects_scrubbed":2,"log_size":481,"log_dups_size":0,"ondisk_log_size":481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:01:17.010816+0000","scrub_duration":0.003456808,"objects_trimmed":0,"snaptrim_duration":0.00035787500000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":2434,"num_read_kb":14959,"num_write":545,"num_write_kb":12213,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":335,"num_bytes_recovered":6920328,"num_keys_recovered":43,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"149'622","reported_seq":2751,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:35:01.626851+0000","last_change":"2024-04-24T15:35:01.626810+0000","last_active":"2024-04-24T15:35:01.626851+0000","last_peered":"2024-04-24T15:35:01.626851+0000","last_clean":"2024-04-24T15:35:01.626851+0000","last_became_active":"2024-04-24T15:34:23.825759+0000","last_became_peered":"2024-04-24T15:34:23.825759+0000","last_unstale":"2024-04-24T15:35:01.626851+0000","last_undegraded":"2024-04-24T15:35:01.626851+0000","last_fullsized":"2024-04-24T15:35:01.626851+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"149'622","last_scrub_stamp":"2024-04-24T15:35:01.626716+0000","last_deep_scrub":"149'622","last_deep_scrub_stamp":"2024-04-24T15:35:01.626716+0000","last_clean_scrub_stamp":"2024-04-24T15:35:01.626716+0000","objects_scrubbed":6,"log_size":622,"log_dups_size":0,"ondisk_log_size":622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:02:25.346853+0000","scrub_duration":0.012008629,"objects_trimmed":2,"snaptrim_duration":0.0055639879999999997,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":4324,"num_read_kb":21128,"num_write":677,"num_write_kb":10933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":5713612,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"149'486","reported_seq":3243,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:46.639033+0000","last_change":"2024-04-24T15:34:46.638989+0000","last_active":"2024-04-24T15:34:46.639033+0000","last_peered":"2024-04-24T15:34:46.639033+0000","last_clean":"2024-04-24T15:34:46.639033+0000","last_became_active":"2024-04-24T15:34:15.657660+0000","last_became_peered":"2024-04-24T15:34:15.657660+0000","last_unstale":"2024-04-24T15:34:46.639033+0000","last_undegraded":"2024-04-24T15:34:46.639033+0000","last_fullsized":"2024-04-24T15:34:46.639033+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"149'486","last_scrub_stamp":"2024-04-24T15:34:46.638853+0000","last_deep_scrub":"149'486","last_deep_scrub_stamp":"2024-04-24T15:34:46.638853+0000","last_clean_scrub_stamp":"2024-04-24T15:34:46.638853+0000","objects_scrubbed":1,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:11:13.324969+0000","scrub_duration":0.003648166,"objects_trimmed":0,"snaptrim_duration":0.00066763,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3120,"num_read_kb":27458,"num_write":430,"num_write_kb":12267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":237,"num_bytes_recovered":5299136,"num_keys_recovered":49,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"129'551","reported_seq":1987,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:38.371801+0000","last_change":"2024-04-24T15:34:38.371767+0000","last_active":"2024-04-24T15:34:38.371801+0000","last_peered":"2024-04-24T15:34:38.371801+0000","last_clean":"2024-04-24T15:34:38.371801+0000","last_became_active":"2024-04-24T15:34:15.668717+0000","last_became_peered":"2024-04-24T15:34:15.668717+0000","last_unstale":"2024-04-24T15:34:38.371801+0000","last_undegraded":"2024-04-24T15:34:38.371801+0000","last_fullsized":"2024-04-24T15:34:38.371801+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"129'551","last_scrub_stamp":"2024-04-24T15:34:38.371640+0000","last_deep_scrub":"129'551","last_deep_scrub_stamp":"2024-04-24T15:34:38.371640+0000","last_clean_scrub_stamp":"2024-04-24T15:34:38.371640+0000","objects_scrubbed":0,"log_size":551,"log_dups_size":0,"ondisk_log_size":551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:19:54.567694+0000","scrub_duration":0.001344725,"objects_trimmed":0,"snaptrim_duration":0.00039775200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2839,"num_read_kb":21248,"num_write":721,"num_write_kb":8473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":141,"num_bytes_recovered":1490474,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"149'1049","reported_seq":2519,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:52.641192+0000","last_change":"2024-04-24T15:34:52.641161+0000","last_active":"2024-04-24T15:34:52.641192+0000","last_peered":"2024-04-24T15:34:52.641192+0000","last_clean":"2024-04-24T15:34:52.641192+0000","last_became_active":"2024-04-24T15:34:23.825039+0000","last_became_peered":"2024-04-24T15:34:23.825039+0000","last_unstale":"2024-04-24T15:34:52.641192+0000","last_undegraded":"2024-04-24T15:34:52.641192+0000","last_fullsized":"2024-04-24T15:34:52.641192+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"149'1049","last_scrub_stamp":"2024-04-24T15:34:52.641062+0000","last_deep_scrub":"149'1049","last_deep_scrub_stamp":"2024-04-24T15:34:52.641062+0000","last_clean_scrub_stamp":"2024-04-24T15:34:52.641062+0000","objects_scrubbed":3,"log_size":1049,"log_dups_size":0,"ondisk_log_size":1049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:09:37.232015+0000","scrub_duration":0.00545701,"objects_trimmed":0,"snaptrim_duration":0.0029127150000000002,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":6311,"num_read_kb":16558,"num_write":1677,"num_write_kb":12129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":54,"num_bytes_recovered":1219753,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.d","version":"149'299","reported_seq":2584,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:54.716410+0000","last_change":"2024-04-24T15:34:54.716378+0000","last_active":"2024-04-24T15:34:54.716410+0000","last_peered":"2024-04-24T15:34:54.716410+0000","last_clean":"2024-04-24T15:34:54.716410+0000","last_became_active":"2024-04-24T15:34:23.823763+0000","last_became_peered":"2024-04-24T15:34:23.823763+0000","last_unstale":"2024-04-24T15:34:54.716410+0000","last_undegraded":"2024-04-24T15:34:54.716410+0000","last_fullsized":"2024-04-24T15:34:54.716410+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"149'299","last_scrub_stamp":"2024-04-24T15:34:54.716280+0000","last_deep_scrub":"149'299","last_deep_scrub_stamp":"2024-04-24T15:34:54.716280+0000","last_clean_scrub_stamp":"2024-04-24T15:34:54.716280+0000","objects_scrubbed":1,"log_size":299,"log_dups_size":0,"ondisk_log_size":299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:19:20.347461+0000","scrub_duration":0.003803912,"objects_trimmed":0,"snaptrim_duration":0.00047837299999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":2121,"num_read_kb":46483,"num_write":303,"num_write_kb":20603,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":4246835,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"129'563","reported_seq":2406,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:44.539365+0000","last_change":"2024-04-24T15:34:44.539331+0000","last_active":"2024-04-24T15:34:44.539365+0000","last_peered":"2024-04-24T15:34:44.539365+0000","last_clean":"2024-04-24T15:34:44.539365+0000","last_became_active":"2024-04-24T15:34:23.824037+0000","last_became_peered":"2024-04-24T15:34:23.824037+0000","last_unstale":"2024-04-24T15:34:44.539365+0000","last_undegraded":"2024-04-24T15:34:44.539365+0000","last_fullsized":"2024-04-24T15:34:44.539365+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"129'563","last_scrub_stamp":"2024-04-24T15:34:44.539210+0000","last_deep_scrub":"129'563","last_deep_scrub_stamp":"2024-04-24T15:34:44.539210+0000","last_clean_scrub_stamp":"2024-04-24T15:34:44.539210+0000","objects_scrubbed":0,"log_size":563,"log_dups_size":0,"ondisk_log_size":563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:42:19.986755+0000","scrub_duration":0.001982106,"objects_trimmed":0,"snaptrim_duration":0.001084025,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3022,"num_read_kb":13469,"num_write":696,"num_write_kb":6185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":113,"num_bytes_recovered":1676557,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"149'909","reported_seq":3319,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:52.269827+0000","last_change":"2024-04-24T15:34:52.269796+0000","last_active":"2024-04-24T15:34:52.269827+0000","last_peered":"2024-04-24T15:34:52.269827+0000","last_clean":"2024-04-24T15:34:52.269827+0000","last_became_active":"2024-04-24T15:34:16.655733+0000","last_became_peered":"2024-04-24T15:34:16.655733+0000","last_unstale":"2024-04-24T15:34:52.269827+0000","last_undegraded":"2024-04-24T15:34:52.269827+0000","last_fullsized":"2024-04-24T15:34:52.269827+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"149'909","last_scrub_stamp":"2024-04-24T15:34:52.269703+0000","last_deep_scrub":"149'909","last_deep_scrub_stamp":"2024-04-24T15:34:52.269703+0000","last_clean_scrub_stamp":"2024-04-24T15:34:52.269703+0000","objects_scrubbed":2,"log_size":909,"log_dups_size":0,"ondisk_log_size":909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:54:41.487566+0000","scrub_duration":0.0037041299999999999,"objects_trimmed":0,"snaptrim_duration":0.0018480180000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":5733,"num_read_kb":14488,"num_write":1011,"num_write_kb":8995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":223,"num_bytes_recovered":630687,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"130'244","reported_seq":1072,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:52.850262+0000","last_change":"2024-04-24T15:34:52.850233+0000","last_active":"2024-04-24T15:34:52.850262+0000","last_peered":"2024-04-24T15:34:52.850262+0000","last_clean":"2024-04-24T15:34:52.850262+0000","last_became_active":"2024-04-24T15:34:15.663081+0000","last_became_peered":"2024-04-24T15:34:15.663081+0000","last_unstale":"2024-04-24T15:34:52.850262+0000","last_undegraded":"2024-04-24T15:34:52.850262+0000","last_fullsized":"2024-04-24T15:34:52.850262+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"130'244","last_scrub_stamp":"2024-04-24T15:34:52.850135+0000","last_deep_scrub":"130'244","last_deep_scrub_stamp":"2024-04-24T15:34:52.850135+0000","last_clean_scrub_stamp":"2024-04-24T15:34:52.850135+0000","objects_scrubbed":0,"log_size":244,"log_dups_size":0,"ondisk_log_size":244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:06:40.625024+0000","scrub_duration":0.0016600580000000001,"objects_trimmed":0,"snaptrim_duration":0.0010945530000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":384,"num_read_kb":4092,"num_write":235,"num_write_kb":5715,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":1989443,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.11","version":"149'707","reported_seq":2175,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:42.781526+0000","last_change":"2024-04-24T15:34:42.781482+0000","last_active":"2024-04-24T15:34:42.781526+0000","last_peered":"2024-04-24T15:34:42.781526+0000","last_clean":"2024-04-24T15:34:42.781526+0000","last_became_active":"2024-04-24T15:33:57.644309+0000","last_became_peered":"2024-04-24T15:33:57.644309+0000","last_unstale":"2024-04-24T15:34:42.781526+0000","last_undegraded":"2024-04-24T15:34:42.781526+0000","last_fullsized":"2024-04-24T15:34:42.781526+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"149'707","last_scrub_stamp":"2024-04-24T15:34:42.781389+0000","last_deep_scrub":"149'707","last_deep_scrub_stamp":"2024-04-24T15:34:42.781389+0000","last_clean_scrub_stamp":"2024-04-24T15:34:42.781389+0000","objects_scrubbed":2,"log_size":707,"log_dups_size":0,"ondisk_log_size":707,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:56:03.138605+0000","scrub_duration":0.005289134,"objects_trimmed":0,"snaptrim_duration":0.0012982219999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":2615,"num_read_kb":21488,"num_write":891,"num_write_kb":14014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":5651177,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"149'332","reported_seq":5240,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:46.291941+0000","last_change":"2024-04-24T15:34:46.291914+0000","last_active":"2024-04-24T15:34:46.291941+0000","last_peered":"2024-04-24T15:34:46.291941+0000","last_clean":"2024-04-24T15:34:46.291941+0000","last_became_active":"2024-04-24T15:34:15.653205+0000","last_became_peered":"2024-04-24T15:34:15.653205+0000","last_unstale":"2024-04-24T15:34:46.291941+0000","last_undegraded":"2024-04-24T15:34:46.291941+0000","last_fullsized":"2024-04-24T15:34:46.291941+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"149'332","last_scrub_stamp":"2024-04-24T15:34:46.291812+0000","last_deep_scrub":"149'332","last_deep_scrub_stamp":"2024-04-24T15:34:46.291812+0000","last_clean_scrub_stamp":"2024-04-24T15:34:46.291812+0000","objects_scrubbed":1,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:01:55.256353+0000","scrub_duration":0.003426279,"objects_trimmed":0,"snaptrim_duration":0.0013576079999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":4982,"num_read_kb":42072,"num_write":333,"num_write_kb":14625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":3758016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"149'242","reported_seq":2015,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:55.668018+0000","last_change":"2024-04-24T15:34:55.667990+0000","last_active":"2024-04-24T15:34:55.668018+0000","last_peered":"2024-04-24T15:34:55.668018+0000","last_clean":"2024-04-24T15:34:55.668018+0000","last_became_active":"2024-04-24T15:34:23.823962+0000","last_became_peered":"2024-04-24T15:34:23.823962+0000","last_unstale":"2024-04-24T15:34:55.668018+0000","last_undegraded":"2024-04-24T15:34:55.668018+0000","last_fullsized":"2024-04-24T15:34:55.668018+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"149'242","last_scrub_stamp":"2024-04-24T15:34:55.667892+0000","last_deep_scrub":"149'242","last_deep_scrub_stamp":"2024-04-24T15:34:55.667892+0000","last_clean_scrub_stamp":"2024-04-24T15:34:55.667892+0000","objects_scrubbed":1,"log_size":242,"log_dups_size":0,"ondisk_log_size":242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:42:05.504651+0000","scrub_duration":0.0028992699999999998,"objects_trimmed":0,"snaptrim_duration":0.000635438,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":1312,"num_read_kb":18878,"num_write":253,"num_write_kb":9844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":108,"num_bytes_recovered":2694373,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"149'451","reported_seq":6607,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:45.807425+0000","last_change":"2024-04-24T15:34:45.807385+0000","last_active":"2024-04-24T15:34:45.807425+0000","last_peered":"2024-04-24T15:34:45.807425+0000","last_clean":"2024-04-24T15:34:45.807425+0000","last_became_active":"2024-04-24T15:34:16.659071+0000","last_became_peered":"2024-04-24T15:34:16.659071+0000","last_unstale":"2024-04-24T15:34:45.807425+0000","last_undegraded":"2024-04-24T15:34:45.807425+0000","last_fullsized":"2024-04-24T15:34:45.807425+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"149'451","last_scrub_stamp":"2024-04-24T15:34:45.807282+0000","last_deep_scrub":"149'451","last_deep_scrub_stamp":"2024-04-24T15:34:45.807282+0000","last_clean_scrub_stamp":"2024-04-24T15:34:45.807282+0000","objects_scrubbed":1,"log_size":451,"log_dups_size":0,"ondisk_log_size":451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:49:42.830953+0000","scrub_duration":0.0030214170000000002,"objects_trimmed":0,"snaptrim_duration":0.0013356920000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":7366,"num_read_kb":44142,"num_write":407,"num_write_kb":9850,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":233,"num_bytes_recovered":4458876,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"130'312","reported_seq":1586,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:50.936162+0000","last_change":"2024-04-24T15:34:50.936113+0000","last_active":"2024-04-24T15:34:50.936162+0000","last_peered":"2024-04-24T15:34:50.936162+0000","last_clean":"2024-04-24T15:34:50.936162+0000","last_became_active":"2024-04-24T15:34:16.655084+0000","last_became_peered":"2024-04-24T15:34:16.655084+0000","last_unstale":"2024-04-24T15:34:50.936162+0000","last_undegraded":"2024-04-24T15:34:50.936162+0000","last_fullsized":"2024-04-24T15:34:50.936162+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"130'312","last_scrub_stamp":"2024-04-24T15:34:50.936005+0000","last_deep_scrub":"130'312","last_deep_scrub_stamp":"2024-04-24T15:34:50.936005+0000","last_clean_scrub_stamp":"2024-04-24T15:34:50.936005+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:30.062675+0000","scrub_duration":0.0015637629999999999,"objects_trimmed":0,"snaptrim_duration":0.003384743,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":837,"num_read_kb":16692,"num_write":319,"num_write_kb":15488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":277,"num_bytes_recovered":16228001,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.16","version":"149'377","reported_seq":2467,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-24T15:34:48.393753+0000","last_change":"2024-04-24T15:34:48.393706+0000","last_active":"2024-04-24T15:34:48.393753+0000","last_peered":"2024-04-24T15:34:48.393753+0000","last_clean":"2024-04-24T15:34:48.393753+0000","last_became_active":"2024-04-24T15:34:15.668678+0000","last_became_peered":"2024-04-24T15:34:15.668678+0000","last_unstale":"2024-04-24T15:34:48.393753+0000","last_undegraded":"2024-04-24T15:34:48.393753+0000","last_fullsized":"2024-04-24T15:34:48.393753+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"149'377","last_scrub_stamp":"2024-04-24T15:34:48.393610+0000","last_deep_scrub":"149'377","last_deep_scrub_stamp":"2024-04-24T15:34:48.393610+0000","last_clean_scrub_stamp":"2024-04-24T15:34:48.393610+0000","objects_scrubbed":2,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:52:36.292086+0000","scrub_duration":0.003892805,"objects_trimmed":0,"snaptrim_duration":0.0018886770000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":3004,"num_read_kb":17777,"num_write":338,"num_write_kb":8036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":48,"num_read":114807,"num_read_kb":727542,"num_write":20290,"num_write_kb":356328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4413,"num_bytes_recovered":105175530,"num_keys_recovered":235,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15242,"internal_metadata":0},"log_size":17303,"ondisk_log_size":17303,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":83,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":83,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":8,"acting":8,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":16196,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574892,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299140,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072700,"statfs":{"total":96636764160,"available":96330444800,"internally_reserved":0,"allocated":1830912,"data_stored":1500719,"data_compressed":5677,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 15:35:03 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.47899999999999998,"15min":0.46800000000000003},"min":{"1min":0.26500000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.73099999999999998,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.5},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.54200000000000004,"15min":0.55600000000000005},"min":{"1min":0.40799999999999997,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.73999999999999999,"5min":2.4710000000000001,"15min":2.4710000000000001},"last":0.59099999999999997}]},{"osd":1,"last update":"Wed Apr 24 15:35:03 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.56000000000000005,"15min":0.58799999999999997},"min":{"1min":0.38700000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.71099999999999997,"5min":2.4729999999999999,"15min":2.4729999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.52200000000000002,"15min":0.52600000000000002},"min":{"1min":0.376,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.746,"5min":2.4020000000000001,"15min":2.4020000000000001},"last":0.55200000000000005}]},{"osd":2,"last update":"Wed Apr 24 15:35:03 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.47599999999999998,"15min":0.46400000000000002},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.627},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.55000000000000004,"15min":0.55600000000000005},"min":{"1min":0.307,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.752,"5min":2.363,"15min":2.363},"last":0.52800000000000002}]},{"osd":3,"last update":"Wed Apr 24 15:35:03 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.53500000000000003,"15min":0.52300000000000002},"min":{"1min":0.34499999999999997,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.79600000000000004,"5min":1.157,"15min":1.157},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.55500000000000005,"15min":0.53700000000000003},"min":{"1min":0.41799999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.78700000000000003,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.69899999999999995}]},{"osd":4,"last update":"Wed Apr 24 15:35:03 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.41699999999999998,"15min":0.42099999999999999},"min":{"1min":0.32300000000000001,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.66700000000000004,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.43099999999999999,"15min":0.42799999999999999},"min":{"1min":0.35999999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.64000000000000001,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.41799999999999998}]},{"osd":5,"last update":"Wed Apr 24 15:35:03 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.45900000000000002,"15min":0.45700000000000002},"min":{"1min":0.34599999999999997,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.70799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.52700000000000002,"15min":0.52300000000000002},"min":{"1min":0.41499999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.72699999999999998,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.36199999999999999}]},{"osd":6,"last update":"Wed Apr 24 15:34:43 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.46700000000000003}]}]},{"osd":6,"up_from":104,"seq":446676598805,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":305884,"kb_used_data":1748,"kb_used_omap":43,"kb_used_meta":304084,"kb_avail":94065956,"statfs":{"total":96636764160,"available":96323538944,"internally_reserved":0,"allocated":1789952,"data_stored":910351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":44735,"internal_metadata":311382337},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 15:34:43 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.52900000000000003}]},{"osd":1,"last update":"Wed Apr 24 15:34:43 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.8640000000000001,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":0.41499999999999998}]},{"osd":2,"last update":"Wed Apr 24 15:34:43 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.40000000000000002}]},{"osd":3,"last update":"Wed Apr 24 15:34:43 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":2.6389999999999998,"5min":2.6389999999999998,"15min":2.6389999999999998},"last":0.5}]},{"osd":4,"last update":"Wed Apr 24 15:34:43 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.46400000000000002}]},{"osd":5,"last update":"Wed Apr 24 15:34:43 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.441}]},{"osd":7,"last update":"Wed Apr 24 15:34:43 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.51700000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574891,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307288,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":305534,"kb_avail":94064552,"statfs":{"total":96636764160,"available":96322101248,"internally_reserved":0,"allocated":1785856,"data_stored":909395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312867160},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 15:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.35699999999999998,"15min":0.35799999999999998},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.371,"15min":0.374},"min":{"1min":0.224,"5min":0.216,"15min":0.216},"max":{"1min":0.59999999999999998,"5min":1.375,"15min":1.375},"last":0.51300000000000001}]},{"osd":2,"last update":"Wed Apr 24 15:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.34699999999999998,"15min":0.34300000000000003},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.40200000000000002,"15min":0.42299999999999999},"min":{"1min":0.251,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.51100000000000001,"5min":1.494,"15min":1.494},"last":0.47999999999999998}]},{"osd":3,"last update":"Wed Apr 24 15:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.41699999999999998,"15min":0.42899999999999999},"min":{"1min":0.25800000000000001,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.63100000000000001,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.38600000000000001,"15min":0.375},"min":{"1min":0.27800000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.76600000000000001,"5min":1.319,"15min":1.319},"last":0.41999999999999998}]},{"osd":4,"last update":"Wed Apr 24 15:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.53700000000000003,"15min":0.54100000000000004},"min":{"1min":0.41199999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.82099999999999995,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57599999999999996,"15min":0.58199999999999996},"min":{"1min":0.35199999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.85199999999999998,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.82699999999999996}]},{"osd":5,"last update":"Wed Apr 24 15:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.54000000000000004,"15min":0.53800000000000003},"min":{"1min":0.40200000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.747,"5min":1.454,"15min":1.454},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.54900000000000004,"15min":0.54500000000000004},"min":{"1min":0.48499999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.93700000000000006,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.81200000000000006}]},{"osd":6,"last update":"Wed Apr 24 15:34:44 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.83999999999999997}]},{"osd":7,"last update":"Wed Apr 24 15:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.41599999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.1379999999999999,"5min":1.407,"15min":1.407},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.55800000000000005,"15min":0.55900000000000005},"min":{"1min":0.48799999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.90400000000000003,"5min":0.999,"15min":0.999},"last":0.76900000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574892,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303256,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94068584,"statfs":{"total":96636764160,"available":96326230016,"internally_reserved":0,"allocated":1851392,"data_stored":1500802,"data_compressed":4842,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Apr 24 15:35:03 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.433,"15min":0.47499999999999998},"min":{"1min":0.223,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":1.3300000000000001,"5min":3.7890000000000001,"15min":3.7890000000000001},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.44500000000000001,"15min":0.48099999999999998},"min":{"1min":0.19700000000000001,"5min":0.191,"15min":0.191},"max":{"1min":1.2030000000000001,"5min":2.7890000000000001,"15min":2.7890000000000001},"last":0.38200000000000001}]},{"osd":2,"last update":"Wed Apr 24 15:35:03 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.496,"15min":0.54800000000000004},"min":{"1min":0.245,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":1.3600000000000001,"5min":3.9950000000000001,"15min":3.9950000000000001},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.41899999999999998,"15min":0.435},"min":{"1min":0.27700000000000002,"5min":0.253,"15min":0.253},"max":{"1min":1.2250000000000001,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":0.43099999999999999}]},{"osd":3,"last update":"Wed Apr 24 15:35:03 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.53500000000000003,"15min":0.52200000000000002},"min":{"1min":0.25800000000000001,"5min":0.223,"15min":0.223},"max":{"1min":7.9429999999999996,"5min":7.9429999999999996,"15min":7.9429999999999996},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.57699999999999996,"15min":0.64200000000000002},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.4039999999999999,"5min":3.2109999999999999,"15min":3.2109999999999999},"last":0.45600000000000002}]},{"osd":4,"last update":"Wed Apr 24 15:35:03 2024","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.629,"15min":0.59099999999999997},"min":{"1min":0.33400000000000002,"5min":0.308,"15min":0.308},"max":{"1min":8.0329999999999995,"5min":8.0329999999999995,"15min":8.0329999999999995},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.63200000000000001,"15min":0.66700000000000004},"min":{"1min":0.38100000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.383,"5min":3.282,"15min":3.282},"last":0.61099999999999999}]},{"osd":5,"last update":"Wed Apr 24 15:35:03 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.60799999999999998,"15min":0.61499999999999999},"min":{"1min":0.42999999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.1120000000000001,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":1.0089999999999999,"5min":0.71399999999999997,"15min":0.68500000000000005},"min":{"1min":0.371,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":7.9969999999999999,"5min":7.9969999999999999,"15min":7.9969999999999999},"last":0.74199999999999999}]},{"osd":6,"last update":"Wed Apr 24 15:34:43 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":8.0619999999999994,"5min":8.0619999999999994,"15min":8.0619999999999994},"last":0.83699999999999997}]},{"osd":7,"last update":"Wed Apr 24 15:35:03 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.60699999999999998,"15min":0.61099999999999999},"min":{"1min":0.47999999999999998,"5min":0.375,"15min":0.375},"max":{"1min":1.177,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.69599999999999995,"15min":0.66500000000000004},"min":{"1min":0.46000000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":8.0899999999999999,"5min":8.0899999999999999,"15min":8.0899999999999999},"last":0.64500000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574892,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303192,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94068648,"statfs":{"total":96636764160,"available":96326295552,"internally_reserved":0,"allocated":1785856,"data_stored":909395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 15:35:09 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.316,"15min":0.316},"min":{"1min":0.22,"5min":0.16200000000000001,"15min":0.16200000000000001},"max":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.33600000000000002,"15min":0.34200000000000003},"min":{"1min":0.219,"5min":0.185,"15min":0.185},"max":{"1min":0.45400000000000001,"5min":0.45800000000000002,"15min":0.45800000000000002},"last":0.27300000000000002}]},{"osd":1,"last update":"Wed Apr 24 15:35:09 2024","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.36499999999999999,"15min":0.375},"min":{"1min":0.25,"5min":0.222,"15min":0.222},"max":{"1min":0.59899999999999998,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.38},{"interface":"front","average":{"1min":0.375,"5min":0.438,"15min":0.47399999999999998},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.53600000000000003,"5min":3.1110000000000002,"15min":3.1110000000000002},"last":0.36099999999999999}]},{"osd":3,"last update":"Wed Apr 24 15:35:09 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.39300000000000002,"15min":0.41499999999999998},"min":{"1min":0.23499999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.58699999999999997,"5min":2.056,"15min":2.056},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.39900000000000002,"15min":0.40500000000000003},"min":{"1min":0.27500000000000002,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.56799999999999995,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.40799999999999997}]},{"osd":4,"last update":"Wed Apr 24 15:35:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.52600000000000002,"15min":0.52700000000000002},"min":{"1min":0.45900000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.53900000000000003,"15min":0.54600000000000004},"min":{"1min":0.42099999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.66900000000000004,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.53700000000000003}]},{"osd":5,"last update":"Wed Apr 24 15:35:09 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.55200000000000005,"15min":0.54900000000000004},"min":{"1min":0.40400000000000003,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.76500000000000001,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.627,"15min":0.65100000000000002},"min":{"1min":0.48999999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81799999999999995,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":0.55200000000000005}]},{"osd":6,"last update":"Wed Apr 24 15:34:46 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.495},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.46000000000000002}]},{"osd":7,"last update":"Wed Apr 24 15:35:09 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.621,"15min":0.65200000000000002},"min":{"1min":0.42999999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.85299999999999998,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.62,"5min":0.57799999999999996,"15min":0.57999999999999996},"min":{"1min":0.41099999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.82799999999999996,"5min":1.502,"15min":1.502},"last":0.59799999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574892,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303200,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94068640,"statfs":{"total":96636764160,"available":96326287360,"internally_reserved":0,"allocated":1794048,"data_stored":910370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 15:35:04 2024","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.33900000000000002,"15min":0.35299999999999998},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.443,"5min":1.224,"15min":1.224},"last":0.311},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.33700000000000002,"15min":0.34499999999999997},"min":{"1min":0.22700000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.56799999999999995,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.32500000000000001}]},{"osd":1,"last update":"Wed Apr 24 15:35:04 2024","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.33200000000000002,"15min":0.33800000000000002},"min":{"1min":0.214,"5min":0.187,"15min":0.187},"max":{"1min":0.54900000000000004,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.41199999999999998,"15min":0.42999999999999999},"min":{"1min":0.26700000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.58799999999999997,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.44600000000000001}]},{"osd":2,"last update":"Wed Apr 24 15:35:04 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.38800000000000001,"15min":0.40100000000000002},"min":{"1min":0.26100000000000001,"5min":0.219,"15min":0.219},"max":{"1min":0.61599999999999999,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.376},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.40799999999999997,"15min":0.41799999999999998},"min":{"1min":0.28000000000000003,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.56899999999999995,"5min":1.252,"15min":1.252},"last":0.41599999999999998}]},{"osd":4,"last update":"Wed Apr 24 15:35:04 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.53500000000000003,"15min":0.53900000000000003},"min":{"1min":0.32800000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.448,"5min":1.448,"15min":1.448},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.52200000000000002,"15min":0.53300000000000003},"min":{"1min":0.31,"5min":0.308,"15min":0.308},"max":{"1min":0.70799999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.69399999999999995}]},{"osd":5,"last update":"Wed Apr 24 15:35:04 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.56399999999999995,"15min":0.57199999999999995},"min":{"1min":0.34899999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.59799999999999998,"15min":0.60599999999999998},"min":{"1min":0.47699999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.4810000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.63700000000000001}]},{"osd":6,"last update":"Wed Apr 24 15:34:45 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.76400000000000001}]},{"osd":7,"last update":"Wed Apr 24 15:35:04 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.59999999999999998,"15min":0.60799999999999998},"min":{"1min":0.40500000000000003,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.3180000000000001,"5min":1.635,"15min":1.635},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.56899999999999995,"15min":0.57099999999999995},"min":{"1min":0.42299999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":0.70999999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574893,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307292,"kb_used_data":1748,"kb_used_omap":1,"kb_used_meta":305534,"kb_avail":94064548,"statfs":{"total":96636764160,"available":96322097152,"internally_reserved":0,"allocated":1789952,"data_stored":908139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312867160},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 15:35:01 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.40400000000000003,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.61899999999999999,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.56100000000000005,"15min":0.55500000000000005},"min":{"1min":0.36899999999999999,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.752,"5min":5.6239999999999997,"15min":5.6239999999999997},"last":0.52800000000000002}]},{"osd":1,"last update":"Wed Apr 24 15:35:01 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.58899999999999997,"15min":0.57299999999999995},"min":{"1min":0.497,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.67500000000000004,"5min":5.7149999999999999,"15min":5.7149999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.54000000000000004,"15min":0.52900000000000003},"min":{"1min":0.53100000000000003,"5min":0.31,"15min":0.31},"max":{"1min":0.72499999999999998,"5min":1.913,"15min":1.913},"last":0.58399999999999996}]},{"osd":2,"last update":"Wed Apr 24 15:35:01 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.56100000000000005,"15min":0.53300000000000003},"min":{"1min":0.41099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.1519999999999999,"5min":5.6660000000000004,"15min":5.6660000000000004},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53100000000000003,"15min":0.54300000000000004},"min":{"1min":0.36599999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.72399999999999998,"5min":1.819,"15min":1.819},"last":0.54900000000000004}]},{"osd":3,"last update":"Wed Apr 24 15:35:01 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.53000000000000003,"15min":0.53400000000000003},"min":{"1min":0.34899999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.79200000000000004,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.56100000000000005,"15min":0.55200000000000005},"min":{"1min":0.44400000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.92200000000000004,"5min":2.3149999999999999,"15min":2.3149999999999999},"last":0.66100000000000003}]},{"osd":5,"last update":"Wed Apr 24 15:35:01 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.57999999999999996,"15min":0.51600000000000001},"min":{"1min":0.34200000000000003,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.66800000000000004,"5min":14.667999999999999,"15min":14.667999999999999},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.5,"15min":0.50800000000000001},"min":{"1min":0.36399999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.76400000000000001,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":0.379}]},{"osd":6,"last update":"Wed Apr 24 15:34:43 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.44},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.47499999999999998}]},{"osd":7,"last update":"Wed Apr 24 15:35:01 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54300000000000004,"15min":0.53200000000000003},"min":{"1min":0.35199999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.72999999999999998,"5min":5.2759999999999998,"15min":5.2759999999999998},"last":0.499},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.65500000000000003,"15min":0.57799999999999996},"min":{"1min":0.40300000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.78700000000000003,"5min":14.773,"15min":14.773},"last":0.68000000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574892,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311388,"kb_used_data":1748,"kb_used_omap":1,"kb_used_meta":309630,"kb_avail":94060452,"statfs":{"total":96636764160,"available":96317902848,"internally_reserved":0,"allocated":1789952,"data_stored":910351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317061464},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 15:35:03 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.48499999999999999,"15min":0.48699999999999999},"min":{"1min":0.246,"5min":0.193,"15min":0.193},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.32800000000000001,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.438}]},{"osd":1,"last update":"Wed Apr 24 15:35:03 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.47099999999999997,"15min":0.47399999999999998},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.48799999999999999,"15min":0.47699999999999998},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.41899999999999998}]},{"osd":2,"last update":"Wed Apr 24 15:35:03 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.49099999999999999,"15min":0.47099999999999997},"min":{"1min":0.36599999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.751,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.501,"15min":0.497},"min":{"1min":0.35399999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.546,"5min":1.546,"15min":1.546},"last":0.45400000000000001}]},{"osd":3,"last update":"Wed Apr 24 15:35:03 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.50700000000000001,"15min":0.501},"min":{"1min":0.34499999999999997,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.40300000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.81499999999999995,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":0.47299999999999998}]},{"osd":4,"last update":"Wed Apr 24 15:35:03 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.443,"15min":0.45500000000000002},"min":{"1min":0.32200000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.752,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.5,"5min":0.48599999999999999,"15min":0.48699999999999999},"min":{"1min":0.311,"5min":0.252,"15min":0.252},"max":{"1min":0.78200000000000003,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.48299999999999998}]},{"osd":6,"last update":"Wed Apr 24 15:34:45 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.5},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.59099999999999997}]},{"osd":7,"last update":"Wed Apr 24 15:35:03 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.52700000000000002,"15min":0.53100000000000003},"min":{"1min":0.34699999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.78000000000000003,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.53000000000000003,"15min":0.53800000000000003},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.76500000000000001,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.52700000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4842,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":5677,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":5677,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15242,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T15:35:20.748 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-24T15:35:21.083 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-24T15:35:21.083 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-24T15:35:21.083 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-24T15:35:21.083 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T15:35:21.117 INFO:tasks.ceph.osd.0:Stopped 2024-04-24T15:35:21.117 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-24T15:35:21.117 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T15:35:21.184 INFO:tasks.ceph.osd.1:Stopped 2024-04-24T15:35:21.184 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-24T15:35:21.184 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T15:35:21.253 INFO:tasks.ceph.osd.2:Stopped 2024-04-24T15:35:21.253 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-24T15:35:21.253 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T15:35:21.324 INFO:tasks.ceph.osd.3:Stopped 2024-04-24T15:35:21.325 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-24T15:35:21.325 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T15:35:21.367 INFO:tasks.ceph.osd.4:Stopped 2024-04-24T15:35:21.368 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-24T15:35:21.368 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T15:35:21.405 INFO:tasks.ceph.osd.5:Stopped 2024-04-24T15:35:21.405 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-24T15:35:21.405 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T15:35:21.475 INFO:tasks.ceph.osd.6:Stopped 2024-04-24T15:35:21.475 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-24T15:35:21.475 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T15:35:21.545 INFO:tasks.ceph.osd.7:Stopped 2024-04-24T15:35:21.545 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-24T15:35:21.545 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-24T15:35:21.545 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T15:35:21.568 INFO:tasks.ceph.mgr.y:Stopped 2024-04-24T15:35:21.568 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-24T15:35:21.569 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T15:35:21.609 INFO:tasks.ceph.mgr.x:Stopped 2024-04-24T15:35:21.609 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-24T15:35:21.609 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-24T15:35:21.609 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T15:35:21.619 INFO:tasks.ceph.mon.a:Stopped 2024-04-24T15:35:21.619 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-24T15:35:21.619 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T15:35:21.669 INFO:tasks.ceph.mon.c:Stopped 2024-04-24T15:35:21.670 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-24T15:35:21.670 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T15:35:21.681 INFO:tasks.ceph.mon.b:Stopped 2024-04-24T15:35:21.681 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-24T15:35:21.682 DEBUG:teuthology.orchestra.run.smithi096:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | head -n 1 2024-04-24T15:35:21.741 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi096.front.sepia.ceph.com 2024-04-24T15:35:21.741 DEBUG:teuthology.orchestra.run.smithi096:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-24T15:35:21.952 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi096.front.sepia.ceph.com 2024-04-24T15:35:21.952 DEBUG:teuthology.orchestra.run.smithi096:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-24T15:35:22.038 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi096.front.sepia.ceph.com 2024-04-24T15:35:22.038 DEBUG:teuthology.orchestra.run.smithi096:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-24T15:35:22.119 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi096.front.sepia.ceph.com 2024-04-24T15:35:22.119 DEBUG:teuthology.orchestra.run.smithi096:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-24T15:35:22.187 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi151.front.sepia.ceph.com 2024-04-24T15:35:22.187 DEBUG:teuthology.orchestra.run.smithi151:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-24T15:35:22.382 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi151.front.sepia.ceph.com 2024-04-24T15:35:22.383 DEBUG:teuthology.orchestra.run.smithi151:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-24T15:35:22.458 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi151.front.sepia.ceph.com 2024-04-24T15:35:22.458 DEBUG:teuthology.orchestra.run.smithi151:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-24T15:35:22.525 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi151.front.sepia.ceph.com 2024-04-24T15:35:22.526 DEBUG:teuthology.orchestra.run.smithi151:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-24T15:35:22.590 INFO:tasks.ceph:Archiving mon data... 2024-04-24T15:35:22.591 DEBUG:teuthology.misc:Transferring archived files from smithi096:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuriw-2024-04-24_14:09:24-rbd-wip-yuriw4-testing-20240423.151325-quincy-distro-default-smithi/7671570/data/mon.a.tgz 2024-04-24T15:35:22.591 DEBUG:teuthology.orchestra.run.smithi096:> mktemp 2024-04-24T15:35:22.609 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/tmp.Ex92iwvZkl 2024-04-24T15:35:22.609 DEBUG:teuthology.orchestra.run.smithi096:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.Ex92iwvZkl 2024-04-24T15:35:22.757 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 0666 /tmp/tmp.Ex92iwvZkl 2024-04-24T15:35:22.844 DEBUG:teuthology.orchestra.remote:smithi096:/tmp/tmp.Ex92iwvZkl is 371KB 2024-04-24T15:35:22.940 DEBUG:teuthology.orchestra.run.smithi096:> rm -fr /tmp/tmp.Ex92iwvZkl 2024-04-24T15:35:22.957 DEBUG:teuthology.misc:Transferring archived files from smithi096:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/yuriw-2024-04-24_14:09:24-rbd-wip-yuriw4-testing-20240423.151325-quincy-distro-default-smithi/7671570/data/mon.c.tgz 2024-04-24T15:35:22.957 DEBUG:teuthology.orchestra.run.smithi096:> mktemp 2024-04-24T15:35:23.013 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/tmp.mxdNZJbFRv 2024-04-24T15:35:23.014 DEBUG:teuthology.orchestra.run.smithi096:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.mxdNZJbFRv 2024-04-24T15:35:23.180 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 0666 /tmp/tmp.mxdNZJbFRv 2024-04-24T15:35:23.269 DEBUG:teuthology.orchestra.remote:smithi096:/tmp/tmp.mxdNZJbFRv is 520KB 2024-04-24T15:35:23.369 DEBUG:teuthology.orchestra.run.smithi096:> rm -fr /tmp/tmp.mxdNZJbFRv 2024-04-24T15:35:23.386 DEBUG:teuthology.misc:Transferring archived files from smithi151:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/yuriw-2024-04-24_14:09:24-rbd-wip-yuriw4-testing-20240423.151325-quincy-distro-default-smithi/7671570/data/mon.b.tgz 2024-04-24T15:35:23.386 DEBUG:teuthology.orchestra.run.smithi151:> mktemp 2024-04-24T15:35:23.408 INFO:teuthology.orchestra.run.smithi151.stdout:/tmp/tmp.AdVYesA8HP 2024-04-24T15:35:23.408 DEBUG:teuthology.orchestra.run.smithi151:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.AdVYesA8HP 2024-04-24T15:35:23.616 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod 0666 /tmp/tmp.AdVYesA8HP 2024-04-24T15:35:23.703 DEBUG:teuthology.orchestra.remote:smithi151:/tmp/tmp.AdVYesA8HP is 520KB 2024-04-24T15:35:23.802 DEBUG:teuthology.orchestra.run.smithi151:> rm -fr /tmp/tmp.AdVYesA8HP 2024-04-24T15:35:23.820 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-24T15:35:23.820 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-24T15:35:23.822 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-24T15:35:23.886 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-04-24T15:35:23.886 DEBUG:teuthology.orchestra.run.smithi096:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-04-24T15:35:23.891 DEBUG:teuthology.orchestra.run.smithi151:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-04-24T15:35:23.917 INFO:teuthology.orchestra.run.smithi096.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-04-24T15:35:23.953 INFO:teuthology.orchestra.run.smithi151.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-04-24T15:35:23.959 INFO:tasks.ceph:Archiving crash dumps... 2024-04-24T15:35:23.960 DEBUG:teuthology.misc:Transferring archived files from smithi096:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2024-04-24_14:09:24-rbd-wip-yuriw4-testing-20240423.151325-quincy-distro-default-smithi/7671570/remote/smithi096/crash 2024-04-24T15:35:23.961 DEBUG:teuthology.orchestra.run.smithi096:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-24T15:35:23.991 DEBUG:teuthology.misc:Transferring archived files from smithi151:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2024-04-24_14:09:24-rbd-wip-yuriw4-testing-20240423.151325-quincy-distro-default-smithi/7671570/remote/smithi151/crash 2024-04-24T15:35:23.992 DEBUG:teuthology.orchestra.run.smithi151:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-24T15:35:24.025 INFO:tasks.ceph:Compressing logs... 2024-04-24T15:35:24.025 DEBUG:teuthology.orchestra.run.smithi096:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-24T15:35:24.036 DEBUG:teuthology.orchestra.run.smithi151:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-24T15:35:24.061 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69515.log 2024-04-24T15:35:24.061 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49685.log 2024-04-24T15:35:24.092 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.69515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50791.log 2024-04-24T15:35:24.092 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69515.log.gz 2024-04-24T15:35:24.092 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.49685.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49685.log.gz 2024-04-24T15:35:24.092 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50048.log 2024-04-24T15:35:24.093 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68927.log 2024-04-24T15:35:24.093 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.50791.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50791.log.gz 2024-04-24T15:35:24.093 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.50048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69858.log 2024-04-24T15:35:24.093 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50048.log.gz 2024-04-24T15:35:24.093 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.68927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67506.log 2024-04-24T15:35:24.093 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68927.log.gz 2024-04-24T15:35:24.093 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.69858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56861.log 2024-04-24T15:35:24.093 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69858.log.gz 2024-04-24T15:35:24.093 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.67506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50391.log 2024-04-24T15:35:24.093 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67506.log.gz 2024-04-24T15:35:24.093 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.56861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67457.log 2024-04-24T15:35:24.093 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56861.log.gz 2024-04-24T15:35:24.093 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.50391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55758.log 2024-04-24T15:35:24.093 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50391.log.gz 2024-04-24T15:35:24.093 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.67457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66763.log 2024-04-24T15:35:24.094 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67457.log.gz 2024-04-24T15:35:24.094 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.55758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67065.log 2024-04-24T15:35:24.094 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55758.log.gz 2024-04-24T15:35:24.094 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.66763.log: gzip -5 --verbose -- 0.0%/var/log/ceph/ceph-client.admin.65703.log 2024-04-24T15:35:24.094 INFO:teuthology.orchestra.run.smithi096.stderr: -- replaced with /var/log/ceph/ceph-client.admin.66763.log.gz 2024-04-24T15:35:24.094 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.67065.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54766.log 2024-04-24T15:35:24.094 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67065.log.gz 2024-04-24T15:35:24.094 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.65703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60752.log 2024-04-24T15:35:24.094 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65703.log.gz 2024-04-24T15:35:24.094 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.54766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64076.log 2024-04-24T15:35:24.094 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54766.log.gz 2024-04-24T15:35:24.094 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.60752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66083.log 2024-04-24T15:35:24.094 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60752.log.gz 2024-04-24T15:35:24.094 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.64076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64990.log 2024-04-24T15:35:24.095 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64076.log.gz 2024-04-24T15:35:24.095 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.66083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50342.log 2024-04-24T15:35:24.095 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66083.log.gz 2024-04-24T15:35:24.095 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67114.log 2024-04-24T15:35:24.095 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.64990.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64990.log.gz 2024-04-24T15:35:24.095 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.50342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56797.log 2024-04-24T15:35:24.095 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50342.log.gz 2024-04-24T15:35:24.095 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.67114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58999.log 2024-04-24T15:35:24.095 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67114.log.gz 2024-04-24T15:35:24.095 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.56797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56335.log 2024-04-24T15:35:24.095 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56797.log.gz 2024-04-24T15:35:24.095 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.58999.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58999.log.gz 2024-04-24T15:35:24.095 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67310.log 2024-04-24T15:35:24.095 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.56335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67359.log 2024-04-24T15:35:24.096 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56335.log.gz 2024-04-24T15:35:24.096 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.67310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63646.log 2024-04-24T15:35:24.096 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67310.log.gz 2024-04-24T15:35:24.096 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.67359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68388.log 2024-04-24T15:35:24.096 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67359.log.gz 2024-04-24T15:35:24.096 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.63646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57612.log 2024-04-24T15:35:24.096 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63646.log.gz 2024-04-24T15:35:24.096 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.68388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51729.log 2024-04-24T15:35:24.096 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68388.log.gz 2024-04-24T15:35:24.096 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.57612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44088.log 2024-04-24T15:35:24.096 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57612.log.gz 2024-04-24T15:35:24.096 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.51729.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59602.log 2024-04-24T15:35:24.096 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51729.log.gz 2024-04-24T15:35:24.096 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.44088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51217.log 2024-04-24T15:35:24.097 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44088.log.gz 2024-04-24T15:35:24.097 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.59602.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-24T15:35:24.097 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59602.log.gz 2024-04-24T15:35:24.097 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.51217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62518.log 2024-04-24T15:35:24.097 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51217.log.gz 2024-04-24T15:35:24.097 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67783.log 2024-04-24T15:35:24.097 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-osd.2.log: /var/log/ceph/ceph-client.admin.62518.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-24T15:35:24.097 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62518.log.gz 2024-04-24T15:35:24.097 INFO:teuthology.orchestra.run.smithi096.stderr:gzip /var/log/ceph/ceph-client.admin.67783.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.48948.log 2024-04-24T15:35:24.097 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67783.log.gz 2024-04-24T15:35:24.097 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52927.log 2024-04-24T15:35:24.097 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.48948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56247.log 2024-04-24T15:35:24.098 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48948.log.gz 2024-04-24T15:35:24.098 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.52927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67779.log 2024-04-24T15:35:24.098 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52927.log.gz 2024-04-24T15:35:24.098 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.56247.log: gzip -5 --verbose 0.0%-- /var/log/ceph/ceph-client.admin.69074.log -- replaced with /var/log/ceph/ceph-client.admin.56247.log.gz 2024-04-24T15:35:24.098 INFO:teuthology.orchestra.run.smithi096.stderr: 2024-04-24T15:35:24.098 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.67779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68116.log 2024-04-24T15:35:24.098 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67779.log.gz 2024-04-24T15:35:24.098 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.69074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61614.log 2024-04-24T15:35:24.098 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69074.log.gz 2024-04-24T15:35:24.098 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.68116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63192.log 2024-04-24T15:35:24.099 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68116.log.gz 2024-04-24T15:35:24.099 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.61614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64682.log 2024-04-24T15:35:24.099 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61614.log.gz 2024-04-24T15:35:24.099 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.63192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67786.log 2024-04-24T15:35:24.099 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63192.log.gz 2024-04-24T15:35:24.099 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.64682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49850.log 2024-04-24T15:35:24.099 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64682.log.gz 2024-04-24T15:35:24.099 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.67786.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68512.log 2024-04-24T15:35:24.099 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67786.log.gz 2024-04-24T15:35:24.099 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.49850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68175.log 2024-04-24T15:35:24.099 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49850.log.gz 2024-04-24T15:35:24.099 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.68512.log: gzip 0.0%-5 -- replaced with /var/log/ceph/ceph-client.admin.68512.log.gz--verbose 2024-04-24T15:35:24.099 INFO:teuthology.orchestra.run.smithi096.stderr:-- /var/log/ceph/ceph-client.admin.68117.log 2024-04-24T15:35:24.099 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.68175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48625.log 2024-04-24T15:35:24.099 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68175.log.gz 2024-04-24T15:35:24.100 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.68117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68518.log 2024-04-24T15:35:24.100 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68117.log.gz 2024-04-24T15:35:24.100 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.48625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50673.log 2024-04-24T15:35:24.100 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48625.log.gz 2024-04-24T15:35:24.100 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.68518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55688.log 2024-04-24T15:35:24.100 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68518.log.gz 2024-04-24T15:35:24.100 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.0.50673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49556.log 2024-04-24T15:35:24.100 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.55688.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55688.log.gz 2024-04-24T15:35:24.100 INFO:teuthology.orchestra.run.smithi096.stderr:gzip 94.6%-5 --verbose -- replaced with /var/log/ceph/ceph-client.0.50673.log.gz-- 2024-04-24T15:35:24.100 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.43708.log 2024-04-24T15:35:24.100 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-24T15:35:24.100 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.49556.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49556.log.gz 2024-04-24T15:35:24.100 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.43708.log: gzip -5 --verbose -- 0.0%/var/log/ceph/ceph-client.admin.63554.log 2024-04-24T15:35:24.100 INFO:teuthology.orchestra.run.smithi096.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43708.log.gz 2024-04-24T15:35:24.101 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58624.log 2024-04-24T15:35:24.101 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-client.admin.63554.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63554.log.gz 2024-04-24T15:35:24.101 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50440.log 2024-04-24T15:35:24.101 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.58624.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.58624.log.gz-5 2024-04-24T15:35:24.101 INFO:teuthology.orchestra.run.smithi096.stderr:--verbose -- /var/log/ceph/ceph-client.admin.54840.log 2024-04-24T15:35:24.101 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57394.log 2024-04-24T15:35:24.101 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.50440.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50440.log.gz 2024-04-24T15:35:24.101 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.54840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67163.log 2024-04-24T15:35:24.101 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54840.log.gz 2024-04-24T15:35:24.101 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.57394.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57394.log.gz 2024-04-24T15:35:24.101 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52758.log 2024-04-24T15:35:24.101 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.67163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59109.log 2024-04-24T15:35:24.101 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67163.log.gz 2024-04-24T15:35:24.101 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.52758.log: 93.2% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-24T15:35:24.101 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65256.log 2024-04-24T15:35:24.101 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52758.log.gz 2024-04-24T15:35:24.102 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.59109.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57441.log 2024-04-24T15:35:24.102 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59109.log.gz 2024-04-24T15:35:24.102 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58142.log 2024-04-24T15:35:24.102 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.65256.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65256.log.gz 2024-04-24T15:35:24.102 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.57441.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57441.log.gz 2024-04-24T15:35:24.102 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68178.log 2024-04-24T15:35:24.102 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 /var/log/ceph/ceph-client.admin.58142.log: --verbose -- /var/log/ceph/ceph-client.admin.69221.log 2024-04-24T15:35:24.102 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58142.log.gz 2024-04-24T15:35:24.102 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.68178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66861.log 2024-04-24T15:35:24.102 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68178.log.gz 2024-04-24T15:35:24.102 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.69221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56990.log 2024-04-24T15:35:24.102 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69221.log.gz 2024-04-24T15:35:24.102 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.66861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62595.log 2024-04-24T15:35:24.102 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66861.log.gz 2024-04-24T15:35:24.103 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.56990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69907.log 2024-04-24T15:35:24.103 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56990.log.gz 2024-04-24T15:35:24.103 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.62595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67781.log 2024-04-24T15:35:24.103 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62595.log.gz 2024-04-24T15:35:24.103 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.69907.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69907.log.gz 2024-04-24T15:35:24.103 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49172.log 2024-04-24T15:35:24.103 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48946.log /var/log/ceph/ceph-client.admin.67781.log: 2024-04-24T15:35:24.103 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67781.log.gz 2024-04-24T15:35:24.103 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.49172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66714.log 2024-04-24T15:35:24.103 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49172.log.gz 2024-04-24T15:35:24.103 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.48946.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48946.log.gz 2024-04-24T15:35:24.103 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50489.log 2024-04-24T15:35:24.103 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62793.log 2024-04-24T15:35:24.103 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.66714.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66714.log.gz 2024-04-24T15:35:24.103 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.50489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52443.log 2024-04-24T15:35:24.104 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50489.log.gz 2024-04-24T15:35:24.104 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.62793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68517.log 2024-04-24T15:35:24.104 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62793.log.gz 2024-04-24T15:35:24.104 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-24T15:35:24.104 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.52443.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52443.log.gz 2024-04-24T15:35:24.104 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.68517.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68517.log.gz 2024-04-24T15:35:24.104 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69564.log 2024-04-24T15:35:24.104 INFO:teuthology.orchestra.run.smithi096.stderr:gzip /var/log/ceph/ceph-osd.3.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.68829.log 2024-04-24T15:35:24.104 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.69564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59906.log 2024-04-24T15:35:24.104 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69564.log.gz 2024-04-24T15:35:24.104 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.68829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52079.log 2024-04-24T15:35:24.104 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68829.log.gz 2024-04-24T15:35:24.104 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.59906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69613.log 2024-04-24T15:35:24.104 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59906.log.gz 2024-04-24T15:35:24.104 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.52079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54183.log 2024-04-24T15:35:24.105 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52079.log.gz 2024-04-24T15:35:24.105 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.69613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59553.log 2024-04-24T15:35:24.105 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69613.log.gz 2024-04-24T15:35:24.105 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.54183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66910.log 2024-04-24T15:35:24.105 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54183.log.gz 2024-04-24T15:35:24.105 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.59553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55063.log 2024-04-24T15:35:24.105 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59553.log.gz 2024-04-24T15:35:24.105 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.66910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53647.log 2024-04-24T15:35:24.105 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66910.log.gz 2024-04-24T15:35:24.105 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.55063.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-24T15:35:24.105 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55063.log.gz 2024-04-24T15:35:24.105 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.53647.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.53647.log.gz-5 2024-04-24T15:35:24.105 INFO:teuthology.orchestra.run.smithi096.stderr:--verbose -- /var/log/ceph/ceph-client.admin.60250.log 2024-04-24T15:35:24.105 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49997.log 2024-04-24T15:35:24.106 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.60250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48950.log 2024-04-24T15:35:24.106 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60250.log.gz 2024-04-24T15:35:24.106 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.49997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64299.log 2024-04-24T15:35:24.106 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49997.log.gz 2024-04-24T15:35:24.106 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.48950.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48950.log.gz 2024-04-24T15:35:24.106 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.64299.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64299.log.gz 2024-04-24T15:35:24.106 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69662.log 2024-04-24T15:35:24.106 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53287.log 2024-04-24T15:35:24.106 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.69662.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69662.log.gz 2024-04-24T15:35:24.106 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.53287.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53287.log.gz 2024-04-24T15:35:24.106 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67212.log 2024-04-24T15:35:24.106 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.37474.log 2024-04-24T15:35:24.106 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.67212.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67212.log.gz 2024-04-24T15:35:24.106 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph.tmp-client.admin.37474.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.37474.log.gz 2024-04-24T15:35:24.107 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62049.log 2024-04-24T15:35:24.107 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58385.log 2024-04-24T15:35:24.107 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.62049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44218.log 2024-04-24T15:35:24.107 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62049.log.gz 2024-04-24T15:35:24.107 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.58385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56559.log 2024-04-24T15:35:24.107 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58385.log.gz 2024-04-24T15:35:24.107 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.44218.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44218.log.gz 2024-04-24T15:35:24.107 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.56559.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56559.log.gz 2024-04-24T15:35:24.107 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55687.log 2024-04-24T15:35:24.107 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57346.log 2024-04-24T15:35:24.107 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.55687.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.55687.log.gz-5 2024-04-24T15:35:24.107 INFO:teuthology.orchestra.run.smithi096.stderr:--verbose -- /var/log/ceph/ceph-client.admin.65428.log 2024-04-24T15:35:24.107 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56404.log 2024-04-24T15:35:24.108 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.57346.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57346.log.gz 2024-04-24T15:35:24.108 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.65428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66959.log 2024-04-24T15:35:24.108 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65428.log.gz 2024-04-24T15:35:24.108 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.56404.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56404.log.gz 2024-04-24T15:35:24.118 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-24T15:35:24.118 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43602.log 2024-04-24T15:35:24.118 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-24T15:35:24.118 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.52856.log 2024-04-24T15:35:24.118 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.43602.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43602.log.gz 2024-04-24T15:35:24.118 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43749.log 2024-04-24T15:35:24.118 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/objectstore_tool.52856.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.52033.log 2024-04-24T15:35:24.118 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.43749.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43749.log.gz 2024-04-24T15:35:24.119 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43848.log 2024-04-24T15:35:24.119 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.54635.log 2024-04-24T15:35:24.119 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/objectstore_tool.52033.log: /var/log/ceph/ceph-client.admin.43848.log: 88.0% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-24T15:35:24.119 INFO:teuthology.orchestra.run.smithi151.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43848.log.gz 2024-04-24T15:35:24.119 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/objectstore_tool.54635.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-24T15:35:24.119 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43554.log 2024-04-24T15:35:24.119 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-osd.7.log: /var/log/ceph/ceph-client.admin.43554.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43554.log.gz 2024-04-24T15:35:24.119 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-24T15:35:24.119 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43699.log 2024-04-24T15:35:24.119 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-mgr.x.log: /var/log/ceph/ceph-client.admin.43699.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43699.log.gz 2024-04-24T15:35:24.119 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43898.log 2024-04-24T15:35:24.119 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.43898.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43898.log.gz 2024-04-24T15:35:24.119 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool..log 2024-04-24T15:35:24.120 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/objectstore_tool..log: 95.2% -- replaced with /var/log/ceph/objectstore_tool.52856.log.gz 2024-04-24T15:35:24.120 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43650.log 2024-04-24T15:35:24.120 INFO:teuthology.orchestra.run.smithi151.stderr: 95.3% -- replaced with /var/log/ceph/objectstore_tool.52033.log.gz 2024-04-24T15:35:24.120 INFO:teuthology.orchestra.run.smithi151.stderr: 95.2%gzip -5 --verbose -- replaced with /var/log/ceph/objectstore_tool.54635.log.gz-- 2024-04-24T15:35:24.120 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-mon.b.log 2024-04-24T15:35:24.120 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.43650.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43650.log.gz 2024-04-24T15:35:24.120 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.53840.log 2024-04-24T15:35:24.120 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-24T15:35:24.120 INFO:teuthology.orchestra.run.smithi151.stderr: 93.0% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-24T15:35:24.120 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/objectstore_tool.53840.log: /var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-24T15:35:24.120 INFO:teuthology.orchestra.run.smithi151.stderr: 83.3% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-24T15:35:24.120 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-24T15:35:24.120 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-osd.4.log: /var/log/ceph/ceph-osd.5.log: 95.3% -- replaced with /var/log/ceph/objectstore_tool.53840.log.gz 2024-04-24T15:35:24.120 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43800.log 2024-04-24T15:35:24.120 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.43800.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43800.log.gz 2024-04-24T15:35:24.130 INFO:teuthology.orchestra.run.smithi151.stderr: 95.3% -- replaced with /var/log/ceph/objectstore_tool..log.gz 2024-04-24T15:35:24.140 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.66959.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66959.log.gz 2024-04-24T15:35:24.140 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57216.log 2024-04-24T15:35:24.140 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49948.log 2024-04-24T15:35:24.140 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.57216.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57216.log.gz 2024-04-24T15:35:24.140 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.49948.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49948.log.gz 2024-04-24T15:35:24.140 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64731.log 2024-04-24T15:35:24.140 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50097.log 2024-04-24T15:35:24.141 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.64731.log: gzip -5 --verbose -- 0.0%/var/log/ceph/ceph-client.admin.67261.log 2024-04-24T15:35:24.141 INFO:teuthology.orchestra.run.smithi096.stderr: -- replaced with /var/log/ceph/ceph-client.admin.64731.log.gz 2024-04-24T15:35:24.141 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.50097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55127.log 2024-04-24T15:35:24.141 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50097.log.gz 2024-04-24T15:35:24.141 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.67261.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67261.log.gz 2024-04-24T15:35:24.141 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.55127.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55127.log.gz 2024-04-24T15:35:24.141 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50146.log 2024-04-24T15:35:24.141 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68118.log 2024-04-24T15:35:24.141 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.50146.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50146.log.gz 2024-04-24T15:35:24.141 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.68118.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68118.log.gz 2024-04-24T15:35:24.141 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68511.log 2024-04-24T15:35:24.141 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48723.log 2024-04-24T15:35:24.141 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.68511.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68511.log.gz 2024-04-24T15:35:24.141 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.48723.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48723.log.gz 2024-04-24T15:35:24.141 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61662.log 2024-04-24T15:35:24.142 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52328.log 2024-04-24T15:35:24.142 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.61662.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61662.log.gz 2024-04-24T15:35:24.142 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.52328.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52328.log.gz 2024-04-24T15:35:24.142 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69025.log 2024-04-24T15:35:24.142 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66812.log 2024-04-24T15:35:24.142 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.69025.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69025.log.gz 2024-04-24T15:35:24.142 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.66812.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66812.log.gz 2024-04-24T15:35:24.142 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53530.log 2024-04-24T15:35:24.142 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50788.log 2024-04-24T15:35:24.142 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.53530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66587.log 2024-04-24T15:35:24.143 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53530.log.gz 2024-04-24T15:35:24.143 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.50788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64778.log 2024-04-24T15:35:24.143 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50788.log.gz 2024-04-24T15:35:24.143 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.66587.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66587.log.gz 2024-04-24T15:35:24.143 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53411.log 2024-04-24T15:35:24.143 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68780.log 2024-04-24T15:35:24.143 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.64778.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64778.log.gz 2024-04-24T15:35:24.143 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.53411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48951.log 2024-04-24T15:35:24.143 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53411.log.gz 2024-04-24T15:35:24.143 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.68780.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68780.log.gz 2024-04-24T15:35:24.144 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62895.log 2024-04-24T15:35:24.144 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67775.log 2024-04-24T15:35:24.144 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.48951.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48951.log.gz 2024-04-24T15:35:24.144 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.62895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59375.log 2024-04-24T15:35:24.144 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62895.log.gz 2024-04-24T15:35:24.144 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.67775.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67775.log.gz 2024-04-24T15:35:24.144 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49608.log 2024-04-24T15:35:24.144 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66490.log 2024-04-24T15:35:24.144 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.59375.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59375.log.gz 2024-04-24T15:35:24.144 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.49608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66464.log 2024-04-24T15:35:24.144 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49608.log.gz 2024-04-24T15:35:24.144 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.66490.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66490.log.gz 2024-04-24T15:35:24.144 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63749.log 2024-04-24T15:35:24.144 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65757.log 2024-04-24T15:35:24.145 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.66464.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66464.log.gz 2024-04-24T15:35:24.145 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.63749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68127.log 2024-04-24T15:35:24.145 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63749.log.gz 2024-04-24T15:35:24.145 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.65757.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65757.log.gz 2024-04-24T15:35:24.145 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68976.log 2024-04-24T15:35:24.145 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.68127.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68127.log.gz 2024-04-24T15:35:24.145 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.68976.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68976.log.gz 2024-04-24T15:35:24.145 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56196.log 2024-04-24T15:35:24.145 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52105.log 2024-04-24T15:35:24.145 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.56196.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56196.log.gz 2024-04-24T15:35:24.145 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.52105.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52105.log.gz 2024-04-24T15:35:24.145 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48518.log 2024-04-24T15:35:24.145 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66588.log 2024-04-24T15:35:24.145 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.48518.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48518.log.gz 2024-04-24T15:35:24.145 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58093.log 2024-04-24T15:35:24.145 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61179.log 2024-04-24T15:35:24.146 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.66588.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66588.log.gz 2024-04-24T15:35:24.146 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.58093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67785.log 2024-04-24T15:35:24.146 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58093.log.gz 2024-04-24T15:35:24.146 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.61179.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61179.log.gz 2024-04-24T15:35:24.146 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.67785.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67785.log.gz 2024-04-24T15:35:24.146 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54512.log 2024-04-24T15:35:24.146 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55564.log 2024-04-24T15:35:24.146 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.54512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60036.log 2024-04-24T15:35:24.146 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54512.log.gz 2024-04-24T15:35:24.146 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.55564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60259.log 2024-04-24T15:35:24.146 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55564.log.gz 2024-04-24T15:35:24.146 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67555.log 2024-04-24T15:35:24.146 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.60036.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60036.log.gz 2024-04-24T15:35:24.146 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.60259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49307.log 2024-04-24T15:35:24.147 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60259.log.gz 2024-04-24T15:35:24.147 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.67555.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67555.log.gz 2024-04-24T15:35:24.147 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65477.log 2024-04-24T15:35:24.147 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49686.log 2024-04-24T15:35:24.147 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.49307.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49307.log.gz 2024-04-24T15:35:24.147 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.65477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62111.log 2024-04-24T15:35:24.147 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65477.log.gz 2024-04-24T15:35:24.147 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.49686.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49686.log.gz 2024-04-24T15:35:24.147 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.62111.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62111.log.gz 2024-04-24T15:35:24.147 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48934.log 2024-04-24T15:35:24.147 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49302.log 2024-04-24T15:35:24.147 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.48934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69760.log 2024-04-24T15:35:24.147 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48934.log.gz 2024-04-24T15:35:24.147 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.49302.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49302.log.gz 2024-04-24T15:35:24.148 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.69760.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69760.log.gz 2024-04-24T15:35:24.148 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68124.log 2024-04-24T15:35:24.148 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48468.log 2024-04-24T15:35:24.148 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.68124.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68124.log.gz 2024-04-24T15:35:24.148 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.48468.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48468.log.gz 2024-04-24T15:35:24.148 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61061.log 2024-04-24T15:35:24.148 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69956.log 2024-04-24T15:35:24.148 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.61061.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61061.log.gz 2024-04-24T15:35:24.148 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63096.log 2024-04-24T15:35:24.148 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67784.log 2024-04-24T15:35:24.148 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.69956.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69956.log.gz 2024-04-24T15:35:24.148 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.63096.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-24T15:35:24.148 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63096.log.gz 2024-04-24T15:35:24.148 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.67784.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67784.log.gz 2024-04-24T15:35:24.148 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67408.log 2024-04-24T15:35:24.149 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70109.log 2024-04-24T15:35:24.149 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.67408.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67408.log.gz 2024-04-24T15:35:24.149 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.70109.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70109.log.gz 2024-04-24T15:35:24.149 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50195.log 2024-04-24T15:35:24.149 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49270.log 2024-04-24T15:35:24.149 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.50195.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50195.log.gz 2024-04-24T15:35:24.149 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.49270.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49270.log.gz 2024-04-24T15:35:24.149 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55613.log 2024-04-24T15:35:24.149 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55808.log 2024-04-24T15:35:24.149 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.55613.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55613.log.gz 2024-04-24T15:35:24.149 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.55808.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55808.log.gz 2024-04-24T15:35:24.149 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68085.log 2024-04-24T15:35:24.149 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49318.log 2024-04-24T15:35:24.149 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.68085.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68085.log.gz 2024-04-24T15:35:24.149 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.49318.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49318.log.gz 2024-04-24T15:35:24.150 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49300.log 2024-04-24T15:35:24.150 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-24T15:35:24.150 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.49300.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49300.log.gz 2024-04-24T15:35:24.150 INFO:teuthology.orchestra.run.smithi096.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-24T15:35:24.150 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose /var/log/ceph/ceph-mon.c.log: -- /var/log/ceph/ceph-client.admin.48954.log 2024-04-24T15:35:24.150 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60534.log 2024-04-24T15:35:24.150 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.48954.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48954.log.gz 2024-04-24T15:35:24.150 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.60534.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60534.log.gz 2024-04-24T15:35:24.150 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68563.log 2024-04-24T15:35:24.150 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64427.log 2024-04-24T15:35:24.150 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.68563.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68563.log.gz 2024-04-24T15:35:24.151 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.64427.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64427.log.gz 2024-04-24T15:35:24.151 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69809.log 2024-04-24T15:35:24.151 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51475.log 2024-04-24T15:35:24.151 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.69809.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69809.log.gz 2024-04-24T15:35:24.151 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.51475.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51475.log.gz 2024-04-24T15:35:24.151 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61390.log 2024-04-24T15:35:24.151 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69466.log 2024-04-24T15:35:24.151 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.61390.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61390.log.gz 2024-04-24T15:35:24.151 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.69466.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69466.log.gz 2024-04-24T15:35:24.151 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51284.log 2024-04-24T15:35:24.151 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53914.log 2024-04-24T15:35:24.151 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.51284.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51284.log.gz 2024-04-24T15:35:24.151 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.53914.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53914.log.gz 2024-04-24T15:35:24.151 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68581.log 2024-04-24T15:35:24.151 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64891.log 2024-04-24T15:35:24.152 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.68581.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68581.log.gz 2024-04-24T15:35:24.152 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.64891.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64891.log.gz 2024-04-24T15:35:24.152 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55444.log 2024-04-24T15:35:24.152 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-24T15:35:24.152 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.55444.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55444.log.gz 2024-04-24T15:35:24.152 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph.log: 90.9% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-24T15:35:24.152 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57509.log 2024-04-24T15:35:24.152 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67768.log 2024-04-24T15:35:24.152 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.57509.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57509.log.gz 2024-04-24T15:35:24.152 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.67768.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67768.log.gz 2024-04-24T15:35:24.152 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48947.log 2024-04-24T15:35:24.152 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53260.log 2024-04-24T15:35:24.153 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.48947.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48947.log.gz 2024-04-24T15:35:24.153 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.53260.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53260.log.gz 2024-04-24T15:35:24.153 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52878.log 2024-04-24T15:35:24.153 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50789.log 2024-04-24T15:35:24.153 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.52878.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52878.log.gz 2024-04-24T15:35:24.153 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.50789.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50789.log.gz 2024-04-24T15:35:24.153 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69711.log 2024-04-24T15:35:24.153 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70062.log 2024-04-24T15:35:24.153 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.69711.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69711.log.gz 2024-04-24T15:35:24.153 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.70062.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70062.log.gz 2024-04-24T15:35:24.153 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66027.log 2024-04-24T15:35:24.153 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54026.log 2024-04-24T15:35:24.153 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.66027.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66027.log.gz 2024-04-24T15:35:24.154 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.54026.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54026.log.gz 2024-04-24T15:35:24.154 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60960.log 2024-04-24T15:35:24.154 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63865.log 2024-04-24T15:35:24.154 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.60960.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60960.log.gz 2024-04-24T15:35:24.154 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.63865.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63865.log.gz 2024-04-24T15:35:24.154 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67008.log 2024-04-24T15:35:24.154 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57714.log 2024-04-24T15:35:24.154 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.67008.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67008.log.gz 2024-04-24T15:35:24.154 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.57714.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57714.log.gz 2024-04-24T15:35:24.154 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50293.log 2024-04-24T15:35:24.161 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.50293.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50293.log.gz 2024-04-24T15:35:24.161 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50538.log 2024-04-24T15:35:24.162 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50244.log 2024-04-24T15:35:24.162 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.50538.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50538.log.gz 2024-04-24T15:35:24.162 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.50244.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50244.log.gz 2024-04-24T15:35:24.162 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68878.log 2024-04-24T15:35:24.162 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48674.log 2024-04-24T15:35:24.162 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.68878.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68878.log.gz 2024-04-24T15:35:24.162 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.48674.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48674.log.gz 2024-04-24T15:35:24.162 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52442.log 2024-04-24T15:35:24.162 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62267.log 2024-04-24T15:35:24.162 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.52442.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52442.log.gz 2024-04-24T15:35:24.162 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.62267.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62267.log.gz 2024-04-24T15:35:24.162 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49493.log 2024-04-24T15:35:24.162 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50614.log 2024-04-24T15:35:24.162 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.49493.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49493.log.gz 2024-04-24T15:35:24.163 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.50614.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50614.log.gz 2024-04-24T15:35:24.163 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49687.log 2024-04-24T15:35:24.163 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64940.log 2024-04-24T15:35:24.163 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.49687.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49687.log.gz 2024-04-24T15:35:24.163 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.64940.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64940.log.gz 2024-04-24T15:35:24.163 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54844.log 2024-04-24T15:35:24.163 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69172.log 2024-04-24T15:35:24.163 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.54844.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54844.log.gz 2024-04-24T15:35:24.163 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60650.log 2024-04-24T15:35:24.163 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.69172.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69172.log.gz 2024-04-24T15:35:24.163 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59025.log 2024-04-24T15:35:24.163 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.60650.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60650.log.gz 2024-04-24T15:35:24.163 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.59025.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59025.log.gz 2024-04-24T15:35:24.163 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49301.log 2024-04-24T15:35:24.163 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69417.log 2024-04-24T15:35:24.164 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.49301.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49301.log.gz 2024-04-24T15:35:24.164 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.69417.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69417.log.gz 2024-04-24T15:35:24.164 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48949.log 2024-04-24T15:35:24.164 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68516.log 2024-04-24T15:35:24.164 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.48949.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48949.log.gz 2024-04-24T15:35:24.164 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.68516.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68516.log.gz 2024-04-24T15:35:24.164 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66660.log 2024-04-24T15:35:24.164 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58190.log 2024-04-24T15:35:24.164 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.66660.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66660.log.gz 2024-04-24T15:35:24.164 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.58190.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58190.log.gz 2024-04-24T15:35:24.164 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51825.log 2024-04-24T15:35:24.164 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63145.log 2024-04-24T15:35:24.164 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.51825.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51825.log.gz 2024-04-24T15:35:24.164 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.63145.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63145.log.gz 2024-04-24T15:35:24.164 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54464.log 2024-04-24T15:35:24.165 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69319.log 2024-04-24T15:35:24.165 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.54464.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54464.log.gz 2024-04-24T15:35:24.165 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.69319.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69319.log.gz 2024-04-24T15:35:24.165 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60856.log 2024-04-24T15:35:24.165 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53978.log 2024-04-24T15:35:24.165 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.60856.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60856.log.gz 2024-04-24T15:35:24.165 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.53978.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53978.log.gz 2024-04-24T15:35:24.165 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49899.log 2024-04-24T15:35:24.165 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48397.log 2024-04-24T15:35:24.165 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.49899.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49899.log.gz 2024-04-24T15:35:24.165 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.48397.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48397.log.gz 2024-04-24T15:35:24.165 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-24T15:35:24.166 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70005.log 2024-04-24T15:35:24.166 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-osd.1.log: /var/log/ceph/ceph-client.admin.70005.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70005.log.gz 2024-04-24T15:35:24.166 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65907.log 2024-04-24T15:35:24.166 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.65907.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65907.log.gz 2024-04-24T15:35:24.166 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69270.log 2024-04-24T15:35:24.166 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.69270.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69270.log.gz 2024-04-24T15:35:24.166 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64181.log 2024-04-24T15:35:24.166 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.64181.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64181.log.gz 2024-04-24T15:35:24.166 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57912.log 2024-04-24T15:35:24.167 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.57912.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57912.log.gz 2024-04-24T15:35:24.167 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49607.log 2024-04-24T15:35:24.167 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.49607.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49607.log.gz 2024-04-24T15:35:24.167 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69123.log 2024-04-24T15:35:24.167 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.69123.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69123.log.gz 2024-04-24T15:35:24.167 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51755.log 2024-04-24T15:35:24.167 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.51755.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51755.log.gz 2024-04-24T15:35:24.167 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64806.log 2024-04-24T15:35:24.167 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.64806.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64806.log.gz 2024-04-24T15:35:24.167 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69368.log 2024-04-24T15:35:24.167 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.69368.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69368.log.gz 2024-04-24T15:35:24.167 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57886.log 2024-04-24T15:35:24.167 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.57886.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57886.log.gz 2024-04-24T15:35:24.201 INFO:teuthology.orchestra.run.smithi151.stderr: 91.0% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-24T15:35:24.250 INFO:teuthology.orchestra.run.smithi096.stderr: 91.0% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-24T15:35:24.354 INFO:teuthology.orchestra.run.smithi096.stderr: 93.2% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-24T15:35:24.427 INFO:teuthology.orchestra.run.smithi096.stderr: 90.5% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-24T15:35:24.694 INFO:teuthology.orchestra.run.smithi151.stderr: 95.2% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-24T15:35:25.016 INFO:teuthology.orchestra.run.smithi151.stderr: 95.2% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-24T15:35:25.153 INFO:teuthology.orchestra.run.smithi096.stderr: 94.6% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-24T15:35:25.292 INFO:teuthology.orchestra.run.smithi096.stderr: 94.7% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-24T15:35:25.936 INFO:teuthology.orchestra.run.smithi151.stderr: 95.1% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-24T15:35:26.413 INFO:teuthology.orchestra.run.smithi151.stderr: 95.0% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-24T15:35:26.414 INFO:teuthology.orchestra.run.smithi151.stderr: 2024-04-24T15:35:26.415 INFO:teuthology.orchestra.run.smithi151.stderr:real 0m2.326s 2024-04-24T15:35:26.415 INFO:teuthology.orchestra.run.smithi151.stderr:user 0m5.566s 2024-04-24T15:35:26.415 INFO:teuthology.orchestra.run.smithi151.stderr:sys 0m0.246s 2024-04-24T15:35:28.546 INFO:teuthology.orchestra.run.smithi096.stderr: 95.0% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-24T15:35:28.547 INFO:teuthology.orchestra.run.smithi096.stderr: 2024-04-24T15:35:28.547 INFO:teuthology.orchestra.run.smithi096.stderr:real 0m4.497s 2024-04-24T15:35:28.547 INFO:teuthology.orchestra.run.smithi096.stderr:user 0m7.249s 2024-04-24T15:35:28.547 INFO:teuthology.orchestra.run.smithi096.stderr:sys 0m0.375s 2024-04-24T15:35:28.548 INFO:tasks.ceph:Archiving logs... 2024-04-24T15:35:28.548 DEBUG:teuthology.misc:Transferring archived files from smithi096:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-04-24_14:09:24-rbd-wip-yuriw4-testing-20240423.151325-quincy-distro-default-smithi/7671570/remote/smithi096/log 2024-04-24T15:35:28.549 DEBUG:teuthology.orchestra.run.smithi096:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-24T15:35:29.753 DEBUG:teuthology.misc:Transferring archived files from smithi151:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-04-24_14:09:24-rbd-wip-yuriw4-testing-20240423.151325-quincy-distro-default-smithi/7671570/remote/smithi151/log 2024-04-24T15:35:29.754 DEBUG:teuthology.orchestra.run.smithi151:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-24T15:35:30.540 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-24T15:35:30.549 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-24T15:35:30.550 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-24T15:35:30.552 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-24T15:35:30.635 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-24T15:35:30.644 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-24T15:35:30.644 DEBUG:teuthology.orchestra.run.smithi096:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T15:35:30.646 DEBUG:teuthology.orchestra.run.smithi151:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T15:35:30.663 INFO:teuthology.orchestra.run.smithi096.stderr:bash: ntpq: command not found 2024-04-24T15:35:30.665 INFO:teuthology.orchestra.run.smithi151.stderr:bash: ntpq: command not found 2024-04-24T15:35:30.668 INFO:teuthology.orchestra.run.smithi096.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T15:35:30.668 INFO:teuthology.orchestra.run.smithi096.stdout:=============================================================================== 2024-04-24T15:35:30.668 INFO:teuthology.orchestra.run.smithi096.stdout:^- hv01.front.sepia.ceph.com 2 6 377 24 +564us[ +564us] +/- 574ms 2024-04-24T15:35:30.668 INFO:teuthology.orchestra.run.smithi096.stdout:^* hv02.front.sepia.ceph.com 3 6 377 25 -1183us[-1182us] +/- 68ms 2024-04-24T15:35:30.668 INFO:teuthology.orchestra.run.smithi096.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 26 +1206us[+1207us] +/- 65ms 2024-04-24T15:35:30.669 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T15:35:30.669 INFO:teuthology.orchestra.run.smithi151.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T15:35:30.670 INFO:teuthology.orchestra.run.smithi151.stdout:=============================================================================== 2024-04-24T15:35:30.670 INFO:teuthology.orchestra.run.smithi151.stdout:^- hv01.front.sepia.ceph.com 2 6 377 24 +564us[ +563us] +/- 574ms 2024-04-24T15:35:30.670 INFO:teuthology.orchestra.run.smithi151.stdout:^* hv02.front.sepia.ceph.com 3 6 377 24 -1177us[-1178us] +/- 68ms 2024-04-24T15:35:30.670 INFO:teuthology.orchestra.run.smithi151.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 24 +1220us[+1220us] +/- 65ms 2024-04-24T15:35:30.670 INFO:teuthology.orchestra.run.smithi151.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T15:35:30.671 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-24T15:35:30.680 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-24T15:35:30.680 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-24T15:35:30.689 DEBUG:teuthology.orchestra.run.smithi096:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-24T15:35:30.790 DEBUG:teuthology.orchestra.run.smithi151:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-24T15:35:30.863 DEBUG:teuthology.orchestra.run.smithi096:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-24T15:35:30.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T15:35:30.895 DEBUG:teuthology.orchestra.run.smithi151:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-24T15:35:30.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T15:35:30.926 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-24T15:35:30.936 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-24T15:35:30.945 INFO:teuthology.task.internal:Duration was 840.147236 seconds 2024-04-24T15:35:30.945 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-24T15:35:30.954 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-24T15:35:30.954 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-24T15:35:30.956 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-24T15:35:31.001 INFO:teuthology.orchestra.run.smithi096.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T15:35:31.014 INFO:teuthology.orchestra.run.smithi151.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T15:35:31.298 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-24T15:35:31.298 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi096.front.sepia.ceph.com 2024-04-24T15:35:31.298 DEBUG:teuthology.orchestra.run.smithi096:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-24T15:35:31.321 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi151.front.sepia.ceph.com 2024-04-24T15:35:31.321 DEBUG:teuthology.orchestra.run.smithi151:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-24T15:35:31.343 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-24T15:35:31.343 DEBUG:teuthology.orchestra.run.smithi096:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-24T15:35:31.363 DEBUG:teuthology.orchestra.run.smithi151:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-24T15:35:31.427 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-24T15:35:31.427 DEBUG:teuthology.orchestra.run.smithi096:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-24T15:35:31.429 DEBUG:teuthology.orchestra.run.smithi151:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-24T15:35:31.541 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-24T15:35:31.552 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-24T15:35:31.552 DEBUG:teuthology.orchestra.run.smithi096:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-24T15:35:31.579 DEBUG:teuthology.orchestra.run.smithi151:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-24T15:35:31.606 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-24T15:35:31.616 DEBUG:teuthology.orchestra.run.smithi096:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-24T15:35:31.622 DEBUG:teuthology.orchestra.run.smithi151:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-24T15:35:31.645 INFO:teuthology.orchestra.run.smithi096.stdout:kernel.core_pattern = core 2024-04-24T15:35:31.672 INFO:teuthology.orchestra.run.smithi151.stdout:kernel.core_pattern = core 2024-04-24T15:35:31.722 DEBUG:teuthology.orchestra.run.smithi096:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-24T15:35:31.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T15:35:31.739 DEBUG:teuthology.orchestra.run.smithi151:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-24T15:35:31.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T15:35:31.779 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-24T15:35:31.791 INFO:teuthology.task.internal:Transferring archived files... 2024-04-24T15:35:31.792 DEBUG:teuthology.misc:Transferring archived files from smithi096:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-04-24_14:09:24-rbd-wip-yuriw4-testing-20240423.151325-quincy-distro-default-smithi/7671570/remote/smithi096 2024-04-24T15:35:31.792 DEBUG:teuthology.orchestra.run.smithi096:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-24T15:35:31.836 DEBUG:teuthology.misc:Transferring archived files from smithi151:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-04-24_14:09:24-rbd-wip-yuriw4-testing-20240423.151325-quincy-distro-default-smithi/7671570/remote/smithi151 2024-04-24T15:35:31.837 DEBUG:teuthology.orchestra.run.smithi151:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-24T15:35:31.875 INFO:teuthology.task.internal:Removing archive directory... 2024-04-24T15:35:31.875 DEBUG:teuthology.orchestra.run.smithi096:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-24T15:35:31.877 DEBUG:teuthology.orchestra.run.smithi151:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-24T15:35:31.927 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-24T15:35:31.941 INFO:teuthology.task.internal:Not uploading archives. 2024-04-24T15:35:31.941 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-24T15:35:31.950 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-24T15:35:31.951 DEBUG:teuthology.orchestra.run.smithi096:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-24T15:35:31.953 DEBUG:teuthology.orchestra.run.smithi151:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-24T15:35:31.968 INFO:teuthology.orchestra.run.smithi096.stdout: 525357 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 24 15:35 /home/ubuntu/cephtest 2024-04-24T15:35:31.985 INFO:teuthology.orchestra.run.smithi151.stdout: 525931 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 24 15:35 /home/ubuntu/cephtest 2024-04-24T15:35:31.986 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-24T15:35:31.999 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-24T15:35:32.022 INFO:teuthology.run:Summary data: description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zstd supported-random-distro$/{centos_8} thrashers/cache thrashosds-health workloads/rbd_fsx_rate_limit} duration: 840.1472356319427 flavor: default owner: scheduled_yuriw@teuthology success: true 2024-04-24T15:35:32.023 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-24T15:35:32.100 INFO:teuthology.run:pass